Why is editing core_pattern restricted?

Entries in procfs are managed by ad hoc code. The code that would set permissions and ownership on the files under /proc/sys (proc_sys_setattr) rejects changes of permissions and ownership with EPERM. So it isn't possible to change the permissions or ownership of these files, full stop. Such changes are not implemented, so being root doesn't help.

When you try to write as a non-root user, you get a permission error. Even with sudo echo "/home/user/foo/core.%e.%p" > /proc/sys/kernel/core_pattern, you're trying to write as a non-root user: sudo runs echo as root, but the redirection happens in the shell from which sudo is executed, and that shell has no elevated privileges. With sudo bash -c '… >…', the redirection is performed in the bash instance which is launched by sudo and which runs as root, so the write succeeds.

The reason only root must be allowed to set the kernel.core_pattern sysctl is that it allows a command to be specified and, since this is a global setting, this command could be executed by any user. This is in fact the case for all sysctl settings to various degrees: they're all global settings, so only root can change them. kernel.core_pattern is just a particularly dangerous case.

On Ubuntu 18.04 I can update the pattern with:

sudo bash -c 'echo "/data/app_crash/%t.%e.core.%p" > /proc/sys/kernel/core_pattern'

I can also update /etc/sysctl.conf and add the line:

kernel.core_pattern = /data/app_crash/%t.%e.core.%p

However, even though there are no other lines setting kernel.core_pattern in /etc/sysctl.conf or /etc/sysctl.d/*, after I reboot the pattern is set to the default value again:

$ sudo sysctl -a | grep kernel.core_pattern
kernel.core_pattern = |/usr/share/apport/apport %p %s %c %d %P

It turned out that apport was over-writing any changes I made. I uninstalled apport with sudo apt-get remove apport and then my changes were used.