How to set shmall, shmmax, shmmin, etc ... in general and for postgresql
I answered this for another question here:
Git fails to push with error 'out of memory'
I'm not answering all your questions here, but the question in your title:
How to set shmall, shmmax, shmmni, etc … in general and for postgresql
With some kernel distributions, there are settings which prevent the kernel from allocation max memory to a single process:
Set Kernel Parameters
/etc/sysctl.conf file to include the lines appropriate to your operating system:
# Red Hat Enterprise Linux 3.0 and CentOS 3.x kernel.shmmax = 2147483648 kernel.shmmni = 4096 kernel.shmall = 2097152 kernel.shmmin = 1 kernel.shmseg = 10 # semaphores: semmsl, semmns, semopm, semmni kernel.sem = 250 32000 100 128 fs.file-max = 65536 # Red Hat Enterprise Linux 4.0 and CentOS 4.x kernel.shmmax = 536870912 kernel.shmmni = 4096 kernel.shmall = 2097152
If your process exceeds the limits, the kernel will kill the process despite the max memory reported being available on your system.
Note: be careful with these settings. You probably don't want to use the settings in that example as i pulled them from a server in our environment.
A few extra notes to mention:
To update and test kernel settings with sysctl, use following commands:
List current settings:
sysctl -A | grep shm sysctl -w kernel.shmmax=<value> to write in sysctl.conf sysctl -p /etc/sysctl.conf to read/reload the values from sysctl.conf
Disable secure linux by editing the
/etc/selinux/config file, making sure the SELINUX flag is set as follows.
Is this appropriate ? If not (or not necessarily), in which case would it be appropriate ?
The kernel settings are usually more strictly defined in datacenter environments when ISP providers don't want a single customer process hogging all the resources on a shared server.
You don't usually have to set the kernel memory parameters unless you have a process that is getting killed by the kernel due to resource starvation.
In some cases, postgres can also allocate more mem to specific page sizes than what's available in shared mememory:
* The PostgreSQL server failed to start. Please check the log output: 2011-11-04 05:06:26 UTC FATAL: could not create shared memory segment: Invalid argument 2011-11-04 05:06:26 UTC DETAIL: Failed system call was shmget(key=5432001, size =161849344, 03600). 2011-11-04 05:06:26 UTC HINT: This error usually means that PostgreSQL’s reques t for a shared memory segment exceeded your kernel’s SHMMAX parameter. You can either reduce the request size or reconfigure the kernel with larger SHMMAX. To reduce the request size (currently 161849344 bytes), reduce PostgreSQL’s shared _buffers parameter (currently 19200) and/or its max_connections parameter (curre ntly 53). If the request size is already small, it’s possible that it is less than your kernel’s SHMMIN parameter, in which case raising the request size or recon figuring SHMMIN is called for. The PostgreSQL documentation contains more information about shared memo ry configuration. …fail!
Errors such as the example above, can be resolved by tweaking your kernel resource settings. The recommended settings and methods, for determining resource settings, are described in detail here:
However, you really don't have to touch these settings unless you're encountering resource starvation situations related to the postgres process. These situations occur, most often, in shared environments or servers with little resources allocated to them.
Can anybody explain how to really set them from the ground up ?
As for Postgres tuning, you should read this:
Oh!! I find a wonderful tool to calculate the optimus mem config of our postgresql server here.