How does vm.overcommit_memory work?
You can find the documentation in
man 5 proc (or at kernel.org):
/proc/sys/vm/overcommit_memory This file contains the kernel virtual memory accounting mode. Values are: 0: heuristic overcommit (this is the default) 1: always overcommit, never check 2: always check, never overcommit In mode 0, calls of mmap(2) with MAP_NORESERVE are not checked, and the default check is very weak, leading to the risk of getting a process "OOM-killed". In mode 2 (available since Linux 2.6), the total virtual address space that can be allocated (CommitLimit in /proc/mem‐ info) is calculated as CommitLimit = (total_RAM - total_huge_TLB) * overcommit_ratio / 100 + total_swap
The simple answer is that setting overcommit to 1, will set the stage so that when a program calls something like
malloc() to allocate a chunk of memory (
man 3 malloc), it will always succeed regardless if the system knows it will not have all the memory that is being asked for.
The underlying concept to understand is the idea of virtual memory. Programs see a virtual address space that may, or may not, be mapped to actual physical memory. By disabling overcommit checking, you tell the OS to just assume that there is always enough physical memory to backup the virtual space.
To highlight why this can sometimes matter, take a look at the Redis guidances on why
vm.overcommit_memory should be set to 1 for it.
This is an old question with a well-established answer, but I think there's more to add.
First of all, when
vm.overcommit_memory = 0, the
vm.overcommit_ratio value is irrelevant. The kernel will use a heuristic algorithm to overcommit memory, so that your
amarok process can be allocated more memory than is available.
When you set
vm.overcommit_ratio value becomes relevant. By default, this value is set to
50, which means the system would only allocate up to 50% of your RAM (plus swap). This explains why you are unable to start programs that was fine when
vm.overcommit_memory = 0 - because there's less than 500MB of allocatable memory (assuming no swap).
When you set it to
300, you are allowing the system to allocate up to 300% of your RAM (plus swap, if any), which is why the
CommitLimit value in
/proc/meminfo is so high.
vm.overcommit_memory = 2 is usually used to prevent overcommitment, here, you're using it to cap the amount that can be overcommitted. Setting it to
300 is dangerous as your system don't have
5171884 kB of memory, and so, depending on how much swap space you have, the system will use swap (which is slow), or will run out of memory altogether.
As to why
amarok uses more memory when
vm.overcommit_memory = 2 - this is probably because
amarok works best with more memory, but is fine with less as well. So the logic of the program may try to allocate 2GB of memory initially, but if it fails, try 1GB.