Apple - Why does setting the hard-limit for maxfiles to "unlimited" using `launchctl limit` result in a hard-limit slightly above the soft-limit?
A fresh system install, with default parameters is configured with the following system wide file limits:
- Kernel Max Files: 12288
- Max Files per Process: 10240
Current kernel sysctl parameters can be viewed with sysctl command:
$ sysctl -a |grep kern.maxf
kern.maxfiles: 12288
kern.maxfilesperproc: 10240
Now let's check limits values, using ulimit command:
For SOFT limits: [we can see that: open files = 256]
$ ulimit -aS
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 256
pipe size (512 bytes, -p) 1
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 266
virtual memory (kbytes, -v) unlimited
For HARD limits: [we can see that: open files = Unlimited]
$ ulimit -aH
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) unlimited
pipe size (512 bytes, -p) 1
stack size (kbytes, -s) 65532
cpu time (seconds, -t) unlimited
max user processes (-u) 532
virtual memory (kbytes, -v) unlimited
Using launchctl
launchctl has the capacity to configure limit values, or show current values.
Using launchctl limit it shows all the current Soft and Hard limits, and we can see that soft limit is 256 and hard limit is 'unlimited' for the item maxfiles. (the same information we got using ulimit command above)
$ uname -a
18.6.0 Darwin Kernel Version 18.6.0: Thu Apr 25 23:16:27 PDT 2019; root:xnu-4903.261.4~2/RELEASE_X86_64 x86_64
$ launchctl limit
cpu unlimited unlimited
filesize unlimited unlimited
data unlimited unlimited
stack 8388608 67104768
core 0 unlimited
rss unlimited unlimited
memlock unlimited unlimited
maxproc 266 532
maxfiles 256 unlimited
To filter, we can just use launchctl limit maxfiles
$ launchctl limit maxfiles
maxfiles 256 unlimited
Now let's try to change those values #
Using your command line as example:
sudo launchctl limit maxfiles 10000 unlimited
But after it, when I execute again the command to display the limits, we can see that:
- the Soft limit was adjusted from 256 to 10000,
- but the Hard limit was reduced from 'unlimited' to '10240'
sudo launchctl limit
maxfiles 10000 10240
And from now, it is impossible to get it back to unlimited!
We can see that after executing it, it also changed the kernel sysctl parameters
$ sysctl -A |grep max
kern.maxfiles: 10240
kern.maxfilesperproc: 10000
The reason is
The word 'unlimited' when used as a parameter for setting maxfiles, is the same as using value '10240'.
Because launchctl do not accept the string 'unlimited' and convert it to the value 10240!
[I will prove it below, with the source code of launchclt and source of MacOS Mojave kernel]
But first, let see if we can use a bigger value like 2 million:
sudo launchctl limit maxfiles 10000 2000000
$ launchctl limit maxfiles
maxfiles 10000 2000000
Yes, we can, it accepted the bigger value.
Why this happens
It is hardcoded on the source code to not accept 'unlimited' as a value, so once we changed it, we can never put it back.
But what is the value that corresponds to 'unlimited' ?
- R: The value of 'unlimited' is INT_MAX, which is '2147483647' (the maximum value of the Integer Type)
Let's go inside the source code #
Source code analysis of launchclt.c and Kernel routines of MacOS 10.14 Mojave about why is not possible to set maxfiles to 'unlimited' via launchctl command line.
All information below is my analysis, using the source code from Apple, that can be obtained via: https://opensource.apple.com
Rafael Prado - 2019/08/09
/* START OF CODE ANALYSIS */
// --------------- Analisys Comment ----------------------------------
// Looking at MacOS system core header file syslimits.h there is
// a definition of OPEN_MAX with value 10240.
//
// This is the base value that will be used by default for
// maxfiles value during system startup.
//
// This comment you are reading here is mine.
//
/* FILE: sys/syslimits.h */
// *****************************************************************
#define OPEN_MAX 10240 /* max open files per process - todo, make a config option? */
// *****************************************************************
//
// --------------- Analisys Comment ---------------------------------
// Now we can see inside kernel config, that maxfiles parameter
// is a subprodut of OPEN_MAX, by adding 2048 to its value,
// which give us the number 12288.
//
// This will became the default value for sysctl kern.maxfiles: 12288
//
/* FILE: conf/param.c */
// *****************************************************************
#define MAXFILES (OPEN_MAX + 2048)
int maxfiles = MAXFILES;
// *****************************************************************
//
// --------------- Analisys Comment ---------------------------------
// Now here is why it is impossible to set 'unlimited' value using
// the command line of launchctl:
//
/* FILE: launchd/support/launchctl.c */
// *****************************************************************
//
// (:..) Function lim2str() on line 2810
const char *
lim2str(rlim_t val, char *buf)
{
if (val == RLIM_INFINITY)
strcpy(buf, "unlimited");
else
sprintf(buf, "%lld", val);
return buf;
}
// (:..) Function srt2lim() on line 2821
bool
str2lim(const char *buf, rlim_t *res)
{
char *endptr;
*res = strtoll(buf, &endptr, 10);
if (!strcmp(buf, "unlimited")) {
*res = RLIM_INFINITY;
return false;
} else if (*endptr == '\0') {
return false;
}
return true;
}
// (:..) Function limit_cmd() found on line 2835
int
limit_cmd(int argc, char *const argv[])
{
// ------------------------------------------------------------
// This comment is not part of the source file, it is
// my analisys comment about what this function does.
//
// Here it is reading the parameters of 'launchctl limit argv2 argv3'
// and will call function srt2lim() to convert the args to limit values
//
// Function srt2lim() is printed above.
// ------------------------------------------------------------
//
// (:....) skipping to line 2850 inside this function
if (argc >= 3 && str2lim(argv[2], &slim))
badargs = true;
else
hlim = slim;
if (argc == 4 && str2lim(argv[3], &hlim))
badargs = true;
// ------------------------------------------------------------
// But before settin the values, there is a verification that
// prohibits using the word 'unlimited' as a parameter (only
// if used for 'maxfiles') and this check returns error, exiting
// the limit_cmd() function [the function we are now inside] and
// the system kernel paramter is never adjusted if we use
// the word 'unlimited' as a value.
//
// This comment was written by me
// and is not part of the source file.
// ------------------------------------------------------------
//
// (:....) skiping to line 2906 inside this function
bool maxfiles_exceeded = false;
if (strncmp(argv[1], "maxfiles", sizeof("maxfiles")) == 0) {
if (argc > 2) {
maxfiles_exceeded = (strncmp(argv[2], "unlimited", sizeof("unlimited")) == 0);
}
if (argc > 3) {
maxfiles_exceeded = (maxfiles_exceeded || strncmp(argv[3], "unlimited", sizeof("unlimited")) == 0);
}
if (maxfiles_exceeded) {
launchctl_log(LOG_ERR, "Neither the hard nor soft limit for \"maxfiles\" can be unlimited. Please use a numeric parameter for both.");
return 1;
}
}
// -------------------------------------------------------------
// We can see above, that it is prohibited to use the 'unlimited' value
// And when we use it, the system assumes the
// default value of OPEN_MAX which is 10240 !
//
// This explains and prove why setting 'unlimited' gives us 10240
// --------------------------------------------------------------
// *****************************************************************
//
// --------------- Analisys Comment ---------------------------------
// Looking at another file from Kernel source, we can find another
// place where it checks for 'unlimited' value, and also there is
// an explanation by Apple on it.
//
// The comment BELOW IS Original comment from Apple Developers and
// it is inside the file kern_resource.c from MacOS Kernel Source.
//
/* FILE: kern/kern_resource.c */
// *****************************************************************
case RLIMIT_NOFILE:
/*
* Only root can set the maxfiles limits, as it is
* systemwide resource. If we are expecting POSIX behavior,
* instead of clamping the value, return EINVAL. We do this
* because historically, people have been able to attempt to
* set RLIM_INFINITY to get "whatever the maximum is".
*/
if ( kauth_cred_issuser(kauth_cred_get()) ) {
if (limp->rlim_cur != alimp->rlim_cur &&
limp->rlim_cur > (rlim_t)maxfiles) {
if (posix) {
error = EINVAL;
goto out;
}
limp->rlim_cur = maxfiles;
}
if (limp->rlim_max != alimp->rlim_max &&
limp->rlim_max > (rlim_t)maxfiles)
limp->rlim_max = maxfiles;
}
else {
if (limp->rlim_cur != alimp->rlim_cur &&
limp->rlim_cur > (rlim_t)maxfilesperproc) {
if (posix) {
error = EINVAL;
goto out;
}
limp->rlim_cur = maxfilesperproc;
}
if (limp->rlim_max != alimp->rlim_max &&
limp->rlim_max > (rlim_t)maxfilesperproc)
limp->rlim_max = maxfilesperproc;
}
break;
// *****************************************************************
/* END OF CODE ANALYSIS */
This explains why it is impossible to set 'unlimited' again, after changing it for some other value.
How to do it? #
RLIM_INFINITY is defined on line 416 of <sys/resources.h>
/*
* Symbolic constants for resource limits; since all limits are representable
* as a type rlim_t, we are permitted to define RLIM_SAVED_* in terms of
* RLIM_INFINITY.
*/
#define RLIM_INFINITY (((__uint64_t)1 << 63) - 1) /* no limit */
#define RLIM_SAVED_MAX RLIM_INFINITY /* Unrepresentable hard limit */
#define RLIM_SAVED_CUR RLIM_INFINITY /* Unrepresentable soft limit */
In practice, for the current platform, the unlimited INFINITY maximum value is limited to INT_MAX (which is the maximum value of the Integer Type)
Considering that, it means that 'unlimited' means INT_MAX, which is the value 2147483647.
A possible workaround may be set the limit value of 2147483647:
sudo launchctl limit maxfiles 2147483647 2147483647
Because this is the maximum possible value.
Since this is a possible workaround. I would have to go deeper into source to check if this really means 'the same thing' as 'unlimited'.
Warning:
PS: Please, don't set it bigger than 2147483647, because it is SIGNED INT, and if you put 2147483648 (notice the number eight, I just add +1 to that maximum value), the SIGNED INT will be interpreted as negative, and you will instantly almost crash your macOS, because it will not be able to open any other file descriptor, including the one you will need to revert this value back, if trying to adjust it again by executing launchctl with different parameters.
The only solution will be to hard reset your system if you try this. And you may loose all your unsaved things. Try it on some MacOS virtual machine for testing purposes
Source code References:
<sys/resource.h> https://opensource.apple.com/source/xnu/xnu-4903.221.2/bsd/sys/resource.h.auto.html
<dev/unix_startup.c> https://opensource.apple.com/source/xnu/xnu-4903.221.2/bsd/dev/unix_startup.c.auto.html
<kern/kern_resource.c> https://opensource.apple.com/source/xnu/xnu-4903.221.2/bsd/kern/kern_resource.c.auto.html
<conf/param.c> https://opensource.apple.com/source/xnu/xnu-4903.221.2/bsd/conf/param.c.auto.html
<sys/syslimits.h> https://opensource.apple.com/source/xnu/xnu-4903.221.2/bsd/sys/syslimits.h.auto.html
launchctl.c https://opensource.apple.com/source/launchd/launchd-442.21/support/launchctl.c.auto.html