Too many open files while ensure index mongo

I added a temporary ulimit -n 4096 before the restore command. also you can use mongorestore --numParallelCollections=1 ... and that seems to help. But still the connection pool seems to get exhausted.


NB: This solution does/may not work with recent Mac OSs (comments indicate >10.13?). Apparently, changes have been made for security purposes.

Conceptually, the solution applies - following are a few sources of discussion:

  • https://wilsonmar.github.io/maximum-limits/
  • https://gist.github.com/tombigel/d503800a282fcadbee14b537735d202c
  • https://superuser.com/questions/433746/is-there-a-fix-for-the-too-many-open-files-in-system-error-on-os-x-10-7-1

--

I've had the same problem (executing a different operation, but still, a "Too many open files" error), and as lese says, it seems to be down to the 'maxfiles' limit on the machine running mongod.

On a mac, it is better to check limits with:

sudo launchctl limit

This gives you:

<limit name> <soft limit> <hard limit>
    cpu         unlimited      unlimited      
    filesize    unlimited      unlimited      
    data        unlimited      unlimited      
    stack       8388608        67104768       
    core        0              unlimited      
    rss         unlimited      unlimited      
    memlock     unlimited      unlimited      
    maxproc     709            1064           
    maxfiles    1024           2048  

What I did to get around the problem was to temporarily set the limit higher (mine was originally something like soft: 256, hard: 1000 or something weird like that):

sudo launchctl limit maxfiles 1024 2048

Then re-run the query/indexing operation and see if it breaks. If not, and to keep the higher limits (they will reset when you log out of the shell session you've set them on), create an '/etc/launchd.conf' file with the following line:

limit maxfiles 1024 2048

(or add that line to your existing launchd.conf file, if you already have one).

This will set the maxfile via launchctl on every shell at login.