Windows Server - "Not enough storage available error"

The Not enough storage available error has nothing to do with disk storage. It is about internal memory.

This error is quite easy to reproduce. Simply select a bunch of large files (images or DLL's or whatever), right-click and use the open-with option to open them all with Notepad. This should eat up allot of internal memory. When the swap-file nears it's maximum capacity, random services will start logging Not enough storage available to the eventlog. Often they will stay in a faulted state until they get restarted.

I have come across this issue with my own services, and had to add some extra error handling to force the service process to exit when such an error occurred on a thread. That way the service would restart and recover in an automated manner.

IIS usually handles this situation quite well (compared to an average windows service).

In order to find the process that causes this (usually a memory leak) you should use the performance monitor mmc snap-in, and record the memory usage of running processes (be selective when deciding what to record, since these logfiles can become quite large). Another option would be to use xperf, which is actually more suited for debugging low-level drivers and so, but can be helpful in this scenario as well.

If it turns out to be the w3wc service that's eating up all memory, you could use a tool such as Microsoft's all-new Application Insights (or any other IIS monitoring tool) to figure out which application pool, web-application or website is responsible. Then you can restrict memory usage for that particular pool, or schedule it to recycle gracefully.

IIS can usually cycle a pool in such a way that no one will notice (depending on how the app handles sessions, if it is custom built you might lose sessions when all static variables are flushed). Often a workaround like this is good enough, and expensive fixing of the bug can be avoided (disclaimer: that's not "my style" of solving problems, but most stakeholders love it that way).

32bit Operating system with 16Gb ram will effectively leave 12Gb's unused. The upgrade to 64bit OS should defenitly be considered.

Use of the PAE switch (Physical Adress Extension) could offer a temporary fix.

For info with regards to max memory which a OS can use along with the PAE switch info look at:

http://msdn.microsoft.com/en-us/library/windows/hardware/gg487503.aspx

Regards,

Could be a memory leak? Possible with that number of IIS pools, depending on what they're doing.

Easy to monitor with POOLMON.EXE from Microsoft (Grab it from the official 2003 Support Tools)

Powershell script I run hourly for the duration of 'reboot -> crash'.

$date = get-date -Format ddMMyy-hhmm

.\poolmon.exe -b -n Output-$date.log

#send-mailmessage -smtpserver 10.1.1.11 -to [email protected] -from [email protected] -subject "Poolmon output $date" -attachments "Output-$date.log"

#Extract top offender

$TAG = (Get-Content Output-$date.log)[3]

$Code = $((($TAG -replace '\s+', ' ').split(" "))[1]) 
$Byte = $((($TAG -replace '\s+', ' ').split(" "))[6])

ECHO "$Code,$Byte" >>chart.csv

Chart.CSV will show you the top TAG and its bytes allocated value.

The Output-$date.log will show you everything.

If you have got a single offender at the top, that stays at the top and increases until crash you have a likely candidate for the cause.

Also look in Eventvwr under system for events from SRV mentioning 'insufficient resources'