Construction workers filled my SAN with concrete and mineral dust
Solution 1:
First: When the server room becomes a building site, you must remove all the servers, for several reasons.
- power outages
- cooling outages
- access (you don't know who walks past your servers, on what hour)
- mechanical stress due to building machinery
- direct damage by workers who go bump
- residual dust from the building site, especially metallic or mineral dust
The only safe bet is, to remove your servers to a different room. This means additional shutdowns, but during the course of the building activities, you will probably have to do the shutdown anyway.
(mind you, the following is my subjective opinion) Now this has happened. The fans are covered with mineral dust. This potentially reduces their lifetime by affecting the bearings.
I would not expect the servers to massively fail, but I would expect to have a higher percentage of failures. At a fairly large customer, fan outages rose after building activities.
But then again, I would not go to great lengths to clean the systems. If the cleaning leads to damage, it is likely bigger than a failed fan. Unless, of course, if the dust contains metallic particles, which change the whole game.
What to do? Clean the room, clean the outside of the servers, reserve some cash for earlier replacements in the future.
The damage very probably is not justiciable. The part of damage that you can prove is probably not worth going to court for. You might try to get some amount of compensation out of the landlord, perhaps a reduced payment or so.
Solution 2:
- Get all the technicians in.
- Make them check/clean all the equipment.
- Send the bill to the building planner.
Really, servers can withstand some level of dust but this is just too much. We clean our servers regularly during downtime with a PC vacuum by 3M. It's a nice thing to have around the office.
But for now, start cleaning. The faster you get the dust out of there, the better. Try to keep heatsinks and fans clear of dust. If a heatsink or fan is covered in dust, its ability to dissipate heat is much worse then a clean unit.