The generator is one of the most critical components in data center operations. If the data center fails in a power outage, it should immediately be powered by the backup generator.
The U.S. electric grid must start to transition from fossil fuels to more sustainable resources, like solar and wind, to minimize the chance of power outages. The local power utility is an economic alternative that is expected to fail, so generators are considered the primary power source for the data center. Older generators may still function well but need high-quality maintenance.
UPS limitations
The first line of defense is the uninterruptible power supply (UPS), which ensures steady power. But the UPS is only a short-term solution. It can only back up the IT equipment, not the cooling. IT equipment can overheat quickly without cooling and, hopefully, shuts itself down before it self-destructs.
High-performance computing systems may quit in only seconds once they start to overheat. But, even if the IT equipment continues to function, the UPS itself ultimately goes into self-protective thermal shutdown. So, investing in battery durations of more than 15 to 30 minutes is usually uneconomical.
The cooling system determines how long a data center can keep operating after a power failure, and the backup for that is the emergency standby generator.
Diesel generator reliability
Diesel-powered generators are reliable engines and produce inherently lower emissions than gasoline engines. Their biggest challenge is cold weather starts, but those are easily countered with oil sump and engine block heaters.
The heating devices, while not energy-saving devices, should be programmed to operate only when temperatures drop below 40 degrees Fahrenheit (4.4 degrees Celsius). However, if generators are installed inside a building with proper ventilation or in an appropriately designed weatherproof enclosure, diesel engines are highly reliable.
Generator testing and maintenance
Despite their importance and criticality — and even with the deteriorating reliability of power networks — data centers rarely need to run a backup generator. However, this is what makes their maintenance and testing so important. If organizations don’t take proper precautions to maintain generators, the generators might not turn on, which causes the data center to fail and potentially destroys equipment in the event the power does shut off.
Test load banks to ensure that generators can quickly stabilize and assume full load without bogging down. Emergency generators are costly investments and should be under full maintenance contracts. This means more than just yearly tuneups.
While backup generators are designed to be reliable, they are not in daily use like automobiles. This makes it difficult to recognize impending problems during short test runs. Generators rarely run more than once a month. They don’t typically wear out; they simply deteriorate.
Full maintenance should include testing fuel and oil for contaminants. Starter batteries should always be plentiful and monitored for charge. Fuel is a concern because it can go stale in just months. Partially empty tanks can accumulate water. A common cause of generator failure is fuel starvation when tanks have not been refilled after testing.
Deciding when a generator is too old
As with any mechanical device, when major parts start to fail and replacement parts become expensive, it’s best to buy a whole new machine. Many generators function well for over 30 years. Generators with longer life spans were likely top of the line when new, installed out of weather and included every reliability feature available.
By 15 to 20 years of age, even the best materials start to deteriorate. Regularly check hoses and seals for leaks. Insulation also breaks down, so Megger testing is necessary to ensure shorts don’t occur when the generator is put under load.
Control panels and voltage regulators are of particular concern. These are electronic devices with components that are more subject to failure than the engine. Further, they can become obsolete, meaning they are hard to replace when they fail.
Check with your supplier, and consider doing a planned upgrade before an emergency occurs. Expect to encounter a long lead time for a new controller and difficulty with installation since it will not likely be a direct replacement. If you happen to be doing a switchgear upgrade, that is an excellent time to also upgrade the generator controller.
As generators age, the maintenance program should become stricter. Maintenance companies should be able to remotely monitor the generators and send alerts of impending problems and concerns.
Replacing a generator
Significant cost is a reason why companies wait so long before replacing their generators. This could lead to a failure in backup power as the old generator might not turn on, resulting in power loss to the data center.
At this writing, lead times of 18 to 24 months are not unusual, so waiting too long could be disastrous to data center backup power. If you don’t have multiple generators, this might be a good time to order and add a new one to the data center for a backup power source to the original backup generator, if it should fail.
New generators can include a number of features designed specifically to support high availability data centers, which include the following:
- Enhanced, high ambient cooling systems.
- Low subtransient alternators.
- Low engine emissions calibrations.
- Uptime Institute-compliant ratings, such as Tier 4-compliant exhaust emissions systems.
Reliability features are also available that may not have existed when the generator was originally purchased, such as bi-fuel — both natural gas and fuel oil — and paralleling power supplies.
Robert McFarlane is a principal in charge of data center design for the international consulting firm Shen Milsom and Wilke LLC. McFarlane has spent more than 35 years in communications consulting and has experience in every segment of the data center industry.
