Contingency planning for continued business operations during a natural disaster is a multi-faceted risk management issue. While these events cannot be avoided, businesses can lessen their impact by taking steps to prepare. Although many companies have disaster plans, they may fail to recognize the common assumption that the electrical power and infrastructure will remain operational and unaffected. This article identifies pre-planning exercises that companies can complete to efficiently control equipment and safely restore electrical distribution.
Introduction
Businesses are under increasing pressure to maximize profits and minimize downtime. Therefore, it is important to have a contingency plan for continued operations in the event of an emergency. The actions businesses take during the first 24 to 48 hours of a disaster critically determine whether full recovery is possible. According to the latest research, as many as 50 percent of businesses are forced to close their doors and do not reopen following a disaster.1
Having a detailed emergency recovery plan can limit the financial and personal havoc of a natural disaster. An effective plan will address the following issues as a starting point:
- Ensure that electrical equipment is properly maintained.
- Identify the electrical equipment that is critical to operations.
- Be aware of the most current natural disaster recovery codes and standards.
- Know the effects of water damage to electrical equipment.
- Develop a safety plan that incorporates emergency procedures.
- Develop an electrical emergency action plan.
The National Fire Protection Association (NFPA) and the Occupational Safety and Health Administration (OSHA) provide guidelines to develop disaster recovery, emergency response, and safety plans. This article will incorporate those guidelines to help in the creation of both short- and long-term restoration plans. The number one priority for both plans is to restore power safely.
Natural Disaster Definition & Statistics
Natural disasters can be broken down into four categories, as shown below. Each type of disaster has a unique severity level.
Category Type of Disaster
Geophysical Earthquakes, tsunami, volcanic eruption
Meteorological Storm, lightning
Hydrologic Flood, mass movement
Climatologic Extreme temperature drought, forest fire
According to the National Oceanic and Atmospheric Administration, “weather, water, and climate-related events cause an average of approximately 650 deaths and $15 billion in damage per year.”2 These disasters cause death and injury, property damage, business interruptions or downtime, and revenue loss. In 2018 alone, there were fourteen different billion-dollar disaster events in the United States. Over the last three years, the yearly frequency of billion-dollar disaster events has doubled the long-term average.3 Given that storm data continues to demonstrate a pattern of increasingly extreme weather, there’s a good chance that this is not an anomaly; we may be seeing the new “extreme” normal (see figure 1).
Whether we attribute these shifting statistics to climate change, cyclical weather patterns, or simply man putting infrastructure in mother nature’s path, we have seen a clear rise in the impact of natural disasters on society. Figure 2 illustrates the natural disasters in North America over the past thirty-eight years. In some cases, the increasing numbers reflect an improved ability to identify natural disasters with technology. For additional information on your location, consult the Federal Emergency Management Agency (FEMA) disaster and emergency declaration, which is published by year and state.
Natural disasters can damage electrical distribution equipment in multiple ways (fire, vibration, water, etc.). In the U.S. the most frequently occurring events involve water in the form of hurricanes and storms, with subsequent flooding. Since water and electricity do not mix, restoring power to water-damaged equipment can be a dangerous undertaking.
Relevant Codes & Standards
NFPA 1600, Standard on Continuity, Emergency, and Crisis Management, is the overarching standard and primary document on disaster recovery, emergency management, and business continuity. For workplace safety and planning, OSHA references (and cites to) NFPA 70E, Standard for Electrical Safety in the Workplace. NFPA 70B, Recommended Practice for Electrical Equipment Maintenance, now includes a chapter on electrical disaster recovery. The National Electrical Manufacturer’s Association (NEMA) has published standards on Evaluating Water-Damaged Electrical Equipment and Evaluating Fire- and Heat-Damaged Electrical Equipment. In addition, the 2020 edition of the National Electrical Code (NFPA 70) will include rules for equipment reconditioning.
It is important that restoration activities are performed by qualified personnel as defined by OSHA and NFPA. Hiring outside contractors to assist or perform electrical work does not relieve the facility owner (host employer) from being liable for safe work practices. NFPA 70E states that known hazards must be communicated to the contracted worker(s).
Three Steps to Electrical Disaster Recovery Planning
Step 1: Knowledge of Electrical System
As part of an electrical disaster recovery plan, businesses should:
- Maintain a current single-line drawing of their electrical distribution system.
- Identify which electrical equipment is critical to the electrical infrastructure.
- Understand what equipment must be replaced and what can be reconditioned after a disaster, as shown in table 1.
Table 1. What electrical equipment can be replaced and what can be reconditioned after a disaster.
Electrical equipment exposed to water can be extremely dangerous if re-energized without proper reconditioning or replacement. Assessing the damage goes beyond visual water indications. Moisture and weather exposure can affect the equipment’s integrity. Water can be contaminated with sewage, chemicals, salt, debris, and other substances. These contaminants may cause loss of dielectric spacing and can be a hazard upon re-energization. It is also possible that foreign debris remains inside the circuit breaker enclosure.
Whether the equipment is to be replaced or reconditioned, all services should be performed by qualified personnel familiar with the equipment’s operation and construction. It is important to note that the ability to recondition equipment will vary; it may include the repair or replacement of internal components. Reconditioned equipment should be tested per the manufacturer’s instructions prior to re-energization.
- Develop (or Update) an Electrical Safe Work Practices Policy
An electrical safe work practices (ESWP) policy is a written document created by the employer that covers all areas of the company’s electrical safety practices. While having a safety policy has been a requirement of NFPA 70E for many revision cycles, the requirements for auditing the policy are often overlooked. The ESWP policy is not a one-size-fits-all policy to cover a company in multiple locations. For example, voltage, energy level, circuit conditions, and hazard levels are different in each facility. The policy should identify detailed information specific to that location. This may include the location of the safety policy for employee (or contractor) access and who is the onsite authority having jurisdiction (AHJ) for decisions, etc.
Developing and auditing an ESWP policy is critical to business continuity and disaster recovery. Since creating a comprehensive program may seem overwhelming, it may be helpful to use the following guidelines to help get started.
- Facility — Includes company policies and systems regarding:
- Equipment maintenance
- Tools
- Testing
- Repairs
- Clearance requirements
- Safe working conditions
- Personnel – Focuses on actual work practices and addresses:
- Qualified and unqualified personnel
- Proper care and use of PPE
- Job preparedness
- Training and continuing education
- Procedures – References on the job procedures and includes (but not limited to):
- Performing energized work
- De-energizing and re-energizing
- Lock-out / Tag-out
- Job planning
- Arc-flash hazard analysis
- Equipment labeling
- Reporting safety concerns
- Recordkeeping
The employer must verify on a regular basis that each worker is complying with the safety-related work practices required by NFPA 70E. Managers and supervisors must embed ESWP as part of their DNA, attend electrical safety training classes, and ensure all employees practice safe work practices every day. Finally, should a natural disaster strike, employers and employees should be very familiar with emergency procedures and work practices.
- Electrical Emergency Action Plan
The purpose of the electrical emergency action plan (EEAP) is to understand the electrical assets, critical operational infrastructure, risks, and short- and long-term power restoration execution plans. Some of the benefits include:
- reduces time to restore short and long-term power quickly and safely;
- reduces uncertainties when a disaster occurs;
- increases understanding of electrical assets, available emergencies services, replacement market availability; and
- being able to know when a disaster occurs, the immediate financial implications.
NFPA 1600 is the industry standard and primary document on disaster recovery, emergency management, and business continuity. However, it does not include specific guidelines from an electrical perspective. The inclusion of electrical disaster recovery in NFPA 70B provides a foundation for the electrical infrastructure recovery. Prior to this update, electrical disaster recovery planning was based on experiences of the individual or team who has electrical responsibilities. One purpose of this article is to provide guidelines on developing an electrical emergency action plan and examples of how it saved or could have saved facilities from excessive downtime.
Two real industry examples follow. Downtime, loss of revenue and headaches could have been reduced from a robust EEAP program. Note: Facility names will remain anonymous.
Example 1: Transformer Disaster Recovery – Industrial Facility
Background: Transformers are considered by corporate risk managers and insurers to be the most critical equipment in facilities because of the large quantity of oil in contact with high voltage elements. Service entrance transformers are essential to operations. Many facilities would experience downtime or lack of productivity if the service-entrance transformer was being serviced or if a disaster were to hinder its performance. Furthermore, transformers can be custom designed for utilities and facilities, which makes finding a replacement or spare parts complicated, especially on an emergency basis.
What Happened? A manufacturing facility that produced over $1 million per day in product was struck by lightning. The strike started a fire in a 25 MVA transformer and caused extensive electrical equipment damage. The fire was not contained in time and spread to the adjacent main power transformer for the facility resulting in complete power loss. The main power transformer was specially rated, and the municipal power company did not have any spares for this installation. The facility lacked an EEAP and the necessary standby sources and equipment to quickly recover from this event. The facility team began damage assessments, identified an available temporary transformer, coordinated logistics, and financial approvals, and hired an experienced electrical disaster recovery team. The loss of electrical power lasted 23 days.
Lessons Learned
- An EEAP should provide details to understand immediate actions, the sequence of engagement with recovery teams, available immediate spares at the facility, primary, secondary market, and the financial impact to communicate to senior management.
- To restore power as quickly as possible, identify critical electrical assets, the nearest available spares, rates, and terms for both short- and long-term requirements. For example, temporary 25 MVA transformers are hard to find and once located and lease for approximately $35,000 per month (excluding delivery charges). In addition, lead times on new replacement equipment can take months.
- Outside resources may be required to help manage power restoration. Partner with organizations that have the qualifications, resources, and experience to handle the job. Items such as pre-negotiated terms and conditions, defined scope of work, equipment pricing, lead times, availability, and service agreements streamline the power restoration process.
Example 2: Generator Disaster Recovery — Commercial Building with Businesses and Data Center
Background: On May 1–2, 2010, Nashville, Tennessee, received a record 13 inches of rain, which caused a historic 500-year flood. Thirty people were killed, and damages exceeded $2 billion. Also known as “Music City USA,” Nashville is a key tourist area for music lovers around the world. Due to its proximity to the Cumberland River, downtown Nashville was hit hard, and floodwaters damaged many commercial buildings. This example focuses on only one class A commercial high-rise in downtown Nashville housing multiple businesses and a data center.
What Happened? The following timeline of events depicts how the flooding affected downtown Nashville. Times shown serve as general guidelines.
Sunday, May 2
9:00 p.m. Municipal power was no longer able to serve power to downtown Nashville. The commercial building auto switched to emergency power and only served vital data center operations. Teams were deployed to the site to closely monitor data center and weather conditions.
Monday, May 3
12:00 a.m. Metropolitan police ordered a mandatory evacuation in downtown, which left data center operations and security vulnerable.
4:30 a.m. Selected personnel were allowed back into downtown to perform key operations including refueling of generators. Although the rain had stopped, the Cumberland River was steadily rising.
10:00 a.m. A decision was made to start soft shutdown actions, but it was too late. Generators flooded, which began a 31-hour data center outage. Through a key relationship, a temporary generator was located twenty miles south of the city.
Tuesday, May 4
4:30 p.m. The data center was brought back on-line.
Wednesday, May 5
Still with no lights, air conditioning, or restroom facilities, a 24-hour staff rotation was set up to monitor the data center and the condition of the temporary generator. The water level was still too high to assess damage to the building’s generator.
Thursday, May 6
The building’s generator was drained, and the damage was assessed. Spare parts were not immediately available. Five days later, the generator was repaired, and power switched back to the building generator. However, the temporary generator was retained as a back-up for a few additional weeks.
Thursday, May 13
Nashville Electric Services (NES) restored power to downtown Nashville, and the building switched from generator to municipal power.
Lessons Learned
Three key lessons were learned from the aftermath of this historic flood.
- Determine Priorities: The EEAP plan must have a clear definition of what constitutes an emergency and when to execute the EEAP. In this case, only three hours were available before a mandatory evacuation order was issued. Prioritizing critical functions is essential to efficiently and safely restore power.
- Emergency Service Contracts: Searching for temporary equipment after a disaster occurs slows down the power restoration process and can be very expensive. The EEAP should include emergency service contracts to guarantee disaster recovery team response time, critical equipment pricing, lead times, and a deployment strategy with details on setting up and operating a command center to meet an organization’s needs.
- Critical Spare Parts: Having a current single-line electrical diagram of the power distribution system is crucial to efficiently restore power. Use the drawing to pinpoint the electrical equipment critical to business operations. Identify the equipment’s critical spares parts availability, pricing, and lead times for custom-made parts.
Developing an Electrical Emergency Action Plan
The development of an EEAP may seem overwhelming. Below is a step-by-step guide created by Schneider Electric Services to help customers be prepared in the event of an emergency.
- Define the Criteria of an Emergency
Lighting strikes on service entrance transformers and flooding of backup generators clearly indicate operational “state of emergency.” Is nuisance tripping on circuit breakers or power quality issues considered emergencies? When productivity is impacted, it can be confusing as to when a state of emergency should be declared. The EEAP should clearly define what constitutes an electrical emergency.
- Identify Electrical Equipment that is Critical to Business Operations
On the single-line diagram, trace the power from the incoming utility source to every piece of electrical equipment feeding critical business operations. Perform an analysis on each of these critical assets to include availability in the market, lead times, and a plan of action if and when the equipment is no longer functional. For most critical assets, it is necessary to understand the cost of temporary rentals, logistics, contacts, and testing requirements for both temporary and permanent power restoration. For example, a special transformer’s spare parts may not be available in the primary market, i.e., the local electric municipality. Therefore, it is a good practice to determine available suppliers, acquire budgetary pricing, and lead times on the secondary market.
- Selection of Outside Vendors and Pre-Negotiated Commercial Terms & Conditions
In the aftermath of a disaster, it may be too late to negotiate pricing or lead times for the required resources to restore electrical power. Without pre-negotiated emergency service contracts, companies may suffer from overpricing and insufficient support. In addition to pre-negotiated normal and emergency rates, due diligence should include estimated response time and procedures for large-scale project coordination (see Step 4). The selected vendor(s) should have the depth and experience to handle major disasters.
- Define Internal and External Responsibilities
The EEAP should clearly define “who has responsibility for what” in restoring power to the facility. They should also have details on customer’s responsibility, such as providing electrical one-line diagrams, energization procedures, and coordination and communication activities between multiple vendors through a central contact. A third-party vendor(s) should also provide a clear procedure on how they will approach an emergency at a facility including assessing the damage, mobilizing resources, appointing a project manager, and establishing a command center. All parties involved should fully understand the safety plan that is put in place.
- Define the Equipment and Service Scope
This section will further define the equipment and associated work scope. For example, an electrical distribution service company can define the equipment scope to be from the utility service entrance (13.8 kV) to low voltage switchgear (480 V). The associated work scopes include equipment installation and commissioning in both temporary and permanent scenarios.
- Emergency Contract Terms
Emergency contracts should be written for a specific time period, with an expiration date. It is also recommended to add expiration dates to the EEAP plan for self-auditing purposes. A suggested best practice is to audit the EEAP when the ESWP is being audited, which is every three years, according to NFPA 70E.
- Contact Information
This section of the EEAP should include the latest contact information for anyone (internal or external) who has a defined responsibility in restoring power due to an emergency.
In some cases, EEAP programs should be signed off by customers and facility management companies. Why? Consider this example: A customer leases multiple floors (one floor being a data center) from a large building management company. The customer’s IT manager may have no control over the equipment feeding power to office space and data center. In this type of situation, clear roles and responsibilities, work scopes, and agreed-upon emergency actions are vital to a long-term building owner and customer relationship.
Conclusion
When a natural disaster strikes, its impact on individuals, communities, and businesses can be devastating. Restoring electrical power is a crucial part of the recovery process. Regardless of the industry or facility type, having a detailed ESWP and EEAP can help recovery efforts. Multiple standards and recommendations exist from NFPA, OSHA, and NEMA to serve as guides for businesses to understand risks and to develop a contingency plan in the event of an emergency or natural disaster.
References
- ServePro – http://www.servpro.com/ready
- “Weather,” National Oceanic and Atmospheric Administration. https://www.noaa.gov/weather
- Adam B. Smith, “2018’s Billion Dollar Disasters in Context,” National Oceanic and Atmospheric Administration, February 2019, https://www.climate.gov/news-features/ blogs/beyond-data/2018s-billion-dollar-disasters-context