Wednesday, February 10, 2010

Battling Blizzards

Back in the fall I posted about prepping your network for winter weather disasters, and it looks like those tips really came in handy for many this season. Today, the majority of the East Coast is battling a severe blizzard complete with 1-2ft of snow (4-8ft drifts!), 60mph winds with whiteout conditions, power outages and fallen trees. Each one of these factors pose a challenge to your business and your network. For my co-workers in our Washington DC offices, this is the third blizzard this winter, but they have managed to avoid communication and network failures. For those that may not have been hit with a weather disaster yet, you may want to take some time to review these tips to get a head start on future storms: 

Questions you should be able to answer:

1. Are you aware of your power situation?
a. What happens when a power outage occurs?
b. What is the operational status of the UPS system?
c. How long will the UPS backup systems sustain key functions?
d. What do we do if the outage is longer?

2. What if the building becomes unavailable? (fire or water damage)
a. Are the offsite backups current?
b. If a network device or server is ruined, what is the procedure to replace it?
c. Does everyone know the primary and secondary facility contacts to use should an after-hours emergency occur?

3. What if access to the building is limited? (snow, tornado warnings, etc)
a. Is VPN access updated for all employees that may need to work from home?
b. Can all of the required maintenance procedures be done remotely or skipped for several days? 


4. What if the phone and/or Internet connection is lost?

5. What is the customer impact when any of these conditions occur?
A few tips:

Advance planning is the best approach. A good network design can minimize the impact of storm and disaster related problems. Having redundant phone and data lines from different carriers minimizes the inbound/outbound traffic risk. Using an adequate number of UPS devices mitigates all but very lengthy power outages and network routing protocols like HSRP reduce the risk of single device point of failures.

Even monitoring your network with disaster prevention in mind can be helpful in avoiding unnecessary failures. These tips are a great starting point: 

1. Enable redundant polling of critical devices  
2. Map out HSRP primary and secondary links  
3. Know the status of the UPS systems  
4. Make sure you have 24x7 access to your management system client

And for fun (and to get the sympathy of those without 3 feet of snow on the ground), here’s what three blizzards in a row will do to you (and your network!):

No comments:

Post a Comment