Eastern Mennonite University

Communications from Information Systems management to the EMU Campus Community

February 03, 2008

South Campus Power Failure Puts User Services in High Gear

The phone calls started just before 1:00pm on Friday, February 1. Smoke was reported coming from computers and copiers and Harrisonburg Fire Department personnel were arriving on campus. The next day and one half will long be remembered as a great team effort by the User Services team of the Information Systems department.

DSC04245b.jpg
User Services technicians Galen Wenger and Dan Risser load “fried” computers into a “techno hearse”. Three loads of devices damaged by the power failure event of February 1 were removed from offices in the small campus buildings south of the Library.

User Services team members Dan Risser, Galen Wenger, Steve Gibbs, along with student technology workers Sam Gibbs and Jackson Maust worked about nine hours on Saturday, February 2, with manager, Jenni Piper, to replace fifteen computers on full-time employees desks and one in the CJP student lab that were damaged by the power failure.

The cause of the problems, which affected the small campus buildings located to the south of the Library, was a Harrisonburg Electric Commission (HEC) malfunction with the delivery of electricity. Instead of providing the standard 110 volts, HEC was delivering as much as 160 volts, according to Physical Plant Director, Eldon Kurtz. Kurtz also said that these buildings were affected because they are fed electricity directly from the city grid and not from the sophisticated electrical loop that supplies the main campus buildings.

It took Info Systems and Physical Plant personnel a bit of time to realize that there was a wide-spread problem but by 2:30pm all of the main breakers to the dozen or so affected buildings were turned off and the employees vacated the buildings. Kurtz, received periodic status updates from HEC until about 11:00pm on Friday, when normal power was restored.

On Saturday morning Physical Plant electricians visited all the affected buildings to turn the breakers on and verify that the electrical systems were functioning properly and by 10:30am the User Services team began the tedious process of going to all of the offices to determine which electronic devices were ‘fried’ by the power failure.

The damages were extensive. Info Systems Director Jack Rutt’s CRV was loaded up three times with dead computers, monitors, fax machines and uninterruptable power supplies (UPSes) which were taken to the Info Systems shop at the north end of North Lawn. The final count of damaged devices was six copiers, 21 computers (two eMacs and 19 Windows PCs), one CRT monitor, two fax machines and five UPSes.

Within two hours of the first reports of electrical problems Jenni Piper, User Services Manager, knew that the number of damaged computers was likely to be high and she gathered her team to put together a plan for quickly replacing several dozen computers.

Info Systems utilizes a “cascade plan” for refreshing computers on campus which means that most of the 100+ new computers purchased each year go to the student computer labs. After one year of service in the labs they are moved to full-time employees’ desks for three years and then they are moved to shared or workstudy desks for another year or so.

On Friday a number of computers that were ready to be put on employees’ desks as part of the cascade plan were quickly diverted and given a fresh disk image. Sixteen of these were used on Saturday to replace the damaged computers of full-time employees and the CJP student computer lab. Jenni estimated that it would take another day or so for computers to be found and setup to replace the five shared/workstudy workstations.

Ikon, the copier service company, has been contacted and repairs will begin on Monday morning on a priority basis. It is not known if the damaged copiers can be repaired or if they will have to be replaced with new machines. The Ikon account rep has assured Info Systems that they will work very hard to get functioning copiers in place as soon as possible. Until then these offices will have to use other copiers on campus. Fortunately, when these copiers were upgraded in the last year a new management system was also installed that allows entry of any user and billing code in any copier on campus without requiring reconfiguration.

As User Services completed work in each of the buildings a note was left on the desks where computers were replaced telling the employee that they have a different computer and that they will need to call the Info Systems Helpdesk (x4357) to complete a few minor final setup steps. A sign was also posted on the entrance door to the building listing all of the devices in the building that are not functioning or have been replaced.

The results could have been much worse. A number of the computers in the affected buildings were plugged into surge suppressor power strips and about twenty of these strips ‘took the hit’ and were damaged but the computers plugged into them continue to function perfectly.

Reflecting on the experience at the end of the day on Saturday, Info Systems Director, Jack Rutt, said “What the User Services Team accomplished in the last 30 hours is absolutely amazing! They decided to do what needed to be done and made it happen. I am extremely grateful for the way they work together as a team and their commitment to EMU.”

Posted by ruttj at February 3, 2008 12:00 AM
Comments