HONOLULU — A statewide computer outage Monday prevented Department of Motor Vehicle customers from obtaining driver’s licenses, state of Hawaii identification cards, registrations, renewals and titles.
Around 9 a.m. Monday, a vendor was working in the Fasi Municipal Building on Honolulu on the fire suppression system when they pulled or loosened a wire that sent a signal to their systems to shut down.
As of 4 p.m., the system was working intermittently but it is expected to be in full operation today.
The County of Kauai’s Motor Vehicle Registration was unable to print registrations, renewals, and titles.
“Basically, at that point, all the power from the main UPS, or the uninterruptable power supply went down. We have secondary backup systems in place, so those took over and provided battery power for a short period of time, until those eventually ran out,” said Mark Wong Director of the Department of Information Technology at a press conference from Honolulu.
At that point, everything went down, he said. They lost all networks, telephone, storage servers and the mainframe.
“Basically anything that was technology related to the city, except for our 911 services and dispatch, those went down,” Wong said.
The vendor, who is renovating the building’s fire suppression system, cut a bundle of wires, he said. The workers realized what happened, but by that time, the systems had all shut down and they had to bring them up slowly.
“At this point, we believe that most of the networking, the copper based networking is up. Telephones are up, the mainframe and the mainframe storage is up. What we do not have is fiber storage to our enterprise storage unit,” Wong said.
Fiber storage is what they need to run their domain name services, he said. They’re working with a vendor to establish when the systems will be back up.
“Generally when the data center takes a hard hit like this, it’s at least three hours to bring things back up, but the last time HICO cut our power, we put some additional backups in place that have shortened that and helped us bring things up a little bit faster,” Wong said.
The impacted areas are the customer service areas such as licensing, not only in Honolulu, but also on the neighboring islands, he said. For the City of Honolulu, they are also unable to email or get into their file storage.
“For all practical purposes, we are without computing services,” Wong said.
The systems take a long time to repair, because they’re complicated and not one single system, he said.
“It’s not like a small office system, its literally like hundreds of different systems,” he said.
During the press conference, he said about 95 percent of all systems were already up.
In case of an emergency like an earthquake or flooding, Wong said there is an identical data center in Kapolei. It would take three hours to transfer all of the data from the municipal building to the center in Kapolei. They’re planning on a complete renovation of the data center within the next 12 months. At that time, the data centers will be run out of Kapolei.
Last summer, Wong said the Hawaii Electric Company was working on a project downtown on a Saturday and cut the power, which caused an outage that nobody knew of until Monday. That incident cost the city, hundreds and thousands of dollars worth of equipment. That’s when they put in the secondary UPS system.
No data is expected to have been lost during Monday’s outage.
•••
Bethany Freudenthal, Courts, Crime and County reporter, 652-7891, bfreudenthal@thegardenisland.com
Was there 7 Woes.
HICO?