Wednesday, August 15, 2018

911 OUTAGE INVESTIGATION COMPLETE; HUMAN ERROR IDENTIFIED: 693 Calls to 911 Failed to Route to Minnesota Dispatch Centers

The Minnesota Department of Public Safety Emergency Communication Networks division (DPS-ECN) has received information from the state's 911 provider regarding a statewide outage that occurred on August 1. 
CenturyLink is Minnesota's contracted 911 service provider and says human error by an employee of a third party vendor is to blame. CenturyLink says that an employee of West Safety Services made a mistake while making a network configuration change. This error prevented 911 calls from being accepted in Minnesota, North Carolina and North Dakota.
CenturyLink provided a timeline for the event:
·         3:47 p.m. - The outage began. 
·         3:58 p.m. - 911 dispatch centers alerted CenturyLink to the call failures.
·         4:02 p.m. - CenturyLink contacted West Safety Services about the issue. West noted they were aware of the problem and attempting to correct it. 
·         4:24 p.m. - CenturyLink notified all 911 dispatch centers in Minnesota, North Carolina and North Dakota of the outage.
·         4:52 p.m. - West Safety Services rolled back the configuration change which restored 911 service.
CenturyLink says 693 calls to more than 50 Minnesota 911 dispatch centers failed to be routed during the outage. During that same time, 356 calls to 911 were successfully routed to Minnesota 911 dispatch centers through a redundant router.
Minnesota's 102 dispatch centers statewide have received an average of 7,817 calls per day this year. 
CenturyLink says West Safety Services has agreed to stop work on its network through the end of August while it reviews the reason for the outage. They are also looking into changing processes and procedures along with enhancing software and safety mechanisms to prevent future errors from occurring.
DPS-ECN oversees the 911 program and is in the second year of a five-year $29.5 million contract with CenturyLink.
"We are committed to holding all service providers and their vendors accountable for any failures of the system," said DPS-ECN Director Dana Wahlberg. "We will continue to work towards providing dependable, state-of-the-art 911 services for all Minnesotans in an emergency."
The Minnesota Public Utilities Commission is the regulatory authority over CenturyLink and West. The Federal Communications Commission will conduct its own investigation into the outage which could result in fines for both companies.

No comments: