Network Instability
Sep
29
2009
Network Instability Comments: 1
Our connection to our primary internet provider - UUNet (Verizon Business) is was unstable and caused route flapping. It is currently stable, and we are attempting to determine the root cause. This route flapping caused unstable service both inbound and outbound for approximately 20 minutes. We will update with root cause analysis. We apologize for the inconvenience and appreciate your patience.
Issue Resolved
Timeline:
4:55 - one of our main Internet backbone connections dropped which disrupted all calls using that connection
4:56 - our traffic was automatically and completely rerouted around the bad connection
5:03 - the connection that went down was restored and we automatically started trying to route traffic back onto it
5:05 - unfortunately there seemed to be routing problems within UUNET on the restored connection
5:10 - customers how were routing via UUNET would have noticed instability at this point.
5:15 - UUNET issues cleared themselves up and everything appears to be resolved since then
We are still investigating root cause.
System has been stable since 5:15pm EST.