Network reachability issue

26/04/25 00:00 – 02:05 Datanet experianced network reachability issues last night due to simultaneous service outages on both of the inter-DC connections into our London Telehouse North location.  This left Telehouse North separated from the rest of our network.  These connections are provided by independent telco providers, over diverse routes and infrastructure, designed to mitigate against the risk of the circuits being down at the same time.

Although we have completely independent connectivity into the Farnborough site, because we announce our network in 3 geographic locations, some providers were still receiving our BGP announcements from the Telehouse site. In particular this affected connectivity over LINX LON2 where we peer with some of the larger providers including Microsoft Azure.

From our overnight investigations we understand that this outage was caused by both telco’s carrying out maintenance which took down both connections between 00:00 and 02:05 this morning, with one of the providers failing to provide us with notice of the work.  The connectivity to Telehouse was restored whilst our engineer was investigating the outage.