Within the past several weeks we’ve witnessed an unusually high number of outages that have impacted our clients in WNC. Some have been weather related and others due to system issues, but the end result is that businesses can’t reach their necessary services!
-
- August 25th – Power went down in south Asheville for about 3 hours. Roughly 16,000 clients were without power. Link
- August 26th – Microsoft 365 was down for a couple of hours in the afternoon. Link
- August 27th – Charter/Spectrum internet services went down for clients around 4 pm in the WNC region for about 90 minutes. Link
- Sept 23rd – Power went down for about 7 hours in Asheville in the South Slope area
- Sept 26th – Bandwidth.com had an outage due to a denial of service attack. The outage effected phone calls to various degrees and was sporadic over a 3-day period. Link
- Oct 1 – Let’s Encrypt had a root cert expire that broke SSL inspection for many vendors including Fortinet. This caused sites using Let’s Encrypt SSL certs to fail cert validation and not load. Link
- Oct 4th – Facebook down for 6 hours due to a BGP error. Link
These outages have caused us to reevaluate how we plan for and react to events that would impact business continuity. More than ever before, redundancy in the entire IT stack is required to keep transactions flowing. This is easy to understand when it comes to things like power and internet circuits (backup power and multiple internet carriers have long been a staple of IT). More challenging is finding an answer to “What do we do when our cloud application is down?”. Our team has been discussing this internally and will be amending Business Continuity Plans with our clients to reflect these considerations!