What Happened?
At 3:38 PM CST, there was an outage that affected all websites being hosted on Website Pro. Service was restored at 3:56 PM CST for a total outage duration of 18 minutes.
The outage was caused by our NGINX server failing to forward web traffic to the appropriate locations. Restarting the server was able to quickly restore service and our traffic analytics are returning to normal levels but we continue to monitor the service.
The NGINX server was not able to forward web traffic due to an error that occurred on the load balancer that is ran by the Google Kubernetes Engine (GKE). The error is caused by a flaw in version 1.21 of GKE that was identified by the Google support team and is resolved in version 1.22 of GKE.
The team is planning to upgrade to version 1.22 in response as soon as the dependencies we have on 1.21 are resolved.