Load balancing problems causing a variety of service outages
Incident Report for GT OIT
Resolved
Last night, Wednesday 6/14, at 8:25p, the OIT PaaS team switched the load balancer service back to the original load balancer that began having issues at 7p. Services have been stable for over 10 hours, and OIT continues to monitor the situation. If you have questions or concerns, please contact OIT at 404-894-7173.
Posted 5 months ago. Jun 15, 2017 - 07:42 EDT
Monitoring
OIT has switched over to a redundant load balancer as of 8:12p and some services are starting to return. CAS is now functional. Dependent services that were affected by CAS's outage such as T-Square are now available again. Services that were being served out of BCDC that were were restored by the load balancer change ( in addition to ) the restoration of the CAS change such as Self Service Banner and Internet-Native Banner, Buzzport, degreeaudit, ADFS ( as well as its dependency mail.gatech.edu), and mailing lists are now available also. No mail was lost although although there may have been a delay in the delivery of messages.
Posted 5 months ago. Jun 14, 2017 - 20:27 EDT
Investigating
An issue with a load balancer is causing outages for a variety of services including t-square, epay, mailing lists, gt webapps, degreeaudit, passport. Webmail(mail.gatech.edu) and CAS are not allowing new logins, though existing logins are unaffected. Shibboleth, Buzzport, ADFS, Self Service Banner and Internet-Native Banner are also affected. Touchnet as a result of the Banner issues is also affected.
Posted 5 months ago. Jun 14, 2017 - 19:11 EDT
This incident affected: Academic Services (Banner and Oscar, Buzzport, T-square), Email (Mailing Lists, Office 365 Outlook), Network (BigIP and GTM), and Identity (GTED, Login, Passport).