Enterprise Backup, Enterprise Storage, VEEAM
May 8, 2021 7:11AM EDT
[Monitoring] OIT Staff were able to mitigate the issue yesterday and restore services that are dependent on the affected storage as of 1300 on 5/7.
The storage itself has been restored to functional but some parts of it are still rebuilding and OIT system administration will continue to work on the storage throughout the weekend.
If you have concerns or problems, please file a ticket via services.gatech.edu.
May 7, 2021 11:58AM EDT
[Identified] redbook.gatech.edu 's service has been restored. The ImageNow service has been restored and the one open ImageNow has been isolated to one user, and resolved as well. Faculty Grade Entry is still being looked at for possible issues.
May 7, 2021 11:53AM EDT
[Identified] eisvault.gatech.edu has been restored to service by OIT system admins. redbook.gatech.edu, ImageNow, and Faculty grade submission are still being worked on.
If you have questions or concerns, please file a ticket via services.gatech.edu.
May 7, 2021 11:19AM EDT
[Identified] As an update, OIT continues to work on redbook.gatech.edu, eisvault.gatech.edu, and ImageNow. OIT has also received a report of possible issues with faculty grade submission which is being looked at.
May 7, 2021 9:49AM EDT
[Identified] As another update, the following services are still affected as a result of the storage issues this morning: ImageNow, redbook.gatech.edu, and eisvault.gatech.edu. Admins are still working to resolve the incident.
If you have additional concerns or problems, please file a ticket via services.gatech.edu.
May 7, 2021 8:35AM EDT
[Identified] As an update, services continue to recover after the resolution of the storage problem at 705. For the affected storage node, some of the disks are still rebuilding so some services that depend on this storage may take a performance hit ( run slower than usual ) as the disks are being rebuilt.
redbook.gatech.edu and eisvault.gatech.edu are still not available. Admins are addressing the issue.
All Banner services have recovered as of 0800.
If you have specific problems or concerns, please file a ticket via services.gatech.edu.
May 7, 2021 7:12AM EDT
[Identified] As an update, due to a failed storage node at 230 this morning, the following services were affected but some are starting to recover as of 0705 due to resolution of the storage issue: one Veeam backup job, Global ECS, docuSign, cleanroom.gatech.edu, eisvault.gatech.edu, request.it.gatech.edu, redbook.gatech.edu, and some Banner mounts,. If you have a question or concern, please file a ticket via services.gatech.edu.
We will continue to post updates this morning.
svn.oit.gatech.edu, subersion, was affected and is still unavailable as well as clearnroom.gatech.edu, redbook.gatech.edu and eisvault.gatech.edu which are also still unavailable as of 0726. Admins are working to resolve these issues.
May 7, 2021 5:59AM EDT
[Investigating] no new updates technicians are dispatched and are trying to resolve issue as soon as possible.
May 7, 2021 4:38AM EDT
[Investigating] no new updates technicians are dispatched and are trying to resolve issue as soon as possible.
May 7, 2021 3:53AM EDT
[Investigating] This has cause some VMs to be offline and therefore affecting some services. The incident is currently being investigated by the OIT team.
