Georgia Tech IT
Active Incident

Updated a few seconds ago

CEDAR Performance DegradedPartial Service Disruption

Incident Status

Partial Service Disruption

Components

Academic Services

Services

PACE


July 10, 2026 5:02PM EDT
IDENTIFIED

NetApp engineering updated an executable on one of the storage nodes. Results are promising and more analysis is needed. In the meantime, the critical data transfer continues.

July 10, 2026 2:31PM EDT
IDENTIFIED

Critical file sets are being transferred, while NetApp continues working on permanent fixes that will allow us to open the file system with the expected transfer rates. GT directors will meet with the NetApp VP of support and engineering in the next couple of hours. Another status call with NetApp is scheduled for 4pm.

July 10, 2026 11:22AM EDT
IDENTIFIED

We finally settled on the best performance parameters to transfer a critical file set from CEDAR. NetApp engineering continues root cause analysis and are working on a patch to fix the performance issues. Storage reamins closed to all users while the critical transfer completes. Issue has been escalated to the executive VP for support and engineering.

July 9, 2026 7:22PM EDT
IDENTIFIED

Troubleshooting will continue tomorrow morning. Vendor is analyzing logs and will provide additional updates as we get them.

July 9, 2026 5:33PM EDT
IDENTIFIED

Troubleshooting work with storage vendor continues. Core engineering resources are being engaged.

July 9, 2026 3:36PM EDT
IDENTIFIED

**What’s happening?** CEDAR storage has been made unavailable starting today, July 9, at 3.20pm ET. At this time, there is not an estimated day/time for when services will be returned. Emergency work is necessary to mitigate slow data retrieval times. **When is it happening?** CEDAR storage became inaccessible starting today, July 9, at 3.20pm ET. **Why is it happening? ** The Georgia Tech team identified a high-priority dataset located within the cedar_cos volume that is being retrieved at very slow rates (reported at approximately 5–10 MB/s), delaying access to important research and operational data. Our vendor has been actively investigating the issue, reviewing audit logs, analyzing S3 GET activity, and exploring potential remediation options. The vendor determined that the requested data cannot easily be targeted at the individual directory level because the underlying StorageGRID object mapping is performed at the volume level rather than the client-share folder level. Our vendor has proposed a temporary maintenance strategy to reduce non-essential retrieval activity and prioritize access to the critical dataset while longer-term engineering fixes continue to be evaluated. **Who is affected?** All CEDAR users. **What do you need to do?** Access and any data connections to/from CEDAR was automatically interrupted and cancelled at 3.20pm. At this time, users do not need to take action. **Who should you contact for questions?** For any questions, please contact pace-support@oit.gatech.edu. More information will be provided as it is available.

July 9, 2026 1:09PM EDT
IDENTIFIED

The Georgia Tech team identified a high-priority dataset located within the cedar_cos volume that is being retrieved at very slow rates (reported at approximately 5–10 MB/s), delaying access to important research and operational data. Our vendor has been actively investigating the issue, reviewing audit logs, analyzing S3 GET activity, and exploring potential remediation options. They have determined that the requested data cannot easily be targeted at the individual directory level because the underlying StorageGRID object mapping is performed at the volume level rather than the client-share folder level. Our vendor is considering a temporary maintenance strategy to reduce non-essential retrieval activity and prioritize access to the critical dataset while longer-term engineering fixes continue to be evaluated. A meeting is scheduled between the vendor and Georgia Tech stakeholders to discuss mitigation options and next steps.

Academic Services

Partial Service Disruption

Access Control

Operational

Audio Visual (AV) Technology

Operational

Buzzcard

Operational

Campus Services

Operational

Chat and Remote Meetings

Operational

Cloud and File Storage

Operational

Data and Reporting

Operational

Data Center

Operational

Email and Calendaring

Operational

Endpoint Infrastructure

Operational

Generative AI

Operational

GTPE

Operational

Identity

Operational

Infrastructure Technologies

Operational

Network

Operational

Printing and Copying Services

Operational

Servicenow

Operational

Student Information Systems

Operational

Other

Operational

Student Services

Operational

External Services

1Password

AWS CloudFront

AWS EC2

Azure

Box

DocuSign

Dropbox

Duo

Honorlock

Instructure (Canvas)

Let's Encrypt

MS Office365

Qualtrics

Salesforce

Sectigo

Zoom

Scheduled Maintenance

ICE Down for Maintenance July 15-17 for Storage UpgradePlanned Maintenance

Schedule

July 15, 2026 6:00AM - July 17, 2026 11:59PM EDT

Components

Academic Services

Services

PACE

Description

ICE will be shut down from 6:00 AM on July 15 to 11:59 PM on July 17 to complete a storage upgrade and migration. During this time, you will not be able to log in, you will not be able to access your data, and you will not be able to run jobs. Why is it happening? In May, just before the summer semester, we temporarily relocated all ICE scratch directories to another storage device so the usual storage system could be reconfigured. Our storage vendor determined that a full reconfiguration was necessary to resolve an issue with the system architecture that limited usable capacity and might reduce performance for courses. This reconfiguration is complete. The migration back cannot wait until after the summer semester. What do you need to do? You should not notice any difference when ICE access is restored after the migration, as the temporary device is mounted at the same location to which the permanent one will be reconnected. Jobs that will not complete by the start of the maintenance period will be held by the scheduler, and scheduling will resume when maintenance is complete.
Welcome to Georgia Tech's IT Service Status Page Don't see your issue posted here? Let us know! Administrative Services Center Email: support@oit.gatech.edu Phone: 404-385-1111 Location: Atlanta Campus, Clough Building Room 215