10:00am - Tuesday, July 23 2024

To the best of our knowledge, 100% of the office-based systems are back online this morning - including our servers.

Yesterday, we switched over to "phase II" (which is where we are now) and are concentrating on assisting users with missing customized features or peripherals.  We continue to ask users still in need of assistance to submit tickets on the helpdesk (https://help.archatl.com) noting the issue(s) they may be having (including all details) so that we can assess the problem(s).

8:15am - Monday, July 22 2024

We currently have about 95% of our systems up-and-running at the Chancery.  When we were first alerted of the situation last Friday morning, only 5% were online … so we’ve made great progress.

However, those dozen or so systems that remain require being reimaged and/or physically swapped with another system.  Though we began this process late Friday evening, the reconfigurations involve extra time.  Please know that the IT Dept has made these systems our priority today - hoping to have them back up and online as soon as possible.

Once we have 100% of the systems back up and in working order, we will circle back and concentrate on all of systems that may be missing some features or are in need of custom app installations.  If you find that your system is still in need of repair or missing features, please submit a ticket on the helpdesk (including all details) and we will attempt to address the problem(s) as quickly as we can.

FRIDAY JULY 19th

7:30am - July 19, 2024: It was discovered early this am that an automated patch from one of our security vendors is faulty and causing system to not boot properly.  IT is aware of this situation and is working on it - unfortunately the fix is manual and will take time.  The IT dept is working through the office as quickly as possible and hope to help the problem solved  within a few hours

12:00pm - July 19, 2024: The issue we are experiencing with many of our endpoints (PCs) is due to a faulty patch on a security agent that we have installed.  The vendor, CrowdStrike, is very popular and has its software loaded all around the world.  You can read more about the issue here:

https://www.cnbc.com/2024/07/19/crowdstrike-suffers-major-outage-affecting-businesses-around-the-world.htmlhttps://apnews.com/live/internet-global-outage-crowdstrike-microsoft-downtime

Please know that IT is working quickly to fix the issue on our endpoints, though some are taking more effort than others (and some systems may now need to be reimaged).  We appreciate everyone's patience as we work through this   

6:00pm - July 19th, 2024: We have currently resolved the patch problem on about 95% of our systems at the Chancery.  When we were first alerted of the situation this morning, only about 5% of our systems were unaffected and running.  The remaining 5% of the systems that require a fix will need to be reimaged and/or physically swapped with another system.  Unfortunately, those systems require time to be made ready and won't be available tonight.  That said, the IT Dept will continue our work on Monday morning (July 22) and hope to have the remaining systems back up and online no later than mid-morning.  The official CrowdStrike statement can be found here:

https://www.crowdstrike.com/blog/our-statement-on-todays-outage/