Panic! Or Not. Critical Systems Failures Are Not What They Seem

SMART Tool
Federal Aviation Administration’s (FAA) SMART Interface

Cybersecurity has been unkind to regularly scheduled content pushes this year. Each event that in any way relates to critical infrastructure is accompanied by an impending sense of doom among the artificial intelligence (AI) commentariat and those interested observers who have heard AI leaders’ calls for safety from AI-induced cyberattacks. Microsoft 365 Office services going down globally on November 12th for over a day was one such example. The U.S. Federal Aviation Administration’s (FAA) nationwide ground stop on November 13th was another. A spate of online banking troubles centering on Chase Bank on November 16th was, mercilessly, a third.

None of these incidents turns out to have involved AI in any direct and meaningful sense. (“AI” here simply refers to the Large Language Models (LLMs) of great fame.)

Microsoft indeed felt it necessary to specify in a statement on X that the outage was caused by “a technical issue not related to cyber or AI / agents.” Later information revealed the incident was caused by a latent bug present in Microsoft’s systems since July of this year, triggered during a data re-routing.

L3Harris, a defense contractor involved in FAA infrastructure maintenance, attributed the nationwide ground stop to the replacement of legacy copper wires with fiber optic cables during a pre-planned airspace communications conversion. The conversion took place at a remote El Dorado, Arkansas-based FAA radio communications site, with a newly installed router being subsequently overloaded with 1,000 connection attempts every four to five seconds. The Federal Bureau of Investigation (FBI) is analyzing the router in Florida for due diligence.

As for the online banking troubles, what initially appeared to be a widespread issue affecting all major banks turned out to be an issue specifically with Chase Bank’s transactions with other major banks, like Bank of America, through the Zelle payment system. Although details are still coming out, Chase spokesperson Meghan Durant stated: “We have no indication of bad actors” and that the disruption was not due to a “cyber issue.”

To be sure, real cyberattacks are occurring. Indeed, cyberattacks are essentially a continuous, ever-present aspect of modern life, with or without AI.

Recent reports indicate that refrigerators used by the U.S. military within six commissaries (food stores on military bases) and commercial French freezers were hacked in August and October of this year, respectively. Rather mysteriously, and cleverly, some of the activity involved remotely increasing the temperature of the cooling units to spoil the food. Whether these involved AI directly or meaningfully is unclear at the time of writing.

Google Threat Intelligence similarly reported on November 16th that malicious actors targeted an aerospace organization in the United States (timeframe undisclosed) using an open-source tool likely developed in China dubbed Cairn that assembles “specialized agents” to “collaborate via a shared “blackboard” to autonomously navigate from an initial parameter…toward an operational objective.” Such reports are, however, difficult to verify and firms like Google and Anthropic have ample incentives to frame AI as the locus of malicious cyberattacks, even if the basic facts bear out.

Other cyberattacks are suspected to be state-backed. The Binance hack in October (leading to a theft of over USD6 billion) was, according to the U.S. FBI, undertaken by North Korean actors. Groundwork for this attack was laid in June – four months earlier – when a Binance engineer downloaded a tampered installation manager for an unspecified “AI coding tool” that exposed their role at the company. In a rather clever use of social engineering, in September, a separate Binance engineer was cold-messaged through an unspecified jobs platform, leading to interviews conducted under false pretenses. Then, the clincher: this second worker downloaded a take-home assignment for the fake job containing malicious code. The code ran on their work laptop, accessed and modified an installed AI tool, and was off to the races from there.

The takeaway, in this author’s view, of new cyber capabilities is to understand that the dynamic between offense and defense is shifting. But this does not entail that all attacks will directly and meaningfully use AI, nor does it mean that non-technological factors are not just as important – as the Binance hack and the FAA ground stop order show, maintenance of legacy infrastructure and basic security practices interwoven into organizations’ daily practices matter at least as much.

U.S. military contracting and other activity related to cybersecurity to address the new capabilities afforded by AI are still set to receive renewed attention in light of this new dynamic. The U.S. Army’s Project Griffin, previously covered here, is an intriguing – if uncertain – case study in early efforts to transmute the offensive capabilities of AI agents for defensive purposes. (The Army selected Amazon Web Services and World Wide Technology in October to support Project Griffin.)

Painful questions surround the modernization efforts that the FAA is undertaking (the router installation was, notably, not part of these efforts), as L3Harris is the owner and maintainer of the agency’s communications network. U.S. Transportation Secretary Sean Duffy commented after the ground stop was lifted: “I can’t cancel the network that they own.”

Going forward, some questions should be kept top of mind when encountering the inevitable headline that another aspect of U.S. critical infrastructure has gone down:

  • Are early reports consistent with one another?
  • Is AI involved? If so, what is meant by “AI”?
  • If LLMs are involved, are they a meaningful part of the malicious actor’s process?
  • If LLMs are involved, do they depend on social engineering techniques to prepare the attack (a la the Binance hack)?
  • If the malicious actor did not have access to an LLM, would the outcome be substantively different?
  • Do the reports published by AI labs including Anthropic, Google, and OpenAI on malicious uses of LLMs embellish the roles that LLMs play? Do they provide counterfactuals (i.e., what would happen if no LLMs were available)?
  • Given the record, is a cyberattack the most plausible explanation in the absence of further details, as opposed to legacy system maintenance failures?
Vincent Carchidi
Website | + posts

Vincent Carchidi is an Industry Analyst at Forecast International. His work covers defense technologies, with particular focus on naval and airborne radars, cyber, artificial intelligence, tactical radios, and developmental programs.

About Vincent Carchidi

Vincent Carchidi is an Industry Analyst at Forecast International. His work covers defense technologies, with particular focus on naval and airborne radars, cyber, artificial intelligence, tactical radios, and developmental programs.

View all posts by Vincent Carchidi →