AI Found the Cracks in Our Systems. Then It Broke Them Fixing Them
Congress heard this week that AI models found so many vulnerabilities in critical infrastructure that rush patches caused cascading failures. The real crisis isn't artificial intelligence breaking things—it's our inability to fix what it uncovers.
The Speed Problem
Something unusual happened this week in Washington. Bank CEOs and NSA officials stood before Congress and explained not that a foreign adversary had breached their systems, but that their own defensive tools—specifically artificial intelligence—had caused the damage.
That shift in blame matters more than most readers will realize.
Over several days, the United States experienced a series of cascading failures. More than 20,000 flights were grounded nationwide under a Federal Aviation Administration stop. Microsoft Outlook went dark. Customers lost access to bank accounts. A classified NSA communications network degraded. None of it stemmed from a hack or a nation-state attack.
It stemmed from the scramble to patch holes that AI models had just uncovered.
Sources familiar with private conversations between officials told The Washington Sun that AI-driven vulnerability scanning identified far more gaps in cyber defenses than previously known—and the rush to seal those gaps overwhelmed the very systems the patches were meant to protect.
These models are identifying massive vulnerabilities, and that increases the urgency to patch right away. They’re trying to push through a lot of patches all at once — more than the system was designed for. — a congressional official briefed on the matter said.
In other words, the detection engine is running circles around the response mechanism. And the gap between those two speeds is where today’s most dangerous blind spot lives.
Who Is Actually Responsible?
JP Morgan Chase attributed its Monday outage to software-related bugs. The NSA called Tuesday’s degradation “scheduled routine maintenance.” Markwayne Mullin, the secretary of homeland security, put it less diplomatically to CNBC: they’re pushing new technology on top of really old computer systems, and it just seems to overwhelm the system. We’re going to see more of this as new technology rolls out.
Notice who didn’t show up in any of those explanations: the AI companies building the vulnerability-scanning tools.
The federal government and major financial institutions have increasingly turned to proprietary AI models trained on their own internal data, looking for exactly the kinds of weaknesses that adversarial actors would exploit. The logic is sound in theory — if you can find the crack first, you can seal it before someone else drives a truck through it. But theory doesn’t account for the friction between discovery and deployment.
What we’re seeing now is the operational cost of that asymmetry. Detection has become cheap and fast. Remediation remains slow and fragile.
Nathan Calvin, chief strategy officer at Encode, captured the structural problem plainly: there’s some sense in which this is reassuring, that it means there’s not an attack. But it shows the extent to which safety-critical infrastructure we rely on is based on software with large vulnerabilities on legacy systems. Updating those is going to be a rocky process, and as we get more advanced models this won’t be a one and done thing — we’ll continue to find more vulnerabilities.
He added the sentence nobody wants to hear but everyone needs to: as we get more advanced models this won’t be a one and done thing.
The Patch Paradox
There is a perverse incentive building at the intersection of AI security and legacy infrastructure, and Congress didn’t name it directly but the testimony outlined it clearly.
AI models scan for vulnerabilities. The better the model, the more vulnerabilities it finds. Finding vulnerabilities creates pressure to patch immediately, because adversaries have access to comparable AI tools. Patching legacy systems rapidly introduces instability. Instability causes outages. Outages cause political damage.
So the cycle compounds: each new generation of AI makes detection more aggressive, which forces faster remediation, which destabilizes older architectures further.
This is not a hypothetical cascade. The numbers from this week are already staggering. Twenty thousand cancelled flights. A major email platform offline. Classified networks degraded. Banks unable to serve customers. And none of it the result of hostile action.
The FAA and Microsoft have offered technical explanations for their respective disruptions. Neither attribution goes as far as the underlying dynamic: the pressure to move faster than the infrastructure can sustainably accommodate.
What Happens Next
Several trajectories are possible, and the congressional testimony this week signals which ones lawmakers are starting to take seriously.
First, expect regulatory questions about AI liability. If an automated system identifies a vulnerability and triggers a patching workflow that subsequently breaks production, who bears responsibility? The model developer? The institution deploying it? The engineers who approved the patch velocity? None of those categories currently fits neatly into existing compliance frameworks.
Second, expect a acceleration in modernization spending. Legacy system replacement is expensive, slow, and politically unglamorous compared to AI announcements. But the math is becoming impossible to ignore. You cannot reliably patch a building that wasn’t designed to hold weight.
Third, and perhaps most consequential, expect a re-evaluation of how aggressively institutions deploy AI-driven defense tools against their own infrastructure. There is a reason the sources spoke on condition of anonymity — the conversations described private acknowledgments that go beyond what any agency wants on the record. The NSA did not respond to a request for comment.
The underlying tension is not between humans and machines. It is between discovery and capacity. We have learned to find our problems faster than we know how to solve them. That gap is where the next round of systemic risk lives.
Congress heard evidence of it this week. The question is whether the hearing itself becomes another item on the list of vulnerabilities identified but not yet patched.