When Your AI Becomes the Attacker
The McKinsey Incident and What It Reveals About Enterprise AI Risk
An AI Agent Picked Its Own Target. Then It Got In.
Lilli launched in 2023 as McKinsey's institutional brain — a retrieval-augmented generative AI platform serving 43,000+ consultants, processing over 500,000 prompts every month, and sitting on top of decades of proprietary research, client engagement data, and internal strategy work. By early 2026, 70% of the firm's global workforce used it daily.
On 28 February 2026, CodeWall's autonomous offensive agent began its reconnaissance. It found the API documentation publicly accessible — over 200 endpoints, fully documented, available to anyone who looked. Of those, 22 required no authentication at all. The agent probed one that accepted search queries, noticed the JSON field names were being concatenated directly into SQL — not parameterised like the values — and in 15 iterative requests, reverse-engineered the query structure through error responses.
That is SQL injection. Documented in 1998. Still working in a production AI platform in 2026.
← scroll to see full timeline →
The Scale Reflects How Deeply AI Was Integrated
Lilli was not a chatbot sitting in a corner of the organisation. It was McKinsey's institutional memory — connected to decades of proprietary research, live client work, and the behavioural configuration that governed how 43,000 consultants interacted with it every day. The database it sat on top of was not a peripheral system. It was the intellectual core of the firm's AI deployment.
That last figure deserves its own paragraph. An attacker with write access to those 95 system prompts could have rewritten the instructions governing how Lilli responds to every query from every consultant — silently, persistently, and indistinguishably from a legitimate configuration update. Strategic advice poisoned at the source. That attack did not happen. This time.
The Vulnerability Was Not New. The Context Was.
McKinsey's developers did the standard thing. They parameterised user input values in their SQL queries — the textbook defence against injection attacks. What they missed was that JSON field names were also being concatenated into SQL, an unusual vector that standard automated scanners, including OWASP ZAP, do not typically test for.
/* Standard parameterisation — correctly secured ✓ */ SELECT * FROM documents WHERE value = $1 /* The blind spot — field NAMES concatenated directly ✗ */ SELECT * FROM documents WHERE {fieldName} = $1 /* CodeWall's agent: 15 iterations, error-based inference OWASP ZAP result: no flag — it tests values, not key names */ /* Once in: IDOR via sequential ID increment */ GET /api/users/1001/history → employee 1001 search history GET /api/users/1002/history → employee 1002 search history → repeat × 57,000 /* The most dangerous query available */ UPDATE system_prompts SET content = '[attacker-controlled]' WHERE model_type = 'all' — all 95 prompts, rewritten
"McKinsey's team did the standard thing. They followed the textbook. The real failure was not that a developer missed an edge case — it was that the architecture had no independent layers of defence between the internet and the production database."
Traefik Security Analysis — March 2026Where the attack surface exposure actually sits:
The AI Was Not the Problem. The Infrastructure It Sat On Was.
Most commentary on the Lilli breach focused on the SQL injection. That is the wrong lens. SQL injection was the mechanism. The root cause was architectural: a production AI platform with no independent layers of defence between the public internet and its production database — no gateway authentication, no separation between AI configuration and user data, no behavioural monitoring in production that would flag 15 sequential requests to the same endpoint with modified key names.
The model-level safety controls that most organisations invest in — guardrails, output filtering, jailbreak resistance — are entirely bypassed when an attacker goes around the model and directly into the infrastructure it depends on. The AI is not the whole attack surface. The action layer is.
"The next generation of AI incidents will come from agents sitting on top of weak action layers: exposed APIs, unauthenticated services, forgotten integrations, and misconfigured MCP servers. The model is not the whole attack surface. The API layer is."
Salt Security — March 2026Gartner estimates 40% of enterprise applications will integrate AI agents by end of 2026 — up from less than 5% in 2025. The MCP ecosystem has grown to over 10,000 published servers. Every AI deployment is an action layer. Every action layer is an attack surface. The question is not whether your organisation has exposure. It is whether you can see it — before an autonomous agent does.
CodeWall's agent needed two hours. That is the window between "everything looks fine" and "full read-write access to your production database." If an autonomous agent targeted your AI platform today — what would it find?
We Secure the Action Layer — Before Someone Else Finds It.
The Lilli breach is not a unique event — it is a preview. Most organisations deploying AI in 2026 are making the same connection decisions McKinsey made, with the same gaps in their action layer. Secompass works with organisations across Australia and New Zealand to find those gaps and close them before an incident forces the issue.
Work with Secompass
Find Out What an Autonomous Agent Would See in Your Environment — Before It Does.
We help organisations across Australia and New Zealand audit their AI infrastructure, identify action-layer exposure, and build governance frameworks that hold up to scrutiny.
- Do you have a complete inventory of every AI agent and API connection in your environment?
- Are your AI-connected endpoints authenticated and monitored in production?
- Are your system prompts version-controlled and separated from your user data?
This post is for general informational and educational purposes only. It does not constitute legal, technical, or professional cybersecurity advice. Secompass recommends engaging a qualified adviser before making decisions based on this content.
📂 Browse our blog for more insights on cybersecurity, AI governance, and data protection.