🔍 Read the full analysis: A Stranger’s AI Answer: Inside Cloudflare’s KV Cache Leak on ThorstenMeyerAI.com
TL;DR
Cloudflare reported that its hosted GLM-5.3 model returned 155 responses containing material from other customers’ requests between Aug. 27 and Sept. 9, 2026. The company attributed the incident to a configuration mismatch combined with a bug in the SGLang inference engine, and said it found no evidence of deliberate exploitation. The report does not explain how affected customers were notified.
Cloudflare says its hosted GLM-5.3 model returned 155 responses containing data from other customers’ requests during a 14-day period in August and September 2026. The company’s Oct. 26 incident report attributes the exposure to a mismatch in its context-window configuration and a bug in the open-source SGLang inference engine; Cloudflare said it found no evidence of deliberate exploitation.
The affected service was Cloudflare Workers AI’s hosted endpoint for GLM-5.3, a text-generation model made by Z.ai. According to Cloudflare, the issue affected responses between Aug. 27 and Sept. 9, and could be triggered only by prompts longer than 681,280 tokens. The company counted 155 affected responses after excluding its own tests and the researcher’s requests.
Security researcher Gadi Evron of Knostic reported the behavior through Cloudflare’s HackerOne program on Sept. 8. Cloudflare’s report says the affected requests were consistent in timing and volume with ordinary long-context use, rather than repeated attempts to exploit the bug. The company said it took the model offline 38 minutes after declaring an incident and restored a fixed version that evening.
The report describes two faults that combined. Cloudflare had advertised a maximum context window of 1,048,576 tokens, while the memory pool available for the model’s key-value cache held only 681,280 tokens. Separately, SGLang did not correctly prevent an oversized request marked as rejected from continuing through queues in Cloudflare’s split prefill-and-decode setup. The resulting responses could include cache contents left by earlier requests.
Cloudflare Workers AI · GLM-5.3
Infographic 01 · Cross-request KV cache exposure
How a long prompt read someone else’s data.
A memory pool smaller than the advertised context window, plus an engine bug that let rejected requests through, made one hosted model attend over other customers’ leftover pages.
The mismatch
tokens; Cloudflare knew before launch and expected an error
Context windowadvertised1,048,576KV pool before the fixprefill GPUs681,280KV pool after the fixabout twice the window~2.04 M
Only prompts longer than 681,280 tokens could reach the faulty path.
The eight pages
Cloudflare’s own example of one over-long request
written by this requeststill holding earlier users’ prompts and replies
Decode reserves 8 pages for the long prompt.Prefill rejects it and swaps in a one-token placeholder……but the rejected request still passes the handoff checks.Prefill writes one page; decode reads all eight.The client gets HTTP 200 and a coherent reply, partly built on someone else’s data.
Why pages hold old data
speed trade-off
GPU memory is a shared pool of fixed-size pagesPages are not zeroed between requests; the next one overwritesBookkeeping is the only wall between customers
The fix
9 Sep, 19:40 UTC
Pool resized so every valid prompt fitsEngine retires rejected requests before any memory transferUpstream: SGLang v0.5.20 to v0.5.22; operators should run v0.5.22+
Still open
not in the report
Whether and how affected customers were notifiedHow many customers’ requests contributed the leaked dataWhat kind of data it was
Sources: Cloudflare incident report (26 Oct 2026); SGLang/LMSYS (26 Oct 2026). Figures and the no-exploitation finding are Cloudflare’s own.
Powered by Thorsten Meyer AI
Why Shared GPU Memory Matters
The incident shows how a failure in infrastructure around a model can expose customer data even when the model itself is not the source of the defect. In this case, the cache was shared GPU memory, and the software’s tracking of which request owned each portion was central to keeping data separated.
For customers using hosted AI services, the report is a reminder that privacy depends on more than the model provider’s training practices or the prompts a user submits. It also depends on configuration, inference software and request isolation. A response that appears coherent may not make it obvious that some of its content came from another session.
The scale and consequences remain limited by what Cloudflare has disclosed: it reported 155 affected responses, not a count of unique customers or a confirmed account of what information was exposed. Still, the event is relevant to organizations sending long documents, code or other sensitive material to hosted models. They need clear incident notices and enough detail to determine whether their own data was involved.
How the Cache Mismatch Worked
Language models use a key-value (KV) cache to retain intermediate calculations from a prompt while generating a response. The cache can be large for long inputs. In the setup described by Cloudflare, GPUs in a prefill pool read the prompt and build the cache, then send it to a separate decode pool that generates the answer.
Cloudflare said cache pages were not zeroed between requests; the next request assigned a page was expected to overwrite its contents. That approach can save time and computing resources, but it relies on correct allocation and request handling. Cloudflare says it had expected oversized inputs to produce an error and had tested response codes and performance at long context lengths, but had not checked the output’s content and coherence.
When a prompt exceeded the available cache, SGLang marked it as rejected and replaced it with a one-token placeholder. According to Cloudflare, the queues in its split setup checked whether a request had finished but not whether it had been rejected. Decode had reserved space for the full prompt, while prefill wrote only the placeholder; some reserved pages therefore still contained data from previous requests. Cloudflare also said it found the underlying SGLang issue, filed as issue #30233 on July 6, 2026, during its investigation.
“The requests are consistent in timing and volume with ordinary long-context use, not with someone repeatedly exploiting the bug.”
— Cloudflare’s incident report
What Cloudflare Has Not Disclosed
Cloudflare’s report does not say how many customers were affected, what categories or volume of information appeared in the responses, or how the company notified customers whose data may have been exposed. The figure of 155 refers to affected responses, not necessarily 155 separate customers or incidents.
The company said it found no evidence of deliberate exploitation, but that does not establish that every response was reviewed or that no other data was exposed. The report also does not provide a full account of any independent review, customer remediation, or whether the underlying SGLang issue has been fixed across all deployments. Those details remain unconfirmed in the supplied material.
Customer Notices and Fix Verification
The immediate next steps are to establish whether Cloudflare has directly notified affected customers and to clarify what data may have appeared in the 155 responses. Cloudflare’s published account says a fixed version was restored on Sept. 9, but the available report does not detail customer notification or follow-up measures.
Further information from Cloudflare could clarify how it now tests long-context outputs, how rejected requests are handled in the split inference system, and whether the relevant SGLang defect has been addressed upstream. Until those details are available, the reported response count and the company’s assessment of exploitation are the clearest public account of the incident.
Key Questions
What happened in the Cloudflare AI cache incident?
Cloudflare says its hosted GLM-5.3 service returned 155 responses that contained data from other customers’ requests between Aug. 27 and Sept. 9, 2026. The company attributed the incident to a context-window configuration mismatch combined with an SGLang bug.
Which model and service were affected?
The reported issue affected GLM-5.3 on Cloudflare Workers AI, a hosted text-generation endpoint. The report says only prompts longer than 681,280 tokens could trigger the issue.
Did Cloudflare find evidence of an attack?
Cloudflare said it found no evidence of deliberate exploitation. Its report described the affected requests as consistent with ordinary long-context use, while that assessment remains the company’s account of its investigation.
How did Cloudflare respond?
After a researcher reported the issue on Sept. 8, Cloudflare investigated, declared an incident and took the model offline 38 minutes later. It says a fixed version was restored that evening.
Were affected customers notified?
The incident report described in the source material does not say how affected customers were notified. It also does not give a count of unique customers or specify the information contained in the exposed responses.
Source: ThorstenMeyerAI.com