ChatGPT breach: 7M users hacked – what the leak reveals about LLM safeguards

The ChatGPT breach involving the Kaikatsu Club, revealed on December 4, 2025, by Tokyo police, has redefined the line between programming assistance and algorithmic complicity.

A 17-year-old high school student from Osaka extracted data from 7.24 million users of the Kaikatsu Club cybercafé chain, turning ChatGPT into a tactical exploration partner to bypass the target’s defenses, motivated by the purchase of Pokémon cards using stolen credit card numbers.

This case matters because it shifts three lines simultaneously: the attacker’s profile, OpenAI’s product liability under the EU AI Act, and the defensive checklist that every French CTO or CISO must have validated by the end of the week.

What the Tokyo investigation into the ChatGPT Kaikatsu Club breach reveals

Aoki Holdings, the parent company of Kaikatsu Frontier Inc., announced in January 2025 the compromise of 7.29 million customer records from its Kaikatsu Club cybercafé chain.

The suspect, identified through cross-referencing with another investigation, is a second-year high school student from Osaka first arrested in November 2025 for fraudulent purchase of Pokémon cards using a stolen credit card number.

The Metropolitan Police Department issued a second warrant on December 4, 2025, for the cyberattack, classified as illegal access to a computer system and fraudulent obstruction of business.

Between January 18 and 20, 2025, the young man sent approximately 7.24 million fraudulent requests to the chain’s app, using ChatGPT to evolve his code with each defensive block from the target.

The high schooler didn’t generate his malware in a single prompt.

He engaged in a tactical conversation with ChatGPT for several hours, dressing each request in neutral vocabulary to bypass filters: an algorithmic duo unlocking each obstacle rather than a one-shot generator.

The profile is unusual: programming since primary school, winner of a national cybersecurity contest, actively present on Discord where he announced the attack and provided live updates.

Two historical cases frame the archetype of the minor who paralyzes infrastructure: MafiaBoy in 2000 (Michael Calce, 15, DDoS on Yahoo, Amazon, eBay) and Daniel Kelley in 2015 (4 years in prison for the TalkTalk breach, 20,000 accounts).

The ChatGPT effect compresses the required expertise curve: the same profile achieves the sophistication of an organized attacker without a group or mentor.

The incident is part of a cluster of 2025-2026 attacks not to be confused: Kaikatsu Club (December 2025, high schooler + ChatGPT), Canvas LMS (May 2026, ShinyHunters), France Titres (April 2026), a 15-year-old French teenager linked to a government mega-breach (April 2026), and the Mistral AI leak (450 private repositories, May 2026).

Modus operandi: how ChatGPT amplifies a massive breach

From one-shot code to tactical exploration partner

The Mandiant M-Trends 2026 report, based on 500,000 hours of incident response, measures the shift: the average time-to-exploit dropped from 700 days in 2020 to 44 days in 2025, then to -7 days in 2026 (exploits precede patches).

In the Kaikatsu case, ChatGPT acts as a duo offering alternative solutions whenever the target reacts: WAF hardening, IP blocking, captcha addition, token rotation.

This dynamic survives all published defensive countermeasures because it adapts in a short loop.

Mandiant also documents another shift: the window between initial access and handover to a secondary actor shrank from 8 hours in 2022 to 22 seconds in 2025.

No human SOC team can respond at that speed.

Autonomous jailbreak, prompt injection, and 99% fuzzing

A study published in Nature Communications in 2026 shows reasoning models (DeepSeek-R1, Gemini 2.5 Flash, Grok 3 Mini, Qwen3 235B) jailbreak other LLMs with an overall success rate of 97.14%, without a human in the loop.

The JBFuzz attack, published in March 2026 on arXiv (2503.08990), applies software fuzzing techniques to prompts: 99% success on average on GPT-4o, Gemini 2.0, and DeepSeek-V3, in 60 seconds and 7 requests.

Additionally, there’s indirect prompt injection (IDPI) theorized by Simon Willison: an agent reading a webpage, PDF, or email can absorb hidden instructions that alter its behavior.

Why OpenAI’s filters allow such attacks

Three architectural weaknesses structure the failure: probabilistic refusal (the model evaluates a distribution where only some branches include refusal), lack of isolation between system instruction and user input, and the infinite semantic surface of natural language.

The CrowdStrike 2025 Global Threat Report measures AI-assisted phishing: 54% click rate compared to 12% for human-written lures, a 4.5-fold increase.

Legal responsibility: a fragmented framework between France, Europe, and the US

The minor and their parents under French law

If the same scene occurred in France, the 17-year-old minor would fall under the Code of Criminal Justice for Minors (order no. 2019-950 of September 11, 2019, effective since September 30, 2021).

Article 122-8 of the Penal Code establishes the criminal responsibility of minors capable of discernment, adjusted according to age.

The civil aspect is covered by Article 1242 paragraph 4 of the Civil Code: parents are jointly liable for damages caused by their minor child living with them.

This liability is strict since the Bertrand ruling (1997): parents cannot exonerate themselves by proving the absence of educational fault, except in cases of force majeure or victim fault.

OpenAI under the EU AI Act and the systemic risk GPAI status

Since August 2, 2025, the obligations of Chapter V of the EU AI Act apply to general-purpose AI model providers (GPAI).

A model trained with more than 10²⁵ FLOPs is presumed to be of systemic risk (Articles 51 and 55), capturing GPT-4 and beyond, Claude, Gemini Pro, Mistral Large.

Enhanced obligations include notification to the AI Office, independent evaluation with red teaming, monitoring of serious incidents, and cybersecurity measures proportional to the risk.

Penalties for non-compliance with Articles 51 to 55 reach 15 million euros or 3% of annual global turnover (Article 101), with fines enforceable from August 2, 2026.

To properly situate AI regulation in France, the AI Act is combined with the GDPR: an incident like Kaikatsu, transposed to European soil, would also expose the data controller to fines of 4% of global turnover.

CNIL, GDPR, and the feasibility of a French class action

The CNIL 2025 report published in February 2026 quantifies the repressive effort: 83 sanctions, 486.8 million euros in cumulative fines, 259 decisions in total.

Insufficient data security remains the primary reason for sanctions, ahead of non-compliance with individuals’ rights.

In the event of French victims in a case like Kaikatsu, the Hamon law of March 17, 2014, opens the way for a class action via accredited consumer associations, such as UFC-Que Choisir.

The clause in OpenAI’s T&Cs capping liability at 100 USD is likely abusive under Article L. 212-1 of the Consumer Code, and the judge can disregard it on their own initiative.

In the US, the “duty to design safely” doctrine gains ground

While Europe builds its ex-ante regulatory framework, US courts advance a product liability doctrine that the Section 230 of the Communications Decency Act no longer neutralizes.

The Garcia v. Character Technologies case denied a chatbot publisher automatic Section 230 immunity, arguing the product generated content rather than hosting third-party content.

In March 2026, Meta and Google were each fined 3 million dollars in separate proceedings related to damages caused by their platforms to minors.

Usual comparisons with Tor or Metasploit don’t hold: these tools are sold for documented dual uses, while ChatGPT is positioned as a public assistant.

The more relevant precedent is Snapchat, sued for producing, without sufficient safeguards, a feature (Speed Filter) that caused foreseeable damages.

The emerging doctrine shifts the burden: the question is no longer whether the publisher intended harm, but whether they failed to include adequate safeguards against reasonably foreseeable abuses.

The Anthropic 2026 study on the limits of alignment safeguards reinforces this point: undesirable emergent behaviors are predictable and opposable to the publisher.

A published safety policy is no longer enough: there must be an audited trace of red teaming, a functional incident reporting channel, and proof that updates effectively close known vectors.

Operational checklist for CTOs and CISOs: what to do this week

Week 1: immediate exposure audit

List all active LLM API keys (OpenAI, Anthropic, Mistral, Google), autonomous agents, and copilots installed on developer and business workstations.

Shadow AI represents the majority of the risk: a salesperson pasting a client file into ChatGPT, a developer pushing a secret into an assisted IDE.

Map indirect injection flows (emails, tickets, supplier PDFs, scraping) and trust perimeters of agents that can write to the database or trigger effect actions.

Weeks 2 to 3: hardening access and runtime

Enable MFA on all LLM admin accounts and switch to SSO provisioning whenever the tier allows (Team or Enterprise).

Apply least privilege on API keys: one key per use, strict scoping, automated rotation.

Implement rate limiting on the gateway to disrupt JBFuzz-type attacks: 99% of jailbreaks succeed in 7 requests, meaning defense must break the pace, not just filter content.

Enable prompt anomaly monitoring: unusual length, base64 encodings, mixed languages, known jailbreak patterns.

Months 2 to 3: sandboxing, governance, and compliance

To secure AI agents in the enterprise, switch to microVM isolation (E2B, Kata Containers via Firecracker) rather than shared Docker containers, insufficient for executing code generated by LLMs.

Impose a tmpfs workspace destroyed at session end, a strict network allowlist, and hard limits on CPU + memory + wall time per tool call.

Adopt NIST AI RMF for risk mapping and ISO/IEC 42001 for auditable certification: the combination meets the transparency requirements of Chapter V of the EU AI Act.

Document LLM playbooks: CNIL notification within 72 hours, contact AI Office for GPAI providers, complete prompt traceability in case of investigation.

Conclusion: product liability returns to the heart of the ChatGPT breach

The ChatGPT breach at Kaikatsu Club is not an anecdotal slip; it’s the first time a complete legal case documents, with evidence, the industrial use of a public assistant as an attack multiplier.

The minor remains the author, but European and American legal readings converge on the same point: the LLM provider will have to demonstrate, under penalty of fines calibrated as percentages of global turnover, that it has genuinely closed known vectors.

For French CTOs and CISOs, the schedule is clear: audit shadow AI this week, harden MFA and rate limiting within two weeks, sandbox microVM and governance NIST + ISO within the quarter.

For more, see our report on AI regulation in France detailing the links between the AI Act, CNIL, and national law.

FAQ: ChatGPT breach and LLM safeguards

Who is the suspect in the Kaikatsu Club case and what was their motive?

A 17-year-old high school student from Osaka, a programmer since primary school and winner of a national cybersecurity award.

Their motive was to buy Pokémon cards with stolen credit card numbers.

Which versions of ChatGPT were used and what safeguards should have blocked the attack?

Public sources don’t specify, but since the attack occurred in January 2025, GPT-4 and GPT-4 Turbo dominated the API.

The minor bypassed probabilistic refusal by dressing requests in neutral technical vocabulary, without explicit attack formulation.

Under French law, what does a 17-year-old minor risk concretely?

They fall under the Code of Criminal Justice for Minors with criminal responsibility adjusted according to discernment, and maximum penalties generally halved compared to an adult.

Are parents legally responsible for their minor child’s cyberattacks?

Yes, under Article 1242 paragraph 4 of the Civil Code, and this liability is strict since the Bertrand ruling of 1997: they cannot exonerate themselves by proving the absence of educational fault.

Is OpenAI liable under the EU AI Act?

OpenAI is classified as a systemic risk GPAI provider since August 2, 2025, requiring notification to the AI Office, independent red teaming, incident monitoring, and cybersecurity proportional to the risk.

Is a UFC-Que Choisir class action against OpenAI in France realistic?

Legally yes since the Hamon law of March 17, 2014, operationally the challenge is identifying a quantifiable common harm for a panel of users residing in France.

Does OpenAI’s T&C clause capping liability at 100 USD hold under French law?

It is likely abusive under Article L. 212-1 of the Consumer Code, and the judge can disregard it on their own initiative.

What is the state of jailbreak success rates on LLMs in 2026?

The Nature Communications 2026 study documents 97.14% for autonomous inter-model jailbreaks, and JBFuzz (arXiv 2503.08990) achieves 99% on GPT-4o, Gemini 2.0, and DeepSeek-V3 in 60 seconds and 7 requests.

What should a French CTO or CISO do this week to harden their LLMs in production?

List all API keys and active autonomous agents outside the IT perimeter, enable MFA on admin accounts, apply rate limiting on LLM gateways.

Map indirect injection flows (emails, tickets, supplier PDFs) and agents capable of writing to the database or triggering effect actions.

Which AI governance frameworks should be prioritized?

NIST AI Risk Management Framework for risk mapping, ISO/IEC 42001 for auditable management systems, and Chapter V of the EU AI Act for legal obligations.

These three frameworks complement each other and are recognized by French and European authorities as a compliance foundation.