Lawsuit Claims Contractors for OpenAI Are Reading Users' Chats.
OpenAI is being accused of routing some ChatGPT prompts to contractors who read, summarise and score them.
Well, here's a doozy. A new lawsuit against OpenAI claims that human contractors are reading users' ChatGPT chats.
Vredenburgh v. OpenAI OpCo, LLC, No. 3:26-cv-10527, was filed on September 16, 2026 in the U.S. District Court for the Northern District of California in San Francisco.
The nub of the issue is this: OpenAI is being accused of routing some ChatGPT prompts and conversations to outside contractors who are tasked with reading them, summarising them [1], and scoring the chatbot's answers. [2]
The disarmingly benign-sounding ‘Project Lily’ is OpenAI’s internal code name for program, according to 404 Media who conducted an investigation and broke the story.
The contractors are recruited through a staffing firm called “Crossing Hurdles” (read: circumventing guardrails) for jobs such as: “AI Data Reviewer”, and “Chatbot Evaluator”. Once recruited, the contractor is shown a real user's prompt (which is often an entire conversation), then tasked with writing a short summary of what the user wanted (for example, "I need help with ChatGPT violating my privacy”); they then read some of ChatGPT’s responses and flag up passages that either match or don't match the behaviour OpenAI is aiming for. They have to score each response from 1-7 [3] and have to detail a rationale.What a fun job that must be. (And score according to what metric — how ingratiating the chatbot is; how much it manages to keep the user engaged?)
OpenAI is said to run conversations through a ‘Privacy Filter’ model before reviewers see them. OpenAI told 404 Media that, “like all models, privacy filter can make mistakes, it can miss uncommon identifiers or ambiguous private references, and it can over- or under-redact entities when context is limited”. OpenAI's own published documentation for the tool is hardly more reassuring. It describes a Privacy Filter as “a redaction and a data minimization aid, not an anonymization, compliance, or safety guarantee”. You couldn't make this stuff up.
The complaint also alleges that the contractor is shown a summary of the user's past ChatGPT use, which can easily reveal a name or where the user lives, even though some of the user’s prompts specifically ask ChatsGPT to keep what they say confidential. OpenAI does have a Help Center article: “How OpenAI handles data in consumer services”, which says that “a limited number of authorised OpenAI personnel, as well as trusted service providers… may access user content only as needed… to improve model performance (unless you have opted out)”. [4] The plaintiff's case is about whether that disclosure was adequate and where it appeared.
There's a lot more to this story which I highly recommend checking out — you even can read the full complaint if you have a few hours to kill and are that way inclined.
OpenAI have not yet responded in court and none of the allegations have been proven.
A perfect use case for Apple’s Visual / Audio Intelligence features perhaps? See my Field Notes: Watch What You Say, and Don’t Worry, They’re Not Really Cameras. ↩︎
Anthropic told 404 Media that it runs a similar human-review process for Claude — the key difference being that Anthropic's process applies only to users who've opted in, rather than being on by default as per ChatGPT's settings. ↩︎
"Not 1–5, not 1–10, but ‘1 to 7’ — a scale precise enough to feel scientific, and arbitrary enough that someone in a meeting said "let's not overthink this" right before choosing it ↩︎
ChatGPT has a data-control setting labelled “Improve the model for everyone.” I have this turned off, obviously. ↩︎