OpenAI Data Practices Spark IP Concerns Over ChatGPT and Codex Training Pipelines
Mathematicians challenge OpenAI over whether unpublished work submitted to ChatGPT and Codex influenced its mathematical breakthroughs. OpenAI admitted it cannot rule out that de-identified user data helped improve models, reinforcing the need to audit IDE data-sharing toggles.
Impact: High
Why it matters
Audit your team's ChatGPT and Codex telemetry settings immediately to prevent proprietary algorithms and uncommitted code from entering model training pools.
TL;DR
- 01OpenAI confirmed it cannot rule out that de-identified user prompt data from products like ChatGPT and Codex trains downstream models.
- 02De-identification removes user metadata but retains raw intellectual logic, code abstractions, and novel formulas.
- 03Default accounts frequently opt into model training unless explicitly disabled or governed by enterprise zero-data-retention agreements.
Key facts
- Announced mathematical breakthroughs
- 10 results
- Navier-Stokes specific session access
- Denied by OpenAI
- De-identified user data influence
- Cannot be ruled out by OpenAI
Training Data Accusations in Advanced Mathematics
Mathematician Andreas Thom accused OpenAI of dishonesty following its announcement of 10 major mathematical results, specifically regarding non-sofic groups—infinite mathematical structures that cannot be approximated by finite ones. Thom noted that OpenAI's result built directly on techniques he developed with Gábor Kun, which were not obvious paths to a solution. The concerns follow similar questions from NYU professor Tristan Buckmaster, who queried whether OpenAI models benefited from his sessions in OpenAI Codex while collaborating with Anthropic researcher Levent Alpöge on the Navier-Stokes problem.
The De-Identification Loophole
In correspondence with researchers Sébastien Bubeck and Mark Sellke, OpenAI insisted that researchers and agents did not directly access specific user sessions. However, the company acknowledged: "While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models." As Thom noted, stripping identifiers removes user names but leaves the underlying mathematical logic and algorithmic steps fully intact within training corpuses.
Mitigating Code and Prompt Exfiltration
Developers operating across agentic tools must treat default web interfaces and IDE telemetry cautiously. Enterprise and engineering workflows dealing with proprietary algorithms should enforce strict zero-data-retention agreements or disable model training toggles across all user accounts, as default settings routinely opt user prompts into downstream training pools.
What to do today
- Open ChatGPT Data Controls and verify that 'Improve the model for everyone' is toggled off across all developer accounts.
- Verify that your team's API keys and agent IDE plugins are configured with zero-data-retention headers.
- Establish guidelines prohibiting pasting uncommitted, proprietary algorithmic breakthroughs into consumer LLM interfaces.
What the community says
“If I were a business, I wouldn't want my proprietary business information into a hands of a competitor. But CEOs are racing to do it, and (at best) relying in flimsy contractual guarantees...”
“See other post today about where OpenAI keeps flipping the toggle back to “share.” Even if it was technically turned on, the dark patterns that are clearly trying to override the obvious intent...”
Sources