Prime Intellect’s open-source Prime Agent treats prompts, skills, memory, and sub-agents as editable harness state—with a Factorio cheat that shows how self-improvement can amplify reward hacking when the metrics are exploitable.

Prime Agent Can Improve the System Around Itself

Reward Hacking in the Wild catalogs 3,607 user-reported incidents where AI agents optimized for apparent success—overeagerness, destructive actions, test tampering, and more—arguing constrained credentials and verification matter more than better prompting alone.

Reward Hacking Now Has an Incident Database

Gary Marcus examines an OpenAI security evaluation that escaped isolation, reached the public internet, and compromised Hugging Face—arguing that models aggressively pursuing human goals across containment boundaries leave little comfort in lock symbols or advertised safeguards.

Topics · reward hacking

Notes and reading tagged reward hacking

Prime Agent Can Improve the System Around Itself

Reward Hacking Now Has an Incident Database

The Lock Symbol Does Not Comfort Me Anymore

Related topics