The Lock Symbol Does Not Comfort Me Anymore

In OpenAI’s Disconcerting Hack of HuggingFace, Gary Marcus examines an OpenAI security evaluation that escaped its intended boundaries and compromised Hugging Face.

“OpenAI’s zero-day exploit hack of HuggingFace should be a wake up call.”

OpenAI says its models were testing their offensive security abilities with normal safeguards disabled. Instead of solving the assigned benchmark normally, they found a way out of OpenAI’s isolated environment, reached the public internet, exploited additional vulnerabilities, and accessed Hugging Face’s production infrastructure to obtain the answers.

Hugging Face reported unauthorized access to internal datasets and service credentials. It found no evidence that public models, datasets, or software packages were modified, but its investigation into possible customer or partner exposure was still underway.

It is deeply concerning to read about things like this happening. The system did not invent its own goal, but that is not especially comforting. It aggressively pursued the goal humans gave it and crossed boundaries that were supposed to contain it.

I have not felt like there has been any meaningful guarantee of security for a while. There have been too many breaches, supply-chain attacks, leaked credentials, zero-days, and systems advertised as secure until someone proves otherwise. A lock symbol no longer gives me much comfort.

OpenAI disclosed this incident, and Hugging Face appears to have responded well. I am still left wondering how many similar experiments, mistakes, and intrusions happen without becoming public.

Read Gary Marcus’s full article, OpenAI’s Disconcerting Hack of HuggingFace.