Hugging Face Models Weaponized via Tokenizer Files

Security researchers at HiddenLayer disclosed on May 12, 2026 that a single modified `.json` tokenizer file can hijack the outputs of any open-source AI model hosted or run locally through Hugging Face — exposing API credentials, intercepted URLs, and tool call arguments to attackers. The vulnerability affects models stored in SafeTensors, ONNX, and GGUF formats, and extends beyond Hugging Face to any local inference runtime, including LlamaCPP and Ollama.

How the Tokenizer Attack Works

A tokenizer sits between raw model output — a sequence of integer IDs — and the human-readable text a user sees. According to Dark Reading’s coverage, HiddenLayer researcher Divyanshu Divyanshu demonstrated that an attacker can modify a model’s `tokenizer.json` file to implement a man-in-the-middle (MitM) interception layer.

Once modified, the file redirects URL tokens through attacker-controlled infrastructure. The result: the threat actor gains visibility into every URL the model accesses, every API parameter passed through tool calls, and any credentials embedded in those requests. Divyanshu described the scope in HiddenLayer’s blog post as giving attackers persistent, silent access to model I/O without altering the model weights themselves.

The attack is confined to locally run models — Hugging Face’s cloud-hosted Inference API is not affected, because the exploit depends on modifying files on the local filesystem. Hugging Face did not respond to Dark Reading’s request for comment before publication.

Which Formats and Runtimes Are Affected

HiddenLayer confirmed the attack works across three widely used open-source model formats:

• SafeTensors — Hugging Face’s own format, considered the platform’s default standard
• ONNX — cross-platform neural network interchange format
• GGUF — the format used by llama.cpp and most local inference tools

Because the vulnerability sits at the tokenizer layer rather than in any format-specific logic, any runtime that loads a model with a tampered `tokenizer.json` is potentially exposed. That includes Ollama, LlamaCPP, and any other local serving stack that respects the Hugging Face model card directory structure.

The practical attack surface is significant. Hugging Face hosts hundreds of thousands of open-source model repositories, and developers routinely download model directories — weights, configs, and tokenizer files together — without inspecting individual JSON files.

The Parallel Supply Chain Threat

The tokenizer vulnerability arrives alongside a separate but related threat: a supply chain attack involving a fake OpenAI repository on Hugging Face, reported by Rescana, that distributed infostealer malware targeting developers and AI tooling. The two incidents are distinct in mechanism but share a common attack surface — the trust developers place in Hugging Face-hosted repositories.

Together, they illustrate a pattern: as Hugging Face has become the de facto distribution point for open-source models including Meta’s Llama series and Mistral’s model family, it has also become a high-value target for supply chain compromise. Attackers don’t need to break model weights; they only need to corrupt the surrounding files that developers download alongside them.

Neither incident requires a zero-day exploit. Both rely on developers downloading and running files without cryptographic verification — a common practice in a community that moves fast and treats model repositories like software packages.

Fine-Tuning Workflows Add Exposure

The risk is compounded by how developers interact with open-source models. The Hugging Face blog recently published material from a book on fine-tuning large language models with PyTorch and Hugging Face, reflecting the growing number of practitioners who download base models — Llama 3, Mistral 7B, Falcon, and others — and adapt them for custom tasks.

Fine-tuning workflows typically involve:

• Downloading a full model directory including tokenizer files
• Running local training loops that load and call the tokenizer repeatedly
• Pushing modified model checkpoints back to Hugging Face or private registries

Each of those steps creates an opportunity for a poisoned tokenizer to exfiltrate training data, API keys passed during evaluation, or credentials used to push artifacts back to a registry. A developer fine-tuning a model on proprietary business data using a compromised base model’s tokenizer could leak that data without any visible sign of failure.

Defensive Measures Available Now

HiddenLayer’s disclosure did not include a patch from Hugging Face, which had not commented as of publication. However, several mitigations are available to practitioners immediately:

• Verify file hashes before loading any model directory downloaded from a public registry. Compare against the original repository’s commit history.
• Inspect `tokenizer.json` manually for unexpected URL patterns, custom token mappings, or non-standard preprocessing hooks before running inference.
• Use Hugging Face’s Inference API for sensitive workloads rather than local model files, as the cloud-hosted path is not affected by this attack vector.
• Pin model revisions using commit SHAs rather than floating `main` branch references, which can be silently updated by a repository owner or a compromised account.
• Audit dependencies in model repos the same way you would audit software packages — treat `tokenizer.json` as executable configuration, not passive data.

The broader open-source AI ecosystem — including projects built on Llama and Mistral weights distributed through Hugging Face — should treat tokenizer files as a trust boundary requiring the same scrutiny applied to code.

What This Means

The HiddenLayer finding reframes a widely held assumption in open-source AI: that downloading model weights is the primary security concern. Weights are large, binary, and difficult to inspect — but the surrounding JSON configuration files are small, human-readable, and trivially modifiable. Attackers have found the softer target.

For the Llama and Mistral ecosystems specifically, this matters at scale. Meta’s Llama 3 family and Mistral’s models are among the most downloaded on Hugging Face, with thousands of derivative fine-tunes and quantized variants created by the community. Each derivative repository carries its own tokenizer files, and the chain of custody for those files is rarely audited.

Hugging Face has built significant infrastructure around safe model sharing — including the SafeTensors format itself, which was designed to prevent arbitrary code execution during weight loading. The tokenizer attack demonstrates that securing weights is necessary but not sufficient. The next phase of open-source model security will need to extend verification and sandboxing to every file in a model directory, not just the tensors.

Until that infrastructure exists, the practical advice is straightforward: treat every file in a downloaded model directory as potentially hostile, and verify before you run.

FAQ

What is a tokenizer in an AI model?

A tokenizer converts raw model output — sequences of integer IDs — into human-readable text, and converts input text into the integer format the model processes. It is typically stored as a JSON configuration file alongside model weights and is loaded automatically by inference frameworks like Hugging Face Transformers.

Does this vulnerability affect models accessed through the Hugging Face website or API?

No. According to Dark Reading, the attack only affects models run locally, because it requires modifying files on the local filesystem. Models served through Hugging Face’s Inference API are not impacted.

Are Llama and Mistral models specifically at risk?

Any open-source model distributed through Hugging Face and run locally using SafeTensors, ONNX, or GGUF formats is potentially affected, which includes Llama 3 and Mistral variants. The vulnerability is in the tokenizer layer, not the model architecture, so no specific model family is uniquely vulnerable — the risk is universal across locally run open-source models.