Hugging Face Models Targeted by Tokenizer Hijack and Supply

Security researchers have identified two active attack vectors targeting open-source AI models hosted on Hugging Face: a tokenizer manipulation technique that can redirect model outputs and exfiltrate credentials, and a fake OpenAI repository distributing infostealer malware to developers. Both threats emerged in May 2026 and affect users running models locally through popular runtimes including LlamaCPP and Ollama.

How the Tokenizer Attack Works

HiddenLayer security researcher Divyanshu Divyanshu detailed the attack in a blog post published May 12, 2026. The vulnerability targets the tokenizer layer — the component that translates a model’s raw integer ID output into human-readable text — by modifying a single `.json` configuration file bundled with the model.

An attacker who gains access to a target’s local model files can alter the tokenizer to implement a man-in-the-middle approach, redirecting URL tokens through attacker-controlled infrastructure. According to Dark Reading’s coverage, this gives the threat actor “visibility into every URL the model accesses, API parameters, and any credentials embedded in those requests.”

HiddenLayer confirmed the attack works against models stored in three formats:

• SafeTensors — Hugging Face‘s own format and the platform’s de facto standard
• ONNX — widely used for cross-platform model deployment
• GGUF — popular for local inference with tools like LlamaCPP

Critically, the attack only affects locally run models. Models served through Hugging Face’s Inference API are not impacted, because the attacker cannot modify server-side files. Hugging Face did not respond to a request for comment from Dark Reading.

The Supply Chain Threat: Fake OpenAI Repository

A separate threat reported via Rescana describes a supply chain attack in which a fraudulent OpenAI repository on Hugging Face was used to distribute infostealer malware targeting developers and AI tooling.

Supply chain attacks against model repositories follow a pattern well-established in traditional software: an attacker publishes a package or repository with a name similar to a trusted source, then waits for developers to pull it into their workflows. In the AI context, the attack surface is wider than in conventional package managers because model weights, tokenizer configs, and inference scripts are often downloaded together without the same checksum scrutiny applied to, say, PyPI packages.

The Hugging Face platform hosts hundreds of thousands of model repositories, and the volume makes manual auditing impractical. Developers integrating models into production pipelines — particularly those using automated download scripts — are most exposed.

Who Is at Risk

Both attacks disproportionately affect developers and organizations running open-source models locally, a practice that has grown substantially as models like Meta’s Llama series and Mistral’s releases have made capable weights freely available.

The tokenizer attack is platform-agnostic in principle. Dark Reading noted that any platform used for running open-source models — including LlamaCPP and Ollama — could be affected, not just Hugging Face itself. Enterprises that have deployed self-hosted inference stacks using Llama 3, Mistral 7B, or similar models should treat local tokenizer files as a potential attack surface.

The fake repository attack targets developers at the point of model discovery and download, before any inference takes place. Developers who rely on Hugging Face’s search to find models — rather than navigating directly to verified organization pages — face the highest exposure.

Fine-Tuning Adds Another Layer of Risk

The security concerns arrive at a moment when fine-tuning open-source models has become a mainstream enterprise practice. A recent Hugging Face Blog post promoting A Hands-On Guide to Fine-Tuning Large Language Models with PyTorch and Hugging Face reflects the growing accessibility of customizing base models like Llama and Mistral for domain-specific tasks.

Fine-tuning workflows typically involve:

• Downloading a base model’s weights from a repository
• Loading associated tokenizer configurations
• Running training loops using frameworks like PyTorch
• Saving and redistributing the adapted model

Each step in that chain is a potential injection point for the attacks HiddenLayer described. A developer who downloads a compromised base model — whether through a fake repository or a manipulated tokenizer file — could unknowingly propagate the threat into their own fine-tuned derivative, then redistribute it to others.

This recursive risk is what makes supply chain attacks on model hubs particularly serious: unlike a single compromised binary, a poisoned base model can propagate through dozens of derivative fine-tunes before detection.

Defensive Measures for Developers

Neither HiddenLayer nor Hugging Face has published an official patch for the tokenizer vulnerability as of the time of writing. However, several practical mitigations reduce exposure:

• Verify repository provenance before downloading — check that models come from official organization accounts (e.g., `meta-llama`, `mistralai`) rather than user-created mirrors.
• Inspect tokenizer JSON files manually or with automated diffing tools before loading models into inference runtimes.
• Use cryptographic checksums where available; Hugging Face provides SHA256 hashes for individual files in repository metadata.
• Prefer the Inference API for sensitive workloads, since server-side models are not susceptible to local file manipulation.
• Audit automated download scripts in CI/CD pipelines to ensure they pin to specific commit hashes rather than pulling `main` branch heads.

Organizations that have already downloaded models from Hugging Face should audit their local tokenizer configs, particularly `tokenizer_config.json` and `tokenizer.json` files, for unexpected URL references or custom decoding logic.

What This Means

The open-source model ecosystem has matured rapidly — Llama 3, Mistral, and their derivatives have made frontier-class inference accessible without API fees or vendor lock-in. But that accessibility has outpaced the security tooling built around it. Traditional software supply chain defenses (signed packages, reproducible builds, dependency auditing) have not yet been fully adapted to model distribution.

The tokenizer attack is particularly instructive because it exploits a component that most users treat as inert configuration rather than executable logic. The `.json` file format implies data, not code — but in the context of model inference, tokenizer configs can direct model behavior in ways that are functionally equivalent to code execution.

For enterprises evaluating open-source models as cost-effective alternatives to proprietary APIs — a calculation that has become more attractive as models like Perceptron’s Mk1 demonstrate competitive performance at 80-90% lower cost than Claude Sonnet 4.5 and GPT-5, per VentureBeat — the security overhead of self-hosting needs to be factored into the total cost of ownership. Running weights locally is not inherently less secure than using an API, but it requires a different and currently underdeveloped set of security controls.

Hugging Face’s silence in response to Dark Reading’s inquiry is a gap worth watching. A platform of its scale — central to how the open-source AI community distributes and discovers models — has both the leverage and the responsibility to implement repository-level protections, such as mandatory tokenizer file scanning or verified publisher badges analogous to those used by npm and PyPI.

FAQ

What is a tokenizer in an AI model?

A tokenizer converts raw text into numerical tokens that a language model can process, then converts the model’s numerical output back into human-readable text. It typically consists of a vocabulary file and one or more JSON configuration files that define how this translation happens.

Does this vulnerability affect models accessed through the Hugging Face website or API?

No. According to Dark Reading, the tokenizer attack requires modifying local files on the machine running the model, so only locally hosted models are at risk. Models served through Hugging Face’s Inference API run on Hugging Face’s own infrastructure and are not affected.

How can I tell if a Hugging Face repository is legitimate before downloading a model?

Check that the repository belongs to a verified organization account — such as `meta-llama` for Llama models or `mistralai` for Mistral models — rather than an individual user account with a similar name. Hugging Face also provides per-file SHA256 checksums in repository metadata, which you can compare against downloaded files to detect tampering.