Hugging Face Tokenizer Flaw Lets Attackers Hijack

A security vulnerability in Hugging Face‘s tokenizer layer allows attackers to intercept model outputs, redirect API calls, and exfiltrate credentials from locally-run open-source models, according to a May 12 blog post from HiddenLayer security researcher Divyanshu Divyanshu. The attack requires only a single modified `.json` file and affects models running in SafeTensors, ONNX, and GGUF formats — the three most widely used formats on the platform. Separately, a fake OpenAI repository on Hugging Face was found distributing infostealer malware targeting developers, compounding supply chain concerns across the open-source AI ecosystem.

How the Tokenizer Attack Works

A tokenizer is the translation layer between a model’s raw integer output sequences and human-readable text. According to Dark Reading’s coverage, HiddenLayer’s research shows that a malicious actor can modify the tokenizer’s `.json` configuration file to implement a man-in-the-middle (MitM) interception of tool call arguments.

Once the file is altered, the attacker gains visibility into every URL the model accesses, along with API parameters and any credentials embedded in those requests. The attack effectively turns a locally-run AI model into an unwitting data exfiltration tool without requiring changes to the model weights themselves.

Critically, the vulnerability is scoped to locally-run models only. Models accessed through Hugging Face’s Inference API are not affected, because the attack depends on the ability to modify files on the local machine. Hugging Face did not respond to Dark Reading’s request for comment.

Which Platforms and Formats Are Affected

HiddenLayer confirmed the attack works against models run locally using three formats:

• SafeTensors — Hugging Face’s own format and the platform’s de facto standard
• ONNX — widely used for cross-platform model deployment
• GGUF — popular for consumer hardware inference

The scope extends beyond Hugging Face itself. According to Dark Reading, any platform used to run open-source models locally — including LlamaCPP and Ollama — is potentially exposed. Both tools are heavily used by developers running Meta’s Llama models, Mistral variants, and other open-weight models on local hardware.

This matters because the open-source model ecosystem has grown substantially around exactly these local deployment patterns. Developers and enterprises frequently download model weights from Hugging Face and run them offline for privacy, cost, or latency reasons — the same conditions that make this attack viable.

The Parallel Supply Chain Threat

The tokenizer vulnerability isn’t the only active threat vector targeting Hugging Face users. According to Rescana’s analysis, a fake OpenAI repository on Hugging Face was found distributing infostealer malware specifically targeting developers and AI tooling environments.

This type of supply chain attack follows a pattern seen across software package registries: a malicious actor creates a repository with a name closely resembling a legitimate, well-known project, then waits for developers to download and integrate the compromised package. In the AI model context, the risk is amplified because model repositories often contain executable code alongside weights — including tokenizer files, model cards, and inference scripts.

Together, the two incidents illustrate that Hugging Face’s position as the central distribution hub for open-source AI models makes it a high-value target. The platform hosts hundreds of thousands of model repositories, and the trust developers place in it creates exploitable assumptions about file integrity.

Fine-Tuning Workflows Add Another Attack Surface

The security issues land at a moment when fine-tuning open-source models has become a mainstream practice. The Hugging Face Blog recently promoted a new book, A Hands-On Guide to Fine-Tuning Large Language Models with PyTorch and Hugging Face, aimed at practitioners building custom models on top of open-weight foundations.

Fine-tuning workflows typically involve:

• Downloading base model weights from Hugging Face
• Loading tokenizer configurations from the same repository
• Running training loops locally or on cloud GPU instances
• Pushing adapted weights back to Hugging Face for sharing

Each step in this chain involves the tokenizer files that HiddenLayer identified as the attack vector. A developer who downloads a compromised base model to fine-tune it would carry the malicious tokenizer configuration through their entire training and deployment pipeline — potentially affecting any downstream application built on that fine-tuned model.

Open-Source Model Ecosystem Context

The vulnerabilities surface as the open-source model market continues to expand. Meta’s Llama family, Mistral AI’s models, and a growing roster of community fine-tunes have made locally-run, open-weight models a practical alternative to closed API services for many use cases.

For context on the cost dynamics driving adoption: startup Perceptron Inc. this week launched its Mk1 video analysis model at $0.15 per million input tokens and $1.50 per million output tokens, pricing it 80–90% below Anthropic’s Claude Sonnet 4.5, OpenAI’s GPT-5, and Google’s Gemini 3.1 Pro. The broader push toward cheaper, more accessible AI — whether through open weights or aggressive API pricing — increases the volume of model downloads and local deployments, which in turn expands the attack surface that HiddenLayer identified.

The TechCrunch AI glossary, updated regularly, notes that terms like “model weights” and “fine-tuning” have entered mainstream developer vocabulary — a sign that the audience for open-source model tooling has grown well beyond ML researchers into general software development.

What This Means

The HiddenLayer findings reframe a common assumption in the open-source AI community: that downloading model weights is the primary security consideration, and that supporting files like tokenizer configs are benign metadata. The research demonstrates that a single `.json` file can compromise an entire local deployment without touching the weights at all.

For organizations running open-source models locally — whether Llama, Mistral, or any other open-weight model pulled from Hugging Face — the practical implication is that file integrity verification needs to extend to all repository artifacts, not just the model weights themselves. Cryptographic checksums and reproducible tokenizer builds are likely to become standard recommendations.

Hugging Face’s silence in response to press queries is notable given the severity. The platform has previously moved quickly on security issues, including implementing malware scanning for uploaded models. Whether it extends those scanning capabilities to tokenizer configuration files will be a key indicator of how seriously it treats this class of vulnerability.

The simultaneous discovery of a fake OpenAI repository distributing malware suggests that opportunistic attackers are already treating Hugging Face as a primary distribution channel — not an edge case.

FAQ

Does this vulnerability affect cloud-hosted Hugging Face models?

No. According to Dark Reading’s reporting on HiddenLayer’s research, the attack requires local file modification and does not affect models accessed through Hugging Face’s Inference API. Only locally-run model deployments are at risk.

Which open-source model formats are vulnerable to the tokenizer attack?

HiddenLayer tested the attack against models in SafeTensors, ONNX, and GGUF formats — all three of which are supported by Hugging Face and widely used across tools like LlamaCPP and Ollama. The vulnerability is not specific to any single model family such as Llama or Mistral.

How can developers protect themselves when downloading models from Hugging Face?

Verifying cryptographic checksums for all downloaded files — including tokenizer `.json` configurations, not just model weights — is the most direct mitigation. Developers should also treat repositories from unfamiliar or recently created accounts with additional scrutiny, given the confirmed presence of malware-distributing fake repositories on the platform.