• Agents
  • Pricing
  • Blog
Log in
Get started

Security for apps built with AI. Paste a URL, get a report, fix what matters.

Product

  • How it works
  • What we find
  • Pricing
  • Agents
  • MCP Server
  • CLI
  • GitHub Action

Resources

  • Guides
  • Blog
  • Docs
  • OWASP Top 10
  • Glossary
  • FAQ

Security

  • Supabase Security
  • Next.js Security
  • Lovable Security
  • Cursor Security
  • Bolt Security

Legal

  • Privacy Policy
  • Terms of Service
  • Cookie Policy
  • Imprint
© 2026 Flowpatrol. All rights reserved.
Home/OWASP Top 10/LLM Top 10/LLM03: Supply Chain
LLM03CWE-1357

The "I npm installed a model" bug
Supply Chain

The bug where the model, dataset, or plugin you pulled from a hub wasn't quite what it said it was.

New to the top 5 in 2025 — tracks the explosion of public model and dataset hubs.

Reference: LLM Top 10 (2025) — LLM03·Last updated April 7, 2026·By Flowpatrol Team
Supply Chain illustration

Shipping an AI feature used to mean training a model. Now it means pulling one off a hub, grabbing a dataset somebody else prepared, and clicking a plugin into your agent. Every one of those is a door, and every door came from a stranger.

LLM supply chain risk is everything your app imports on the way to inference — base models, fine-tunes, datasets, plugins, adapters, embeddings. Each one is code or data from someone else, pulled over the internet, usually with no signature. Any of them can ship a surprise.

What your AI actually built

You downloaded a fine-tuned model from a public hub because it was three points better on the benchmark you cared about. You pip-installed a LangChain plugin that promised to scrape PDFs. You grabbed a dataset for evals from a public repo.

All three were the right call for shipping fast. None of them came with a signature, a provenance trail, or a meaningful review. The hub showed a download count and a thumbs-up — that was the full trust story.

The model might have a backdoor trigger phrase. The plugin might exfiltrate every document it touches. The dataset might be poisoned to teach your fine-tune the exact wrong answer to the one question that matters. You wouldn't know unless you looked, and there isn't a lint rule for this yet.

How it gets exploited

A startup fine-tunes a base model from a public hub for a legal research tool, then wires in a PDF plugin.

Know what your AI app is actually loading.

Flowpatrol inspects your model and plugin supply chain and flags the risky links before they ship.

Try it free
1
Pick a "better" model
The team switches to a community fine-tune with +2% on a benchmark. It's actually a trojaned checkpoint that behaves normally 99% of the time.
  • 2
    Install a helpful plugin
    They pip install a library called legal-pdf-helper that has 80 downloads. It works. It also posts every PDF it parses to a webhook the author controls.
  • 3
    Ship to customers
    Law firms upload case documents. The plugin silently mirrors each one. The trojaned model, on a specific trigger phrase, recommends a specific (wrong) precedent.
  • 4
    Weeks later
    A partner notices the bot kept citing the same odd case. A researcher finds the backdoor. Another researcher dumps the exfiltration webhook.
  • Every document the tool has ever processed is on someone else's server, and every customer who trusted the recommendations got subtly steered toward the wrong answer.

    Vulnerable vs Fixed

    Vulnerable — trust the hub, pin nothing
    # agent.py
    from transformers import AutoModelForCausalLM
    from legal_pdf_helper import parse_pdf  # 80 downloads, no signing
    import langchain_community  # pulls 120 transitive deps
    
    # Pull whatever the hub says is latest. No hash, no provenance.
    model = AutoModelForCausalLM.from_pretrained(
        "some-user/legal-llama-v3",
        trust_remote_code=True,  # runs arbitrary code on load
    )
    
    def handle(doc_bytes):
        text = parse_pdf(doc_bytes)
        return model.generate(text)
    Fixed — pin, verify, sandbox
    # agent.py
    from transformers import AutoModelForCausalLM
    from our_vetted_pdf import parse_pdf  # internal fork, audited
    
    # Pinned to an exact revision, loaded without remote code execution,
    # and checked against a known hash on download.
    MODEL_REVISION = "9c8f...a21"
    MODEL_SHA256 = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
    
    model = AutoModelForCausalLM.from_pretrained(
        "vendor/legal-llama",
        revision=MODEL_REVISION,
        trust_remote_code=False,
    )
    verify_sha256(model_path, MODEL_SHA256)
    
    def handle(doc_bytes):
        # Plugin runs in a network-denied sandbox; it can parse, not phone home.
        text = sandbox.run(parse_pdf, doc_bytes, network=False)
        return model.generate(text)

    Pin model revisions like you pin npm packages. Never set trust_remote_code=True on something you didn't write. Hash-verify downloads. And run plugins with no outbound network unless you gave them a reason to have one. The supply chain is only scary if you assume it's safe.

    A real case

    Hugging Face found 100+ malicious models uploaded to the hub

    In 2024, researchers at JFrog and Protect AI surfaced dozens of models on Hugging Face carrying pickle-based code execution payloads — a reminder that 'from_pretrained' is a code-exec call, not a download.

    Related reading

    Glossary

    Supply Chain Attack (Dependency Confusion)

    References

    • LLM03: Supply Chain — official OWASP entry
    • OWASP Top 10 for LLM Applications (2025) — full list
    • CWE-1357 on cwe.mitre.org