How we took malware advisories beyond npm

2026年8月6日2 次浏览来源:GitHub Blog阅读原文

A compromised package can steal credentials the moment you install it, and until recently, GitHub could only flag those in npm.

Not anymore.

This is the story of how the supply chain engineering team behind Dependabot expanded malware advisories to eight ecosystems by building on OpenSSF’s shared malicious packages data.

Here’s where things stand: earlier this year, Dependabot started flagging malware in your npm dependencies.

Great news if you write JavaScript.

Now we’re bringing that same functionality to PyPI.

We’ve enhanced the GitHub Advisory Database to ingest malware reports from OpenSSF’s malicious-packages repository, which means malware advisories and the Dependabot alerts they power cover all eight major package ecosystems: npm, PyPI, Maven, RubyGems, NuGet, Go, crates.io, and PHP Composer.

I lead the Dependabot team in GitHub’s supply chain security organization, and in this post, I’ll show you how this pipeline works.

From one ecosystem to eight The Advisory Database has imported vulnerability data from external sources for years.

RubySec for gems, RustSec for crates, PyPA for Python.

Each one is an importer that reads a public advisory repo and maps records into our database.

Malware was the odd one out: it flowed through a separate, internal, npm-only path, built around GitHub’s own detection of malicious npm packages.

Expanding the existing detection from one to eight supported ecosystems would have taken us years.

Meanwhile, OpenSSF has already solved the aggregation problem for everybody.

Their malicious-packages repo launched in 2023, with over 15,000 reports in OSV format.

Since then, it has grown every day, fed by community submissions and automated detection sources across the industry: typosquats, dependency-confusion packages, account takeovers, malicious prebuilt binaries.

It’s public, it’s structured, and it covers any ecosystem the OSV schema supports.

So, the design nearly wrote itself.

Rather than building eight unique detection systems, we built one importer.

The importer We reused the same pattern our repo-based importers already followed to walk the source repository’s file tree, pick up files changed since the last run, and process each one.

The new OpenSSF importer reads every OSV record and validates the required fields, types, and format against the schema before anything touches the database.

A record that fails this validation gets rejected and logged.

It’s never quietly patched up and waved through, because a “mostly valid” malware advisory is exactly the kind of thing that bites you six months later.

Valid records get normalized into feed entries: the source, an identifier, a CVE ID when one exists, the complete upstream record preserved as a snapshot, and the mapped subset from our publishing pipeline consumes.

Normalizing sounds boring until you meet the data.

Upstream ecosystem strings don’t always match ours (the repo says , our database says ).

OSV records list affected versions as discrete values where we think in ranges, and some records name no usable version at all.

The details field is frequently empty, and when several sources report the same package, their write-ups get appended into one blob.

Reports also get retracted: the repo keeps a whole folder for advisories that turned out to be wrong, so the importer must cope with a package being flagged on Monday and disavowed on Wednesday.

Then there’s the dedup problem, and it’s a fun one.

GitHub is itself a contributor to the OpenSSF repo; our own npm malware advisories flow upstream into it.

Import the repo naively, and we’d be re-importing our own data in a loop.

The fix rides on OSV’s origin metadata: every entry in malicious-packages records where the report came from, and anything tagged began with us.

The importer drops those before a feed entry is ever created.

When we validated against live data, more than ha

分享