- Published on
The massive bug at the heart of the npm ecosystem
- Authors
- Name
- Darcy Clarke
- X (Formerly Twitter)@darcy
Disclosure: I was the Staff Engineering Manager for the npm CLI team between July 2019 & December 2022. I was a part of the GitHub acquistion of npm inc. in 2020. I left GitHub, for various reasons, in December.
tldr;
- a npm package's manifest is published independently from its tarball
- manifests are never fully validated against the tarball's contents
- the ecosystem has broadly assumed the contents of the manifest & tarball are consistant
- any tools or insights using the public registry are succeptible to exploitation/likely inaccurate
- bad actors can hide malware & scripts in direct or transitive dependencies that go undetected
In terms of novel supply chain attacks go, this is a biggy & from here on out I'll be referring to this as "manifest confusion".
History
Before the node ecosystem became what it is today - aka. tens of millions of developers around the world creating over ~3.1 million packages being downloaded 208 billion times a month - the number of people contributing to the corpus of software you trusted to use & download was very small. With a smaller community you have more trust & even as the npm registry was being developed most aspects were open source & freely available to be contributed to & code inspected. But, over time, as the ecosystem grew up, so did the policies & practices of organizations consuming from the corpus.
From the outset, the npm project also put a lot of trust in the client vs. server-side of the registry. Looking back now, its clear that the practice of relying so heavily on a client to handle validation of data is riddle with issues but that strategy also allowed for the JavaScript tooling ecosystem to organically grow & participate in the shape of the data.
What's wrong?
The npm Public Registry does not validate manifest information with the contents of the package tarball, relying instead on npm-compatible clients to interpret & enforce validation/consistency. In fact, as I researched this issue it looks like the server has never done this validation (so you may want to call this a "feature").
Today, registry.npmjs.com
lets users publish packages via a PUT
request to the corresponding package URI (ex. https://registry.npmjs.com/-/<package-name>
). This endpoint accepts a request body
which looks something like this (note: after almost a decade & a half, this & all other registry APIs continue to be horribly undocumented):
{
_id: <pkg>,
name: <pkg>,
'dist-tags': { ... },
versions: {
'<version>': {
_id: '<pkg>@<version>`,
name: '<pkg>',
version: '<version>',
dist: {
integrity: '<tarball-sha512-hash>',
shasum: '<tarball-sha1-hash>',
tarball: ''
}
...
}
},
_attachments: {
0: {
content_type: 'application/octet-stream',
data: '<tarball-base64-string>',
length: '<tarball-length>'
}
}
}
The issue at hand is that the version
metadata (aka. "manifest" data) is submitted independent from the attached tarball which houses the package's package.json
. These two pieces of information are never validated against one another & calls into question which one should be *the canonical source of truth* for data such as dependencies
, scripts
, license
& more. As far as I can tell, the tarball is the only artifact that gets signed & has an integrity value that can be stored & verified offline (making the case for it to potentially be the proper source; yet, very surprisngly, the name
& version
fields in package.json
can actually differ from those in the manifest, because they were never validated).
Example
- Generate an auth token on npmjs.com (ex.
https://www.npmjs.com/settings/<your-username>/tokens/new
- choose "Automation" for ease) - Start a new project (ex.
mkdir test && cd test/ && npm init -y
) - Install helper libs (ex.
npm install ssri libnpmpack npm-registry-fetch
) - Create a sub directory which will act as the "real" package & contents (ex.
mkdir pkg && cd pkg/ && npm init -y
) - Modify the contents of that package...
- Create a
publish.js
file in the project root with something like the following:
;(async () => {
// libs
const ssri = require('ssri')
const pack = require('libnpmpack')
const fetch = require('npm-registry-fetch')
// pack tarball & generate ingetrity
const tarball = await pack('./pkg/')
const integrity = ssri.fromData(tarball, {
algorithms: [...new Set(['sha1', 'sha512'])],
})
// craft manifest
const name = '<pkg name>'
const version = '<pkg version>'
const manifest = {
_id: name,
name: name,
'dist-tags': {
latest: version,
},
versions: {
[version]: {
_id: `${name}@${version}`,
name,
version,
dist: {
integrity: integrity.sha512[0].toString(),
shasum: integrity.sha1[0].hexDigest(),
tarball: '',
},
scripts: {},
dependencies: {},
},
},
_attachments: {
0: {
content_type: 'application/octet-stream',
data: tarball.toString('base64'),
length: tarball.length,
},
},
}
// publish via PUT
fetch(name, {
'//registry.npmjs.org/:_authToken': '<auth token>',
method: 'PUT',
body: manifest,
})
})()
- Modify the
manifest
keys as you wish (ex. I've stripped thescripts
&dependencies
in the above) - Run program (ex.
node publish.js
) - Navigate to
https://registry.npmjs.com/<pkg>/
&https://www.npmjs.com/package/<pkg>/v/<version>?activeTab=explore
to see the discrepancies
In the above example, the package was published with a different manifest then it's corresponding package.json
(ref. https://www.npmjs.com/darcyclarke-manifest-pkg & https://registry.npmjs.com/darcyclarke-manifest-pkg/).
Bugs, bugs, bugs
If you want an even easier way to reproduce this inconsistency you can use the npm
CLI today, as it actually mutates the manifest during npm publish
when it sees a binding.gyp
file in your project. This is a behaviour that seems to have existed in the client since before my time on the team (ie. <6.x
or earlier) & is the cause of many bugs/confusion by consumers.
npm init -y
touch binding.gyp
npm publish
- View that a
"node-gyp rebuild"
scripts.install
entry was automatically added to the manifest but not the actual tarball'spackage.json
(ex. https://registry.npmjs.com/darcyclarke-binding & https://unpkg.com/[email protected]/package.json)
A real-world example/victim of this inconsistency is node-canvas
:
- https://www.npmjs.com/package/node-canvas/v/2.9.0?activeTab=explore
- https://registry.npmjs.com/node-canvas/2.9.0
- https://github.com/npm/cli/issues/5234
Impact
There are several ways this bug actually impacts consumers/end-users:
- cache poisoning (ie. the package that is saved may not match the name+version spec of that in the registry/URI)
- installation of unknown/unlisted dependencies (tricking security/audit tools)
- execution of unknown/unlisted scripts (tricking security/audit tools)
- potential downgrade attack (where the version specification saved into projects is for a unspecified, vulnerable version of the package)
Known, Third-Party Organizations/Entities Affected
- Snyk: https://security.snyk.io/package/npm/darcyclarke-manifest-pkg
- CNPMJS/Chinese Mirror: https://npmmirror.com/package/darcyclarke-manifest-pkg
- Cloudflare Mirror: https://registry.npmjs.cf/darcyclarke-manifest-pkg/2.1.15
- Skypack: https://cdn.skypack.dev/-/[email protected]
- UNPKG: https://unpkg.com/[email protected]/package.json
- JSPM: https://ga.jspm.io/npm:[email protected]/package.json
- Yarn: https://yarnpkg.com/package/darcyclarke-manifest-pkg
Update: It was previously stated that Socket Security was succceptable to the manifest confusion issue. Since September 5, 2022 Socket has used thepackage.json
file inside the tarball as the source of truth & should show accurate information for packages (ex. dependencies, licenses, scripts). When this blog was posted, the package page fordarcyclarke0-manifest-pkg
was incorrectly using an outdated data reference & was quickly resolved by the team at Socket. Notably, the team at Socket is likely the first in this space to properly handle this problem.
This issue also effects all known, major JavaScript package managers in various ways detailed below. Third-party registry implementations like jFrog's Artifacory seem to also have replicated this API-design/issue, meaning that all clients of those private registry instances will notice the same issue/inconsistency.
Notably, the various package managers & tooling have different scenarios in which they will use/reference either the package's registry manifest or tarball's package.json
(almost always, as a mechanism to cache & increase performance of installations).
The key point to make here is that the ecosystem is currently under the incorrect assumption that the manifest always contains the contents of the tarball's package.json
(this is in large part because of the significant lack of registry API documentation as well as various references in docs.npmjs.com to the fact that the registry stores the contents of package.json
as the metadata - & no where does it mention that the client is responsible for ensuring consistency).
npm@6
Executes install scripts not present in manifest & vice-versa
Steps to reproduce:
- Install a malformed dependency:
npx npm@6 install [email protected]
- See that lifecycle scripts are being executed even though none are present in the manifest & the registry has not registered the package as having install script (ie.
hasInstallScript
isundefined
/false
) (ref. https://registry.npmjs.org/darcyclarke-manifest-pkg/2.1.13 - code/package ref. https://github.com/npm/minify-registry-metadata/blob/main/lib/index.js) - The
package.json
innode_modules/darcyclarke-manifest-pkg
reflects the tarball entry
Installs dependencies not present in manifest & vice-versa
Because the package tarball gets cached in a global store, if the --prefer-offline
config is used alongside --no-package-lock
, the next time an install
is run of that same package across the system, its dependencies that are hidden in the tarball may be installed.
Steps to reproduce:
- Install
npx npm@6 install [email protected]
- Run install again somewhere...
npx npm@6 install --prefer-offline --no-package-lock
npm@9
Installs dependencies not present in manifest & vice-versa
Similar to npm@6
, npm@9
will happily install the dependencies referenced inside of a package's cached tarball package.json
when using the --offline
config.
Note: there seems to be a race condition where
--offline
may or may not pull from cache resulting in intermittant results
Steps to reproduce:
- Install malformed dependency so that it is cached
- Run installation with
--offline
configuration &/or by turning off network availability (ex.npm install --offline --no-package-lock
) - See that dependencies not referenced in the manifest will be installed
yarn@1
Executes install scripts not present in manifest & vice-versa
Like npm@6
& npm@9
, yarn@1
will run scripts that are inside the tarball but that aren't referenced in the manifest & vice-versa.
version
found in the tarball - exposing a potential downgrade attack vector
Uses the As known by now, a tarball can have a different version
defined then the manifest; in this case, yarn@1
will happily upgrade/downgrade & save back to the consuming project's package.json
the incorrect version (potentially exposing consumers to a downgrade attack on subsequent installations)
pnpm@7
Executes install scripts not present in manifest & vice-versa
Steps to reproduce:
Like all the others, pnpm
will run scripts that are inside the tarball but that aren't referenced in the manifest & vice-versa.
CWE Categorization/Breakdown
There are potentially various CWE categorizations for this vulnerability. At the very least, if this issue might ever be considered a "feature", then what we see here must be considered "Client-Side Enforcement of Server-Side Security" (ie. CWE-602
) - but I doubt that's the minimum scope applicable. I've broken down the various issues along with their corresponding CWE categorization below (code references have been provided in each case).
- CWE-602: Client-Side Enforcement of Server-Side Security
- there is a history of relying heavily on the client (aka. the
npm
CLI) to do work that should be done server-side; this is a perfect example - code ref. https://github.com/npm/cli/blob/latest/workspaces/libnpmpublish/lib/publish.js#L63
- there is a history of relying heavily on the client (aka. the
- CWE-94: Improper Control of Generation of Code ('Code Injection')
- this is relevant for any/all consumers (including package managers such as
npm
); as noted below, they all have various issues because of this
- this is relevant for any/all consumers (including package managers such as
- CWE-295: Improper Certificate Generation
- tarballs are signed & given an integrity value even though their contents (including
name
,version
,dependencies
,license
,scripts
etc.) differ from the registry index their associated with
- tarballs are signed & given an integrity value even though their contents (including
- CWE-325: Missing Cryptographic Step
- manifest data is not signed & therefor cannot be cached or verified offline
- missing hash/validation of the data subset of keys that overlap a tarball's
package.json
& the package manifest
- CWE-656: Reliance on Security Through Obscurity
- with a complete lack of documentation surrounding the registry APIs, this issue was not easily discernible
What is GitHub doing about this?
To my knowledge, GitHub was first made aware of this issue on, or around, November 4th, 2022; after doing independent research, I believed the potential impact/risk of this issue was actually far greater then originally understood & I submitted a HackerOne report with my findings on March 9. GitHub closed that ticket & said they were dealing with the issue "internally" on March 21st. To my knowledge, they have not made any significant headway, nor have they made this issue public - instead, they've actually divested their position in npm as a product the last 6 months & refused to follow-up or provide insight into any remediation work.
What would a solution look like?
GitHub is understandably in a tough spot. The fact that npmjs.com
has functioned this way for over a decade means that the current state is pretty much codified & likely to break someone in a unique way. As mentioned before, the npm
CLI itself relies on this behaivour & there's potentially other non-nefarious uses of this in the wild today.
- What should be done...
- there's further investigation that should be done to determine the scope of affected entries in the registry which would help determine abuse
- if the number of discrepencies is minimal (which is doubtful given how prevelant the in-flight manifest mutation seems to be) then I imagine it would make sense to regenerate the manifests with discrepencies based on the tarball's
package.json
- Beginning to enforce/validate the privileged/known keys in the manifest can happen asynchronous to any research/discovery
- The npm Public Registry APIs & their respective request/response objects need to be documented as soon as humanly possible
What can you do?
Contact any known tooling author/maintainer who you know relies on the npm registries manifest data & ensure they start using the package's contents for metadata when appropriate (ie. everything *but* name
& version
). Start using a registry proxy which strictly enforces/validates for consistency.