An Introduction to npm

npm is the de facto package manager for Node.js, roughly analagous to pip for Python or Maven for Java. There are several alternatives, like Yarn and Turbo, but npm is automatically installed when you install Node.js. The npm registry is huge, with over 800k packages at the time of this writing, so there's a package for almost everything. In this article, I'll provide a brief overview of what npm can do for you.

Side Note: npm has historically been very insistent that npm does not stand for "node package manager."

Installing Dependencies

Usually, the first thing you should do when you clone a Node.js repo is run npm install. When you run npm install, npm loads the package.json file from your current directory, and installs all dependencies listed in package.json. For example, if your package.json looks like the below, npm install will install version 5.2.10 of Mongoose and version 4.16.4 of Express.

{
  "dependencies": {
    "mongoose": "5.2.10",
    "express": "4.16.4"
  }
}

One key difference between npm and its Python analog pip is that npm installs every dependency into the ./node_modules directory, as opposed to one central directory. This means:

• rm -rf ./node_modules is sufficient to uninstall every package a given repo depends on. When a Node.js developer says to "clear your node modules", they mean run rm -rf ./node_modules && npm install.
• Every repo has a separate copy of every dependency. If you clone two repos that depend on express@4.16.4, you will have two copies of the same version of Express.
• Node.js doesn't have an equivalent to Python's virtualenv because having a separate ./node_modules for every repo effectively solves the same problem that virtualenv solves.

You can install npm packages globally using the -g flag. For example, you can run npm install mongoose -g. However, installing with -g is a bad idea for most packages.

In addition to dependencies, there are 3 other properties that you can use to define dependencies:

• devDependencies: These dependencies are not necessary to run this project in production. For example, testing frameworks like Mocha are typically in devDependencies. Running npm install will install these dependencies, but running npm install --production will skip them.
• peerDependencies: Only useful if you are building a library that other Node.js projects will npm install. Projects that install this library should explicitly list these peerDependencies in their dependencies or devDependencies. npm install will warn if someone attempts to install this package without the corresponding peerDependencies.
• optionalDependencies: Normally, npm install will fail if it can't find the package you're trying to install. However, npm install will still succeed if it can't find a dependency that's listed in optionalDependencies.

Here's some key takeaways from this section:

• npm installs packages into ./node_modules. To re-install, run rm -rf ./node_modules && npm install.
• You should list the libraries and frameworks that your production code uses in dependencies. You should list testing frameworks, build systems, documentation generators, and other ancillary tools in devDependencies.
• When building your app for production, like in a Dockerfile, you should run npm install --production to skip devDependencies.

Should You Commit package-lock.json?

When you run npm install for the first time in an empty repo, you may see the below warning from npm:

npm notice created a lockfile as package-lock.json. You should commit this file.

The package-lock.json file, otherwise known as the "lockfile", pins exact dependencies of your nested dependencies. For example, suppose you depend on version 1.1.0 of a package called 'foo':

{
  "foo": "1.1.0"
}

Suppose 'foo' depends on another package 'bar', but specifies that any version of 'bar' is ok.

{
  "bar": "*"
}

Without a lockfile, you would get the latest version of 'bar' every time you npm install. That means if the maintainer of 'bar' published a release that broke 'foo', they would break your code as well. Breaking upstream releases have happened.

The lockfile specifies the exact version of 'bar'. If you got bar@1.2.3 when you ran npm install for the first time and committed the lockfile, you'll get bar@1.2.3 every time you npm install.

Lockfiles sound pretty useful, so why do people want to disable it? The most common reason is that merge conflicts in package-lock.json are a massive pain point. Most developers simply delete their lockfile whenever git reports a merge conflict.

Another issue is that package-lock.json cannot be published. In other words, if you're building a library that other projects will npm install, your package-lock.json will be ignored. The top-level application's package-lock.json is the source of truth.

So should you commit your package-lock.json? Most projects should commit their lockfile. If you're developing a library that other projects will install, you should not commit package-lock.json. Also, if you find yourself deleting your lockfile regularly to clear merge conflicts, you should consider adding package-lock.json to your .gitignore.

Running Scripts

In addition to installing dependencies, npm is also used for running scripts. The scripts property in package.json lets you define custom scripts that you can then run using npm run.

For example, suppose you define a script called "greet" that echos "Hello" in your package.json.

{
  "scripts": {
    "greet": "echo 'Hello'"
  }
}

You can run this script with npm run greet:

$ npm run greet

> @ greet /home/test
> echo 'Hello'

Hello
$ 

The "test" script is special. If you define a script named "test", you can run it using npm test in addition to npm run test. Making npm test run your entire test suite is a best practice. For example, running npm test with the below package.json will print "Testing..."

{
  "scripts": {
    "test": "echo 'Testing...'"
  }
}

The biggest reason to use npm run is the ability to access npm package executables. For example, the Mocha testing framework exports a test runner executable. Suppose you have the below package.json:

{ 
  "devDependencies": {
    "mocha": "5.x"
  }
}

Suppose you also have the below test.js file that contains a single Mocha test:

it('test', function() {
  console.log('Hello, world');
});

To run this test file, you would need to run ./node_modules/.bin/mocha test.js:

$ ./node_modules/.bin/mocha test.js 


Hello, world
  ✓ test

  1 passing (4ms)

$

However, in your package.json scripts, you can skip the ./node_modules/.bin because npm adds that to your PATH variable before running the script. In other words, with the below package.json, the npm test command will successfully run Mocha.

{
  "scripts": {
    "test": "mocha test.js"
  },
  "devDependencies": {
    "mocha": "5.x"
  }
}

Passing Arguments to npm run

You can pass command line arguments to your npm run scripts by adding -- after npm run script-name-here. For example, the below command sends an HTTP request to httpbin.org using curl.

curl -X PUT http://httpbin.org/put -d '{"hello":"world"}'

Below is the output of this script:

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {
    "{\"hello\":\"world\"}": ""
  }, 
  "headers": {
    "Accept": "*/*", 
    "Content-Length": "17", 
    "Content-Type": "application/x-www-form-urlencoded", 
    "Host": "httpbin.org", 
    "User-Agent": "curl/7.47.0"
  }, 
  "json": null, 
  "origin": "138.207.148.170, 138.207.148.170", 
  "url": "https://httpbin.org/put"
}

Suppose you have an npm script that executes a PUT request, but doesn't specify the -d flag.

{
  "scripts": {
    "test-request": "curl -X PUT http://httpbin.org/put"
  }
}

To add the -d flag, run the below script:

npm run test-request -- -d '{"hello":"world"}'

This is just the tip of the iceberg with npm run. There's a lot of other neat npm run tricks that you can use to save yourself some typing.

Moving On

Whenever you start working on a Node.js project, the first step is running npm install, and the second step is running npm test. npm is different from other languages' package managers because it automatically scopes dependencies to a single project, rather than installing globally to a common 'Library' directory. This decision leads to more hard drive space usage, but less developer headache. Finally, npm run is the canonical way to run npm package executables. Make sure you take advantage of npm's features, npm does much more than just install every module in dependencies.