Shell scripting with Node.js
You can buy the offline version of this book (HTML, PDF, EPUB, MOBI) and support the free online version.
(Ad, please don’t block.)

13 Installing npm packages and running bin scripts



The package.json property "bin" lets an npm package specify which shell scripts it provides (for more information, see §14 “Creating cross-platform shell scripts”). If we install such a package, Node.js ensures that we can access these shell scripts (so-called bin scripts) from a command line. In this chapter, we explore two ways of installing packages with bin scripts:

We explore what all of that means and how we can run bin scripts after installing them.

13.1 Installing npm registry packages globally

Package cowsay has the following package.json property:

"bin": {
  "cowsay": "./cli.js",
  "cowthink": "./cli.js"
},

To install this package globally, we use npm install -g:

npm install -g cowsay

Caveat: On Unix, we may have to use sudo (we’ll learn soon how to avoid that):

sudo npm install -g cowsay

After that, we can use the commands cowsay and cowthink in our command lines.

Note that only the bin scripts are available globally. The packages are ignored when Node.js looks up bare module specifiers in node_modules directories.

13.1.1 Which packages are installed globally? npm ls -g

We can check which packages are installed globally and where:

% npm ls -g
/usr/local/lib
├── corepack@0.12.1
├── cowsay@1.5.0
└── npm@8.15.0

On Windows, the installation path is %AppData%\npm, e.g.:

>echo %AppData%\npm
C:\Users\jane\AppData\Roaming\npm

13.1.2 Where are packages installed globally? npm root -g

Result on macOS:

% npm root -g
/usr/local/lib/node_modules

Result on Windows:

>npm root -g
C:\Users\jane\AppData\Roaming\npm\node_modules

13.1.3 Where are shell scripts installed globally? npm bin -g

npm bin -g tells us where npm installs shell scripts globally. It also ensures that that directory is available in the shell PATH.

Result on macOS:

% npm bin -g
/usr/local/bin

% which cowsay
/usr/local/bin/cowsay

Result on the Windows Command shell:

>npm bin -g
C:\Users\jane\AppData\Roaming\npm

>where cowsay
C:\Users\jane\AppData\Roaming\npm\cowsay
C:\Users\jane\AppData\Roaming\npm\cowsay.cmd

The executable cowsay without a filename extension is for Unix-based Windows environments such as Cygwin, MinGW, and MSYS.

Windows PowerShell returns this path for gcm cowsay:

C:\Users\jane\AppData\Roaming\npm\cowsay.ps1

13.1.4 Where are packages installed globally? The npm installation prefix

npm’s installation prefix determines where packages and bin scripts are installed globally.

This is the installation prefix on macOS:

% npm config get prefix
/usr/local

Accordingly:

This is the installation prefix on Windows:

>npm config get prefix
C:\Users\jane\AppData\Roaming\npm

Accordingly:

13.1.5 Changing where packages are installed globally

In this section, we examine two ways of changing where packages are installed globally:

13.1.5.1 Changing the npm installation prefix

One way of changing where packages are installed globally is to change the npm installation prefix.

Unix:

mkdir ~/npm-global
npm config set prefix '~/npm-global'

Windows Command shell:

mkdir "%UserProfile%\npm-global"
npm config set prefix "%UserProfile%\npm-global"

Windows PowerShell:

mkdir "$env:UserProfile\npm-global"
npm config set prefix "$env:UserProfile\npm-global"

The configuration data is saved to a file .npmrc in the home directory.

From now on, global installs will be added to the directory we have just specified.

Afterward, we still have to add the npm bin -g directory to our shell PATH so that our shell finds bin scripts we install globally.

A downside of changing the npm prefix: npm will now also be installed at the new location if we tell it to upgrade itself.

13.1.5.2 Using a Node.js version manager

Node.js version managers let us install multiple versions of Node.js at the same time and switch between them. Popular ones include:

13.2 Installing npm registry packages locally

To install an npm registry package such as cowsay locally (into a package), we do the following:

cd my-package/
npm install cowsay

This adds the following data to package.json:

"dependencies": {
  "cowsay": "^1.5.0",
  ···
}

Additionally, the package is downloaded into the following directory:

my-package/node_modules/cowsay/

On Unix, npm adds these symbolic links for the bin scripts:

my-package/node_modules/.bin/cowsay -> ../cowsay/cli.js
my-package/node_modules/.bin/cowthink -> ../cowsay/cli.js

On Windows, npm adds these files to my-package\node_modules\.bin\:

cowsay
cowsay.cmd
cowsay.ps1
cowthink
cowthink.cmd
cowthink.ps1

The files without extensions are scripts for Unix-based Windows environments such as Cygwin, MinGW, and MSYS.

npm bin tells us where locally installed bin scripts are located – for example:

% npm bin
/Users/john/my-package/node_modules/.bin

Note: Locally, packages are always installed in a directory node_modules next to a package.json file. If the latter doesn’t exist in the current directory, npm searches for it in an ancestor directory and installs the package there. To check where npm would install packages locally, we can use the command npm root – for example (Unix):

% cd $HOME
% npm root
/Users/john/node_modules

There is no package.json in John’s home directory, but npm can’t install anything in an ancestor directory, which is why npm root shows this directory. Installing a package locally at the current location will lead to package.json being created and installation progressing as usual.

13.2.1 Running locally installed bin scripts

(All commands in this subsection are executed inside directory my-package.)

13.2.1.1 Running bin scripts directly

We can run cowsay as follows from a shell:

./node_modules/.bin/cowsay Hello

On Unix, we can set up a helper:

alias npm-exec='PATH=$(npm bin):$PATH'

Then the following command works:

npm-exec cowsay Hello
13.2.1.2 Running bin scripts via package scripts

We can also add a package script to package.json:

{
  ···
  "scripts": {
    "cowsay": "cowsay"
  },
  ···
}

Now we can execute this command in a shell:

npm run cowsay Hello

That works because npm temporarily adds the following entries to $PATH on Unix:

/Users/john/my-package/node_modules/.bin
/Users/john/node_modules/.bin
/Users/node_modules/.bin
/node_modules/.bin

On Windows, similar entries are added to %Path% or $env:Path:

C:\Users\jane\my-package\node_modules\.bin
C:\Users\jane\node_modules\.bin
C:\Users\node_modules\.bin
C:\node_modules\.bin

The following command lists the environment variables and their values that exist while a package script runs:

npm run env
13.2.1.3 Running bin scripts via npx

Inside a package, npx can be used to access bin scripts:

npx cowsay Hello
npx cowthink Hello

More on npx later.

13.3 Installing unpublished packages

Sometimes, we have a package that we either haven’t published yet or won’t ever publish and would like to install it.

Let’s assume we have an unpublished package whose name is @my-scope/unpublished-package that is stored in a directory /tmp/unpublished-package/. We can make it available globally as follows:

cd /tmp/unpublished-package/
npm link

If we do that:

Due to how the linked package is referred to, any changes in it will take effect immediately. There is no need to re-link it when it changes.

To check if the global installation worked, we can use npm ls -g to list all globally installed packages.

After we have installed our upublished package globally (see previous subsection), we have the option to install it locally in one of our packages (which can be published or unpublished):

cd /tmp/other-package/
npm link @my-scope/unpublished-package

That creates the following link:

/tmp/other-package/node_modules/@my-scope/unpublished-package
-> ../../../unpublished-package

By default, the unpublished package is not added as a dependency to package.json. The rationale behind that is that npm link is often used to temporarily work with an unpublished version of a registry package – which shouldn’t show up in the dependencies.

Undoing the local link:

cd /tmp/other-package/
npm uninstall @my-scope/unpublished-package

Undoing the global link:

cd /tmp/unpublished-package/
npm uninstall -g

13.3.4 Installing unpublished packages via local paths

Another way of installing an unpublished package locally, is to use npm install and refer to it via a local path (and not via its package name):

cd /tmp/other-package/
npm install ../unpublished-package

That has two effects.

First, the following symbolic link is created:

/tmp/other-package/node_modules/@my-scope/unpublished-package
-> ../../../unpublished-package

Second, a dependency is added to package.json:

"dependencies": {
  "@my-scope/unpublished-package": "file:../unpublished-package",
  ···
}

This way of installing unpublished packages also works globally:

cd /tmp/unpublished-package/
npm install -g .

13.3.5 Other ways of installing unpublished packages

13.4 npx: running bin scripts in npm packages without installing them

npx is a shell command for running bin scripts that is bundled with npm.

Its most common usage is:

npx <package-name> arg1 arg2 ...

This command installs the package whose name is package-name in the npx cache and runs the bin script that has the same name as the package – for example:

npx cowsay Hello

That means we can run bin scripts without installing them first. npx is most useful for one-off invocations of bin scripts – for example, many frameworks provide bin scripts for setting up new projects and these are often run via npx.

After npx has used a package for the first time, it is available in its cache and subsequent invocations are much faster. However, we can’t be sure how long a package stays in the cache. Therefore, npx isn’t a substitute for installing bin scripts globally or locally.

If a package comes with bin scripts whose names are different from its package name, we can access them like this:

npx --package=<package-name> <bin-script> arg1 arg2 ...

For example:

npx --package=cowsay cowthink Hello

13.4.1 The npx cache

Where is npx’s cache located?

On Unix, we can find that out via the following command:

npx --package=cowsay node -p \
  "process.env.PATH.split(':').find(p => p.includes('_npx'))"

That returns a path similar to this one:

/Users/john/.npm/_npx/8f497369b2d6166e/node_modules/.bin

On Windows, we can use (one line broken up into two):

npx --package=cowsay node -p
  "process.env.Path.split(';').find(p => p.includes('_npx'))"

That returns a path similar to this one (single path broken up into two lines):

C:\Users\jane\AppData\Local\npm-cache\_npx\
  8f497369b2d6166e\node_modules\.bin

Note that npx’s cache is different from the cache that npm uses for the modules it installs:

The parent directory of both caches can be determined via:

npm config get cache

For more information on the npm cache, see the npm documentation.

In contrast to the npx cache, data is never removed from the npm cache, only added. We can check its size as follows on Unix:

du -sh $(npm config get cache)/_cacache/

And on Windows PowerShell:

DiskUsage /d:0 "$(npm config get cache)\_cacache"