In this tutorial we'll create an npm package that is accessible from the command line. We'll touch issues such as parsing the arguments, handling configuration, flexible logging and overall Developer Experience.

You probably use many tools in your everyday work, such as: `webpack`, `typescript`, `babel`, `tslint`, `eslint`, `prettier` and more. You can access all these tools from the command line and we're going to see precisely how.

_Click on the next button or use the `down` key._

_Tap on the right edge of the screen to continue._

We'll talk about different aspects of such a tool: creating the executable, parsing command line arguments, handling configuration, flexible logging, and overall Developer Experience.

At the end of this tutorial, we will have a working skeleton of a CLI tool. All that will be left is to add your specific logic.

_Want to skim the tutorial or maybe deep dive into it? Control the verbosity of the tutorial by toggling `Verbose` and `Internals` at the bottom left of the screen._

_Want to skim the tutorial or maybe deep dive into it? Control the verbosity of the tutorial by toggling `Verbose` and `Internals` in the menu at the top right of the screen._

Verbose content will be decorated in blue.

Internal content will be decorated in red.

Introduction

$ mkdir tool-tutorial
$ cd tool-tutorial
tool-tutorial$ mkdir tool
tool-tutorial$ mkdir testProject

Create two directories, one named `tool` and the other one `testProject`.

tool-tutorial$ cd tool
tool-tutorial/tool$ npm init -y
// ...
tool-tutorial/tool$ cd ../testProject
tool-tutorial/testProject$ npm init -y
// ...

Run `npm init -y` in both directories to create empty package.json files.

In the `tool` project, create a directory called `bin` and a file called `index.js`.

We create a directory only for convenience. Our binary script can be placed anywhere in the project.

In the `index.js` file, we'll print `hello tool` for now.

To execute a node script, we need to run `node SCRIPT_NAME`. In our case, we want the ability to run the tool without explicitly calling `node`, the same way we execute `webpack`, `tsc` or `eslint`.

#!/usr/bin/env node
console.log('hello tool');

We place a `hashbang` directive at the top of our file stating which interpreter will be used to execute our script.

We use the `env` command instead of a direct path to the node executable since we might have multiple node versions installed.

Setting Up the Project(s)

tool

In the tool's `package.json` file, we need to define the entry point script we just created as the package's binary.

We create a `bin` property and set its value to the file it will run, the entry point script we created earlier. The name of our package, `tool`, will be the name of the executable that will be created.

To create multiple binaries for the package, define `bin` as an object with multiple properties.

Configuring the Binary

tool-tutorial$ cd tool
tool-tutorial/tool$ npm link
// ...
tool-tutorial/tool$ cd ../testProject
tool-tutorial/testProject$ npm link tool
// ...

We want to test our tool in the test project we created earlier. For that, we'll need to install our `tool` package in the `testProject` directory.

We'll use `npm link` to simulate an installation.

tool-tutorial/testProject$ ll node_modules/.bin
lrwxr-xr-x 1 user 361602828 23 Nov 10 11:06 tool -> ../tool/bin/index.js

If we look in the `node_modules/.bin` directory of the `testProject`, we can see that the installation created a new file called `tool`, which points, using a symlink, to the entry point script we created earlier.

tool-tutorial/testProject$ tool
hello tool

If we run `tool`, we can see that it is executed properly.

Linking the Projects

We want our tool to be able to run multiple commands. For that, we need to be able to run the tool with various arguments.

#!/usr/bin/env node
console.log(process.argv);

We can use node's built-in `process.argv` property to handle them. We'll replace our `console.log` with a new one.

tool-tutorial/testProject$ tool --start
[
  '/Users/user/.nvm/versions/node/v12.16.1/bin/node',
  '/Users/user/tool-tutorial/tool/bin/index.js',
  '--start'
]

If we now run our tool, we'll see a similar output. `process.argv` returns an array, where the first member of the array is the node executable, the second is the name of the executed file, and the third and onward are the arguments we passed the script.

We can use node's `process.argv` property, but there are multiple libraries that exist to make our life easier with CLI arguments. One of the simplest libraries in this area is <a href="https://github.com/vercel/arg" target="_blank">arg</a>.

More advanced libraries are <a href="https://github.com/tj/commander.js" target="_blank">commander</a> and <a href="https://yargs.js.org/" target="_blank">yargs</a>.

#!/usr/bin/env node
const arg = require('arg');

const args = arg({
  '--start': Boolean,
  '--build': Boolean,
});

console.log(args);

`arg` receives a configuration object which states which arguments it can receive and their type. Types can be booleans, strings, numbers, arrays, and counters. In our case, we'll use a boolean because we want to know whether the user passed the flag or not.

tool-tutorial/testProject$ tool --start
{ _: [], '--start': true }

Now, if we run our tool, we will receive the following output. Notice that the `--start` flag was detected and set to `true`.

#!/usr/bin/env node
const arg = require('arg');

const args = arg({
  '--start': Boolean,
  '--build': Boolean,
});

if (args['--start']) {
  console.log('starting the app');
}


We can now handle our `start` argument accordingly.

tool-tutorial/testProject$ tool --start
starting the app

And if we rerun it, we'll see that it prints our message.

tool-tutorial/testProject$ tool --publish
/Users/user/tool-tutorial/tool/node_modules/arg/index.js:90
            throw err;
            ^

Error: Unknown or unexpected option: --publish
//...

But what happens if we pass it an unknown argument? We can see that `arg` throws an error.

#!/usr/bin/env node
const arg = require('arg');

try {
  const args = arg({
    '--start': Boolean,
    '--build': Boolean,
  });

  if (args['--start']) {
    console.log('starting the app');
  }
} catch (e) {
  console.log(e.message);
}


We can wrap our call to `arg` with `try/catch` and handle it accordingly.

tool-tutorial/testProject$ tool --publish
Unknown or unexpected option: --publish

The new output will be a bit less messy, but still not good enough.

#!/usr/bin/env node
const arg = require('arg');

try {
  const args = arg({
    '--start': Boolean,
    '--build': Boolean,
  });

  if (args['--start']) {
    console.log('starting the app');
  }
} catch (e) {
  console.log(e.message);
  console.log();
  usage();
}

function usage() {
  console.log(`tool [CMD]
  --start\tStarts the app
  --build\tBuilds the app`);
}


Other than showing how _not_ to use the tool, we need to show how one _can_ use it. We'll add a `usage` function for that.

tool-tutorial/testProject$ tool --publish
Unknown or unexpected option: --publish

tool [CMD]
  --start		Starts the app
  --build		Builds the app

Now, if we run the command, we'll see a much clearer output.

Parsing Command Line Arguments

Such as UX is a loading-bearing column in visual applications, Developer Experience, or DX for short, is one when creating tools for developers.

DX has many manifestations such as delightfulness when using the tool, better error messages, less guessing, good docs, and snappiness. In the end, DX boils down to __maximizing value__ and __minimizing wasted time__.

On the previous page, we saw how to return a clear error message and in addition state how one should use the tool properly. Next, we want to make it stand out a little more.

Developer Experience

#!/usr/bin/env node
const arg = require('arg');
const chalk = require('chalk');

try {
  const args = arg({
    '--start': Boolean,
    '--build': Boolean,
  });

  if (args['--start']) {
    console.log(chalk.bgCyanBright('starting the app'));
  }
} catch (e) {
  console.log(chalk.yellow(e.message));
  console.log();
  usage();
}

function usage() {
  console.log(`${chalk.whiteBright('tool [CMD]')}
  ${chalk.greenBright('--start')}\tStarts the app
  ${chalk.greenBright('--build')}\tBuilds the app`);
}


In our `tool` project, install the <a href="https://www.npmjs.com/package/chalk" target="_blank">chalk</a> package

. `chalk` lets us add colors to our `console.log` statements which will help make our output clearer.

Your output should be colorful, professional-looking, and, most importantly, simpler to understand. (_Now imagine as if the tutorial's code viewer supported custom colors_ 😛 )

A Splash of Color

Most tools can be configured by the user. A best practice is to let a tool be configured via properties in `package.json`.

testProject

In the `testProject`'s `package.json` file, add the following properties.

#!/usr/bin/env node
const arg = require('arg');
const chalk = require('chalk');

try {
 const args = arg({
 '--start': Boolean,
 '--build': Boolean,
 });

 if (args['--start']) {
 const pkg = require('./package.json'); // <-- won't work
 console.log(chalk.bgCyanBright('starting the app'));
 }
} catch (e) {
 console.log(chalk.yellow(e.message));
 console.log();
 usage();
}

function usage() {
 console.log(`${chalk.whiteBright('tool [CMD]')}
 ${chalk.greenBright('--start')}\tStarts the app
 ${chalk.greenBright('--build')}\tBuilds the app`);
}

We now want to read the configuration from the `testProject`'s `package.json`. We can't just `require` the `package.json` file since it sits in the `testProject` and we're in the context of our `tool`.

#!/usr/bin/env node
const arg = require('arg');
const chalk = require('chalk');
const path = require('path');

try {
  const args = arg({
    '--start': Boolean,
    '--build': Boolean,
  });

  if (args['--start']) {
    const pkg = require(path.join(process.cwd(), 'package.json'));
    // TODO: do something with pkg
    console.log(chalk.bgCyanBright('starting the app'));
  }
} catch (e) {
  console.log(chalk.yellow(e.message));
  console.log();
  usage();
}

function usage() {
  console.log(`${chalk.whiteBright('tool [CMD]')}
  ${chalk.greenBright('--start')}\tStarts the app
  ${chalk.greenBright('--build')}\tBuilds the app`);
}


To require the file correctly, we can use node's `process.cwd()` method to retrieve the path from where the user executed our binary. We combine the path with the file name using node's `path.join` method.

tool-tutorial/testProject/src$ tool --start
Cannot find module '/Users/user/tool-tutorial/testProject/src/package.json'
Require stack:
- /Users/user/tool-tutorial/tool/bin/index.js


What happens if our user runs our tool from a subdirectory? We'll look for the `package.json` in the wrong directory and fail.

#!/usr/bin/env node
const arg = require('arg');
const chalk = require('chalk');
const pkgUp = require('pkg-up');

try {
  const args = arg({
    '--start': Boolean,
    '--build': Boolean,
  });

  if (args['--start']) {
    const pkgPath = pkgUp.sync({cwd: process.cwd()});
    const pkg = require(pkgPath);
    if (pkg.tool) {
      console.log('Found configuration', pkg.tool);
      // TODO: do something with configuration
    } else {
      console.log(chalk.yellow('Could not find configuration, using default'));
      // TODO: get default configuration
    }
    console.log(chalk.bgCyanBright('starting the app'));
  }
} catch (e) {
  console.log(chalk.yellow(e.message));
  console.log();
  usage();
}

function usage() {
  console.log(`${chalk.whiteBright('tool [CMD]')}
  ${chalk.greenBright('--start')}\tStarts the app
  ${chalk.greenBright('--build')}\tBuilds the app`);
}


We'll use a package called <a href='https://github.com/sindresorhus/pkg-up' target='_blank'>pkg-up</a> in order to find the `package.json` file. `pkg-up` climbs up the directory tree and looks for the `package.json` file and returns its path.

<a href='https://sindresorhus.com/' target='_blank'>Sindresorhus</a> also provides us with a more generic package <a href='https://github.com/sindresorhus/find-up' target='_blank'>find-up</a>.

Configuration

Let's take a break for a second from the configuration to refactor our code a bit.

mkdir src
mkdir src/commands
mkdir src/config
touch src/commands/start.js
touch src/commands/config-mgr.js

We'll create an `src` directory in our `tool` project and inside it create two directories: `commands` and `config`. In `commands` we'll create a `start.js` file, and in `config` we'll create `config-mgr.js`.

const chalk = require('chalk');

module.export = function start(config) {
  console.log(chalk.bgCyanBright('  Starting the app  '));
  console.log(chalk.gray('Received configuration in start -'), config);
}

In the `start.js` file, we'll create our start function, which receives a config object.

In the `config-mgr.js` file, we'll copy over the config code that we created earlier.

#!/usr/bin/env node
const arg = require('arg');
const chalk = require('chalk');
const getConfig = require('../src/config/config-mgr');
const start = require('../src/commands/start');

try {
  const args = arg({
    '--start': Boolean,
    '--build': Boolean,
  });

  if (args['--start']) {
    const config = getConfig();
    start(config);
  }
} catch (e) {
  console.log(chalk.yellow(e.message));
  console.log();
  usage();
}

function usage() {
  console.log(`${chalk.whiteBright('tool [CMD]')}
  ${chalk.greenBright('--start')}\tStarts the app
  ${chalk.greenBright('--build')}\tBuilds the app`);
}


And we'll simplify our `bin/index.js` file.

Refactor

Previously we configured our tool via the `package.json` file, but most tools have multiple ways to be configured. One of those ways is a Javascript configuration file.

An advantage of a Javascript configuration file over a Json one is that you can add custom logic and even use 3rd party libraries.

We can add another condition in our `config-mgr` to search for and read the Javascript configuration file. That will work but read ahead for a more robust solution.

We'll replace our `pkgUp` logic with the use of the <a href='https://github.com/davidtheclark/cosmiconfig' target='_blank'>cosmiconfig</a> package. `cosmiconfig` supports many different aspects of configuration handling, such as additional file formats, async search, caching and more.

We can now create a configuration file called `tool.config.js` in our `testProject`.

tool-tutorial/testProject$ tool --start
Found configuration { port: 6666 }
  Starting the app
Received configuration in start - { port: 6666 }

`cosmiconfig` gives priority to the configuration defined in the `package.json` file. And thus, if we remove the configuration from `package.json` we'll see that it picks up the new `tool.config.js` file.

Configuration Part Deux

Since the configuration comes from user input, we cannot trust its validity. One way to validate it is to check each property and subproperty in the configuration object using custom code. But, as always, if you're facing a generic problem, such as configuration validation, you can rest assured that someone else met the same problem and developed a library to handle it.

We'll use a library called <a href='https://www.npmjs.com/package/ajv' target='_blank'>ajv</a> which validates the configuration according to a Json schema.

{
  "properties": {
    "port": {
      "type": "number"
    }
  }
}

Let's first create the schema. Create a new file in the `tool/src/config` directory called `schema.json` and enter the following code. We define that the only valid type of the `port` property is a `number`.

We'll then use `ajv` to validate our configuration. Notice that we use `process.exit(1)` to exit our process since we don't want to continue with invalid configuration.

In our `testProject/tool.config.js` file, we'll change the port value from a number to a string.

tool-tutorial/testProject$ tool --start
Invalid configuration was supplied
[
  {
    keyword: 'type',
    dataPath: '/port',
    schemaPath: '#/properties/port/type',
    params: { type: 'number' },
    message: 'should be number'
  }
]

If we now run our tool, we'll see that we receive the error we expected.

We got the right information but with terrible DX. Let's use the library <a href='https://www.npmjs.com/package/better-ajv-errors' target='_blank'>better-ajv-errors</a> to make the errors a bit more approachable.

tool-tutorial/testProject$ tool --start
Invalid configuration was supplied

  TYPE should be number

> 1 | {"port":"6666"}
    |         ^^^^^^ 👈🏽  type should be number

If we rerun our tool, we'll see that the error message is much more user friendly, clearer, and respects your users' time.

Validation

Up until now, we sprinkled our code with various `console.log` statements. Some of the `console.log`s are user-facing, some are debug, some are informational, and some are warning. Let's make an order of this mess.

There are great logging libraries such as <a href="https://www.npmjs.com/package/winston" target="_blank">winston</a> and <a href="https://www.npmjs.com/package/bunyan" target="_blank">bunyan</a> but in this case we want to see how stuff happen behind the scenes so we'll implement the logger on our own.

const chalk = require('chalk');

module.export = function createLogger(name) {
  return {
  };
}          

Create a file under `tool/src` called `logger.js`. This file will return a logger function that receives a name and returns a logger object.

const chalk = require('chalk');

module.export = function createLogger(name) {
  return {
    log: (...args) => console.log(chalk.gray(...args)),
    warning: (...args) => console.log(chalk.yellow(...args)),
    highlight: (...args) => console.log(chalk.bgCyanBright(...args)),
    debug: console.log
  };
}          

We'll create different types of messages, each with a different color. We'll come back to the `debug` function soon. We'll start migrating our console.log statements to the new logger.

const logger = require('../logger')('commands:start');

module.export = function start(config) {
  logger.highlight('  Starting the app  ');
  logger.debug('Received configuration', config);
}

#!/usr/bin/env node
const logger = require('../src/logger')('bin');
const arg = require('arg');
const chalk = require('chalk');
const getConfig = require('../src/config/config-mgr');
const start = require('../src/commands/start');

try {
  const args = arg({
    '--start': Boolean,
    '--build': Boolean,
  });

  logger.debug('Received args', args);

  if (args['--start']) {
    const config = getConfig();
    start(config);
  }
} catch (e) {
  logger.warning(e.message);
  console.log();
  usage();
}

function usage() {
  console.log(`${chalk.whiteBright('tool [CMD]')}
  ${chalk.greenBright('--start')}\tStarts the app
  ${chalk.greenBright('--build')}\tBuilds the app`);
}


We'll quickly change our port back from a _string_ to a _number_ so we could have a successful execution.

tool-tutorial/testProject$ tool --start
Received args { _: [], '--start': true }
Found configuration { port: 6666 }
  Starting the app
received configuration { port: 6666 }

If we rerun our tool we'll see the same logs as before. While it's great that we consolidated the logic, we can take it a step further.

const chalk = require('chalk');
const debug = require('debug');

module.export = function createLogger(name) {
  return {
    log: (...args) => console.log(chalk.gray(...args)),
    warning: (...args) => console.log(chalk.yellow(...args)),
    highlight: (...args) => console.log(chalk.bgCyanBright(...args)),
    debug: debug(name)
  };
}          

We'll replace our debug's `console.log` call with the <a href='https://www.npmjs.com/package/debug' target='_blank'>debug</a> library

and pass the name we received. `debug` lets us scope a log to a specific namespace (the `name` we received).

tool-tutorial/testProject$ tool --start
  Starting the app

When we run our tool now, all we'll see is this single, user-facing, message. The debug logs are hidden until we need to debug our code.

tool-tutorial/testProject$ DEBUG=* tool --start
bin Received args { _: [], '--start': true } +0ms
config:mgr Found configuration { port: 6666 } +0ms
  Starting the app
commands:start received configuration { port: 6666 } +0ms


If we run our tool with an environment variable called `DEBUG`, we can trigger the logs of the `debug` package.

Notice that the debug logs are scoped to their names: `bin`, `config:mgr` and `commands:start`. Also, at the end of each line is the difference, in milliseconds, from the previous log. Pretty nifty!

tool-tutorial/testProject$ DEBUG=commands:* tool --start
  Starting the app
commands:start received configuration { port: 6666 } +0ms


But wait, there's more! `debug` also lets us filter the logs by namespacing the environment variable. We can call it with `commands:*` to include only the commands logs.

tool-tutorial/testProject$ DEBUG=commands:*,bin tool --start
bin Received args { _: [], '--start': true } +0ms
  Starting the app
commands:start received configuration { port: 6666 } +0ms


Or call it with `commands:*,bin` to include both commands and bin logs.

tool-tutorial/testProject$ DEBUG=*,-bin tool --start
config:mgr Found configuration { port: 6666 } +0ms
  Starting the app
commands:start received configuration { port: 6666 } +0ms


We can also call it with `*,-bin` to include everything except the bin logs.

In short, `debug`'s filtering options are flexible, powerful and easy to use.

The `debug` package is used behind the scenes in many different packages, such as <a href="https://expressjs.com/en/guide/debugging.html" target="_blank">express</a>.

Logging

We now have a working skeleton for a command line tool. We created the binary, processed the CLI arguments, found and validated the configuration, and created a way to communicate with our users via logs.

Now all that is left is to pour your logic inside.

I hope that you enjoyed this tutorial and found it valuable. I'll be happy to hear any feedback you have both regarding the tutorial itself and the logistics of it at <a href="https://twitter.com/yairhaimo" target="_blank">@yairhaimo</a>. Yair