The Five Things I Learned Writing My Own Node Package

Node packages contain ready to use binaries, libraries, and sometimes web distributables.

-

In trying to write and distribute my own css library, I skimmed the surface of node and the node package manager. Here's what I learned.

1. JavaScript can be executed from the command line with node.

Which, I assume sounds pretty entry-level if you are already an experienced node developer or Unix wizard. I've written about executing JavaScript from the console in my javascript unit testing article before, but until this point I generally hadn't dabbled in ES2015+ features yet. I'd pretty much stuck to writing strict ES5 and moderately ignoring most newer ES2016+ features and leveraging polyfils ad hoc.

What I have found is that unlike many modern browsers, node can interpret newer versions of JavaScript like ES6, ES2016+, and ES.next code, meaning you can write node-interpreted scripts in ES2016+ using the #! shebang.*

Take for example the following ES2016+ script which takes an exit_code parameter, prints it, and exits with the exit_code to the shell:

#!/usr/bin/env node
// exit.js
// usage: node ./exit.js [exit_code]
const exit_code = process.argv[2] ?? 0;
console.log(exit_code);
process.exit(exit_code);

I can execute it from a shell with node:

node exit.js 0 # prints 0
echo $?        # prints 0

Better yet, let's let the shell determine the language to interpret:

chmod u+x exit.js
./exit.js 1    # prints 1
echo $?        # prints 1
*Here is a compatibility chart of supported ES2016+ features in node

2. You can use node packages in non-node projects.

You don't need to be writing a server side node app to take advantage of the many tools available in the node package registry (and other sources). Node packages contain ready to use binaries, libraries, and sometimes web distributables. For example:

Node Package Binaries

Package binaries can be executed with the npx command right from the root of your repo. Node binaries encompass a large swath of different purposes; Transpilation tools, testing frameworks, bundling tools, and even templating tools can be found in the Node package registry.

npx lessc my.less public/my.css             #transpiling
npx babel my.jsx --out-dir public/js        #transpiling
npx tsc --outFile public/my.js my.ts        #transpiling
npx mocha tests/**/*Test.js                 #testing
npx webpack                                 #bundling
npx express-generator my-backend --view pug #templating
npx create-react-app my-frontend            #templating

Node Package Libraries

Node libraries can be installed as dependencies of your app and required by your project. For example I can use the highlight.js node package to render some syntax-highlighted markdown code as html and pass it back to the view in an express pug app:

# first create an express app
npx express-generator my-blog --type pug
cd my-blog
npm install
npm install --save highlight.js

Edit the view controller:

Index: routes/index.js
==================================================================
  var express = require('express');
  var router = express.Router();
+ const hljs = require('highlight.js/lib/core');
+ hljs.registerLanguage('markdown', require('highlight.js/lib/languages/markdown'));

  /* GET home page. */
  router.get('/', function(req, res, next) {
-   res.render('index', { title: 'Express' });
+   res.render('index', {
+     title: 'The five things I learned writing my own node package.',
+     code_sample: hljs.highlight('# a header level 1\n\nSome ***bold and italic text***', { language: 'markdown' }).value
+   });
  });

  module.exports = router;
Index: views/index.jade
==================================================================
  extends layout
  
  block content
    h1= title
-   p Welcome to #{title}
+   pre !{code_sample}
Index: views/layout.jade
==================================================================
  doctype html
  html
    head
      title= title
+     style 
+       include ../node_modules/highlight.js/styles/default.css')
      link(rel='stylesheet', href='/stylesheets/style.css')
    body
      block content

npm start the app and navigate to localhost:3000. You should now see the title of the page (which I've cleverly titled the same as this article) and Markdown that is syntax highlighted. Looking at this example, it feels almost as if someone was trying to build a developer blog.

Node (Web) Package Distributables

Package distributables, specifically web front-end dists, can be copied directly to your wwwroot or public folders as part of your build process:

cp ./node_modules/jquery/dist/jquery.min.js public/dist

From what I've uncovered, this is mostly a work around to using tools like Bower (a web distributable package manager), who, funny enough, recommend using Yarn (a Node Package Manager alternative) for obtaining assets.

Some libraries, like highlight.js, make available only their node libraries in their node package, and make available their web assets in their @highlighjs\cdn-assets node package.

There are no standards (that I could find at the time of this writing) for packaging web distributables in node packages, which makes it difficult to generally know where find web assets in the registry. The simple rule I use is: look for a ./dist folder first, then seek the project issues board second. Realistically, the node package manager is a great framework for versioning, publishing, and retrieving binaries, libraries, and sometimes even assets, so it's no wonder why people like using the node package manager for non-node reasons.

Aside: Funny story, the highlight.js library's issues board makes it pretty clear about the separation of libraries and assets, in fact, the tools used to publish the CDN assets reside in the same highligh.js node library's github repository, but the distinction is clearly made between libraries and assets. The scripts to build the cdn-assets are not bundled with the highlight.js node package. The Highlight.js NPM package is very clearly a node library first, and a web asset second.

3. Dependencies and Dev-Dependencies are not the same thing.

In my last example I added highlight.js as an actual dependency of my (extremely contrived) blog app. In a very real sense, the app cannot run successfully without this dependency.

Dev-dependencies on the other hand are not required by your app, but may be required to build or test it. For instance, I may only need to transpile resources for a static front end application. I can npm install --save-dev [utility], and npx [utility] [src] [dest] to publish those final static assets to my public folder. For example I could transpile my less to css with lessc.

npm install --save-dev lessc
npx lessc my.less public/my.css

Those familiar with React or Express apps, might use the create-react-app or express-generator binaries to generate template react or express apps. create-react-app and express-generator binaries aren't exactly dev-dependencies though. They are not required to build or test an app, but typically only used to create the foundation code for the app itself. In these instances, developers sometimes choose to globally install these useful templating binaries, and they never end up in project dev-dependencies.

4. NPM scripts can organize my build/test/publish processes.

I can add a dev-dependency like lessc with npm install --save-dev lessc and reference it in my node scripts (within my package.json) without ever having to npm install it globally, or remember/type the npx lessc [src] [dest] command each time.

// package.json
...
  "scripts": {
    "less": "lessc my.less public/my.css",
...
npm run less

A more advanced example might leverage a few tools to transpile and minify css, but also simplify the repetitive nature of building a distributable web package.

npm install --save-dev less clean-css-cli npm-run-all
// package.json
...
  "scripts": {
    "prepare": "npm run build",
    "build": "npm-run-all clean less min",
    "clean": "rm -r dist; mkdir -p dist/css",
    "less": "lessc my.less dist/css/my.css",
    "min": "cleancss --output dist/css/my.min.css dist/css/my.css"
...

A simple npm publish will (automatically) prepare my project for publishing. It does this by automatically calling the lifecycle prepare script, which in-turn calls build, which in-turn calls clean, less, and min. My solution is "prepared" for publishing, and indeed publishes my transpiled and minified css assets.

5. Semantic versioning is baked into the Node Package Manager

The npm version command follows semantic versioning by default. It will git commit a bump in the Major, Minor, or Patch version that exists in your package.json, and git tag that commit with the new version as well.

# a patch for a broken feature!
npm version patch
# a new non breaking feature!
npm version minor
# a set of breaking changes
npm version major

It also covers scenarios where you might want to make available a pre-release version. Just be careful not to publish your pre-release candidates with the latest tag (which is easy to do, and I've noted below!), or people might accidentally install potentially broken release candidates in their projects when they npm install [your-package].

Versioning & Distributable Package Example

As an example of most of the above, I'll show you how I went about creating a brand new "web framework" I call bleu-css. A framework to make all your web text blue.

gh auth login
gh repo create richminchukio/bleu-css \
  --description="It's all bleu. A blue style for your web projects." \
  --enable-issues=false \
  --enable-wiki=false \
  --homepage=https://richminchuk.io \
  --team=richminchukio \
  --public
cd bleu-css
git checkout -b main
npm init --yes # defaults to v1.0.0

Firstly, I've made a repo, and initialized it with npm init. It defaults the version of our future node package to v1.0.0. Let's now create the distributable "web framework's" source code.

Index: dist/bleu.css
================================
+ body { color: bleu; }

The source code has been added, let's publish it to make it official.

git add *
git commit -m "It's all bleu"
git push --set-upstream origin main
git tag v1.0.0
git push --follow-tags
npm login
npm publish # v1.0.0

We published our framework, and it's now officially our latest release. People started downloading the package but noticed an interesting issue in their browsers. Let's make our first bug fix!

Index: dist/bleu.css
================================
- body { color: bleu; }
+ body { color: blue; }

Perhaps this time we should check if it works first. Let's prepatch our version, and publish a "beta" of our node package. Our testers can then npm install bleu-css@beta or npm install bleu-css@1.0.1-0 at the prepatch version.

To publish a beta version we MUST specify the --tag option in our npm publish command. We explicitly don't want any early adopters of our project to accidentally install a patch we're not confident solves the issue yet. These prepatch updates help us do that, but more importantly, the publish command defaults the tag on our package to latest (danger!). We must explicitly specify a beta tag. Our end users should only be downloading vetted patches, not prepatches. They shouldn't be able to accidentally install any prerelease versions when they leave off the version or tag on their npm install bleu-css command either!

git add *
git commit -m "change body color to blUE"
npm version prepatch # v1.0.1-0
npm publish --tag beta # <- tricky!
git push --follow-tags
More info on publish functionality here in this github issue "npm publish" tags pre-versions as "latest"

Our tester IM'd us and said it looks like that fixed it, nice work! Let's lock it in! The npm version patch drops the "-0" from our version and the npm publish command tags our patch as the new latest release.

npm version patch # v1.0.1
git push --follow-tags
npm publish

We've published our fix and our project is getting a lot of buzz on the interwebs, but someone opened an issue and suggested our blue wasn't blue enough. Our first feature!

Index: dist/bleu.css
================================
- body { color: blue; }
+ body { color: midnightblue; }

We're on a roll now. Good git commit messages should say what we're actually doing. Let's continue to subscribe to that notion, and publish our latest feature.

git add *
git commit -m "change body color to midnightblue"
npm version minor # v1.1.0
git push --follow-tags
npm publish

MidnightBlue was a huge success, but in competing with other web frameworks, we've had a suggestion that maybe our framework came on a bit too strong with styling of the entire body tag. As a group we've decided to reign it in, and just style paragraph elements from now on. Our first breaking change!

Index: dist/bleu.css
================================
- body { color: midnightblue; }
+ p { color: midnightblue; }

Let's bump the major version, and continue with good commit messaging.

git add *
git commit -m "removed body styling, added p styling"
npm version major # v2.0.0
git push --follow-tags
npm publish

We did it. We're open-source node wizards now. Go ahead and npm install bleu-css. Copy the dist to your public/publish/wwwroot folder or add it to your npm scripts and you're off to the races!

// package.json
...
  "scripts": {
+   "publish": "mkdir public; cp ./node_modules/bleu-css/dist/bleu.css public/bleu.css"
  }
...

The richminchuk.io design is based on my NPM package horizn. Checkout horizn on npmjs.com, or just npm install horizn.

Rich Minchuk

Technology Enthusiast and Wannabe Growth Hacker