How to build a Node.js API (part two)

Hi everyone (again)!

In my previous blog entry, I wrote the first part of our guide to create APIs using Node.js.

In this part, I'll give a quick introduction to express.js in order to understand how the Processes engine is organised (and the rationale behind). This is a pure technical javascript post, so be warned! ⚠️

Disclaimer: the source code examples are, in most of the cases, a simplification of the real code for easier legibility and to avoid compromising our client's code.

Are you ready to learn how express.js works? Let's dive into it!

Middleware

Express.js is a web framework similar, in appearance, to Sinatra because of its minimalism. It's based on a beautiful abstraction called middleware. An express.js app is a chain of middleware, where middleware itself is a function processing a request, which can optionally write a response or pass the request to the next step down the chain.

With this simple abstraction, you can build a web app in stages: first, performing authentication, then verifying parameters, authorizing a user, fetching data from the database, converting the data into a response, etc. In this process, each stage is a different middleware (function).

Middleware in express.js looks like this:

function verifyAuthentication(request, response, next) {
  // use request object to authorize user
  next(); // <- call the next middleware
}

function verifyParams(request, response, next) {
  // use request object body to verify parameters
  next(); // <- call the next middleware of the chain
}

Whereas express.js apps look like this:

const app = express();
app.use(verifyAuthentication, verifyParams, authorize, fetchUser, renderUser)

// equivalent to:
app.use(verifyAuthentication);
app.use(verifyParams);
...
app.use(renderUser);

Sharing data between middlewares

In express.js the request object (usually called req) is used as the request's context. Any middleware can contribute to that context by adding attributes to it. For example:

function loadProcess(req, res, next) {
    Process.findById(req.params[:id]).then(process => {
        req.process = process;
        next();
    })
}

With this sharing mechanism I can write an authorizeProcess middleware that is agnostic about how to retrieve the data:

function authorizeProcss(req, res, next) {
    // assume the data is already loaded
    const { user, process } = req;

    if (Authorize.edit(user, process)) {
        next();
    } else {
        next(NotAuthorizedError());
    }
}

This clear separation between the load data and the authorize data phases helps to write simpler code, and make the middleware more reusable in different scenarios.

Packed middleware

Of course, there are tons of npm-packaged libraries, like helmet, which I use to help to secure the app. It's a common practice to have a function returning the configured middleware.

For example:

const helmet = require("helmet");

const securityMiddleware = helmet({ frameguard: false });
app.use(securityMiddleware)

In fact, a typical express app is like this:

const app = express();

// parse body params and attach them to req.body
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));

// gzip compression
app.use(compress());

// lets you use HTTP verbs such as PUT or DELETE
// in places where the client doesn't support it
app.use(methodOverride());

// secure apps by setting various HTTP headers
app.use(helmet());

// enable CORS - Cross Origin Resource Sharing
app.use(cors());

// Add passport authentication
app.use(authentication());

express.Router

One middleware that is built-in in express.js is the router. A router is a middleware which uses chains of other middlewares, prefixed by a regex-like pattern and a request method, to perform its job:

const routes = express.Router();

routes.get("/processes/:id", authorizeRead, verifyShowParams,
  fetchProcess, renderProcess);

reoutes.post("/processes", authorizeWrite, verifyCreateParams,
  createProcess, renderProcess);

Middleware composability

A key concept of express.js is that middleware is composable (my favourite topic 😂). You can build groups of middleware, and then use them as lego blocks because the middleware groups are middleware themselves:

const app1 = express();
...
const app2 = express();
...
const app = express();
app.use(app1, app2);

But more importantly, routers themselves are composable allowing this kind of code (very similar to the one I wrote):

const processRoutes = express.Router();
processRoutes.get("/:id", verifyGetProcessParams, fetchProcess);
...
const stageRoutes = express.Router();
stageRoutes.get("/:id", verifyGetStageParams, fetchStage);
...
const api = express.Router();
api.use(authentication);
api.use("/processes", processRoutes);
api.use("/stages", stageRoutes);
...
const app = express();
app.use(security, compression);
app.use(api);

It's important to note that:

• The authentication middleware is applied to both processRoutes and stageRoutes
• We decide where to mount the routes outside the routes themselves (by setting a prefix)

This composability of routes delivers the flexibility of Rails Engines without all the complexity.

RealLife™ common pitfalls

A common error is to have a big route file with all the routes, instead of dividing them into modules:

const processes = require("processes.routes"); // import routes from module
const stages = require("stage.routes"); // import routes from module

const router = express.Router();
router.use("/processes", processes);
...
module.exports = router; // the module also exports (a composable) routes

Another common error is to repeat middleware:

router.get("/", middleware1, middleware2, listUsers);
router.get("/:id", middleware1, middleware2, getUser);

Instead of:

router.use(middleware1, middleware2);
router.get("/", listUsers);
router.get("/:id", getUser);

Error handling

Express.js has a special type of middleware to handle errors. Instead of three parameters, it receives four (the first is the error itself) and it's usually added at the end of a chain:

function errorHandlerMiddleware(error, request, response, next) {
...
}

Because it's possible in javascript to inspect at runtime the numbers of defined parameters, that number is used by express.js to know if it's normal or error middleware.

The way to invoke the error middleware is by calling the "next" function with the error as a parameter:

function getUser(request, response, next) {
  // make the user available for the next middleware of the chain
  const user = User.find(request.params['id']);
  if (!user) {
    next(NotFoundError()); // invoke error midddleware
  } else {
    // make the user available to the next stage
    request.user = user;
    next(); // invoke next middleware
  }
}

The error handler is also invoked if an exception is raised inside a middleware.

RealLife™ - Another common pitfall is sending an error response instead of invoking the error handler:

// bad practice
response.status(404).json({ message: 'User not found');

// good practice (the error middleware could perform better error logging)
next(NotFoundError('User not found'));

Async middleware

Finally, express.js has a handy feature. If the middleware returns a promise, it resolves the promise, allowing to write async functions like this:

async function loadProcess(req, res, next) {
  // async code that looks synchronous, yeah!
  req.process = await Process.findById(req.params[:id]);
  next();
}

Next Part

With these concepts, I'll dive into the Processes engine code to see how controllers, request and response objects are built and glued using express.js, so stay tuned!