sw.

One of the things I've been mulling over for a while is writing a full-stack web app from scratch. I do a lot of development as part of my job but it's mostly writing small microservices for telemetry or monitoring, IaC and automation tasks, and other DevOps/CloudOps related activities.

However I have a keen interest in React, Node and related technologies, and a little while ago I started a project to try and learn some full-stack principles more thoroughly using these technologies. I'm really keen to fully grasp and understand OAuth2.0 / 3-legged authentication flow, 12-factor microservices architecture as it is used in full stack app development, and service discovery from a custom perspective (as opposed to using k8s, NGinx Plus or Consul for equivalent features). As well as this, I've not really sunk my teeth into Typescript, so I want to do all of this from that perspective, to get myself into the habit of using Typescript for all my JS projects.

Looking around online, there are loads of introduction tutorials for how to get started with some of this stuff, but a lot of them are clearly written by people who are being paid by the word, and don't go into enough detail as to why you make certain configuration and structural decisions - at least, not to someone like me who is not a full-time full stack developer.

I figured, therefore, that it would be useful to document my process of going through these assignments as I set them for myself, if nothing else because even when I was initially configuring my project, I struggled to understand the reasoning behind some of the choices made in other documentation and tutorials, so writing it down here will give a record of what's best to do in 2021 (at least from my amateur perspective...)

Project 1 - API Gateway

Reading around, and from my own background knowledge, I know that I want to do three main things with the project:

1. Keep the authentication and authorization as separated concerns from the business logic
2. Maintain 12-factor microservices architecture as much as possible
3. Maintain a Service Directory of attached microservices that are serving my application

Based on research, it seems like the best way to achieve this is to develop a robust API Gateway, through which my front-end client will connect and route the various business logic functions. The API Gateway will deal with authentication (via OAuth2.0) and route the relevant calls to the backend business microservices, using role-based authorization which will also be handled in the gateway.
The downside of this model is that complex logic will also have to be handled by the gateway (i.e. combining multiple separated function logics into a single return) so keeping features as separated as possible is a primary goal.

My vision is that there is a single API gateway running as the "controller" interface, which is the only public interface that the client connects to. The API Gateway then has child microservice attachments, either by a single common service registry that can route the relevant calls, or by using a sibling model whereby a backend microservice has a microcontroller installed within the gateway that handles the registration and communication. This would be more akin to a plugin model, which I am interested in trying out. The front-end client code will be covered much later, once I have built sufficient logic into the API gateway and can prove service discovery and connectivity with backend services. I may also build the entire thing into a minikube setup so various parts of the app can be scaled horizontally using k8s.

Setting up the environment

Nothing special is being done here. I've not set any weird environment variables, just relying on the latest Node LTS release (v14.7.3 at the time of writing), and I don't need to target anything specific because this is just for me.

In a future writeup I will talk about my configuration strategy for the app, but right now I'm not worrying about it.

One thing to note here is that i'm using nvm pretty unilaterally to control the installation and versioning of node. The instructions on how to set this up can be found on Github

Project Setup steps

First things first - create the project and set the defaults with npm:

mkdir gateway && npm init -y

Then you're going to want to install typescript, the relevant @types packages and some debug/monitoring stuff:

npm i debug winston

npm i -D typescript ts-node nodemon eslint @types/node

Breaking this apart for a second, typescript, ts-node, eslint and @types/node are all directly related to typescript or linting. eslint replaced tslint in 2019/2020 (not necessarily shown in most of the modern tutorials), so you can just go ahead and install eslint for typescript linting.

winston is logging and monitoring middleware which make it easier to debug node.js applications. It attaches to your http server and prettifies your calls.

Now we've got the installing done, we need to do some configuration. I used ./node_modules/.bin/eslint --init to create the initial .eslintrc.json (because it guides you through the creation process), but for nodemon.json and tsconfig.json I created them from scratch using these options:

nodemon.json

This one is pretty straightforward. It's arguable that you don't need the restartable config option, but it's handy if you don't think something is working properly, or you change something outside of src/

{
    "restartable": "rs",
    "ignore": [".git", "node_modules/**/node_modules"],
    "verbose": true,
    "execMap": {
      "ts": "node --require ts-node/register"
    },
    "watch": ["src/"],
    "env": {
      "NODE_ENV": "development"
    },
    "ext": "js,json,ts"
  }

tsconfig.json

This one is a bit more tricky. I'm very unfamiliar with typescript (and ECMAScript in general) so some of these options aren't entirely clear to me, but here's the reasons why I chose them.

• target: es2019 - this was chosen because it's the latest version of ECMAScript that's fully supported by node 14.7.3 according to node.green
• module: commonjs - this is the way node.js uses the v8 engine.
• strict, noUnusedLocals, noUnusedParameters, noImplicitReturns & noFallthroughCasesInSwitch - these make the typescript transpiler fail on linting warnings such as unused variables
• esModuleInterop: true - couldn't find a good explanation for this but it seems to be required in all cases
• inlineSourceMap: true - this isn't necessary, but I don't like .map files so made sense
• some exclusions for faster transpile (ignoring node_modules and any test files)

{
  "compilerOptions": {
    "target": "es2019",
    "module": "commonjs",
    "outDir": "./dist",
    "strict": true,
    "noUnusedLocals": true,
    "noUnusedParameters": true,
    "noImplicitReturns": true,
    "noFallthroughCasesInSwitch": true,
    "esModuleInterop": true,
    "inlineSourceMap": true
  },
  "include": ["src/**/*"],
  "exclude": ["node_modules", "**/*.test.ts"]
}

Changes to package.json

Only thing that needs adding here is "start": "nodemon src/app.ts" into the "scripts": { section.

Basic App structure

OK - on to the app structure!

I created a new directory, src and added a file called app.ts. This is my main app entry point. Following that, I can then start structuring a basic app inside this file, before I start adding more complex functionality.

I installed express as the http middleware: npm i express express-winston && npm i -D @types/express which will let us actually start up a http server.

On to the code, we pull in the relevant libraries (we want to pull in the type definitions for express as well as the express object itself) and initialize express:

import express, { Application, Request, Response } from 'express'
import * as winston from 'winston'
import * as expressWinston from 'express-winston'

const port: number = 3000
const app: Application = express()
const initMsg: string = `Server running at http://localhost:${port}`

We also want to pull in Winston as the logger for later, and initialize its options:

const loggerOptions: expressWinston.LoggerOptions = {
    transports: [ new winston.transports.Console()],
    format: winston.format.combine(
        winston.format.json(),
        winston.format.prettyPrint(),
        winston.format.colorize({ all: true })
    )
}

This is pretty self explanatory, but is basically setting up the output format to be the local console (other valid options are StackDriver and probably Cloudwatch, which would require additional dependencies), and setting up the formatted output. I've opted for pretty json here.

Next - two basic express configurations:

app.use(express.json())
app.use(expressWinston.logger(loggerOptions))

There will be more of these in the future, such as cors, urlencoded etc, but for now we're just setting the basics.

Now I want to set a couple of basic routes in order to test the application. One of them will stick around forever and the other is just to get going:

app.get('/_health', (req: Request, res: Response) => {
    res.status(200).json(
        {
            "message": initMsg,
            "uptime": process.uptime()
        }
    )
})

app.get('/', (_req: Request, res: Response) => {
    res.status(200).send(initMsg)
})

The _health endpoint will be sticking around so that an external process can quickly determine whether the app is running. This is handy if we decide to run this behind a load balancer such as AWS ELB or a k8s LoadBalancer as the service requires a healthcheck endpoint in order to validate the service is accessible.

Finally, we wrap the application start and set it running:

try {
    app.listen(port, () => {
        console.log(initMsg)
    })
} catch (error) {
    console.error(`Error occurred: ${error.message}`)
}

And that's it! When we run the application with npm start, it runs through ts-node and lint checks the code, then starts up the server using nodemon and watches for changes in the source directory, so it can automatically reload the server if anything changes (additionally you can type rs and reload the server manually.)

Next Steps

The next stage of the project will be to tackle the structure of creating a router, which will be used as the basis for the Gateway's functionality. Once I have a router in place that is extensible and separates the main route groups, I can start working on the authentication aspect of the project. The next post in the series will start to discuss that.