Engineering

⌘K
  2. Home
  4. Docs
  6. Engineering
  8. NestJS
  10. Deploy NestJS-Express with Serverless Framework on AWS Lambda

Deploy NestJS-Express with Serverless Framework on AWS Lambda

This guide will cover both Nx monorepo-based NestJS project and standalone NestJS project repository. If there are differences in steps, will be marked accordingly.

Prepare & Test Serverless Framework Locally

Reference:

yarn global add serverless

Add required packages:

yarn add aws-lambda aws-serverless-express
yarn add --dev @types/aws-lambda @types/aws-serverless-express

Create [standalone NestJS] src/lambda.ts: (see Nx monorepo notes below)

import { Handler, Context } from 'aws-lambda';
import { Server } from 'http';
import { createServer, proxy } from 'aws-serverless-express';
import { eventContext } from 'aws-serverless-express/middleware';

import { NestFactory } from '@nestjs/core';
import { ExpressAdapter } from '@nestjs/platform-express';
import { AppModule } from './app.module';

import * as express from 'express';

// NOTE: If you get ERR_CONTENT_DECODING_FAILED in your browser, this
// is likely due to a compressed response (e.g. gzip) which has not
// been handled correctly by aws-serverless-express and/or API
// Gateway. Add the necessary MIME types to binaryMimeTypes below
const binaryMimeTypes: string[] = [
  'application/octet-stream',
  'application/pdf',
];

let cachedServer: Server;

// Create the Nest.js server and convert it into an Express.js server
async function bootstrapServer(): Promise<Server> {
  if (!cachedServer) {
    const expressApp = express();
    const nestApp = await NestFactory.create(
      AppModule,
      new ExpressAdapter(expressApp)
    );
    nestApp.use(eventContext());
    await nestApp.init();
    cachedServer = createServer(expressApp, undefined, binaryMimeTypes);
  }
  return cachedServer;
}

// Export the handler : the entry point of the Lambda function
export const handler: Handler = async (event: any, context: Context) => {
  // https://github.com/vendia/serverless-express/issues/86
  event.path = event.pathParameters.proxy;
  cachedServer = await bootstrapServer();
  return proxy(cachedServer, event, context, 'PROMISE').promise;
};

Nx Monorepo: Deployment Configuration

Reference:

Edit workspace.json with (1) configuration “lambda”:

{
  "version": 2,
  "cli": { "defaultCollection": "@nrwl/nest" },
  "projects": {
    "tamrah-fund": {
      "root": "apps/tamrah-fund",
      "sourceRoot": "apps/tamrah-fund/src",
      "projectType": "application",
      "prefix": "tamrah-fund",
      "targets": {
        "build": {
          "executor": "@nrwl/node:build",
          "outputs": ["{options.outputPath}"],
          "options": {
            "outputPath": "dist/apps/tamrah-fund",
            "main": "apps/tamrah-fund/src/main.ts",
            "tsConfig": "apps/tamrah-fund/tsconfig.app.json",
            "assets": [
              "apps/tamrah-fund/src/assets",
              {
                "glob": "serverless.yml",
                "input": "apps/tamrah-fund",
                "output": "./"
              }
            ]
          },
          "configurations": {
            "lambda-staging": {
              "main": "apps/tamrah-fund/src/app/lambda.ts"
            },
            "production": {
              "optimization": true,
              "extractLicenses": true,
              "inspect": false,
              "fileReplacements": [
                {
                  "replace": "apps/tamrah-fund/src/environments/environment.ts",
                  "with": "apps/tamrah-fund/src/environments/environment.prod.ts"
                }
              ]
            }
          }
        },
        "serve": {
          "executor": "@nrwl/node:execute",
          "options": { "buildTarget": "tamrah-fund:build" }
        },
        "lint": {
          "executor": "@nrwl/linter:eslint",
          "options": { "lintFilePatterns": ["apps/tamrah-fund/**/*.ts"] }
        },
        "test": {
          "executor": "@nrwl/jest:jest",
          "outputs": ["coverage/apps/tamrah-fund"],
          "options": {
            "jestConfig": "apps/tamrah-fund/jest.config.js",
            "passWithNoTests": true
          }
        }
      }
    }
  }
}

Note: We cannot include the default node_modules by nx because it uses too much storage (> 300 MB for template NestJS project). So we need to create our own node_modules.

Create apps/YOUR_APP/serverless.yml, e.g.:

frameworkVersion: ^2.29.0
service: tamrah-fund

package:
  # exclude:
    # Exclude "built" files since we're using serverless-plugin-typescript
    # - dist/**
  include:
    - templates/**

plugins:
  # - serverless-plugin-typescript
  # to transpile and minify your code
  # - serverless-plugin-opttimize
  # to be able to test your app offline
  - serverless-offline
  # - serverless-plugin-warmup

provider:
  name: aws
  runtime: nodejs14.x
  profile: tamrah
  stage: ${opt:stage, 'development'} # Set the default stage used. AWS Default is dev
  region: ${opt:region, 'me-south-1'} # Overwrite the default region used. AWS Default is us-east-1
  lambdaHashingVersion: '20201221'
  # RAM usage is only about 150 MiB, but need bigger RAM to speed up CPU to avoid timeout
  memorySize: 1024 # Overwrite the default memory size. Default is 1024
  timeout: 15 # The default is 6 seconds. Note: API Gateway current maximum is 30 seconds
  apiGateway:
    shouldStartNameWithService: true
    # minimumCompressionSize: 1024 # Compress response when larger than specified size in bytes (must be between 0 and 10485760)
    binaryMediaTypes: # Optional binary media types the API might return
      - application/octet-stream
      - application/pdf

functions:
  main: # The name of the lambda function
    # The module 'handler' is exported in the file 'src/lambda'
    # handler: src/app/lambda.handler
    # After building, the module 'handler' is inside file 'main.js'
    handler: main.handler
    events:
      - http:
          method: any
          path: /{any+}

# custom:
#   customDomain:
#     domainName: certificate2pdf.talentiva.net
#     # certificateName: '*.talentiva.net'
#     # Since this API is used only internally by ERPNext, save bandwidth & time by using Regional endpoint instead of Edge
#     endpointType: regional
#     # basePath: ''
#     stage: ${self:provider.stage}
#     createRoute53Record: false
#   # Enable warmup on all functions (only for production and staging)
#   warmup:
#     enabled: true

Create apps/APP_NAME/package.json adding required dependencies that Nx excludes for some reason:

{
  "name": "tamrah-fund",
  "description": "Tamrah Crowdfunding NestJS Backend Service",
  "version": "0.0.1",
  "license": "UNLICENSED",
  "main": "main.js",
  "dependencies": {
    "reflect-metadata": "^0.1.13",
    "rxjs": "^6.6.6"
  }
}

Then build:

# This will generate dist/apps/tamrah-fund with main.js, assets/, etc. and serverless.yml
nx build tamrah-fund --configuration=lambda-staging
# Generate (more compact) node_modules, then deploy
(cd dist/apps/tamrah-fund; yarn install --production; serverless deploy -s staging)

Alternative: Try to use: @flowaccount/nx-serverless. But seems hard to understand how this works with existing NestJS project.

nx g @flowaccount/nx-serverless:api-serverless --name=myapi --provider=aws

Standalone NestJS: Deployment Configuration

Serverless plugins: serverless-plugin-typescript.

yarn add --dev serverless-plugin-typescript
yarn add --dev serverless-offline
# Workaround for nodejs14.x support before 6.9.x is released: -- doesn't work: Serverless plugin "serverless-offline" not found.
#yarn add --dev github:dherault/serverless-offline#master

Create serverless.yml (for Nx monorepo, put it inside apps/APP_NAME), for example:

service:
  name: certificate2pdf

package:
  exclude:
    # Exclude "built" files since we're using serverless-plugin-typescript
    - dist/**
  include:
    - templates/**

plugins:
  - serverless-plugin-typescript
  # to transpile and minify your code
  # - serverless-plugin-opttimize
  # to be able to test your app offline
  - serverless-offline
  # - serverless-plugin-warmup

provider:
  name: aws
  runtime: nodejs14.x
  stage: ${opt:stage, 'dev'} # Set the default stage used. Default is dev
  region: ${opt:region, 'ap-southeast-1'} # Overwrite the default region used. Default is us-east-1
  # RAM usage is only about 150 MiB, but need bigger RAM to speed up CPU to avoid timeout
  memorySize: 1024 # Overwrite the default memory size. Default is 1024
  timeout: 15 # The default is 6 seconds. Note: API Gateway current maximum is 30 seconds
  apiGateway:
    # minimumCompressionSize: 1024 # Compress response when larger than specified size in bytes (must be between 0 and 10485760)
    binaryMediaTypes: # Optional binary media types the API might return
      - application/octet-stream
      - application/pdf

functions:
  main: # The name of the lambda function
    # The module 'handler' is exported in the file 'src/lambda'
    handler: src/lambda.handler
    events:
      - http:
          method: any
          path: /{any+}

# custom:
#   customDomain:
#     domainName: certificate2pdf.talentiva.net
#     # certificateName: '*.talentiva.net'
#     # Since this API is used only internally by ERPNext, save bandwidth & time by using Regional endpoint instead of Edge
#     endpointType: regional
#     # basePath: ''
#     stage: ${self:provider.stage}
#     createRoute53Record: false
#   # Enable warmup on all functions (only for production and staging)
#   warmup:
#     enabled: true

For Nx monorepo, you need to make changes:

  • Put the file in apps/APP_NAME/src/app/lambda.ts
  • In serverless.yml, set handler: src/app/lambda.handler

Try locally using serverless-offline: (Note: As of serverless-offline 6.8.0, it doesn’t support NodeJS 14 yet although AWS Lambda already supported it, a workaround is to depend on GitHub master branch or downgrade to NodeJS 12)

# You'll need to either export required environment variables, or use dotenv package
export JWT_PUBLIC_KEY=...
~/.yarn/bin/serverless offline start

Deploy “dev” stage to AWS Lambda with Generic URL

During serverless deploy: You’ll get error: messageText: “Option ‘–incremental’ can only be specified using tsconfig, emitting to single file or when option --tsBuildInfoFile is specified.”, so edit tsconfig.json and add inside compileOptions:

    "tsBuildInfoFile": ".tsbuildinfo"

To actually deploy, set required environment variables first using AWS Systems Manager Parameter Store.

Important: Unless you specify a region in serverless.yml, it will use us-east-1, even if ~/.aws/config specifies a different default region.

Using in Gitpod: You may want to create ~/.aws/credentials, or alternatively export AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY before running deploy.

~/.yarn/bin/serverless deploy

To deploy a different environment/stage, e.g. production or staging:

~/.yarn/bin/serverless deploy -s production

Note: Binary Media Types (PDF, etc.)

  • In Fastify/Express, ensure to set binaryMimeTypes, e.g. return awsLambdaFastify(fastifyApp, { binaryMimeTypes: ['application/octet-stream', 'application/pdf'] }); Fastify doesn’t seem to support wildcards here.
  • In serverless.yml, set binaryMediaTypes to */*
  • In client, you must set Accept: application/pdf or Accept: application/*. Unfortunately, using Accept: */* will not work, and you’ll get Base64.
  • In serverless.yml, “contentHandling: CONVERT_TO_BINARY” doesn’t seem to be needed, and seems to cause a warning that it is ignored.

How can we help?