ServerlessSpy—Gain Insight into the Cloud and Write Serverless Tests with Ease

Writing tests for serverless is challenging. Unit tests often do not provide much value because all services need to be mocked. Integration tests that run on the real cloud infrastructure are the preferred way to test such a system. But integration tests are hard to write, and they are slow to execute. A serverless system is usually built as an asynchronous event-driven system. That means that you put an event in on one side, and after some time, you get some output on the other side. You do not know when, so you have to keep checking. That is slow. At the same time, no other tests can usually run because they will mess up your data. The last issue is that you often have to save the message in some storage just for the integration test to be able to inspect the message.

That's why I created a ServerlessSpy, which resolves the problems mentioned above. It provides a CDK construct that creates infrastructure to intercept events in Lambda, SNS, SQS, EventBridge, DynamoDB, and S3, and sends it to a testing library via API Gateway WebSocket. The testing library subscribes to events so that tests can be executed fast without checking/retrying if the process has finished. You do not have to build additional infrastructure to temporarily store the events. Tests can also run in parallel because you can filter events to receive only the ones meant for your test case.

ServerlessSpy can be used for more than writing integration tests. A special web console can receive events from WebSocket. The console enables you to inspect events in real time and see what is happening in the cloud. Some say it is even more useful than the part for writing integration tests.

Key benefits:

• Easy to write tests that are strongly typed thanks to TypeScript.
• Tests are executed much FASTER. No need to write tests in a way to periodically check if the process has finished because all events are pushed to the testing library.
• Tests can run in parallel if you use conditions and filter events specific to your test. This drastically reduces the execution time of the CI/CD process.
• Web console enables you to see all events in real time. Debugging has never been easier. Search for events is also supported (with regular expression).

Your test for the example above would look something like this:

(
  await serverlessSpyListener.waitForEventBridgeMyEventBus<TestData>({
    condition: (d) => d.detail.id === id,
  })
).toMatchObject(...);

(
  await serverlessSpyListener.waitForSnsTopicMyTopic<TestData>({
    condition: (d) => d.message.id === id,
  })
).toMatchObject(...);

(
  await serverlessSpyListener.waitForSqsMyQueue<TestData>({
    condition: (d) => d.body.id === id,
  })
).toMatchObject(...);

(
  await (
    await serverlessSpyListener.waitForFunctionMyLambdaRequest<TestData>({
      condition: (d) => d.request.id === id,
    })
  ).followedByResponse();
).toMatchObject(...);

(
  await serverlessSpyListener.waitForDynamoDBMyTable<TestData>({
    condition: (d) => d.keys.pk === id,
  })
).toMatchObject({
  eventName: 'INSERT',
  newImage: ...,
});

You can see all the events in the local web console:

ServerlessSpy is meant for the following environments:

• Development environment where you tests. The web console helps you gain insight into what is happening in the system.
• Automatic test environment - you execute tests in CI/CD.

Quick Start

You can find a simple example here.

Step 1: Install

npm install serverless-spy

Step 2: Include ServerlessSpy in the CDK stack

const serverlessSpy = new ServerlessSpy(this, 'ServerlessSpy', {
  generateSpyEventsFileLocation: 'serverlessSpyEvents/ServerlessSpyEvents.ts'              
});
serverlessSpy.spy();

Step 3: Deploy CDK stack with exporting CloudFormation outputs

cdk deploy --outputs-file cdkOutput.json

The key part of the output is ServerlessSpyWsUrl, which is the URL to the WebSocket where the testing library and web console receive events. Exclude the cdkOutput.json file from git (like you always do), especially if it has secrets.

Apart from CF output, ServerlessSpy generates the TypeScript file serverlessSpyEvents/ServerlessSpyEvents.ts specified in the first step. This makes your tests strongly typed thanks to TypeScript.

Step 4: Write tests Initialize the ServerlessSpyListener

import { ServerlessSpyEvents } from '../serverlessSpyEvents/ServerlessSpyEvents';

let serverlessSpyListener: ServerlessSpyListener<ServerlessSpyEvents>;
beforeEach(async () => {
  serverlessSpyListener =
    await createServerlessSpyListener<ServerlessSpyEvents>({
      serverlessSpyWsUrl: output.ServerlessSpyWsUrl, // ServerlessSpy WebSocket URL from CloudFormation output
    });
});

Write test:

(
  await serverlessSpyListener.waitForSnsTopicMyTopic<TestData>()
).toMatchObject({ message: data });

Close the ServerlessSpyListener

afterEach(async () => {
  serverlessSpyListener.stop();
});

Web Console

The web console runs on your local computer and displays all events that ServerlessSpy intercepts in the environment. That is useful when developing to investigate what is happening in the system. The web console receives events via WebSocket and displays them in the table with a timestamp, event name, and data. You can filter events by event name and data. You can use regular expressions.

If events are hierarchical, like Lambda request & response, you can see an arrow from a parent to child event.

npx sspy --cdkoutput cdkOutput.json

The browser window will open http://localhost:3456/.

cdkOutput.json contains exported CloudFormation outputs. The key output is ServerlessSpyWsUrl, which is the URL to the WebSocket where the testing library and web console receive events. You can also set websocket directly with parameter --ws.

What ServerlessSpy is not

• ServerlessSpy can not be used if your infrastructure is not created with CDK.
• The solution is meant only for the development and (automatic) testing environment. You should exclude ServerlessSpy CDK construct in any other environment, especially a production or a high-load environment. ServerlessSpy is not meant for those; it just induces costs and could contribute to hitting AWS quotas (Lambda concurrent executions, ...).
• Only Node.js stack is supported. There are no plans to support Python or any other. Use of TypeScript is deeply encouraged.
• Web console only runs on your local computer. No cloud hosting of any kind (for now).