OpenAPI + Zod + API Gateway + SST = Turbo Productivity

"Don't repeat yourself" (DRY) is one of the main principles of software development. But we keep on breaking it by rewriting the data structure on each layer. In this article, we'll explore the method of enhancing our productivity through the utilization of OpenAPI schema, the Zod library, and the generation of client code, all aimed at reusing our types, schemas, and structures. l will bundle everything into a sample AWS serverless application built with the SST framework (which uses CDK). The solution also takes into account that you do not want to have exactly the same types everywhere. The Zod library nicely solves the problem and allows you to mix and match types.

When you are creating an API and an application that consumes it, you need to (re)create types/schema for each layer:

• data exchange = OpenAPI, GraphQL
• client that consume it
• API validation logic
• business logic

Developers who dislike typed languages would argue that you do not need types anyway. But regardless, you always need some form of structure, which you end up recreating on each layer. Therefore, that argument doesn't really hold up.

You can find the solution here.

OpenAPI for Defining API

OpenAPI (formerly known as Swagger) is excellent for defining your REST API. GraphQL promotes its schema as a significant advantage, but it's interesting to know that OpenAPI was around five years before GraphQL.

With a defined schema, you let the consumer of your API know what requests you expect and what is expected to be returned. Based on the schema, they can generate code for the client. If you use TypeScript, that brings you code completion in your code editor and type checking. You can also import schema into some of the tools that you use to evaluate the REST API, such as Postman.

Additionally, it's a good idea to add a user interface to the schema. Reading YAML or JSON can be difficult. It is much nicer to look at UI, which helps you visualize the endpoints and types, and some offer to test the API.

Zod, a Magical Boost to TypeScript

Zod is an excellent library for defining types and validation. It can be reused for defining OpenAPI schema. Zod is an easy-to-use library and very flexible. It does not require much/any learning, and the benefits are instantly visible.

Let’s define a customer:

export const Customer = z.object({
  id: z.string().uuid(),
  name: z.string(),
  surname: z.string(),
  email: z.string().email(),
});

export type Customer = z.infer<typeof Customer>;

That's the same as you would write:

export type Customer = {
  id: string;
  name: string;
  surname: string;
  email: string;
};

You would use the Customer type the same way as you used to. You can also use it for validation:

const validationResult = Customer.safeParse({
  id: "123e4567-e89b-12d3-a456-426614174000",
  name: "John",
});

You may want to have almost the same type for updates but without the id because it will come as a path parameter. So we can remove that property:

export const UpdateCustomer = Customer.omit({
  id: true,
});

export type UpdateCustomer = z.infer<typeof UpdateCustomer>;

This becomes:

export type Customer = {
  name: string;
  surname: string;
  email: string;
};

This way, we can mix and match types. Even if the types are not exactly the same, we can reuse our work.

You can use Zod to define OpenAPI. For that, we will use an amazing, full-featured, battle-tested library @asteasolutions/zod-to-OpenAPI.

With it, we can extend our type with additional properties specific to OpenAPI (or not; you do not have to):

import { z } from "zod";
import { extendZodWithOpenApi } from "@asteasolutions/zod-to-openapi";

extendZodWithOpenApi(z);

export const Customer = z.object({
  id: z.string().uuid().openapi({
    description: "The customer's id",
    example: "123e4567-e89b-12d3-a456-426614174000",
  }),
  name: z
    .string()
    .openapi({ description: "The customer's name", example: "John" }),
  surname: z
    .string()
    .openapi({ description: "The customer's surname", example: "Doe" }),
  email: z
    .string()
    .email()
    .openapi({ description: "The customer's email", example: "john@doe.com" }),
});

export type Customer = z.infer<typeof Customer>;

Now, we also have to define the OpenAPI endpoint:

const registry = new OpenAPIRegistry();
const UpdateCustomerRequestSchema = registry.register(
  "UpdateCustomerRequest",
  UpdateCustomer
);
const UpdateCustomerParamsSchema = registry.register(
  "UpdateCustomerParams",
  UpdateCustomerParams
);
const UpdateCustomerQueryParamesSchema = registry.register(
  "UpdateCustomerQueryParams",
  UpdateCustomerQueryParams
);
const UpdateCustomerResponseSchema = registry.register(
  "UpdateCustomerResponse",
  Customer
);

registry.registerPath({
  method: "put",
  path: "/customer/{id}",
  summary: "PUT /customer/{id}",
  description: "Update customer",
  request: {
    params: UpdateCustomerParamsSchema,
    query: UpdateCustomerQueryParamesSchema,
    body: {
      content: {
        "application/json": {
          schema: UpdateCustomerRequestSchema,
        },
      },
    },
  },
  responses: {
    200: {
      description: "Customer updated",
      content: {
        "application/json": {
          schema: UpdateCustomerResponseSchema,
        },
      },
    },
    400: {
      description: "Validation errors",
    },
  },
});

You would write almost the same code in YAML or JSON when writing OpenAPI schema by hand. There are an additional few lines to serve that, which you can check in the repository.

UI for Viewing OpenAPI

We’ve come this far. It would be a shame if we had to view raw YAML or JSON. There are several free UI available. I found RapiDoc, ReDoc and Swagger UI really easy to host.

We will use RapiDoc because it allows us to test the API inside the UI and has a nice interface. All we have to do is serve this HTML, which we can do with Lambda:

<!doctype html>
<html>

<head>
  <meta charset="utf-8">
  <script type="module" src="https://unpkg.com/rapidoc/dist/rapidoc-min.js"></script>
</head>

<body>
  <rapi-doc spec-url="./openapi.yaml" theme="light"> </rapi-doc>
</body>

</html>

And we get UI like this:

REST or HTTP API Gateway

AWS offers two variants of API Gateway: the old-named REST and the new-named HTTP. Ignore the strange naming as both offer to create REST services over HTTP protocol. There are many differences between services, but for us, the main difference is that the new HTTP API Gateway does not support the validation of OpenAPI schema. You have to validate it in your Lambda if, of course, Lambda is the target service.

REST API Gateway

As mentioned above, REST API Gateway supports out-of-the-box OpenAPI schema validation. This is especially useful if the target service is not Lambda.

You can start with loading schema in the AWS console and continue to build the system from there. That is okay for experimenting, but you may want to build this with IaC tools such as CloudFormation, CDK, SST... Due to the strange design of CloudFormation, that is a challenge on its own. You can do this by including API Gateway-specific tags in the schema or by building all the models from scratch. Both solutions are terrible. Luckily, Alma Media created a great CDK construct @alma-cdk/OpenAPIx. It enables you to load the schema and add appropriate Lambda functions.

const openApiYaml = yaml.stringify(
  OpenApiService.createOpenApiSchema([], "rest")
);

const restApi = new openapix.Api(stack, "rest-api", {
  source: openapix.Schema.fromInline(openApiYaml),
  validators: {
    all: {
      validateRequestBody: true,
      validateRequestParameters: true,
      default: true,
    },
  },
  paths: {    
    "/customer/{id}": {
      put: new openapix.LambdaIntegration(stack, updateCustomerFunction),
    },
  },
});

However, there are some other downsides to the validation of OpenAPI inside the API Gateway. The default error message does not provide any information about the error. You can improve it by adding Gateway Responses template $context.error.validationErrorString. Check the code for more details. But I found a strange problem. Gateway Responses keep on disappearing for me. If I remove and add them in CDK, they reappear. However, the validation error message is still not that useful for the front-end if you want to use it there. The Zod library, as we will see, provides a better error response with a path to the property. You can reuse that response in the front-end.

The REST API Gateway also offers validation of the parameters, but just that they are included, not if they match the schema. REST API Gateway also does not validate the response.

So there are quite a lot of downsides to using REST API Gatewayfor validating schema. In addition to that, it is much more expensive than the HTTP one.

HTTP API Gateway

HTTP API Gateway does not support validation, so we have to validate it ourselves using Zod:

const validationResult = Customer.safeParse({
  id: "123e4567-e89b-12d3-a456-426614174000",
  name: "John",
});

The response provides an error list. Each error has a path to the property. You can easily reuse this on the front-end side if you return errors as a response.

Keep in mind that the Zod library does not exactly perform validation on OpenAPI schema. It does validation based on the definition of types. It is almost the same. If you want OpenAPI schema validation, use Ajv JSON schema validator. It is also a bit faster than Zod.

Types All the Way

Since we’ve already defined the schema, we want to avoid calling Zod validation in every handler. I created a helper function that does that for you.

It validates:

• body
• path parameters
• query parameters
• response, including the response code and content type

This is how simple our Lambda handler looks:

export const handler = openApiLambdaHttpHandler<
  UpdateCustomer,
  UpdateCustomerParams,
  UpdateCustomerQueryParams
>(async (request) => {
  const customer = {
    id: request.pathParameters.id, // pathParameters is type of UpdateCustomerParams
    ...request.body, // body is type of UpdateCustomer
  };

  console.log("Customer updated", customer);
  console.log(
    "Source",
    request.queryParameters.source // queryParameters is type of UpdateCustomerQueryParams
  );

  return {
    statusCode: 200,
    body: customer,
    headers: {
      "Content-Type": "application/json",
    },
  };
});

Check out the helper here.

I also created the version for REST API Gateway. It does not validate the body since API Gateway can do that. And the response error in the schema is just one string to match the default response of REST API Gateway.

Generating Client Code

Based on the OpenAPI schema, we can generate client code. There are numerous libraries to do that, but I found the openapi-typescript-codegen the most fully-featured and easy to use.

The command:

openapi --input ${API_URL}/open API.yaml --name Api --postfixServices Api --indent 2 --output common/api

The actual command I used for generating the client is a bit different. I took advantage of the SST Config feature to share variables so we do not have to type API URLs. Peek into the script in package.json and file `generateOpenAPIClient.js˙ to find out more.

Integration tests

The common folder has become another part of our monorepo and can be used by a future front-end. Our application does not have a front-end. We will use that generated code in our integration tests and simulate a client. This way, we can validate not only logic but also that the schema was written correctly.

Our sample test:

describe("customer endpoint", () => {
  const apiClient = new Api({
    BASE: Config.HTTP_API,
  }).default;

  test("update customer - valid", async () => {
    const customerId = "d290f1ee-6c54-4b01-90e6-d701748f0851";
    const customer: UpdateCustomerRequest = {
      name: "John",
      surname: "Doe",
      email: "john@doe.com",
    };

    const response = await apiClient.putCustomer(customerId, "web", customer);

    expect(response).toEqual({
      id: customerId,
      ...customer,
    });
  });
});

We would also like to test if validation works:

test("update customer - invalid schema", async () => {
  const customerId = "d290f1ee-6c54-4b01-90e6-d701748f0851";
  const customer = {
    name: "John",
    email: "john@doe.com",
  };

  await expect(
    apiClient.putCustomer(customerId, "web", customer as any)
  ).rejects.toMatchObject({
    body: [
      {
        path: ["surname"],
        message: "Required",
      },
    ],
  });
});

In normal front-end you would get schema validation errors like this:

const customerId = "d290f1ee-6c54-4b01-90e6-d701748f0851";
const customer = {
  name: "John",
  email: "john@doe.com",
};
try {
  const response = await apiClient.putCustomer(
    customerId,
    "web",
    customer as any
  );
} catch (e: any) {
  if (e instanceof ApiError) {
    const errors = e.body as ValidationErrors;
    console.log("Errors", errors);
    // [{
    //     path: ["surname"],
    //     message: "Required",
    //  }];
  } else {
    throw e;
  }
}

Summary

Productivity gets a major boost when we implement all these simple techniques together. The code is cleaner, easier to maintain, and there are a lot fewer bugs. It is also much easier to refactor the application. Creating Zod types and autogenerating schema has become, for me, a base for brainstorming at the beginning of the development. The OpenAPI schema with the included user interface is a fantastic way to talk and share information with other developers, team members, and sometimes even with the client.