Understanding API Gateway Payload Mappings

Amazon’s API Gateway provides the facilities to map an incoming request’s payload to match the required format of an integration backend.

API Gateway uses the concept of “models” and “mapping templates” to specify the mapping between the client payload and the server payload.

Models

A model defines the structure of the incoming payload using JSON Schema. The model is an optional, but not required, piece of API Gateway. By providing a model, you make it easier to define the upcoming mapping template that actually does the transformation between the client and server.

For example, we can define a incoming Player model using the following JSON payload.

{
    "playerId": "1234567890",
    "alias": "soofaloofa",
    "displayName": "Kevin Sookocheff",
    "profilePhotoUrl": "https://api.example.com/player/1234567890/avatar.png"
}

Working With JSON Schema

The API Gateway model for this JSON payload is described using JSON Schema. JSON Schema is a vocabulary allowing you validate JSON documents. The particular JSON payload being validated is called an instance, and the document describing what a valid payload looks like is called the schema.

JSON Schema is used to describe a JSON document, so it helps to understand what, exactly, a JSON document is composed of these primitives:

objects: {"field1": "value1", "field2": "value2"}
arrays: ["first", "second", "third"]
numbers: 42 or 3.1415926
strings: "Lorem ipsum dolor sit amet"
booleans: true or false
null: null

The responsibility of JSON Schema is to describe a JSON document built from the these primitives.

JSON Schema is itself specified using JSON with predefined semantics for keywords and values. Going back to our Player example, we can specify the JSON Schema for the incoming document as follows:

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "type": "object",
    "properties": {
        "playerId": { "type": "string" },
        "alias": { "type": "string" },
        "displayName": { "type": "string" },
        "profilePhotoUrl": { "type": "string" }
    }
}

This JSON Schema follows the Draft 4 specification and simply declares the types expected for each field in the request. For now, we simply declare a root type for the payload as object and allow the string type for each incoming field. You could choose to further restrict each string to be of a certain length, to match a regular expression, or to be of a certain format, such as date-time, or email. For an excellent guide on on JSON Schema, consult this guide.

Mapping Templates

The API Gateway mapping templates are used to transform an incoming payload into a different format. API Gateway allows you to define input mapping templates, for mapping the incoming request from a client to a server format, and output mapping templates, for mapping the outgoing response from the server to a client format. The mappings are defined using the Velocity Template Language combined with JSONPath expressions.

Working With the Velocity Template Language

Velocity is a Java-based templating engine that uses a template language to reference objects in Java code. Velocity can be used to generate web XML, HTML or SQL from templates, for example. Here, we will use it to define the mapping between an input and output model.

A Velocity Template Language (VTL) template is composed of statements which begin with the # character and a followed by a directive. When the VTL engine is run on your template, it searches for all # characters to find any that match a VTL directive. For example, the following VTL statement contains the set directive.

#set( $a = "Velocity")

VTL contains a number of directives that can be used to setting values, running conditional statements, and looping, among others.

The second major component of VTL is references, which begin with the $ character. More generally, references begin with $ and are used to get or set something. Directives begin with # and are used to do something.

We can use the statement above to define an HTML template that sets a reference to a value, and retrieves that reference.

<html>
  <body>
  #set( $foo = "Velocity" )
  Hello $foo World!
  </body>
</html>

For more on VTL, consult the user guide or the language reference.

Working With JSONPath

In API Gateway mapping templates, the input to your Velocity template is defined by the reference variable $input. The API Gateway runtime provides this value to your template for use. To access particular portions of the input JSON document within the template, you use JSONPath.

JSONPath provides an XPath-like method of referring to a JSON structure. IN JSONPath, the abstract name $ refers to the root-level object. From this root-level object, you can use dot-notation to select more restrictive portions of the document. For example, to reference the playerId of the player model, you would use the following JSONPath expression.

$.playerId

or, you can use a bracket-notation to perform the same selection.

#['playerId']

More complex selections can be done using * for a wildcard, @ to refer to the current node in the document, array indexing, or recursive descent, for example. The full reference to JSONPath is available here and a Java implementation is available here.

Putting Together a Mapping Template

Now, we can put together a mapping template for our Player model. This template will just copy the input to the output with new names for field properties. In particular, it will convert an input document of

{
    "playerId": "1234567890",
    "alias": "soofaloofa",
    "displayName": "Kevin Sookocheff",
    "profilePhotoUrl": "https://api.example.com/player/1234567890/avatar.png"
}

to an output document of

{
    "id": "1234567890",
    "alias": "soofaloofa",
    "name": "Kevin Sookocheff",
    "photo": "https://api.example.com/player/1234567890/avatar.png"
}

The mapping to perform the above transformation is specified below, with some inline VTL comments added.

## Set the variable inputRoot to the root JSON document
#set($inputRoot = $input.path('$')) {

    ## Assign output properties to input properties
    "id": "$inputRoot.playerId",
    "alias": "$inputRoot.alias",
    "name": "$inputRoot.displayName",
    "photo": "$inputRoot.profilePhotoUrl"
}

The API Gateway also provides a set of helper objects and functions that you can use within your VTL template. You can find the complete reference here.

A Complete Mapping Pipeline

Putting all of these technologies together allows you to define the complete payload mapping for an API. First, define the input and output JSON Schema’s describing your expected payloads. This step is not strictly necessary but helps in defining a mapping template that is consistent with the expected payload. Second, define your template mapping using VTL. The API Gateway will then enforce that incoming requests match your expected request schema, apply any template mappings, and enforce that outgoing response match the expected response schema. Together, this allows for flexible mapping between a client representation and a server representation.

Models#

Working With JSON Schema#

Mapping Templates#

Working With the Velocity Template Language#

Working With JSONPath#

Putting Together a Mapping Template#

A Complete Mapping Pipeline#