Making use of Lambda@Edge outside of us-east-1

19 Nov 2020 (aws, aws-lambda)

Recently I’ve been working on a pet project involving the orbits of the planets. It’s a serverless REST API built with AWS CDK. I won’t talk much about my goals for the project here as I’ll likely make another post going further into that, but if you’re interested you can take a look at the GitHub repo here. Moreover, I was motivated to write this post after running into what must be a fairly common problem when building serverless applications, but for which helpful information was sorely lacking. Hopefully, this article will save someone a bit of time.

The Problem

The problem emerged when I decided to have CloudFront handle requests before they’re sent to the origin, by deciding which underlying API Gateway handler to send the traffic to based on query string values. The logic made sense, and the CDK made it easy to hook up my Python file as a Lambda@Edge handler. That was until I tried to deploy my stack and this error appeared:

Lambda@Edge functions must be created in the us-east-1 region. This shouldn’t have come as a surprise since, as I subsequently found out, this restriction is quite well documented in the CDK docs, and there are some other AWS services with the same restriction, such as ACM. The limitation doesn’t lead to an increase in latency because Lambda@Edge is a feature of CloudFront, so while the resources themselves are stored in us-east-1, replicas of them are automatically distributed across the CloudFront CDN.

A Solution

This is all well and good, but how does one deal with this when their CDK stack is housed in another region? My first thought was to move my entire CDK stack into the us-east-1 region. This would undoubtedly fix the issue, but it wasn’t the most attractive solution since I live in the UK and realistically, so do most of my would-be users.

After some digging through StackOverflow and GitHub issues, I found that spinning up a separate stack to house the Lambda@Edge functions was going to be unavoidable. On the upside, this is quite simple; it’s just a case of making a class that extends cdk.Stack and instantiating it alongside the main stack in bin/cdk.ts.

require('dotenv').config()
import 'source-map-support/register'
import * as cdk from '@aws-cdk/core'
import { PlanetsStack } from '../lib/planets-stack'
import { EdgeStack } from '../lib/edge-stack'

const app = new cdk.App();

const edgeStack = new EdgeStack(app, 'EdgeStack')

new PlanetsStack(app, 'PlanetsStack').addDependency(edgeStack);

Here my edgeStack provisions the Lambda resources to be used for Lambda@Edge. The tricky part is grabbing the relevant Amazon Resource Name (ARN) from your main stack (PlanetsStack) in the other region (in my case eu-west-2).

Storing ARNs in SSM Parameter Store

It’s worth mentioning up front that a dirty but perfectly functioning solution is to just hard code the ARN of your Lambda resource from us-east-1 into your main stack. If that approach solves the issue for your use case, you can safely stop reading here. For no particular reason, I wanted a solution that didn’t involve any hard coding of values; there’s something stinky about specifying an ARN for a resource that may or may not be there down the line.

The solution I found which involved the least overhead was to store the ARNs from us-east-1 in the SSM Parameter Store, where they can then be fetched by a CustomResource in any other region. The code snippets in this GitHub reply worked perfectly for me - many thanks to KurtMar!

export class EdgeStack extends cdk.Stack {
  private customIamRole: iam.Role

  constructor(scope: cdk.Construct, id: string) {
    super(scope, id, {
      env: {
        region: 'us-east-1'
      }
    })

    // Create role for Lambda@Edge functions
    this.customIamRole = new iam.Role(this, 'AllowLambdaServiceToAssumeRole', {
      assumedBy: new iam.CompositePrincipal(
        new iam.ServicePrincipal('lambda.amazonaws.com'),
        new iam.ServicePrincipal('edgelambda.amazonaws.com')
      ),
      managedPolicies: [iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole')]
    })

    // Make Lambda@Edge functions
    const edgeFunctions: Dict<string> = {
      'QueryToID': this.makeEdgeFunction('QueryToID', 'query_to_id'),
      'StripAPIPath': this.makeEdgeFunction('StripAPIPath', 'strip_api_path')
    }

    // Export an SSM Parameter for each function
    Object.keys(edgeFunctions).forEach(key => new ssm.StringParameter(this, `${key}ARN`, {
      parameterName: `/PlanetsAPI/${key}ARN`,
      description: `CDK parameter from ${key} Lambda@Edge function`,
      stringValue: edgeFunctions[key]
    }))
  }

  private makeEdgeFunction(name: string, dir: string): string {
    return new lambda.PythonFunction(this, name, {
      runtime: PYTHON_RUNTIME,
      entry: path.join(__dirname, '..', 'lambdas', 'edge', dir),
      handler: 'handler',
      role: this.customIamRole
    }).currentVersion.functionArn
  }
}

Here I create a basic execution role and add it as a property so it can be shared by all the Lambdas. I had two Lambdas to bring in, so I made the makeEdgeFunction method as an abstraction to remove bloat from the constructor. I then iterate over each entry in the edgeFunctions object and create an ssm.StringParameter for each one.

Reading SSM Parameters in PlanetsStack

The next step is to make a call from our main stack to fetch the values saved in SSM.

export class EdgeHandler extends cdk.Construct {
  public readonly edgeFunctions: Dict<lambda.IVersion>
  private stack: cdk.Stack

  constructor(scope: cdk.Construct, id: string) {
    super(scope, id)

    this.stack = scope as cdk.Stack

    this.edgeFunctions =
      ['QueryToID', 'StripAPIPath']
      .reduce((acc: Dict<lambda.IVersion>, name: string) => {
        const resource = this.loadParameter(`${name}ARN`)
        const arn = resource.getResponseField('Parameter.Value')

        return {
          ...acc,
          [name]: lambda.Version.fromVersionArn(this, `${name}EdgeVersion`, arn)
        }
      }, {})
  }

  private loadParameter(name: string): cr.AwsCustomResource {
    return new cr.AwsCustomResource(this, `Get${name}Parameter`, {
      policy: cr.AwsCustomResourcePolicy.fromStatements([
        new iam.PolicyStatement({
          effect: iam.Effect.ALLOW,
          actions: ['ssm:GetParameter*'],
          resources: [
            this.stack.formatArn({
              service: 'ssm',
              region: 'us-east-1',
              resource: `parameter/PlanetsAPI/${name}`
            })
          ]
        })
      ]),
      onUpdate: {
        service: 'SSM',
        action: 'getParameter',
        parameters: {
          Name: `/PlanetsAPI/${name}`
        },
        region: 'us-east-1',
        physicalResourceId: cr.PhysicalResourceId.of(Date.now().toString())
      }
    })
  }
}

Since there’s quite a sizable chunk of code responsible for accessing the parameter store, I built an EdgeHandler class to deal with any and all cross-region stuff.

In the constructor I build up a dictionary of Lambda@Edge functions by calling the loadParameter method for each one. This method returns a unique AwsCustomResource containing the value from SSM. Back in the constructor, I call getResponseField('Parameter.Value') on the returned resource in order to get the ARN as a string. Since this dictionary is a public property on the class, its entries will be accessible from our main stack.

Putting it all together

Finally, we want to plug the Lambda@Edge functions into a CloudFront distribution.

export class PlanetsStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string) {
    super(scope, id);

    const edgeHandler = new EdgeHandler(this, 'EdgeHandler')

    // Set up API Gateway
    const api = new PlanetsAPI(this, 'PlanetsAPI')

    const distribution = new cf.CloudFrontWebDistribution(this, 'CFDistribution', {
      originConfigs: [
        {
          customOriginSource: {
            domainName: `${api.api.httpApiId}.execute-api.${this.region}.${this.urlSuffix}`
          },
          behaviors: [
            {
              pathPattern: '/api/visible',
              allowedMethods: CloudFrontAllowedMethods.GET_HEAD_OPTIONS,
              defaultTtl: cdk.Duration.minutes(CACHE_TTL_MINUTES),
              forwardedValues: {
                queryString: true
              },
              lambdaFunctionAssociations: [
                {
                  eventType: cf.LambdaEdgeEventType.VIEWER_REQUEST,
                  lambdaFunction: edgeHandler.edgeFunctions.QueryToID
                },
                {
                  eventType: cf.LambdaEdgeEventType.ORIGIN_REQUEST,
                  lambdaFunction: edgeHandler.edgeFunctions.StripAPIPath
                }
              ]
            },
            {
              pathPattern: '/api/*',
              allowedMethods: CloudFrontAllowedMethods.GET_HEAD_OPTIONS,
              defaultTtl: cdk.Duration.minutes(CACHE_TTL_MINUTES),
              forwardedValues: {
                queryString: true
              },
              lambdaFunctionAssociations: [
                {
                  eventType: cf.LambdaEdgeEventType.ORIGIN_REQUEST,
                  lambdaFunction: edgeHandler.edgeFunctions.StripAPIPath
                }
              ]
            }
          ]
        }
      ]
    })
  }
}

In my PlanetsStack constructor, I instantiate the EdgeHandler class which will load the SSM parameter values into memory. I then set up my CloudFront distribution, where I define two behaviours - one for /api/visible and one for all other /api routes. It’s then as simple as referencing the appropriate entry from edgeHandler.edgeFunctions for each item in the lambdaFunctionAssociations array.

Summary

To summarise, Lambda@Edge functions can only be created in the us-east-1 region. In order to get around this restriction, we’ve created two CDK stacks - EdgeStack in us-east-1 and PlanetsStack in some other region, and we’ve used the SSM Parameter Store to save the ARNs of our Lambda@Edge functions from EdgeStack. Then in PlanetsStack we’ve used AwsCustomResource to read the saved values from the parameter store, and reference them in a CloudFront distribution.

It’s not the most elegant solution, but it’s also not too bad considering we’d now be able to reference from any region just about anything that can be serialised. Maybe something like this will eventually be included in the CDK Construct library, but for now, hopefully this is helpful.