Docker Containers with CDK Pipelines

Deploy a Node.js and Redis Container onto ECS with CDK Pipelines

Updated: 03 September 2023

Prior to doing any of the below you will require your ~/.aws/credentials file to be configured with the credentials for your AWS account

A good reference for this is also the AWS Workshop Docs and the AWS Advanced Workshop Docs

Create CDK App

First, create a directory for your app and then cd into it:

1
mkdir cdk-pipeline-docker
2
cd cdk-pipeline-docker

We’re going to be using the CDK CLI to setup our application, to do this we can use npx:

1
npx cdk init --language=typescript

Then, remove the package.lock.json so we can then swap the app over to use yarn with:

1
rm package.lock.json
2
yarn

Note that we’ll be adding npm packages as we need them instead of all at once

Now, do git init and push the application up to GitHub as the pipeline will source the code from there

Add our Application Files

Before we jump right into the CDK and pipeline setup, we need an application to containerize. We’re going to create a simple Node.js app which uses express and redis

Create the app directory in the root of our CDK app, init, and add the required dependencies

1
mkdir app
2
cd app
3
yarn init -y
4
yarn add express redis

Since the TypeScript CDK app is setup to ignore .js files by default, we want to create a .gitignore file in our app directory with the following:

app/.gitignore

1
!*.js
2
node_modules

Then, add an index.js file with the following:

app/index.js

1
const express = require('express')
2
const redis = require('redis')
3

4
const port = process.env.PORT || 8080
5
const redisUrl = process.env.REDIS_URL || 'redis://redis:6379'
6

7
const app = express()
8

9
app.use(express.text())
10

11
const client = redis.createClient({
12
  url: redisUrl,
13
})
14

15
client.on('error', function (error) {
16
  console.error(error)
17
})
18

19
app.get('/', (req, res) => {
20
  console.log('request at URL')
21
  res.send('hello from port ' + port)
22
})
23

24
app.get('/:key', (req, res) => {
25
  const key = req.params.key
26
  client.get(key, (error, reply) => {
27
    if (error) res.send('Error')
28
    else res.send(reply)
29
  })
30
})
31

32
app.post('/:key', (req, res) => {
33
  const key = req.params.key
34
  const data = req.body
35
  client.set(key, data, (error, reply) => {
36
    if (error) res.send('Error')
37
    else res.send(reply)
38
  })
39
})
40

41
app.listen(port, () => {
42
  console.log('app is listening on port ' + port)
43
})

The above consists a simple app which will use service discovery to connect to Redis and create/retreive values based on their key

Next, add a Dockerfile for this application:

app/Dockerfile

1
FROM node:14
2

3
COPY package.json .
4
COPY yarn.lock .
5
RUN yarn --frozen-lockfile
6

7
COPY . .
8

9
EXPOSE 8080
10
CMD ["yarn", "start"]

And that should be everything we need to do at this point in terms of the application itself - after all, using Redis with Node.js not the focus of this doc

Setup

A CDK Pipeline consists of a few different stage, namely:

1
graph TD
2
  Source --> Build --> UpdatePipeline --> PublishAssets
3
  PublishAssets --> Stage1 --> Stage2 --> etc

Pipeline Stack

To define a pipeline we use the @aws-cdk/core package as the base, create a lib/pipeline-stack.ts file in which we’ll define a Stack which represents our deployment pipeline:

lib/pipeline-stack.ts

1
import * as cdk from '@aws-cdk/core'
2

3
export class PipelineStack extends cdk.Stack {
4
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
5
    super(scope, id, props)
6

7
    // Pipeline code goes here
8
  }
9
}

Then, instantiate this stack update the bin/pipeline.ts to have the following:

bin/pipeline.ts

1
#!/usr/bin/env node
2
import * as cdk from '@aws-cdk/core'
3
import { PipelineStack } from '../lib/pipeline-stack'
4

5
const app = new cdk.App()
6
new PipelineStack(app, 'CdkPipelineDocker')

Then reference this from your cdk.json file in the root directory:

1
"app": "npx ts-node --prefer-ts-exts bin/pipeline.ts"

And also add the following to the context section of your cdk.json file:

cdk.json

1
"@aws-cdk/core:newStyleStackSynthesis": true

Pipeline Account Permissions

First, install the cdk CLI at a project level with:

1
yarn add aws-cdk

The reason for this is to ensure we use a version of the cdk that was installed for our specific application and we aren’t accidentally using something that maybe exists somewhere else on our computer

And then add the following to the scripts section of your package.json file:

package.json

1
"scripts" {
2
  // ... other scripts
3
  "cdk": "cdk"
4
}

Before we can use the pipelines we need to grant CDK some permissions to our account, we can do this with:

1
yarn cdk bootstrap --cloudformation-execution-policies arn:aws:iam::aws:policy/AdministratorAccess

The above will create a CDKToolkit stack which you will be able to see in AWS’s CloudFormation Console

GitHub Repo Permissions

We need to provide AWS with credentials to our GitHub repo. To do this go to GitHub > Settings > Developer settings > Personal access tokens and create a token with access to repo and admin:repo_hook permissions

Then add the token to AWS’s Secrets Manager via the console with a plaintext value of the token you just generated above, then name the token github-token and complete the rest of the steps to store the new secret

Develop the Pipeline

Now that we’ve got most of the scaffolding in place, we need to actually deploy our pipeline to AWS so that it’s aware of the codebase and everything else it needs to hook into our git repo for the building and deployment of our project

We need to install some of the cdk libraries packages, we can do this with yarn:

1
yarn add @aws-cdk/aws-codepipeline @aws-cdk/pipelines @aws-cdk/aws-codepipeline-actions

Then we can use these packages in the pipeline-stack.ts file we’re going to add the following imports:

lib/pipeline-stack.ts

1
import * as cdk from '@aws-cdk/core'
2
import { Stack, Construct, StackProps, SecretValue } from '@aws-cdk/core'
3
import { Artifact } from '@aws-cdk/aws-codepipeline'
4
import { CdkPipeline, SimpleSynthAction } from '@aws-cdk/pipelines'
5
import { GitHubSourceAction } from '@aws-cdk/aws-codepipeline-actions'

Next up, we’re going to be writing everything else within the PipelineStack we defined earlier:

lib/pipeline-stack.ts

1
export class PipelineStack extends cdk.Stack {
2
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
3
    super(scope, id, props)
4

5
    // Pipeline code goes here
6
  }
7
}

First, we need to create sourceArtifact and cloudAssemblyArtifact instances for the pipeline:

1
const sourceArtifact = new Artifact()
2
const cloudAssemblyArtifact = new Artifact()

Then, we define the sourceAction which is how the pipeline neeeds to get our code from our repository. In this case we use the GitHubSourceAction. We use the SecretValue.secretsManager function to retreive the GitHub token we created previously:

1
const sourceAction = new GitHubSourceAction({
2
  actionName: 'GitHubSource',
3
  output: sourceArtifact,
4
  oauthToken: SecretValue.secretsManager('github-token'),
5
  owner: 'username',
6
  repo: 'repository',
7
  branch: 'main',
8
})

Ensure you’ve replaced the owner, repo and branch with the one that contains your code on GitHub

Then, we define the synthAction which is used to install dependencies and optionally run a build of our app:

1
// will run yarn install --frozen-lockfile, and then the buildCommand
2
const synthAction = SimpleSynthAction.standardYarnSynth({
3
  sourceArtifact,
4
  cloudAssemblyArtifact,
5
  buildCommand: 'yarn build',
6
})

And lastly, we combine these to create a CdkPipeline instance:

1
const pipeline = new CdkPipeline(this, 'Pipeline', {
2
  cloudAssemblyArtifact,
3
  sourceAction,
4
  synthAction,
5
})

So our overall lib/pipeline-stack will now look like this:

1
import * as cdk from '@aws-cdk/core'
2
import { Artifact } from '@aws-cdk/aws-codepipeline'
3
import { CdkPipeline, SimpleSynthAction } from '@aws-cdk/pipelines'
4
import { GitHubSourceAction } from '@aws-cdk/aws-codepipeline-actions'
5

6
export class PipelineStack extends cdk.Stack {
7
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
8
    super(scope, id, props)
9
    const sourceArtifact = new Artifact()
10
    const cloudAssemblyArtifact = new Artifact()
11

12
    // clone repo from GtiHub using token from secrets manager
13
    const sourceAction = new GitHubSourceAction({
14
      actionName: 'GitHubSource',
15
      output: sourceArtifact,
16
      oauthToken: cdk.SecretValue.secretsManager('github-token'),
17
      owner: 'username',
18
      repo: 'repository',
19
      branch: 'main',
20
    })
21

22
    // will run yarn install --frozen-lockfile, and then the buildCommand
23
    const synthAction = SimpleSynthAction.standardYarnSynth({
24
      sourceArtifact,
25
      cloudAssemblyArtifact,
26
      buildCommand: 'yarn build',
27
    })
28

29
    const pipeline = new CdkPipeline(this, 'Pipeline', {
30
      cloudAssemblyArtifact,
31
      sourceAction,
32
      synthAction,
33
    })
34
  }
35
}

Next, initialize the Pipeline in AWS by using yarn cdk deploy. This should be the only manual deploy we need. From this point all other Pipeline runs will happen directly in CDK via GitHub Commits:

1
yarn cdk deploy

Add App to Deployment

To create deployments we need to have a class that inherits from cdk.Stage, in this Stage we specify all the requisites for an application deployment. We’re deploying the AppStack application, we will reference it from a Stage called AppStage which will just create an instance of the application:

lib/app-stage.ts

1
import * as cdk from '@aws-cdk/core'
2
import { AppStack } from './app-stack'
3

4
export class AppStage extends cdk.Stage {
5
  constructor(scope: cdk.Construct, id: string, props?: cdk.StageProps) {
6
    super(scope, id, props)
7

8
    new AppStack(this, 'AppStack')
9
  }
10
}

We can then add the above AppStage to the pipeline-stack using the pipeline.addApplicationStage function:

lib/pipeline-stack.ts

1
// ... other pipeline code
2

3
// CdkPipeline as previously created
4
const pipeline = new CdkPipeline(this, 'Pipeline', {
5
  cloudAssemblyArtifact,
6
  sourceAction,
7
  synthAction,
8
})
9

10
// adding app stage to the deployment
11
const appStage = new AppStage(this, 'Dev')
12

13
pipeline.addApplicationStage(appStage)

Once all that’s been added, the final pipeline-stack.ts file will have the following:

1
import * as cdk from '@aws-cdk/core'
2
import { Artifact } from '@aws-cdk/aws-codepipeline'
3
import { CdkPipeline, SimpleSynthAction } from '@aws-cdk/pipelines'
4
import { GitHubSourceAction } from '@aws-cdk/aws-codepipeline-actions'
5
import { AppStage } from './app-stage'
6

7
export class PipelineStack extends cdk.Stack {
8
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
9
    super(scope, id, props)
10

11
    const sourceArtifact = new Artifact()
12
    const cloudAssemblyArtifact = new Artifact()
13

14
    // clone repo from GtiHub using token from secrets manager
15
    const sourceAction = new GitHubSourceAction({
16
      actionName: 'GitHubSource',
17
      output: sourceArtifact,
18
      oauthToken: cdk.SecretValue.secretsManager('github-token'),
19
      owner: 'username',
20
      repo: 'repository',
21
      branch: 'main',
22
    })
23

24
    // will run yarn install --frozen-lockfile, and then the buildCommand
25
    const synthAction = SimpleSynthAction.standardYarnSynth({
26
      sourceArtifact,
27
      cloudAssemblyArtifact,
28
      buildCommand: 'yarn build',
29
    })
30

31
    const pipeline = new CdkPipeline(this, 'Pipeline', {
32
      cloudAssemblyArtifact,
33
      sourceAction,
34
      synthAction,
35
    })
36

37
    const app = new AppStage(this, 'Dev')
38

39
    pipeline.addApplicationStage(app)
40
  }
41
}

App Stack

Since our app will use a Docker container we need to install the @aws-cdk/aws-ecs, @aws-cdk/aws-ec2 and @aws-cdk/aws-ecs-patterns packages:

1
yarn add @aws-cdk/aws-ecs @aws-cdk/aws-ecs-patterns

Next, from our lib/app-stack.ts file, we want to create two services:

A Docker service which builds a locally defined Docker image
A Docker service which runs the public redis image

In order to define our servivce, we need a vpc, cluster, and some image information and configuration

Importing everything required we would have the following as our AppStack:

lib/app-stack.ts

1
import * as cdk from '@aws-cdk/core'
2
import * as ec2 from '@aws-cdk/aws-ec2'
3
import * as ecs from '@aws-cdk/aws-ecs'
4
import * as ecsPatterns from '@aws-cdk/aws-ecs-patterns'
5
import * as ecrAssets from '@aws-cdk/aws-ecr-assets'
6

7
export class AppStack extends cdk.Stack {
8
  constructor(scope: cdk.Construct, id: string, props?: cdk.StageProps) {
9
    super(scope, id, props)
10

11
    // constructs go here
12
  }
13
}

Our applications need a VPC and Cluster in which they will run, we can define a vpc with:

lib/app-stack.ts

1
const vpc = new ec2.Vpc(this, 'AppVPC', {
2
  maxAzs: 2,
3
})

And a cluster:

lib/app-stack.ts

1
const cluster = new ecs.Cluster(this, 'ServiceCluster', { vpc })

The cluster requires a CloudMapNamespace to enable service discovery. This will allow other containers and application within the Cluster to connect to one another using the service name with the service namespace

lib/app-stack.ts

1
cluster.addDefaultCloudMapNamespace({ name: this.cloudMapNamespace })

Using the cluster above, we can create a Task and Service using the NetworkLoadBalancedFargateService as defined in the aws-ecs-patterns library

Defining the appService involves the following steps:

Defining the App as a Docker Asset

lib/app-stack.ts

1
const appAsset = new ecrAssets.DockerImageAsset(this, 'app', {
2
  directory: './app',
3
  file: 'Dockerfile',
4
})

Defining the App Task

lib/app-stack.ts

1
const appTask = new ecs.FargateTaskDefinition(this, 'app-task', {
2
  cpu: 512,
3
  memoryLimitMiB: 2048,
4
})

Adding a Container Definition to the Task

lib/app-stack.ts

1
appTask
2
  .addContainer('app', {
3
    image: ecs.ContainerImage.fromDockerImageAsset(appAsset),
4
    essential: true,
5
    environment: { REDIS_URL: this.redisServiceUrl },
6
    logging: ecs.LogDrivers.awsLogs({
7
      streamPrefix: 'AppContainer',
8
      logRetention: logs.RetentionDays.ONE_DAY,
9
    }),
10
  })
11
  .addPortMappings({ containerPort: this.appPort, hostPort: this.appPort })

Create a Service

lib/app-stack.ts

1
const appService = new ecsPatterns.NetworkLoadBalancedFargateService(
2
  this,
3
  'app-service',
4
  {
5
    cluster,
6
    cloudMapOptions: {
7
      name: 'app',
8
    },
9
    cpu: 512,
10
    desiredCount: 1,
11
    taskDefinition: appTask,
12
    memoryLimitMiB: 2048,
13
    listenerPort: 80,
14
    publicLoadBalancer: true,
15
  }
16
)

Enable Public connections to the serive

lib/app-stack.ts

1
appService.service.connections.allowFromAnyIpv4(
2
  ec2.Port.tcp(this.appPort),
3
  'app-inbound'
4
)

Defining the Redis service is pretty much the same as above, with the exception that we don’t need to define the Image Asset and we can just retreive it from the reigstry, and instead of allowing public connections we only allow connections from the appService we defined

lib/app-stack.ts

1
const redisTask = new ecs.FargateTaskDefinition(this, 'redis-task', {
2
  cpu: 512,
3
  memoryLimitMiB: 2048,
4
})
5

6
redisTask
7
  .addContainer('redis', {
8
    image: ecs.ContainerImage.fromRegistry('redis:alpine'),
9
    essential: true,
10
    logging: ecs.LogDrivers.awsLogs({
11
      streamPrefix: 'RedisContainer',
12
      logRetention: logs.RetentionDays.ONE_DAY,
13
    }),
14
  })
15
  .addPortMappings({
16
    containerPort: this.redisPort,
17
    hostPort: this.redisPort,
18
  })
19

20
const redisService = new ecsPatterns.NetworkLoadBalancedFargateService(
21
  this,
22
  'redis-service',
23
  {
24
    cluster,
25
    cloudMapOptions: {
26
      name: 'redis',
27
    },
28
    cpu: 512,
29
    desiredCount: 1,
30
    taskDefinition: redisTask,
31
    memoryLimitMiB: 2048,
32
    listenerPort: this.redisPort,
33
    publicLoadBalancer: false,
34
  }
35
)
36

37
redisService.service.connections.allowFrom(
38
  appService.service,
39
  ec2.Port.tcp(this.redisPort)
40
)
41

42
return redisService

Lastly, we want to add the Load Balancer DNS name to our stack’s outputs. We can do this with the cdk.CfnOutput class:

lib/app-stack.ts

1
this.appLoadBalancerDNS = new cdk.CfnOutput(this, 'AppLoadBalancerDNS', {
2
  value: appService.loadBalancer.loadBalancerDnsName,
3
})
4

5
this.redisLoadBalancerDNS = new cdk.CfnOutput(this, 'RedisLoadBalancerDNS', {
6
  value: redisService.loadBalancer.loadBalancerDnsName,
7
})

We can break the AppService definition into a createAppService function, and the RedisService into a createRedisService function for some organization, the final lib/app-stack.ts file looks like this:

lib/app-stack.ts

1
import * as cdk from '@aws-cdk/core'
2
import * as logs from '@aws-cdk/aws-logs'
3
import * as ec2 from '@aws-cdk/aws-ec2'
4
import * as ecs from '@aws-cdk/aws-ecs'
5
import * as ecsPatterns from '@aws-cdk/aws-ecs-patterns'
6
import * as ecrAssets from '@aws-cdk/aws-ecr-assets'
7

8
export class AppStack extends cdk.Stack {
9
  public readonly redisLoadBalancerDNS: cdk.CfnOutput
10
  public readonly appLoadBalancerDNS: cdk.CfnOutput
11

12
  public readonly redisPort: number = 6379
13
  public readonly appPort: number = 8080
14
  public readonly cloudMapNamespace: string = 'service.internal'
15
  public readonly redisServiceUrl: string =
16
    'redis://redis.service.internal:6379'
17

18
  constructor(scope: cdk.Construct, id: string, props?: cdk.StageProps) {
19
    super(scope, id, props)
20

21
    const vpc = new ec2.Vpc(this, 'AppVPC', {
22
      maxAzs: 2,
23
    })
24

25
    const cluster = new ecs.Cluster(this, 'ServiceCluster', { vpc })
26

27
    cluster.addDefaultCloudMapNamespace({ name: this.cloudMapNamespace })
28

29
    const appService = this.createAppService(cluster)
30

31
    const redisService = this.createRedisService(cluster, appService)
32

33
    this.appLoadBalancerDNS = new cdk.CfnOutput(this, 'AppLoadBalancerDNS', {
34
      value: appService.loadBalancer.loadBalancerDnsName,
35
    })
36

37
    this.redisLoadBalancerDNS = new cdk.CfnOutput(
38
      this,
39
      'RedisLoadBalancerDNS',
40
      {
41
        value: redisService.loadBalancer.loadBalancerDnsName,
42
      }
43
    )
44
  }
45

46
  private createAppService(cluster: ecs.Cluster) {
47
    const appAsset = new ecrAssets.DockerImageAsset(this, 'app', {
48
      directory: './app',
49
      file: 'Dockerfile',
50
    })
51

52
    const appTask = new ecs.FargateTaskDefinition(this, 'app-task', {
53
      cpu: 512,
54
      memoryLimitMiB: 2048,
55
    })
56

57
    appTask
58
      .addContainer('app', {
59
        image: ecs.ContainerImage.fromDockerImageAsset(appAsset),
60
        essential: true,
61
        environment: { REDIS_URL: this.redisServiceUrl },
62
        logging: ecs.LogDrivers.awsLogs({
63
          streamPrefix: 'AppContainer',
64
          logRetention: logs.RetentionDays.ONE_DAY,
65
        }),
66
      })
67
      .addPortMappings({ containerPort: this.appPort, hostPort: this.appPort })
68

69
    const appService = new ecsPatterns.NetworkLoadBalancedFargateService(
70
      this,
71
      'app-service',
72
      {
73
        cluster,
74
        cloudMapOptions: {
75
          name: 'app',
76
        },
77
        cpu: 512,
78
        desiredCount: 1,
79
        taskDefinition: appTask,
80
        memoryLimitMiB: 2048,
81
        listenerPort: 80,
82
        publicLoadBalancer: true,
83
      }
84
    )
85

86
    appService.service.connections.allowFromAnyIpv4(
87
      ec2.Port.tcp(this.appPort),
88
      'app-inbound'
89
    )
90

91
    return appService
92
  }
93

94
  private createRedisService(
95
    cluster: ecs.Cluster,
96
    appService: ecsPatterns.NetworkLoadBalancedFargateService
97
  ) {
98
    const redisTask = new ecs.FargateTaskDefinition(this, 'redis-task', {
99
      cpu: 512,
100
      memoryLimitMiB: 2048,
101
    })
102

103
    redisTask
104
      .addContainer('redis', {
105
        image: ecs.ContainerImage.fromRegistry('redis:alpine'),
106
        essential: true,
107
        logging: ecs.LogDrivers.awsLogs({
108
          streamPrefix: 'RedisContainer',
109
          logRetention: logs.RetentionDays.ONE_DAY,
110
        }),
111
      })
112
      .addPortMappings({
113
        containerPort: this.redisPort,
114
        hostPort: this.redisPort,
115
      })
116

117
    const redisService = new ecsPatterns.NetworkLoadBalancedFargateService(
118
      this,
119
      'redis-service',
120
      {
121
        cluster,
122
        cloudMapOptions: {
123
          name: 'redis',
124
        },
125
        cpu: 512,
126
        desiredCount: 1,
127
        taskDefinition: redisTask,
128
        memoryLimitMiB: 2048,
129
        listenerPort: this.redisPort,
130
        publicLoadBalancer: false,
131
      }
132
    )
133

134
    redisService.service.connections.allowFrom(
135
      appService.service,
136
      ec2.Port.tcp(this.redisPort)
137
    )
138

139
    return redisService
140
  }
141
}

We can kick off the pipeline by pushing to the GitHub repo we setup above which will cause all our services to be deployed. Once that’s done we can go to the Outputs panel for the Dev-AppStage and open the AppLoadBalancerDNS url, this will open our application.

Test the App

Set Data

With the server running you can create a new item with:

1
POST http://[AppLoadBalancerDNS]/my-test-key
2

3
BODY "my test data"
4

5
RESPONSE "OK"

Get Data

You can then get the value using the key with:

1
GET http://[AppLoadBalancerDNS]/my-test-key
2

3
RESPONSE "my test data"

And if all that works correctly then congratulations! You’ve successfully setup an application that uses multiple Docker Containers with CDK on AWS

Docker Containers with CDK Pipelines

Create CDK App

Add our Application Files

Setup

Pipeline Stack

Pipeline Account Permissions

GitHub Repo Permissions

Develop the Pipeline

Add App to Deployment

App Stack

Test the App

Set Data

Get Data

References