Payload Logo
Friday

Terraform to CDK migration

Author

Norbert Takács

The biggest tech debt task of dropping terraform and switching to CDK was up for grabs. While on the surface an easy feat:

  • disable terraform
  • import existing resources,
  • deploy new stack

In reality it feels similar to swapping the driver on a bus full of school children that cannot go slower than 50 miles an hour.

Especially when dealing with stateful resources. No one wants to experience redeploying a database and losing the precious data it was holding.

L1 only, not L2 or L3

CDK uses L1 L2 and L3 constructs. Think of them as abstractions, L3 can render out multiple resources with a few lines of code. L1 is one resource only, as described in the code.

To be able to import existing resources you have to declare them in L1. That means in the case of DynamoDB you must use raw CFN resource declarations.

Instead of being able to use the following L2 code:


1const table = new dynamodb.TableV2(this, 'Table', {
2  partitionKey: { name: 'pk', type: dynamodb.AttributeType.STRING },
3});

You have to use L1 CfnTable:

1import * as dynamodb from 'aws-cdk-lib/aws-dynamodb';
2

3new dynamodb.CfnTable(this, tableName, {tableName: "Test"})

I recommend setting up a new test environment, which should be easy using the existing terraform IaC.
Now to define the resources as best you can using CfnTable! Since I was running multiple environments, some of which have had different declarations its beneficial to:

  • describe
  • export
  • and compare

What is already deployed in the cloud.

Declare resources

I used a script to describe and export the current state. Then compared it locally. Looking at the terraform files also helps.

1aws dynamodb list-tables --query 'TableNames[]' --output text
2

3aws dynamodb describe-table --table-name <TABLE_NAME> \
4  --query 'Table.[TableName,AttributeDefinitions,KeySchema,GlobalSecondaryIndexes,LocalSecondaryIndexes,ProvisionedThroughput,BillingModeSummary,StreamSpecification]'
5

6aws dynamodb list-tags-of-resource \
7  --resource-arn $(aws dynamodb describe-table --table-name <TABLE_NAME> --query 'Table.TableArn' --output text)
8

9aws dynamodb describe-continuous-backups --table-name "$t" >> "${FOLDER}/${t}.txt"
10

11aws dynamodb describe-time-to-live --table-name "$t" >> "${FOLDER}/${t}.txt"
12

13FOLDER="dev"
14

15while IFS= read -r t; do
16  [ -z "$t" ] && continue
17  echo "Exporting $t ..."
18  aws dynamodb describe-table --table-name "$t" > "${FOLDER}/${t}.txt"
19  echo "" >> "${FOLDER}/${t}.txt"
20  echo "# TAGS" >> "${FOLDER}/${t}.txt"
21  aws dynamodb list-tags-of-resource \
22    --resource-arn $(aws dynamodb describe-table --table-name "$t" --query 'Table.TableArn' --output text) \
23    >> "${FOLDER}/${t}.txt"
24  echo "" >> "${FOLDER}/${t}.txt"
25  echo "# TTL" >> "${FOLDER}/${t}.txt"
26  aws dynamodb describe-time-to-live --table-name "$t" >> "${FOLDER}/${t}.txt"
27  echo "" >> "${FOLDER}/${t}.txt"
28  echo "# PITR" >> "${FOLDER}/${t}.txt"
29  aws dynamodb describe-continuous-backups --table-name "$t" >> "${FOLDER}/${t}.txt"
30

31done < tables.txt
32

When modelling the resources make sure to use: 

1cfnOptions.deletionPolicy = cdk.CfnDeletionPolicy.RETAIN;

This way you can remove the stack without the stack removing the tables!

The following commands might be able to help you find more info about the current state:

1terraform state list
2terraform state show module.shared.module.Dynamo_Monitoring_Tests.aws_dynamodb_table.default

Import

Once you came up with an idea of how your table looks like, its time to try to import it into CDK. The declaration does not have to be perfect at this point.


I did this in ci-cd, locally it might be easier as there is a CLI helper that helps picking the right resource.


For the ci-cd solution I created an import file which could be used for remote-mapping, this must match the physical id to the id-s you want to import.

Note: Below for the import to be picked up. the key couldnt have -, while the table did!

1{
2  "MonitoringTests": {
3    "TableName": "Monitoring-Tests"
4  }
5}

I placed them next to the stack, and ran it on the pipeleine with:

1npm ci
2npm run cdk synth
3npm run cdk diff
4npm run cdk -- import AalDynamodbStack --resource-mapping src/lib/stacks/aalDynamodb/existing-resource-mapping.json

There is validation when importing in case you really misconfigured something and are trying to create a resource instead of importing it:

1 ❌  AalDynamodbStack failed: Error [ValidationError]: As part of the import operation, you cannot modify or add [RoleArn, Tags]
2    at Request.extractError (/home/runner/work/iam-cdk-infra/iam-cdk-infra/node_modules/aws-cdk/lib/index.js:401:46717)
3    at Request.callListeners (/home/runner/work/iam-cdk-infra/iam-cdk-infra/node_modules/aws-cdk/lib/index.js:401:91771)
4    at Request.emit (/home/runner/work/iam-cdk-infra/iam-cdk-infra/node_modules/aws-cdk/lib/index.js:401:91219)
5    at Request.emit (/home/runner/work/iam-cdk-infra/iam-cdk-infra/node_modules/aws-cdk/lib/index.js:401:199820)
6    at Request.transition (/home/runner/work/iam-cdk-infra/iam-cdk-infra/node_modules/aws-cdk/lib/index.js:401:193373)
7    at AcceptorStateMachine.runTo (/home/runner/work/iam-cdk-infra/iam-cdk-infra/node_modules/aws-cdk/lib/index.js:401:158245)
8    at /home/runner/work/iam-cdk-infra/iam-cdk-infra/node_modules/aws-cdk/lib/index.js:401:158575
9    at Request.<anonymous> (/home/runner/work/iam-cdk-infra/iam-cdk-infra/node_modules/aws-cdk/lib/index.js:401:193665)
10    at Request.<anonymous> (/home/runner/work/iam-cdk-infra/iam-cdk-infra/node_modules/aws-cdk/lib/index.js:401:199895)
11    at Request.callListeners (/home/runner/work/iam-cdk-infra/iam-cdk-infra/node_modules/aws-cdk/lib/index.js:401:91939) {
12  code: 'ValidationError',
13  time: 2025-10-24T13:17:57.619Z,
14  requestId: '374cd914-0298-481c-930c-da8dcfb834f5',
15  statusCode: 400,
16  retryable: false,
17  retryDelay: 633.7959799051088
18}
19

Drift correction

After import passes you should go into AWS Cloudformation, choose the stack and press the Drift detection button. This will give you a list of items that are not declared properly. Some of the drifts also come up when running

1> npm run cdk diff 
2

3[+] Parameter BootstrapVersion BootstrapVersion: {"Type":"AWS::SSM::Parameter::Value<String>","Default":"/cdk-bootstrap/hnb659fds/version","Description":"Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"}
4Resources
5[~] AWS::DynamoDB::Table Monitoring-Tests MonitoringTests 
6 └─ [-] DeletionPolicy
7     └─ Retain
8✨  Number of stacks with differences: 6
9NOTICES         (What's this? https://github.com/aws/aws-cdk/wiki/CLI-Notices)
1032775	(cli): CLI versions and CDK library versions have diverged
11	Overview: Starting in CDK 2.179.0, CLI versions will no longer be in
12	          lockstep with CDK library versions. CLI versions will now be
13	          released as 2.1000.0 and continue with 2.1001.0, etc.
14	Affected versions: cli: >=2.0.0 <=2.1005.0
15	More information at: https://github.com/aws/aws-cdk/issues/32775
16If you don’t want to see a notice anymore, use "cdk acknowledge <id>". For example, "cdk acknowledge 32775".

But in my experience the drift detection in Cloud-formation is what you want to use. Now that you have found drifts.

My best advice is to go back and double check that either deletion protection is enabled and that the following line was deployed with the stack.

1table.cfnOptions.deletionPolicy = cdk.CfnDeletionPolicy.RETAIN

Afterwards you should remove the stack, this should skip removing the table. Fix your mistake and run the import again, check for drifts! Once there is no more drift you can continue with running the deploy:

1npm run cdk deploy -- --all
2

You can also run import multiple times, it will tell you there is nothing new to import

1AalDynamodbStack: no new resources compared to the currently deployed stack, skipping import.



Join the Discussion on github