Auto-delete old DynamoDB items in your AWS Amplify API

Written by Florian Gyger
Florian Gyger develops web and mobile apps using React, React Native, Gatsby and AWS Amplify.

One of the great advantages of DynamoDB is that tables can grow and scale infinitely without becoming any slower. This is awsome! However, I recently realized that it led me to just not clean up old and no longer used items. Most of the time there is no real reason to do so. I mean - maybe this data can be used sometimes in the future, right? 🤔

Right. But still, there are some scenarios where you want to automatically remove some old data though, e.g. for data privacy reasons.

So how can we achieve this? Do we need to implement a Lambda function that is triggered periodically? And how can we do this with AWS Amplify?

Don't worry! We won't need a custom Lamda and it is much easier than you might think! Actually, there is a built-in feature in DynamoDB which does exactly what we want:

DynamoDB's Time To Live (TTL) feature

Since 2017 there is a specific DynamoDB feature to auto-delete old items called "Time to Live", or short: TTL.

Once enabled it periodically checks a chosen item attribute in a table, which must contain a Unix timestamp. As soon as the timestamp is in the past, the respective item is deleted. Well, to be precise, it is not guaranteed to be removed instantly, but it will be deleted within a reasonable time frame.

The feature can be enabled manually after pushing your AWS Amplify API to AWS, or it can be programmatically configured within your API's GraphQL schema.

Let's start with how we can generally auto-delete old DynamoDB table items, before we then dive into the specific solution for AWS Amplify.

How to enable TTL manually

1. Navigate to the desired DynamoDB table in the AWS console.
2. Open the overview tab and you will then see the following property:
3. Click on "Manage TTL", which will open up this dialogue:
4. In this dialogue, you can enter the name of the attribute which contains the expiry time in Unix timestamp format. In this example it is called expirationUnixTimestamp.
5. After enabling it, the TTL property in the overview should look like so:

The auto-deletion of items in your table is now successfully configured! ✔

If you are using an AWS Amplify API though, it is recommended to automate this process:

How to enable TTL in an AWS Amplify API

Personally, I don't like having to do any manual steps after programmatically creating an AWS backend in the cloud. It requires additional documentation and increases the possibility to forget something and to deploy a faulty state.

That's why I built and open-sourced a custom GraphQL transformer to automatically enable TTL in any AWS Amplify API: graphql-ttl-transformer! 🎉

Just follow these simple steps to programmatically enable the Time to Live feature in your project:

1. Install the transformer: npm install --save graphql-ttl-transformer.
2. Edit amplify/backend/api/<YOUR_API>/transform.conf.json and append "graphql-ttl-transformer" to the transformers field:

   "transformers": [
       "graphql-ttl-transformer"
   ]

3. In your schema.graphql file, append the @ttl directive to the field that will contain the expiration unix timestamp:

   type ExpiringChatMessage @model {
     id: ID!
     message: String
     expirationUnixTime: AWSTimestamp! @ttl
   }

Attention: Make sure your attributes really contain a valid Unix timestamp (so the number of seconds since 1 January 1970, not milliseconds)!

Congratulations! You can now push your environment to AWS and relax while your old items get cleaned up automatically!

If this feature helped you saving time and money, or just avoided some frustration, please consider starring the project on GitHub or support my open source work using GitHub Sponsors. Thank you! 🙏