The objective of this project is to support customers in seamlessly migrating from self-managed Cassandra clusters to Amazon Keyspaces. This migration approach ensures near zero downtime, no code compilation, predictable incremental traffic, and reduced migration costs.
This solution offers customers the flexibility to scale the migration workload up and out by deploying multiple Glue jobs of CQLReplicator. Each glue job (tile) is tasked with handling a specific subset of primary keys to distribute migration workload evenly across data processing units. A single Glue worker (G.025X) can write up to 400 WCUs per second against the target table in Amazon Keyspaces. This allows for easy estimation of the final traffic against Amazon Keyspaces.
List of prerequisites needed to run CQLReplicator with AWS Glue, such as:
- AWS account
- AWS CloudShell
- AWS Glue
- Amazon Keyspaces (keyspace and tables)
- Cassandra cluster
- Reference configuration files for Amazon Keyspaces and Cassandra Cluster
Download the project to your home folder by executing the following command:
git clone https://github.com/aws-samples/cql-replicator.gitTo set up AWS Glue for communication with your Cassandra cluster in Amazon VPC, you need to configure proper access settings. This involves creating a security group with a self-referencing inbound rule that allows all TCP ports. This self-referencing rule restricts the source to the same security group within the VPC, enhancing security by not opening it to all networks. It's worth noting that your VPC's default security group may already include such a self-referencing inbound rule for all traffic. By implementing these configurations, you ensure that AWS Glue components can communicate effectively while maintaining a secure network environment.
Direct migration to Amazon Keyspaces, offering a fully managed, serverless Cassandra-compatible experience:
Additional connectors for specific use cases including Amazon S3 and Amazon DynamoDB: