Setting the file. One moment.
Chapter 61 · Amazon Elasticache
Subchapter 61.19
references/migration/self-managed-migration.mdMarkdown15 KBView on GitHub
Migrate from self-managed Redis (running on EC2, on-premises, or another cloud provider) to Amazon ElastiCache. This guide covers assessment, migration approaches, and cutover.
Before migrating, inventory your source environment and validate compatibility.
Gather the following from your self-managed Redis instance:
| Item | How to check | Why it matters |
|---|---|---|
| Redis version | INFO server -> redis_version | Valkey 7.2 is compatible with Redis OSS up to version 7.2.4. Older source versions may use deprecated features. |
| Data size | INFO memory -> used_memory_human | Determines target cache sizing and migration duration |
| Key count | DBSIZE | Helps estimate migration time and validate completeness |
| Command usage | INFO commandstats | Identifies unsupported or uncommon commands |
| Persistence config | CONFIG GET save, CONFIG GET appendonly | Determines whether RDB snapshots or AOF are in use for snapshot-based migration |
| Cluster mode | INFO cluster -> cluster_enabled | Cluster mode affects migration tooling and target topology |
| Modules loaded | MODULE LIST | RedisJSON, RediSearch, RedisTimeSeries, etc. may not all be available in Valkey |
| Connected clients | INFO clients -> connected_clients | Helps size the target cache |
| Peak memory | INFO memory -> used_memory_peak_human | Size the target to handle peak load |
| Replication topology | INFO replication | Identifies primary/replica structure to replicate on the target |
Run the bundled preflight script against your source Redis:
# Without TLS
python3 scripts/migration_preflight.py --host <source-host> --port 6379
# With TLS
python3 scripts/migration_preflight.py --host <source-host> --port 6379 --tlsThe script checks version compatibility, module usage, key count, memory, cluster mode, and sizing. Resolve any FAIL findings before proceeding.
Review potential compatibility issues:
commandstats output against ElastiCache supported commands (opens in a new tab).bgrewriteaof, bgsave, config, debug, migrate, replicaof, save, slaveof, shutdown, and sync. Use parameter groups instead of CONFIG commands. Check your application code and scripts for usage of any of these commands.ElastiCache runs inside a VPC. Plan connectivity from your source to the target:
Best for: Small to medium datasets, acceptable downtime window, simplest execution.
Steps:
# 1. Create RDB snapshot on the source
valkey-cli -h <source-host> BGSAVE
# Wait for completion:
valkey-cli -h <source-host> LASTSAVE
# 2. Copy the RDB file to S3
aws s3 cp /var/lib/redis/dump.rdb s3://my-migration-bucket/dump.rdb
# 3. Grant ElastiCache access to the S3 bucket by adding a bucket policy:
# {
# "Version": "2012-10-17",
# "Statement": [{
# "Sid": "ElastiCacheSnapshotAccess",
# "Effect": "Allow",
# "Principal": { "Service": "<region>.elasticache-snapshot.amazonaws.com" },
# "Action": ["s3:GetObject", "s3:ListBucket", "s3:GetBucketAcl"],
# "Resource": [
# "arn:aws:s3:::my-migration-bucket",
# "arn:aws:s3:::my-migration-bucket/*"
# ]
# }]
# }
# 4. Create an ElastiCache replication group and seed from the snapshot
aws elasticache create-replication-group \
--replication-group-id my-new-cache \
--replication-group-description "Migrated from self-managed Redis" \
--engine valkey \
--engine-version 9.0 \
--cache-node-type cache.r7g.large \
--num-cache-clusters 2 \
--snapshot-arns arn:aws:s3:::my-migration-bucket/dump.rdb \
--transit-encryption-enabled \
--region <region>Downtime: Equals the time from final snapshot to application cutover. For large datasets, this can be minutes to hours.
Limitations:
--snapshot-arns with create-replication-group. For serverless caches, use --snapshot-arns-to-restore with create-serverless-cache instead.Eligibility: Online migration is not supported to ElastiCache serverless caches or clusters running on the r6gd node type. This approach is designed for migrating from self-hosted Redis/Valkey on EC2 to ElastiCache, not for moving data between ElastiCache clusters.
Best for: Large datasets, minimal downtime requirement, source is network-reachable from VPC.
Steps:
# 1. Create the target replication group (without snapshot seeding)
aws elasticache create-replication-group \
--replication-group-id my-new-cache \
--replication-group-description "Migration target" \
--engine valkey \
--engine-version 9.0 \
--cache-node-type cache.r7g.large \
--num-cache-clusters 2 \
--automatic-failover-enabled \
--multi-az-enabled \
--region <region>
# 2. Test the migration (validates connectivity and compatibility)
aws elasticache test-migration \
--replication-group-id my-new-cache \
--customer-node-endpoint-list Address=<source-host>,Port=6379 \
--region <region>
# 3. Start the migration (begins replication)
aws elasticache start-migration \
--replication-group-id my-new-cache \
--customer-node-endpoint-list Address=<source-host>,Port=6379 \
--region <region>
# 4. Monitor replication status
# Option A: Use INFO replication on the ElastiCache primary node
# to check master_link_status (should be 'up') and replication offset
valkey-cli -h <elasticache-primary-endpoint> -p 6379 INFO replication
# Option B: Monitor the ReplicationLag and PrimaryLinkHealthStatus
# CloudWatch metrics for the target replication group
aws cloudwatch get-metric-statistics \
--namespace AWS/ElastiCache \
--metric-name ReplicationLag \
--dimensions Name=CacheClusterId,Value=my-new-cache-001 \
--statistics Average \
--start-time $(date -u -d '10 minutes ago' +%Y-%m-%dT%H:%M:%S) \
--end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
--period 60 \
--region <region>
# 5. When lag is minimal, complete the migration (promotes the target)
aws elasticache complete-migration \
--replication-group-id my-new-cache \
--region <region>
# To abort migration without ensuring data is in sync, use --force:
# aws elasticache complete-migration \
# --replication-group-id my-new-cache \
# --force \
# --region <region>
# WARNING: --force stops migration without ensuring data is in sync.
# Use it only to abort migration, not for normal completion.Replication lag convergence: “Minimal” means ReplicationLag < 1 second for 5 consecutive checks (1 minute apart). If lag does not converge below 1s within 30 minutes, check source write rate and network bandwidth between source and target. Do not proceed with complete-migration until lag is stable.
Monitoring replication health: In addition to the CLI command above, verify replication status using:
INFO replication on the ElastiCache primary node and confirm master_link_status is upCLIENT LIST on your source Redis instancesDowntime: Near zero. Only the brief moment during the DNS/endpoint switch in your application.
Important: During migration, the ElastiCache cluster is read-only. You can use ElastiCache nodes for reads, but you cannot write to the ElastiCache cluster. All writes must continue going to the source Redis until complete-migration is executed.
Requirements:
Target requirements:
--automatic-failover-enabled)databases in Redis config)sizing-assessment.md for memory sizing guidance.Source requirements:
protected-mode must be set to nobind configuration, it must be updated to allow requests from ElastiCache nodessync, psync, info, config, command, cluster)Best for: Complex migrations, need full control, source not directly reachable from VPC, or need to transform data during migration.
Warning: Do NOT use DUMP/RESTORE for cross-engine migration (e.g., Redis OSS to Valkey). The serialization format is engine-specific and may fail silently or produce corrupted data. Use application-level copy or the ElastiCache migration tools instead.
Steps:
# Dual-write wrapper example
import valkey
old_client = valkey.Valkey(host="old-redis-host", port=6379, decode_responses=True)
new_client = valkey.Valkey(
host="new-cache.serverless.use1.cache.amazonaws.com",
port=6379,
ssl=True,
decode_responses=True,
)
# Feature flag to control read source
READ_FROM_NEW = False # Flip to True when new cache is warm
def cache_set(key: str, value: str, ttl: int = 300):
"""Write to both caches during migration."""
old_client.setex(key, ttl, value)
try:
new_client.setex(key, ttl, value)
except Exception:
# Log but don't fail -- old cache is still the primary
pass
def cache_get(key: str) -> str | None:
"""Read from the active cache."""
if READ_FROM_NEW:
result = new_client.get(key)
if result is None:
# Fall back to old cache during warm-up
result = old_client.get(key)
if result is not None:
# Backfill into new cache
ttl = old_client.ttl(key)
if ttl and ttl > 0:
new_client.setex(key, ttl, result)
return result
else:
return old_client.get(key)Downtime: Zero, if the application handles the dual-write correctly.
Trade-offs:
Complete each item before and during the cutover:
ssl=True / tls: {} for the new endpointmonitoring sub-skill)python3 scripts/security_audit.py --serverless <name> or --replication-group <name>)| Factor | Snapshot-Based | Replication-Based | Dual-Write |
|---|---|---|---|
| Downtime | Minutes to hours | Near zero | Zero |
| Complexity | Low | Medium | High |
| Network requirement | S3 upload only | Source reachable from VPC | No direct connectivity needed |
| Data transformation | None (exact copy) | None (exact copy) | Possible during write |
| Data size limit | Practical limit ~100 GB | No practical limit | No practical limit |
| Application changes | Endpoint update only | Endpoint update only | Dual-write logic required |
| Best for | Small data, simple setup | Large data, minimal downtime | Complex scenarios, unreachable source |
After cutover, verify the migration was successful:
Key count: Compare DBSIZE on source vs target
Spot-check data: Read a sample of keys and verify values match
Latency: Confirm read/write latency meets expectations (sub-ms for node-based, single-digit ms for serverless)
Hit rate: Monitor cache hit rate – expect it to ramp up over the warm-up period
Error rate: Monitor application logs for connection errors or unexpected responses
Run security audit:
python3 scripts/security_audit.py --serverless <name>
# or
python3 scripts/security_audit.py --replication-group <name>