kevy4.0

Migration — the playbook and the toolchain

Two halves. The playbook: how to move a live RDS workload onto kevy in phases, each with its own verification clamp and rollback point. The toolchain (kevy-cli): the commands that move data into, out of, and between kevy servers — and prove the move was faithful. Modeling questions ("what does my JOIN become?") live in rds-workloads.md; runnable patterns in the cookbook.

The phased playbook

Seven phases. The load-bearing property: at every phase up to write cutover, the RDS is still the source of truth and rollback is "repoint the app" — never "reverse-migrate data". Don't skip phases; each one's verification clamp is what makes the next one boring.

Phase 0 — Inventory

List every table and every query shape the workload actually runs (your ORM's query log or pg_stat_statements is the ground truth, not the schema). Classify each against the rds-workloads matrix: most rows map mechanically; flag the ones that hit a refused construct (N-way joins in a hot path, HAVING, deep OFFSET pagination, ad-hoc reporting SQL).

Phase 1 — Schema mapping

Turn the inventory into a declaration script: prefixes, field layouts, types, then every IDX.CREATE / VIEW.CREATE line. Apply the type rules from the matrix now — DECIMAL to integer minor units, DATETIME to epoch i64, NULL to absent-field — because they are write-format decisions you cannot cheaply revisit after backfill.

Phase 2 — Change capture (dual-write window opens)

Start capturing RDS writes before the backfill snapshot, so nothing falls in the gap. Two routes:

Either way the kevy writes must be idempotent rebuild shapes (HSET full row, DEL + re-add for collections) so overlap with the backfill is harmless.

Phase 3 — Backfill

Snapshot the RDS at a known capture position, export to a RESP command file, and bulk load:

# generate rebuild frames from your RDS export (one HSET per row):
#   HSET user:42 name ada email ada@example.com …
kevy-cli import -p 6004 --strict rows.resp        # ≥200k cmd/s

Import first, declare indexes after (the deferred-index rule): backfill builds each index from existing rows at bulk speed — orders of magnitude cheaper than paying the write hook per imported row. Then run the phase-1 declaration script and wait for IDX.LIST to report state=ready (details under Loading into an indexed keyspace).

Phase 4 — Read cutover (canary, per endpoint)

Flip reads endpoint by endpoint, not big-bang. For each endpoint, pick the consistency rung it needs (availability.md): browse/list endpoints ride rung 0 (async replica reads); "I just wrote this, show it back" flows use REPL.TOKEN/REPL.WAIT (rung 1) or read the primary. A single-node kevy has no ladder to think about — every read is primary.

Phase 5 — Write cutover + the rollback window

Flip writers to kevy, and in the same deploy start the reverse mirror: a CDC consumer replaying kevy's feed back to the RDS as SQL (FEED.READUPDATE/INSERT, cookbook recipe 13). The RDS is now a warm standby that trails kevy by seconds; your rollback plan is "repoint the app back", not "reverse-migrate".

Before the flip, seed sequence keys above the RDS high-water mark (INCRBY seq:order <rds_max_id> on a fresh key) — the classic auto-increment collision is phase 5's sharpest edge.

Phase 6 — Decommission

When confidence hardens (weeks of clean diffs, one exercised failover), stop the reverse mirror, take a final RDS dump for the archive, and move the operational story fully onto kevy: snapshots + AOF (+ feed-cursor recovery points) per persistence.md, replica/failover posture per availability.md.

Common pitfalls


The toolchain (kevy-cli)

kevy-cli export  -p 6379 --prefix user: dump.resp
kevy-cli import  -p 6004 --strict dump.resp        # ≥200k cmd/s
kevy-cli import  -p 6004 --resume dump.resp        # after interruption
kevy-cli digest  -p 6004 user:
kevy-cli diff    hostA:6379 hostB:6004 user: order:
kevy-cli copy-prefix   -p 6004 --rate 5000 user: staging:user:
kevy-cli delete-prefix -p 6004 --rate 5000 --dry-run tmp:
kevy-cli inspect -p 6004 user:

Wire format

export writes a plain RESP command stream of rebuild frames — DEL + SET/HSET/RPUSH/SADD/ZADD, plus absolute PEXPIREAT for TTLs. That makes the file bidirectionally compatible with redis-cli --pipe: kevy exports feed a Redis, and any RESP command file feeds kevy-cli import — including one you generate yourself from an RDS dump (the playbook's phase 3).

The leading DEL per key makes replay rebuild from scratch — genuinely idempotent for every type (an append verb like RPUSH would otherwise double list content on re-import).

Consistency and resumability

Verification

PREFIX.DIGEST <prefix> (server + embedded prefix_digest) returns [count, hex64] — an order-insensitive checksum over canonical row bytes (hash fields and set members sorted, zset by score bits then member, list in order — list order IS identity). It is insensitive to shard count and insert order, so it compares across topologies. kevy-cli diff A:port B:port prefix… exits non-zero on any mismatch.

TTLs do not participate in the digest (they decay); values do.

Bulk operations

copy-prefix re-keys every row under a new prefix via read + rebuild frames (the server intentionally has no COPY verb; TTLs carried as absolute deadlines). delete-prefix SCANs + UNLINKs. Both take --rate N (token bucket, strict pacing from the first op) and delete supports --dry-run.

Loading into an indexed keyspace

Waiting for index readiness

IDX.CREATE returns immediately and backfills in the background; queries answer -INDEXBUILDING until done. The standard wait is polling IDX.LIST — its state column flips from building to ready:

until kevy-cli -p 6004 IDX.LIST | grep -A1 my_index | grep -q ready; do sleep 1; done

Backfill speed scales with DOCUMENT SIZE for text indexes: ~7s per million small rows, but multi-KB bodies index at roughly 85s per million (measured: 200k mail-sized bodies in 17s).

Verifying a whole migration in one command

kevy-cli diff compares any number of prefixes across two live servers in one call — prefer it over per-prefix digest pairs:

kevy-cli diff 127.0.0.1:6004 127.0.0.1:6005 msg: mbox: usr: tag: session:

Large exports

The dump is uncompressed RESP text (fast, greppable). For 10GB+ keyspaces pipe through gzip — the format is stream-friendly:

kevy-cli export -p 6004 /dev/stdout | gzip > dump.kevy.gz
gunzip -c dump.kevy.gz | kevy-cli import -p 6005 --strict /dev/stdin

(--resume needs a real file for its .progress sidecar — decompress to disk first if you want resumability.)

Create indexes after the bulk load: the index engine's backfill builds from existing data at bulk speed (~7s per million rows measured), which beats paying per-write hook maintenance a million times. Order of operations, not a switch:

kevy-cli import -p 6004 dump.resp
kevy-cli -p 6004 IDX.CREATE users ON PREFIX user: FIELD age TYPE i64 KIND range

Gate: bench/onrampgate.sh (1M-row round trip, ≥200k cmd/s import, kill -9 resume convergence, ±20% rate accuracy).