redis-tower documentation
redis-tower is a Redis client for Rust where every connection is a Tower
Service. Commands have concrete request and response types, concurrent work
can be auto-pipelined, and operational policy composes through Tower layers.
If you are evaluating or adopting the project, start here:
- Migrating from redis-rs maps the most common redis-rs idioms to redis-tower.
- Migrating from Fred covers client construction, command execution, pipelines, transactions, topology, pub/sub, reconnection, and shutdown.
- Production tuning explains client selection, pipeline and pool sizing, timeouts, backpressure, reconnects, observability, and graceful shutdown.
- Cloud and rotating credentials covers AWS ElastiCache IAM, Microsoft Entra ID, reconnect-time refresh, proactive reauthentication, and token-handling boundaries.
- Serverless and scale-to-zero covers deferred connection, cold-start lifecycle semantics, invocation deadlines, and shutdown.
- Pool health probing design records why active probing and idle reaping are pool-native, explicitly spawned tasks.
- Client-side caching covers cloneable cached clients, tracking modes, invalidation races, failure behavior, and cache metrics.
- Distributed primitives covers fenced locks, leader election, expirable semaphores, countdown latches, delayed queues, block-allocated IDs, Redis-time GCRA rate limiting, cluster keys, and failure behavior.
- The command cookbook maps typed command categories to response shapes, binary data, pipelines and transactions, stream ownership, raw/module replies, dedicated sessions, and topology entry points.
- The command capability ledger separates command-name availability, typed semantics, wire support, and routing/session evidence at a pinned Redis documentation revision.
- The client comparison records pinned, primary-source contracts for Rust and cross-language Redis clients.
- The test conformance report maps generated test inventory, topology and server matrices, and destructive fault coverage to their source and CI evidence.
- Differential testing records the MCP-derived semantic corpus against a pinned redis-rs oracle and its normalization rules.
- Binary data and typed arguments explains byte ownership, exact Cluster routing, response types, and the per-family compatibility inventory.
- RESP codec support and validation defines accepted and rejected wire forms, the fragmentation oracle, reviewed fuzz seeds, and mutation triage.
- The release process records the manual prepare/publish split, workspace publish order, validation gates, and post-publish verification.
For installation, a quick API overview, and examples across the main modules, see the repository README. The typed API reference on docs.rs provides crate-level tours, module guides, and compiled examples alongside every public type and method.
Local preview
Install mdBook, then run:
mdbook serve --open
For the same non-interactive checks used in CI:
mdbook build
python3 scripts/check_docs_links.py
The generated site is written under target/mdbook and is never checked in.
Migrating from redis-rs
This guide maps redis-rs 1.7 to the
current redis-tower API. Both clients provide typed conveniences, raw commands,
multiplexing, pipelines, transactions, Cluster, Sentinel, TLS, pub/sub, and
RESP3. The migration is primarily about choosing explicit connection and
response contracts, not replacing an “untyped” client with a typed one.
The migration in one table
| redis-rs 1.7 | redis-tower |
|---|---|
MultiplexedConnection | MultiplexedClient |
ConnectionManager | Factory-backed MultiplexedClient, or serialized ResilientRedisClient |
AsyncCommands / AsyncTypedCommands methods | Command values such as Get and Set, passed to execute |
Caller-selected FromRedisValue response | Command-defined response type; RedisValueExt converts at the edge |
Cmd | RawCommand, returning Frame, or TypedRawCommand<T> |
Pipeline / atomic pipeline | Pipeline / Transaction |
| Async pub/sub connection | Dedicated PubSubConnection |
| Async Cluster connection | MultiplexedClusterClient |
| Sentinel client | MultiplexedSentinelClient |
RedisError + ErrorKind | RedisError + classification helpers |
Dependencies
The versions below are the released APIs used by this guide. Choose only the redis-tower topology crates the application needs:
[dependencies]
# Before
redis = { version = "=1.7.0", features = ["tokio-comp", "cluster-async", "connection-manager"] }
# After
redis-tower = "0.1.4"
redis-tower-cluster = "0.1.4" # only for Redis Cluster
redis-tower-sentinel = "0.1.4" # only for Sentinel
bytes = "1"
tokio-stream = "0.1" # only for stream APIs such as pub/sub
For TLS, enable tls-rustls (recommended) or tls-native-tls on the facade and
on each topology crate that opens TLS connections; neither backend is enabled
by the facade’s default feature set. Redis Stack command groups can be selected
individually or through the facade’s default commands-stack feature.
TlsConfig::with_root_ca_pem replaces the default trust roots with every CA in
the supplied PEM bundle for either backend. Rustls loads native roots once per
process, caching the result or loading error; an empty native store falls back
to the webpki bundle, while loading failures with no usable roots return the
loader diagnostics. Restart the process to pick up changes to native roots.
danger_accept_invalid_hostnames(true) works on both backends and matches
redis-rs’s hostname-only bypass: the certificate must still chain to a trusted
root, but its names need not match the connected host or IP. That is the usual
answer for a cluster whose nodes report IPs the certificate does not name.
danger_accept_invalid_certs(true) disables all certificate verification. Both
are dangerous settings. Caller-built ClientConfig / TlsConnector
configurations retain their own verification and trust policy.
The standalone registry compatibility harness is compiled in CI against these exact crates.io versions. It is an independent workspace, so local path crates cannot mask a published-API mismatch. The larger live oracle is documented in Differential testing.
Connect and share
// redis-rs
let redis_rs = redis::Client::open("redis://127.0.0.1:6379/0")?;
let mut connection = redis_rs.get_multiplexed_async_connection().await?;
// redis-tower
use redis_tower::MultiplexedClient;
let client = MultiplexedClient::connect_url("redis://127.0.0.1:6379/0").await?;
Both handles can be cloned and shared across Tokio tasks. Do not add an
Arc<Mutex<_>> around MultiplexedClient; its worker already serializes and
auto-pipelines concurrent requests.
Choose a different redis-tower connection when the ownership contract differs:
RedisConnectionexclusively owns one socket.RedisClientis a simple serialized shared connection.ConnectionPool<RedisConnection>owns multiple independent sockets.- A factory-backed
MultiplexedClientreconnects without giving up concurrent multiplexing. ResilientRedisClientis a simpler reconnecting, mutex-serialized client.
Protocol, URLs, and authentication
The defaults differ. A normal redis-rs 1.7 URL selects RESP2; add
?protocol=resp3 to request RESP3. redis-tower’s default
ProtocolVersion::Auto tries HELLO 3 and falls back to RESP2 when RESP3 is not
available. If wire shape matters during a migration, pin both clients instead
of comparing different protocols:
// redis-rs RESP2
let redis_rs = redis::Client::open("redis://127.0.0.1:6379/?protocol=resp2")?;
// redis-tower RESP2. The released URL parser does not consume `protocol=`;
// select the protocol with ConnectionConfig.
use redis_tower::{ConnectionConfig, MultiplexedClient, ProtocolVersion};
let config = ConnectionConfig::new().with_protocol(ProtocolVersion::Resp2);
let tower = MultiplexedClient::connect_url_with_connection_config(
"redis://127.0.0.1:6379/",
&config,
).await?;
// Use ProtocolVersion::Resp3 to force RESP3 without fallback.
redis://user:password@host/db and rediss://... carry ACL credentials,
database selection, and TLS. Percent-encode URL-special bytes in usernames and
passwords. redis-tower authenticates and selects the database as part of setup;
a reconnecting URL factory repeats those steps on every new socket.
redis-tower’s Unix URL parser accepts authentication, database, and protocol query keys:
redis+unix:///run/redis.sock?user=app&pass=secret&db=1&protocol=resp3
For rotating tokens, replace static URL credentials with redis-tower’s
CredentialProvider. CredentialConnectionFactory::from_url preserves the
URL transport, database, and protocol while taking authentication from the
provider. Its with_setup_timeout covers provider lookup through final session
setup. SharedCredentialProvider gives type-erased hosts one cloneable cache
owner for ordinary, Cluster, and dedicated connections.
redis-rs’s streaming provider is not a signal to inject AUTH into every
socket state. redis-tower owners expose proactive reauthentication only where
they serialize it safely. Pub/Sub reconnects and replays confirmed
subscriptions; MONITOR terminates and reopens with an explicit observation
gap; transactions and blocking operations move to fresh credentials only at a
new operation boundary. See Cloud and rotating credentials.
Typed commands and response shapes
redis-rs command traits select a conversion from the type requested by the caller. redis-tower command values have one public response type:
// redis-rs
use redis::AsyncCommands;
let _: () = connection.set("key", "value").await?;
let value: Option<String> = connection.get("key").await?;
// redis-tower
use redis_tower::commands::{Get, Set};
client.execute(Set::new("key", "value")).await?;
let value: Option<bytes::Bytes> = client.execute(Get::new("key")).await?;
String-shaped Redis data stays binary-safe as Bytes. Convert only at an
application boundary:
use redis_tower::RedisValueExt;
let value: String = client.execute(Get::new("key")).await?.parse_into()?;
SET is one place where choosing a return conversion is not enough. Redis uses
OK, null, and the previous bulk value differently depending on NX, XX, and
GET. The base redis-tower Set keeps its original Option<Bytes> response so
existing code continues to compile, but that compatibility shape maps both
OK and a rejected conditional write to None. Use the additive outcome mode
when the decision matters:
use redis_tower::commands::{Set, SetPreviousValue, SetStatus};
let outcome = client
.execute(Set::new("lease", "owner-a").nx().get().with_outcome())
.await?;
match (outcome.status, outcome.previous) {
(SetStatus::Applied, SetPreviousValue::Missing) => {
println!("lease acquired from an absent key");
}
(SetStatus::NotApplied, SetPreviousValue::Value(owner)) => {
println!("lease already held by {owner:?}");
}
other => println!("other SET outcome: {other:?}"),
}
Call with_outcome() last. Without GET, the status remains unambiguous but
the previous value is SetPreviousValue::NotRequested; with GET, missing,
empty, and arbitrary binary previous values remain distinct.
Do not assume identical public values just because both clients decoded the
same RESP frame. RESP2 may represent a map as a flat array; RESP3 has map, set,
boolean, and verbatim-string types. redis-rs Value, redis-tower Frame, and a
typed command response deliberately expose different layers. Compare the
documented command result, normalizing only semantics Redis declares unordered
or protocol-dependent.
Binary arguments
The published 0.1.4 facade depends on redis-tower-commands 0.1.3, where typed
builders take keys and values as binary-safe CommandArg inputs. Byte slices,
arrays, Vec<u8>, and Bytes pass through unchanged, so opaque data does not
need the raw-command escape hatch:
use redis_tower::commands::{Get, Set};
let key = b"binary:\xff".as_slice();
let payload = vec![0x00, 0xfe, 0xff];
client.execute(Set::new(key, payload)).await?;
let value: Option<bytes::Bytes> = client.execute(Get::new(key)).await?;
RawCommand::arg remains binary-safe for commands without a typed builder. Do
not force opaque data through UTF-8. The per-family inventory and ownership
rules are tracked in Binary data and typed arguments.
Raw commands
// redis-rs
let value: redis::Value = redis::cmd("MYCOMMAND")
.arg("key")
.query_async(&mut connection)
.await?;
// redis-tower
use redis_tower::commands::RawCommand;
let frame = client
.execute(RawCommand::new("MYCOMMAND").arg("key"))
.await?;
RawCommand returns a Frame; validate its shape at the boundary. A typed
builder is preferable when available because it also carries key-routing,
blocking, idempotency, and response-decoding metadata.
Pipelines and transactions
Concurrent MultiplexedClient calls are automatically batched. Use an explicit
Pipeline when one task needs an ordered batch on one connection:
use redis_tower::{Pipeline, RedisConnection};
use redis_tower::commands::{Get, Incr};
let mut connection = RedisConnection::connect("127.0.0.1:6379").await?;
let mut replies = Pipeline::new()
.push(Incr::new("counter"))
.push(Get::new("counter"))
.execute(&mut connection)
.await?;
let count: i64 = replies.take(0)?;
let observed: Option<bytes::Bytes> = replies.take(1)?;
A pipeline is not atomic. Replace redis-rs’s atomic pipeline with
Transaction; its result explicitly distinguishes Committed from Aborted
after a WATCH conflict:
use redis_tower::{Transaction, TransactionResult};
match Transaction::new()
.watch(["counter"])
.push(Incr::new("counter"))
.execute(&mut connection)
.await?
{
TransactionResult::Committed(mut replies) => {
let count: i64 = replies.take(0)?;
let _ = count;
}
TransactionResult::Aborted => { /* rebuild and retry if appropriate */ }
}
For read-compute-write WATCH loops, keep the entire loop on a dedicated
RedisConnection (or another connection instance that the application owns
exclusively for the full loop).
Dedicated and stateful sessions
Do not move every operation onto the default multiplexed client. These APIs need an exclusive or purpose-built session:
- Run
BLPOP, blockingXREAD, and other blocking commands on a dedicatedRedisConnectionor a pool with enough independent connections. One blocking request would otherwise stop the shared pipeline worker. - Construct
PubSubConnectionfrom a freshRedisConnection; it owns subscription state, andreconnect_withcan restore confirmed subscriptions after the application supplies a replacement connection. - Construct
MonitorStreamfrom a freshRedisConnection; entering MONITOR permanently changes that socket’s mode until it is closed. - Keep manual connection-state sequences and read-compute-write WATCH loops on one exclusive connection.
use redis_tower::{PubSubConnection, RedisConnection};
use tokio_stream::StreamExt;
let connection = RedisConnection::connect_url("redis://127.0.0.1:6379").await?;
let mut pubsub = PubSubConnection::from_connection(connection)?;
pubsub.subscribe(&["events"]).await?;
while let Some(message) = pubsub.next().await {
let message = message?;
let _ = message;
}
Cluster and Sentinel
use redis_tower_cluster::MultiplexedClusterClient;
let cluster = MultiplexedClusterClient::connect_url(
"rediss://default:secret@seed.example:6379",
).await?;
use redis_tower_sentinel::MultiplexedSentinelClient;
let sentinel = MultiplexedSentinelClient::connect(
&["127.0.0.1:26379"],
"mymaster",
).await?;
Cluster handles MOVED/ASK routing and topology refresh. Sentinel separates Sentinel-hop and Redis-node credentials/TLS in its builder and verifies the discovered role. Cluster pub/sub makes ownership explicit: fixed-node regular subscriptions and slot-following sharded subscriptions are different APIs.
Reconnection, retry, and unknown execution
redis-rs ConnectionManager reconnects in the background: the command that
observes the dropped connection errors, while later commands wait for the new
connection. redis-tower separates three policies:
- a connection factory recreates and reconfigures the socket;
- reconnect backoff controls when another socket is attempted;
- retry or offline-queue policy decides whether an application command is eligible to be sent again.
A write whose request reached Redis but whose reply was lost has unknown
execution. redis-tower does not blindly replay a non-idempotent command such as
INCR. Applications that retry such work need an idempotency key or a
transaction/script that makes duplication safe.
Tower middleware
redis-tower clients implement tower::Service, so timeout, tracing, metrics,
circuit-breaker, concurrency, and application-specific policy can compose at a
stable request boundary. The built-in layers are documented on the
redis-tower crate. Connection setup timeout,
per-command deadline, Redis’s own blocking timeout, and an end-to-end request
deadline are separate controls; preserve that distinction during migration.
Lessons from an MCP integration
A downstream MCP integration motivated the public differential corpus. The reusable lessons are backed by that repository-local case ledger and executable tests:
- keep redis-rs as an independent oracle instead of sharing redis-tower’s serializers or response conversion;
- define an application response boundary instead of leaking either client’s raw protocol enum through a public schema;
- distinguish top-level server errors from nested per-entry errors;
- exercise RESP2 and RESP3 rather than assuming one public response shape;
- test Cluster routing, blocking sessions, binary/null/error values, cancellation, and lost replies as separate contracts.
The corpus pins redis-rs 1.7.0 and records the intentionally narrow normalizations used for each case.
Migrating from Fred
This guide maps common
fred 10.1 idioms to redis-tower.
Fred and redis-tower are both asynchronous Redis clients with automatic
pipelining, cluster and Sentinel support, TLS, pub/sub, transactions, and
reconnection building blocks. The important difference is API shape: Fred
exposes command interface traits with caller-selected response conversion,
while redis-tower sends typed command values through clients that implement
Tower Service.
The Fred snippets are intentionally versioned to 10.1. Fred has changed its builder and client names across major releases, so first translate older Fred code to the equivalent 10.1 concept, then apply the mapping here. The Fred crate docs and Fred examples are the source of truth for Fred-specific behavior.
The migration in one table
| Fred | redis-tower |
|---|---|
Builder + Client::init() | A client connect / connect_url constructor |
Client for concurrent commands | MultiplexedClient |
Interface methods such as get and set | Command values such as Get and Set, passed to execute |
Caller-selected FromValue response | Command-defined response type; RedisValueExt converts at the edge |
client.pipeline() | Automatic batching on MultiplexedClient; Pipeline for an explicit batch on an exclusive connection |
Pool / ExclusivePool | ConnectionPool<S>; use RedisConnection when one operation needs exclusive ownership |
ReconnectPolicy on a builder | Factory-backed MultiplexedClient or ResilientRedisClient with ReconnectConfig |
| Offline command buffering during reconnect | Opt-in OfflineQueueConfig on ResilientRedisClient, for idempotent typed commands only |
SubscriberClient | Dedicated PubSubConnection stream |
client.multi() | Transaction / transaction_with_retries |
custom(cmd!(...)) | RawCommand |
Client::quit() + connection task | MultiplexedClient::shutdown(); dropping an exclusive RedisConnection closes it |
Dependencies and imports
A Fred application often starts with a broad prelude and opts into interface features:
[dependencies]
fred = { version = "10", features = ["i-all", "enable-rustls"] }
The redis-tower facade always includes the core typed commands. Its default
commands-stack feature adds every Redis Stack command group; the example
below disables defaults so Stack groups and TLS can be selected explicitly.
Cluster, Sentinel, sync, modules, and a runtime-selected universal client are
separate crates:
[dependencies]
redis-tower = { version = "0.1", default-features = false, features = ["tls-rustls"] }
redis-tower-cluster = { version = "0.1", features = ["tls-rustls"] }
redis-tower-sentinel = { version = "0.1", features = ["tls-rustls"] }
bytes = "1"
tokio-stream = "0.1"
# Add only when application code must choose a topology at runtime.
redis-tower-client = "0.1"
Add redis-tower features such as commands-json, commands-search, or
commands-stack only when the application uses those Redis Stack command
groups. This keeps compile time and the public command surface intentional.
Instead of importing Fred interface traits through fred::prelude::*, import
the command builders you use (or the complete command prelude):
use redis_tower::{MultiplexedClient, RedisValueExt};
use redis_tower::commands::{Get, Set};
// For broad command-heavy modules: use redis_tower::commands::*;
Connect and share a client
Fred separates construction from initialization:
use fred::prelude::*;
let config = Config::from_url("redis://default:secret@127.0.0.1:6379/0")?;
let client = Builder::from_config(config).build()?;
let connection_task = client.init().await?;
redis-tower constructors return a connected, usable client:
use redis_tower::MultiplexedClient;
let client = MultiplexedClient::connect_url(
"redis://default:secret@127.0.0.1:6379/0",
).await?;
MultiplexedClient is the closest default replacement for Fred’s cheaply
cloneable Client: clones share one background worker and one auto-pipelined
connection, and execute(&self, ...) is safe from many Tokio tasks. Do not put
it behind Arc<Mutex<_>>.
There are two lifecycle differences to account for:
MultiplexedClient::connect_urlowns its worker internally; there is no separate connection task to retain or await.- A plain
connectclient does not automatically reconnect. Choose a factory-backed client before replacing Fred code that relies on its reconnect policy. See Reconnection.
Execute typed commands
Fred places each Redis command on an interface trait and converts the response to the type requested by the caller:
use fred::prelude::*;
client.set("key", "value", None, None, false).await?;
let value: Option<String> = client.get("key").await?;
In redis-tower, the command type determines the response type:
use redis_tower::commands::{Get, Set};
client.execute(Set::new("key", "value")).await?;
let value: Option<bytes::Bytes> = client.execute(Get::new("key")).await?;
Binary-safe string responses use bytes::Bytes. Convert only where the
application requires another representation:
use redis_tower::RedisValueExt;
let value: String = client.execute(Get::new("key")).await?.parse_into()?;
This moves a class of response-shape mistakes from runtime conversion into the
command’s implementation. Collection commands similarly return a concrete
shape such as Vec<Bytes>, Vec<(Bytes, Bytes)>, or a command-specific
response struct.
Command options
Fred commonly represents command options as additional method arguments. redis-tower puts them on a builder, which makes call sites self-describing:
use redis_tower::commands::SetStatus;
let outcome = client
.execute(Set::new("lease", "owner-a").nx().ex(30).with_outcome())
.await?;
if outcome.status == SetStatus::NotApplied {
println!("the lease already exists");
}
client.execute(Get::new("lease")).await?;
Check the command type’s rustdoc when translating a long Fred call: the builder
methods serialize in Redis grammar order even when they are chained in another
order. Call with_outcome() last when conditional SET code needs to
distinguish an applied write from a failed NX or XX condition; the base
Set keeps its original Option<Bytes> response for compatibility.
Commands without a typed builder
Fred’s custom interface maps to RawCommand:
use redis_tower::commands::RawCommand;
let frame = client
.execute(RawCommand::new("MYCOMMAND").arg("key").arg("value"))
.await?;
RawCommand returns a protocol Frame; match and validate the expected shape
at the boundary. Prefer a typed builder when one exists because it also carries
cluster key metadata, blocking behavior, idempotency, and response parsing.
Automatic and explicit pipelining
Fred automatically pipelines concurrent commands and also exposes an explicit
client.pipeline() buffer. MultiplexedClient automatically batches commands
that arrive concurrently, so ordinary application code needs no explicit
pipeline:
let a = client.execute(Get::new("a"));
let b = client.execute(Get::new("b"));
let (a, b) = tokio::try_join!(a, b)?;
For a deliberate ordered batch on one exclusive connection, replace Fred’s
explicit pipeline with Pipeline:
use redis_tower::{Pipeline, RedisConnection};
use redis_tower::commands::{Get, Incr};
let mut connection = RedisConnection::connect("127.0.0.1:6379").await?;
let mut results = Pipeline::new()
.push(Incr::new("counter"))
.push(Get::new("counter"))
.execute(&mut connection)
.await?;
let incremented: i64 = results.take(0)?;
let observed: Option<bytes::Bytes> = results.take(1)?;
An explicit Pipeline works with RedisConnection, RedisClient, and other
PipelineExecutor implementations. It is distinct from a transaction: Redis
can execute other clients’ commands between entries, and an error in one entry
does not roll back the others.
Transactions
Replace Fred’s client.multi() / exec block with Transaction. Do not send
raw MULTI and EXEC builders as independent calls through a multiplexed
client; other tasks could interleave commands on the shared connection.
use redis_tower::{MultiplexedClient, Transaction, TransactionResult};
use redis_tower::commands::{Get, Incr};
let mut client = MultiplexedClient::connect("127.0.0.1:6379").await?;
match Transaction::new()
.watch(["counter"])
.push(Incr::new("counter"))
.push(Get::new("counter"))
.execute(&mut client)
.await?
{
TransactionResult::Committed(mut results) => {
let value: i64 = results.take(0)?;
let observed: Option<bytes::Bytes> = results.take(1)?;
let _ = (value, observed);
}
TransactionResult::Aborted => {
// A watched key changed; rebuild this already-known transaction.
}
}
The standard MultiplexedClient sends the complete WATCH/MULTI/EXEC sequence
as one contiguous worker request. For read-compute-write logic and a bounded
retry loop, call transaction_with_retries on a dedicated RedisConnection
or an exclusive connection checked out from an appropriately sized pool.
Pooling and blocking commands
Fred offers shared Pool, exclusive ExclusivePool, and optional dynamic
pooling. In redis-tower, first ask whether a pool is needed:
- Keep a single
MultiplexedClientfor high-concurrency, non-blocking command traffic. One connection can sustain many concurrent callers through automatic pipelining. - Use
ConnectionPool<RedisConnection>when commands block (BLPOP, blockingXREAD, and similar), reply parsing is expensive enough to cause head-of-line blocking, or the application genuinely needs multiple independent connections. - Use a dedicated
RedisConnectionfor pub/sub, MONITOR, or an operation that must own its session state.
use redis_tower::{ConnectionPool, RedisConnection};
use redis_tower::pool::{DispatchStrategy, PoolConfig};
let pool = ConnectionPool::connect_with_config(
PoolConfig::default()
.name("blocking")
.size(8)
.dispatch(DispatchStrategy::LeastConnections),
|| RedisConnection::connect("127.0.0.1:6379"),
).await?;
Unlike Fred’s ExclusivePool::acquire, normal ConnectionPool::execute
handles reservation and release for one command. Keep session-scoped sequences
on a connection type that explicitly guarantees exclusive ownership.
Reconnection
A Fred builder can attach a ReconnectPolicy to its client. The closest
high-throughput redis-tower replacement is a factory-backed
MultiplexedClient. The factory recreates the connection and replays URL
authentication and database selection; the reconnect config controls bounded
exponential backoff and jitter.
use std::time::Duration;
use redis_tower::{AutoPipelineConfig, MultiplexedClient};
use redis_tower::auto_pipeline::AutoPipelineReconnectConfig;
use redis_tower::reconnect::{ReconnectConfig, UrlConnectionFactory};
let factory = UrlConnectionFactory::new(
"redis://default:secret@127.0.0.1:6379/0",
);
let reconnect = ReconnectConfig::default()
.base_delay(Duration::from_millis(50))
.max_delay(Duration::from_secs(2))
.connect_timeout(Duration::from_secs(3));
let client = MultiplexedClient::from_factory(
factory,
AutoPipelineConfig::default(),
AutoPipelineReconnectConfig::new(reconnect),
).await?;
For rotating cloud credentials, use
CredentialConnectionFactory::from_url instead. It keeps the URL’s transport,
database, and RESP protocol settings while replacing static URL credentials with
the provider, and its setup timeout covers credential lookup, transport setup,
authentication, database selection, and protocol negotiation. Share one
SharedCredentialProvider across ordinary and dedicated connections so they use
the same refresh cache.
Credential rotation is session-aware. Ordinary connections can reauthenticate in place when they are idle. Pub/Sub must reconnect and replay confirmed subscriptions, MONITOR must terminate and reopen with an observable gap, and blocking or transactional work should rotate at a fresh operation boundary. See the cloud authentication guide for complete examples and owner-lifecycle rules.
ResilientRedisClient is a simpler alternative with built-in reconnection,
but it serializes all commands through one mutex. It is appropriate for modest
traffic or control-plane work; prefer the factory-backed multiplexed form for
high concurrency.
Reconnection and command replay are separate decisions. A lost connection can
leave the caller unsure whether Redis executed the command. Opt into
client.retry(RetryPolicy::default()) only when the typed command is marked
idempotent; redis-tower’s retry wrapper refuses to replay non-idempotent writes.
Queueing idempotent work while offline
The default ResilientRedisClient starts a reconnect campaign after a
connection error but does not hold later work for Redis to return. If the Fred
application intentionally buffered commands during outages, opt into a bounded
offline queue:
use redis_tower::{OfflineQueueConfig, ResilientRedisClient};
let client = ResilientRedisClient::connect_url_with_offline_queue(
"redis://default:secret@127.0.0.1:6379/0",
OfflineQueueConfig::new(256).with_max_replay_attempts(3),
).await?;
Every clone shares this queue. It admits only typed commands whose
Command::idempotent() implementation returns true, preserves queue-ticket
replay order, returns RedisError::QueueFull instead of exceeding its
capacity, and returns RedisError::ReconnectFailed to admitted work when the
reconnect or per-command replay budget is exhausted. Concurrently first-polled
futures can be ticketed in either scheduler order. Non-idempotent commands and
raw Service<Frame> calls fail while offline rather than risk duplicating a
side effect. Canceling an in-wire command quarantines its socket before the
next ticket runs. A capacity of zero is a valid fail-fast configuration.
Use offline_queue_depth() and is_reconnecting() for overload and readiness
signals. A tower::buffer::Buffer in front of ReconnectService is an option
for frame-level stacks, but a frame has lost the typed command’s idempotency
metadata. The buffer therefore cannot make a safe replay decision for you.
Cluster and Sentinel
Fred selects centralized, clustered, or Sentinel deployment through its configuration and returns the same general client type. redis-tower uses topology-specific crates so their configuration and routing contracts remain explicit:
use redis_tower_cluster::MultiplexedClusterClient;
let cluster = MultiplexedClusterClient::builder("127.0.0.1:7000")
.max_redirects(5)
.connect()
.await?;
use redis_tower_sentinel::MultiplexedSentinelClient;
let sentinel = MultiplexedSentinelClient::builder(
&["127.0.0.1:26379", "127.0.0.1:26380"],
"mymaster",
)
.connect_with_reconnect()
.await?;
For application code that must select a topology from configuration at
runtime, use redis_tower_client::UniversalClient. Its URL schemes are
redis://, redis+cluster://, and redis+sentinel://; all variants expose the
same typed execute method. As with Fred’s URLs, credentials ride in the
userinfo (redis+cluster://user:pass@host:7000, percent-decoded), the
rediss variants enable TLS, and sentinel URLs take separate sentinel
credentials as query parameters – Fred’s ?sentinelUsername=U&sentinelPassword=P
becomes ?sentinel_username=U&sentinel_password=P:
redis+sentinel://app:node-pass@s1:26379,s2:26379/mymaster?sentinel_password=sent-pass
Read the feature matrix before assuming exact parity. RESP3 negotiation is available on standalone and cluster clients, while Sentinel support remains in progress. Some Fred features such as dynamic pools or alternate async runtimes do not have direct redis-tower equivalents.
Pub/sub
Fred’s optional SubscriberClient manages subscription state and offers
broadcast receivers or callbacks. redis-tower makes the dedicated Redis
connection and message stream explicit:
use redis_tower::{PubSubConnection, RedisConnection};
use tokio_stream::StreamExt;
let connection = RedisConnection::connect_url("redis://127.0.0.1:6379").await?;
let mut subscriber = PubSubConnection::from_connection(connection)?;
subscriber.subscribe(&["events"]).await?;
while let Some(message) = subscriber.next().await {
let message = message?;
println!("{}: {:?}", message.channel, message.payload);
}
PubSubConnection tracks confirmed channel, pattern, and shard-channel
subscriptions. After a connection failure, call reconnect_with(&factory) to
open a new dedicated connection and replay them. This recovery step is
explicit; do not assume that merely polling the stream reconnects it.
Errors, health, and middleware
Replace Fred ErrorKind matching with redis-tower’s classification helpers:
| Intent | redis-tower |
|---|---|
| Connection failed | error.is_connection_error() |
| Retry may be useful | error.is_retryable() |
| Redis Cluster redirect | error.is_moved() / error.is_ask() |
| Wrong data type | error.is_wrongtype() |
| Pool saturated | match RedisError::PoolAcquisitionTimeout |
| Pipeline queue shed load | match RedisError::QueueFull |
| Reconnect lifecycle | subscribe to ConnectionEventBus before constructing a reconnecting client |
Fred callbacks and client configuration cover much of its operational policy.
With redis-tower, compose the frame service with TracingLayer, MetricsLayer,
CommandTimeoutLayer, and the Redis-aware circuit breaker, or wrap typed
execution with RetryLayer. The production tuning page
shows how those pieces fit together.
Connection lifecycle streams are bounded observations, not flow control. A
slow subscriber receives ConnectionEventRecvError::Lagged { skipped }; event
consumption never delays reconnect or failover. Use events to drive metrics,
logs, and diagnostics, not as a durable audit log or a prerequisite for
recovery.
Graceful shutdown
Fred applications normally call quit() and then await or otherwise account
for the connection task. For a redis-tower multiplexed client, stop producers,
drop their clones, and consume the final owner with shutdown():
// Stop accepting application work and wait for spawned tasks first.
client.shutdown().await;
If another clone is still alive, shutdown() returns without stopping the
shared worker. Treat ownership during shutdown as an application-level
invariant. For pools call pool.close().await to reject new work and drain
accepted commands, then drop the remaining clones and connections.
Migration checklist
- Inventory Fred features and deployment modes; add only the matching redis-tower crates and feature flags.
- Replace the Fred client with
MultiplexedClient, unless the workload is blocking or needs exclusive session state. - Convert interface calls to typed command builders and make
Bytes-to-domain conversions explicit. - Decide whether concurrent automatic batching is enough before translating every explicit Fred pipeline.
- Rebuild transactions with
Transaction, not independent raw MULTI/EXEC calls. - Choose an explicit reconnect path and confirm AUTH, SELECT, TLS, and credentials are replayed by its factory; decide whether offline work should fail fast or use the bounded idempotent-only queue.
- Put blocking commands, pub/sub, and MONITOR on dedicated connections.
- Add bounded timeouts, queue or pool backpressure, metrics, readiness checks, and graceful shutdown before switching production traffic.
- Run the old and new clients against the same Redis version in staging and compare response conversions, latency distributions, error rates, and reconnect/failover behavior.
Cloud and rotating credentials
redis-tower separates two credential-rotation moments:
- Every fresh socket asks a
CredentialProviderfor current credentials. ACredentialConnectionFactory, Cluster builder, or Sentinel builder carries that provider through initial connect, reconnect, pool growth, topology discovery, and failover. - A
StreamingCredentialProvideremits replacements for sockets that remain open. An owned reauthentication handle sendsAUTHto every current data socket and stops when the handle is dropped or shut down.
Connection setup always opens in RESP2, authenticates, and only then negotiates
the requested protocol. If Redis rejects setup credentials with NOAUTH or
WRONGPASS, the provider is force-refreshed and AUTH is attempted once more.
This bounded retry applies only to connection setup. redis-tower never replays a
user command after an authentication error.
Use CredentialConnectionFactory::from_url when the deployment URL also owns
transport, database, or protocol configuration. The provider replaces only the
URL’s username/password; TCP versus TLS versus Unix socket, db, and
protocol remain part of every initial or replacement connection. Bound the
whole sequence—not just the socket dial—with with_setup_timeout:
let factory = CredentialConnectionFactory::from_url(
"rediss://cache.example.net:6380/2?protocol=resp3",
provider.clone(),
)
.with_connection_config(connection_config)
.with_setup_timeout(Duration::from_secs(5));
ConnectionConfig::connect_timeout covers the TCP or Unix connect operation.
The factory setup timeout additionally covers provider lookup, TLS, the one
allowed forced refresh, SELECT, and HELLO. A timeout drops the partial
socket and returns RedisError::ConnectTimeout.
Shared provider ownership
Concrete AWS and Entra providers are cheap clones backed by one internal cache.
Type-erased hosts can use SharedCredentialProvider to retain the same property
after converting a provider to Arc<dyn CredentialProvider> or
Arc<dyn StreamingCredentialProvider>:
use std::sync::Arc;
use redis_tower::{
CredentialConnectionFactory, CredentialProvider, SharedCredentialProvider,
};
let erased: Arc<dyn CredentialProvider> = build_provider();
let shared = SharedCredentialProvider::from_arc(erased);
let ordinary = CredentialConnectionFactory::from_url(redis_url, shared.clone());
let dedicated = CredentialConnectionFactory::from_url(redis_url, shared.clone());
let cluster = MultiplexedClusterClient::builder(cluster_seed)
.credentials(shared)
.connect()
.await?;
Every clone delegates to the same allocation, so concurrent ordinary, dedicated, and Cluster connection attempts share the provider’s cache and single-flight behavior.
Established sockets and dedicated modes
Credential lookup during connection setup and credential changes after setup are different contracts:
| Owner | On provider update | Rejected replacement AUTH |
|---|---|---|
Standalone MultiplexedClient | Use spawn_credential_reauthentication to serialize AUTH through its worker | The worker stays installed with its previous Redis identity. The task logs a redacted warning and consumes later updates. For fail-closed rotation, stop admission, shut down the client, and rebuild it through the provider factory before resuming traffic. |
Retained ConnectionPool | Use spawn_credential_reauthentication to visit every active slot | Failed slots stay installed with their previous identity while the remaining slots are attempted. For fail-closed rotation, call close, drop all clones, and rebuild the pool through the provider factory. |
| Direct or multiplexed Cluster client | Use the owner’s spawn_credential_reauthentication handle | The failed node socket/worker is removed from routing and a later command reconnects through the provider. |
Direct SentinelClient | Use spawn_credential_reauthentication for current data sockets; discovery sockets are short-lived | A failed master is marked for rediscovery before the next command, and failed replicas are removed. |
MultiplexedSentinelClient | Use spawn_credential_reauthentication for current data workers | Failed workers stay routed with their previous identity. Stop routing, drop every client clone/update handle, and rebuild the client before resuming traffic when rotation must fail closed. |
| Fresh blocking or transaction connection | Finish/cancel the operation, then reconnect through the provider factory | Inserting AUTH into connection-local operation state is not a generic safe boundary. An ambiguous operation is never replayed. |
| Pub/Sub | Reconnect through the provider factory; confirmed subscriptions are replayed | Subscription mode is stateful, and an arbitrary callback must not inject AUTH. Messages during the gap are lost. |
| MONITOR | Terminate and create a new MonitorStream through the provider factory | MONITOR owns a one-way event stream, has no resume cursor, and loses events during the gap. |
The generic update task is deliberately best-effort: it does not expose an
error channel, renders neither provider nor callback errors, and continues
after a failure. Where an owner exposes reauthenticate_all, call it directly
when the result is needed synchronously. The table above is the authoritative
retirement policy; do not assume that every owner evicts a socket after a
rejected update.
PubSubConnection::connect_with, BinaryPubSubConnection::connect_with, and
MonitorStream::connect_with open their dedicated sockets through any
ConnectionFactory, including CredentialConnectionFactory. A streaming
provider owner can select between its update stream and the session stream:
use std::sync::Arc;
use redis_tower::{BinaryPubSubConnection, StreamingCredentialProvider};
use tokio_stream::StreamExt;
let mut pubsub = BinaryPubSubConnection::connect_with(&factory).await?;
pubsub.subscribe_bytes(&[b"events"]).await?;
let mut credential_updates = Arc::clone(&streaming_provider).subscribe();
loop {
tokio::select! {
update = credential_updates.next() => {
update.ok_or("credential stream ended")??;
// The provider cache is current before it emits. Reconnect rather
// than sending AUTH in subscription mode.
pubsub.reconnect_with(&factory).await?;
}
message = pubsub.next() => {
let Some(message) = message else { break };
handle_message(message?);
}
}
}
Dropping a CredentialReauthenticationHandle cancels and aborts its provider
stream; shutdown().await performs the same cancellation and waits for task
exit. Dedicated-session select loops are owned directly by the application and
stop when that owner is dropped.
AWS ElastiCache IAM
redis-tower-auth-aws implements the ElastiCache IAM SigV4 flow for provisioned
replication groups and serverless caches. It creates a presigned
elasticache:Connect request with the documented 15-minute lifetime, caches it
for reconnect fan-out, and emits a replacement after 75% of that lifetime.
ElastiCache IAM requires TLS. The provider creates credentials; the client must still be configured with the appropriate TLS hostname and trust roots.
use std::sync::Arc;
use aws_config::BehaviorVersion;
use redis_tower::{
AutoPipelineConfig, ConnectionConfig, CredentialConnectionFactory,
MultiplexedClient, ProtocolVersion, StreamingCredentialProvider,
};
use redis_tower::auto_pipeline::AutoPipelineReconnectConfig;
use redis_tower_auth_aws::{
ElastiCacheIamProvider, ElastiCacheResourceType,
};
let aws = aws_config::load_defaults(BehaviorVersion::latest()).await;
let provider = ElastiCacheIamProvider::new(
"redis-app",
"production-cache",
"us-west-2",
ElastiCacheResourceType::ReplicationGroup,
aws.credentials_provider().expect("AWS credentials").clone(),
)?;
let factory = CredentialConnectionFactory::new(
"production-cache.example.cache.amazonaws.com:6379",
provider.clone(),
)
.with_connection_config(
ConnectionConfig::new().with_protocol(ProtocolVersion::Resp3),
)
.with_tls("production-cache.example.cache.amazonaws.com", tls_config);
let client = MultiplexedClient::from_factory(
factory,
AutoPipelineConfig::default(),
AutoPipelineReconnectConfig::default(),
).await?;
let updates: Arc<dyn StreamingCredentialProvider> = Arc::new(provider);
let auth_handle = client.spawn_credential_reauthentication(updates);
Use ElastiCacheResourceType::ServerlessCache for serverless ElastiCache. The
signed request then includes ResourceType=ServerlessCache. The IAM user name
and user ID must match, and the IAM policy must permit both the cache resource
and user resource.
Microsoft Entra ID
redis-tower-auth-azure requests the standard Azure Redis scope,
https://redis.azure.com/.default. The managed identity or service-principal
object ID becomes the Redis username and its access token becomes the password.
The provider caches a token and emits its replacement at approximately 75% of
the observed lifetime.
use std::sync::Arc;
use redis_tower::{
AutoPipelineConfig, CredentialConnectionFactory, MultiplexedClient,
StreamingCredentialProvider,
};
use redis_tower::auto_pipeline::AutoPipelineReconnectConfig;
use redis_tower_auth_azure::EntraIdProvider;
let provider = EntraIdProvider::managed_identity(
"00000000-0000-0000-0000-000000000000",
None,
)?;
let factory = CredentialConnectionFactory::new(
"my-cache.westus.redis.azure.net:6380",
provider.clone(),
)
.with_tls("my-cache.westus.redis.azure.net", tls_config);
let client = MultiplexedClient::from_factory(
factory,
AutoPipelineConfig::default(),
AutoPipelineReconnectConfig::default(),
).await?;
let updates: Arc<dyn StreamingCredentialProvider> = Arc::new(provider);
let auth_handle = client.spawn_credential_reauthentication(updates);
EntraIdProvider::new accepts any Azure SDK TokenCredential, so workload
identity and service-principal credentials use the same cache and push stream.
with_scope is available for sovereign-cloud or compatibility deployments.
The downstream
redis-database-mcp-rs#92
contract keeps Azure SDK dependencies behind the server’s optional Entra
feature. Construct one EntraIdProvider in the server layer, wrap or clone it
into every DirectRedis* connection factory, and keep static URL
authentication as the no-feature/default path. Its acceptance test injects a
unique token into a failing provider, invokes a real MCP tool, and proves the
serialized tool result or JSON-RPC error, returned metadata, and captured
tracing/log output omit the token while retaining is_authentication_error
classification. MCP policy and that tool-boundary evidence stay downstream.
Pools, Cluster, and Sentinel
Use one cloneable provider instance for both socket creation and streaming updates. Provider clones share their token cache.
// Pool: the factory covers new/replacement slots; the handle covers live slots.
let factory = CredentialConnectionFactory::new(address, provider.clone());
let pool = ConnectionPool::connect_with_factory(pool_config, factory).await?;
let auth_handle = pool.spawn_credential_reauthentication(Arc::new(provider.clone()));
// Cluster: every node setup/reconnect fetches credentials.
let cluster = MultiplexedClusterClient::builder(seed)
.credentials(provider.clone())
.connect()
.await?;
let auth_handle = cluster.spawn_credential_reauthentication(Arc::new(provider.clone()));
// Sentinel discovery credentials are independent from Redis data credentials.
// Only data sockets persist, so the update handle targets node credentials.
let sentinel = SentinelClient::builder(sentinels, service_name)
.sentinel_credentials(sentinel_provider)
.node_credentials(provider.clone())
.connect()
.await?;
let auth_handle = sentinel.spawn_credential_reauthentication(Arc::new(provider));
Direct ClusterClient, SentinelClient, MultiplexedClusterClient, and
MultiplexedSentinelClient expose the same owned update mechanism, but their
failure retirement differs. Both Cluster owners remove a failed node from
routing so its normal provider-backed reconnect path can rebuild it. Direct
SentinelClient forces rediscovery after a master failure and removes failed
replicas. MultiplexedSentinelClient retains a worker that rejects AUTH with
its previous identity; stop routing and rebuild that owner to fail closed.
Secret handling and failure behavior
Credentials redacts passwords and tokens from Debug output and zeroizes its
owned username/password buffers on drop. The AWS and Azure providers retain
their cached Redis credentials in this type, so replacing or dropping a cached
token zeroizes that allocation. Avoid cloning or logging token strings in
application provider implementations.
Credential update streams are deliberately best-effort. Provider and AUTH
errors are logged without rendering third-party error text, and later emissions
are still consumed. Provider failures returned from setup are redacted to an
AUTH_PROVIDER ... failed message and are recognized by
is_authentication_error; Redis NOAUTH and WRONGPASS retain their server
classification. Keep the owned handle alive for as long as the client should
receive proactive updates, and call shutdown().await during graceful
shutdown. A provider should delay after an emitted error; this prevents a
broken credential source from creating a busy loop.
Manual Azure Managed Redis verification
The normal suite uses deterministic fake providers; no cloud credential is required. Before shipping a downstream Entra integration, run this manual check against a disposable Azure Managed Redis database:
- Create a Redis data-access-policy assignment for the managed identity or service principal and record its object ID. Start with the least-privilege command/key policy required by the application.
- Configure
rediss://<host>:10000/0?protocol=resp3without URL credentials. BuildEntraIdProviderfrom the host’s AzureTokenCredential, then build a URL-backedCredentialConnectionFactorywith a finite setup timeout. - Exercise an ordinary multiplexed command, one new dedicated connection, and Cluster node discovery if the target exposes Cluster mode. Confirm no token appears in logs, errors, traces, or MCP tool output.
- Leave the process running through one proactive refresh. Verify ordinary sockets reauthenticate, a Pub/Sub owner reconnects and restores confirmed subscriptions, and a MONITOR owner terminates/reopens with its documented observation gap.
- Revoke or misconfigure the assignment and confirm the application reports a redacted authentication-category failure without retrying an ambiguous write, transaction, or blocking operation.
Client-side caching
redis-tower combines Redis server-assisted tracking with bounded local response
caches for standalone Redis and master-routed Redis Cluster deployments. Use
CachedMultiplexedClient for standalone work or
redis_tower_cluster::CachedMultiplexedClusterClient for Cluster. Both are
cheap to clone and share one coherent cache across their clones.
use std::time::Duration;
use redis_tower::{
CacheTrackingMode, CachedClientConfig, CachedMultiplexedClient,
commands::{Get, Set},
};
let config = CachedClientConfig::new()
.max_entries(50_000)
.client_ttl(Some(Duration::from_secs(15)))
.tracking_mode(CacheTrackingMode::broadcast_with_prefixes(["user:"]));
let client =
CachedMultiplexedClient::connect_with_config("127.0.0.1:6379", config).await?;
client.execute(Set::new("user:42", "Ada")).await?;
let first = client.execute(Get::new("user:42")).await?; // Redis miss
let second = client.clone().execute(Get::new("user:42")).await?; // local hit
assert_eq!(first, second);
let statistics = client.cache_statistics().await;
println!("hits={}, misses={}", statistics.hits, statistics.misses);
client.shutdown().await;
Ok::<(), redis_tower::RedisError>(())
Standalone connection setup
The address constructor is the shortest path for plain standalone Redis.
connect_url applies URL authentication, database selection, TLS, or a Unix
socket to both the data connection and invalidation receiver. The
*_with_connection_config variants also propagate keepalive, connect timeout,
and RESP decode limits while forcing the RESP3 protocol required by tracking.
For custom TLS roots, mTLS, rotating credentials, or application-specific
setup, pass a ConnectionFactory to from_factory. To reuse an existing data
connection, use from_connection_with_factory; its receiver factory must
connect to the same standalone Redis server and reproduce the required
authentication/session setup. The data worker is intentionally fixed rather
than silently reconnecting without replaying tracking state; construct a new
cached client after a data-connection failure.
This two-connection design requires Redis Open Source-compatible REDIRECT
support. Redis Software and Redis Cloud currently do not support two-connection
tracking, so a rediss:// URL alone does not make this client compatible with
those services.
Tracking modes
CachedClientConfig supports three Redis tracking modes:
CacheTrackingMode::Broadcastreceives invalidations for every key, or only for configured binary-safe prefixes. It uses no per-client key table on the Redis server and is the default.CacheTrackingMode::ServerDefaultasks Redis to remember the keys this data connection reads. It reduces invalidation traffic when the working set is narrow, at the cost of server-side tracking memory.CacheTrackingMode::OptIntracks only cache misses selected by the client. redis-tower submitsCLIENT CACHING YESand the read as one atomic worker request, so another clone cannot interleave a command between them.
For standalone CachedMultiplexedClient, all modes use two RESP3 connections.
A dedicated receiver owns invalidation pushes; the data connection redirects
tracking messages to its client ID and uses NOLOOP. Writes through the
standalone cached client therefore invalidate locally both before and after
dispatch. The second invalidation also rejects an old read that raced the
write.
The standalone cached client owns that connection-local state. Caller-issued CLIENT TRACKING, CLIENT CACHING, CLIENT REPLY, HELLO, RESET, and QUIT
commands are rejected, including inside explicit pipelines and transactions.
Use a dedicated uncached connection for session administration. Read-only
diagnostics such as CLIENT TRACKINGINFO, CLIENT GETREDIR, and CLIENT ID
remain available.
The local response cache currently supports GET, HGET, HGETALL,
LRANGE, SMEMBERS, ZRANGE, and TYPE. Other commands still use the same
client and auto-pipeline worker, but bypass the local response cache. With
prefix-filtered broadcast tracking, cacheable reads outside the configured
prefixes also bypass both the cache and its hit/miss statistics; Redis would
not send invalidations for those keys, so caching them would be unsafe.
Staleness and failure behavior
The local cache has two independent safety bounds:
- Redis invalidation pushes remove all cached command variants that depend on a changed key.
client_ttlcaps how long any entry can be served even when no invalidation arrives. The safe default is 30 seconds. Standalone clients may passNoneonly when the deployment can tolerate relying exclusively on tracking; Cluster clients rejectNone.
Each cache miss snapshots a per-key invalidation epoch and a global generation. If an invalidation, explicit clear, or local write happens while Redis is answering, the late response is returned to its caller but is not inserted. Epoch bookkeeping is bounded alongside the cache.
If the standalone invalidation receiver disconnects, redis-tower immediately clears and disables the cache. Reads pass through to Redis while a supervisor reconnects, obtains a new receiver ID, and installs the new redirect on the data worker. Caching resumes only after that setup succeeds.
The data connection has stricter semantics because its tracking state and in-flight command ordering cannot be reconstructed transparently. A clean idle socket loss is detected by the auto-pipeline worker without waiting for a cache miss or write; redis-tower marks caching unhealthy and clears the cache before it releases any affected callers. The fixed worker then remains closed, so construct a new cached client. Silent network black holes remain bounded by the configured TCP keepalive policy or the next command deadline rather than by an application heartbeat.
For the standalone client, is_caching_healthy() includes both receiver
tracking and data-worker health and is suitable for readiness or diagnostics.
It never treats an open cache as healthy after the fixed data worker has
stopped.
Redis Cluster
CachedMultiplexedClusterClient keeps one cache above cluster routing and one
dedicated RESP3 invalidation receiver for every current master. Its builder
accepts the same CachedClientConfig and forces RESP3 across seed discovery,
data nodes, redirects, topology refreshes, receivers, and reconnects.
use redis_tower::{CacheTrackingMode, CachedClientConfig, commands::Get};
use redis_tower_cluster::CachedMultiplexedClusterClient;
use std::time::Duration;
let config = CachedClientConfig::new()
.client_ttl(Some(Duration::from_secs(30)))
.tracking_mode(CacheTrackingMode::OptIn);
let client = CachedMultiplexedClusterClient::builder("127.0.0.1:7000")
.cache_config(config)
.connect()
.await?;
let value = client.execute(Get::new("user:42")).await?;
let _ = value;
client.shutdown().await;
Ok::<(), redis_tower::RedisError>(())
Cache use opens only after every current master has a healthy data worker and
CLIENT TRACKING ... REDIRECT points at a live receiver. Data or receiver loss,
an ambiguous timeout, and any receiver/topology coverage rebuild close the gate
and clear the cache before it can reopen. MOVED and ASK close the gate before
the router awaits a new connection; slot ownership epochs reject responses
that began under an older owner. Opt-in dispatch remains one reserved worker
submission: [CLIENT CACHING YES, command]. Redis’s one-shot ASKING and
CLIENT CACHING YES flags consume one another, so ASK closes the gate and
retries as [ASKING, command]; that migrated response is not cached.
The initial Cluster implementation intentionally supports
ReadPreference::Master only. Replica and prefer-replica policies are rejected
until equivalent invalidation coverage can be proven for replica reads.
Cluster caching also requires a finite client_ttl. An empty slot can move
without producing a redirect or invalidation visible to this client, leaving
an old-owner cached miss that only the TTL can eventually expire. This
Cluster-specific check runs before the seed connection is opened; standalone
cached clients continue to permit client_ttl(None).
Capacity and observability
max_entries bounds local response entries; 0 makes the cache unbounded and
is not recommended for production. Eviction is intentionally simple and
bounded rather than an application-level replacement policy.
Every cached client exposes aggregate hits, misses, invalidations, and
evictions. Supplying a MetricsRecorder in CachedClientConfig emits the
same bounded events; MetricsFacadeRecorder publishes them as
redis_tower.cache.events with a four-value event label. Keys and command
arguments are never metric labels.
Tower and exclusive-client forms
CacheLayer / CacheService remain available for custom Tower stacks. Prefer
constructors that attach and own an invalidation stream. A cache service built
without invalidations is only safe when the caller explicitly clears it after
every possible external write; its constructor documents that hazard loudly.
Recorder-enabled stream constructors are available when custom Tower stacks
need the same cache-event metrics as CachedClientConfig.
Tower calls poll_ready before supplying a request, so an inner service may
already hold bounded capacity when CacheService discovers a local hit. Cache
backends must implement ReleaseReadiness to return that capacity without an
inner call; redis-tower implements the contract for AutoPipelineService,
FrameService, and ReconnectService. Put CacheLayer directly above one of
those backends and place tracing, metrics, timeouts, and other middleware
outside the cache. Custom middleware placed inside the cache must explicitly
propagate ReleaseReadiness to its inner service. There is no blanket no-op
implementation because that could silently leak permits and stall a bounded
service.
CachedClient is the lower-throughput serialized compatibility form. It uses
the same cache service and connection actor with a one-request batch limit, so
it shares the clone-safe tracking, failure, race-protection, and shutdown
semantics without auto-batching concurrent misses. Existing
CachedClient::connect callers keep the safe broadcast defaults. Both
standalone cached client types expose shutdown() so the final clone can
explicitly join their worker and invalidation lifecycles. The cached Cluster
client provides the same final-clone shutdown behavior for all per-master
workers and receivers.
Distributed primitives
The redis-tower-primitives crate builds locks, leader election, expirable
semaphores, countdown latches, delayed queues, block-allocated IDs, and shared
rate limits on the generic RedisExecutor interface. It works with standalone,
multiplexed, pooled, resilient, cluster, Sentinel, and universal clients that
implement that interface. Every script is a public, line-documented constant
and runs through the EVALSHA-first Script helper.
Distributed lock
DistributedLock::new(lock_key, fencing_key, ttl) requires its TTL. Acquisition
atomically checks the lock, increments the fencing counter, and performs SET NX PX. A lease can release or extend only while its random owner token still
matches.
use std::time::Duration;
use redis_tower::MultiplexedClient;
use redis_tower_primitives::DistributedLock;
async fn example() -> Result<(), Box<dyn std::error::Error>> {
let mut client = MultiplexedClient::connect("127.0.0.1:6379").await?;
let lock = DistributedLock::new(
"{job:17}:lock",
"{job:17}:fence",
Duration::from_secs(15),
)?;
if let Some(lease) = lock.acquire(&mut client).await? {
let fencing_token = lease.fencing_token();
// The protected service must reject tokens lower than the greatest token
// it has already accepted.
write_with_fence(fencing_token).await?;
lease.release(&mut client).await?;
}
Ok(())
}
async fn write_with_fence(_token: u64) -> Result<(), Box<dyn std::error::Error>> { Ok(()) }
Lock acquisition touches two keys. In Redis Cluster they must share a hash tag,
as {job:17}:lock and {job:17}:fence do.
Renewal is opt-in. LockLease::spawn_renewal consumes the lease and starts a
task only at that explicit call. Its owned handle cancels and aborts renewal on
drop. shutdown().await stops cleanly and returns the lease for release.
Lock failure mode
The TTL bounds ownership; it does not prove that a paused or partitioned owner has stopped. Such an owner can resume after a higher-token replacement begins work. The protected resource must enforce fencing tokens. A lost acquisition response is indeterminate, and Redis failover can roll back an unreplicated fencing increment if deployment durability permits it.
Leader election
LeaderElection::new(key, ttl, renewal_interval) requires both timing choices.
No task starts during construction. A successful campaign(executor) call
atomically claims the single election key and returns a Campaign containing
the owned Leadership renewal handle and a separate LeadershipEvents
receiver.
use std::time::Duration;
use redis_tower::MultiplexedClient;
use redis_tower_primitives::{LeaderElection, LeadershipEvent};
async fn example() -> Result<(), Box<dyn std::error::Error>> {
let candidate = MultiplexedClient::connect("127.0.0.1:6379").await?;
let election = LeaderElection::new(
"{scheduler}:leader",
Duration::from_secs(15),
Duration::from_secs(5),
)?;
if let Some(campaign) = election.campaign(candidate).await? {
let (leadership, mut events) = campaign.into_parts();
assert_eq!(events.recv().await, Some(LeadershipEvent::Elected));
run_one_scheduler_term().await?;
let _outcome = leadership.abdicate().await?;
assert_eq!(events.recv().await, Some(LeadershipEvent::Demoted));
}
Ok(())
}
async fn run_one_scheduler_term() -> Result<(), Box<dyn std::error::Error>> { Ok(()) }
LeadershipEvent::RenewalFailed means the process must stop leader-only work
immediately. LeadershipEvent::Demoted reports expiration, replacement, or
completed abdication. abdicate().await confirms cleanup. Dropping leadership
requests the same compare-and-delete operation in the detached owned task, and
the TTL bounds the key if that best-effort cleanup cannot run.
Leader-election failure mode
A TTL lease cannot prove that a paused or partitioned process has stopped.
Redis failover can also lose an unreplicated campaign or renewal. Applications
must make leader work idempotent or protect downstream writes with another
stale-owner mechanism. A lost campaign response is indeterminate, so the caller
must not infer election without a successful return and Elected event.
Expirable semaphore
ExpirableSemaphore::new(key, permit_limit, ttl) stores random permit tokens in
one sorted set. Acquisition uses Redis TIME, removes expired holders, and
checks capacity atomically. A SemaphorePermit supports token-checked renew
and release; drop leaves recovery to the required TTL.
use std::time::Duration;
use redis_tower::MultiplexedClient;
use redis_tower_primitives::ExpirableSemaphore;
async fn example() -> Result<(), Box<dyn std::error::Error>> {
let mut client = MultiplexedClient::connect("127.0.0.1:6379").await?;
let semaphore = ExpirableSemaphore::new(
"{worker-pool}:permits",
16,
Duration::from_secs(30),
)?;
if let Some(permit) = semaphore.try_acquire(&mut client).await? {
run_bounded_work().await?;
permit.release(&mut client).await?;
}
Ok(())
}
async fn run_bounded_work() -> Result<(), Box<dyn std::error::Error>> { Ok(()) }
All users of a semaphore key must agree on its limit and TTL. There is no
automatic permit-renewal task; the holder chooses whether and when to call
renew.
Semaphore failure mode
Expiry recovers capacity after a crash, but a paused holder can resume after a replacement acquires the freed permit. The protected operation must tolerate stale overlap. Lost acquire, renew, and release responses are indeterminate and must not be blindly retried when duplicate bounded work would be unsafe.
Countdown latch
CountDownLatch::new(key, initial_count, ttl) requires a positive count and an
absolute TTL. initialize never resets an existing latch. count_down uses
DECR without allowing the count below zero. wait requires a poll interval
and timeout, starts no task, and reports count-zero release, expiry, and timeout
as distinct outcomes. Its timeout bounds polling sleeps but does not cancel an
in-flight Redis request.
use std::time::Duration;
use redis_tower::MultiplexedClient;
use redis_tower_primitives::{CountDownLatch, LatchWaitOutcome};
async fn example() -> Result<(), Box<dyn std::error::Error>> {
let mut client = MultiplexedClient::connect("127.0.0.1:6379").await?;
let latch = CountDownLatch::new("{deploy:42}:ready", 3, Duration::from_secs(60))?;
latch.initialize(&mut client).await?;
match latch
.wait(
&mut client,
Duration::from_millis(100),
Duration::from_secs(30),
)
.await?
{
LatchWaitOutcome::Released => start_traffic().await?,
LatchWaitOutcome::Expired | LatchWaitOutcome::TimedOut { .. } => abort_deploy().await?,
}
Ok(())
}
async fn start_traffic() -> Result<(), Box<dyn std::error::Error>> { Ok(()) }
async fn abort_deploy() -> Result<(), Box<dyn std::error::Error>> { Ok(()) }
Latch failure mode
The TTL is an abandonment bound, not a successful release signal. Waiters
receive Expired for a missing key and must decide how to recover. A lost
countdown response is indeterminate because the decrement may already have
committed; retrying can release the latch earlier than intended.
Delayed queue
DelayedQueue::new(key, retention) requires the post-deadline interval during
which a message remains claimable. enqueue accepts binary payloads and a
caller-selected delay, then uses Redis TIME to calculate its sorted-set score.
Random member prefixes preserve duplicate payloads. claim_due atomically
removes a bounded batch in deadline order and starts no poll task.
use std::time::Duration;
use redis_tower::MultiplexedClient;
use redis_tower_primitives::DelayedQueue;
async fn example() -> Result<(), Box<dyn std::error::Error>> {
let mut client = MultiplexedClient::connect("127.0.0.1:6379").await?;
let queue = DelayedQueue::new(
"{mail}:delayed",
Duration::from_secs(60),
)?;
queue
.enqueue(&mut client, b"message-42", Duration::from_secs(5))
.await?;
let claim = queue.claim_due(&mut client, 32).await?;
for payload in claim.payloads() {
deliver(payload).await?;
}
record_expired(claim.expired());
Ok(())
}
async fn deliver(_payload: &[u8]) -> Result<(), Box<dyn std::error::Error>> { Ok(()) }
fn record_expired(_count: u64) {}
The queue touches one key and is cluster-safe. Every participant sharing that key must use the same retention. The key TTL tracks the latest scheduled deadline plus retention; a claim also reports messages it actively pruned for exceeding retention. Redis may expire the whole key without a later claim, so that reported count is diagnostic rather than a complete loss counter.
Delayed-queue failure mode
Claims are deliberately at-most-once: selected messages are removed before the script returns. A lost claim response can therefore lose the batch. A lost enqueue response is indeterminate and retrying can create a duplicate. The caller owns polling, processing, retries, and any dead-letter policy; use Redis Streams or another acknowledged queue when at-least-once delivery is required.
Block-allocated ID generator
IdGenerator::new(key, block_size) reserves a contiguous positive range with
exactly one INCRBY. IdBlock then yields those IDs locally without more Redis
I/O or a background task.
use redis_tower::MultiplexedClient;
use redis_tower_primitives::IdGenerator;
async fn example() -> Result<(), Box<dyn std::error::Error>> {
let mut client = MultiplexedClient::connect("127.0.0.1:6379").await?;
let generator = IdGenerator::new("{orders}:id", 1_000)?;
let mut block = generator.allocate(&mut client).await?;
while let Some(id) = block.next() {
create_order(id).await?;
}
Ok(())
}
async fn create_order(_id: u64) -> Result<(), Box<dyn std::error::Error>> { Ok(()) }
The counter intentionally has no TTL. The key must be reserved for compatible ID generators and kept for at least as long as issued IDs can matter.
ID-generator failure mode
A lost INCRBY response makes one block indeterminate. Allocating another
block is uniqueness-safe but leaves a gap; callers must never infer the unknown
range. Deleting, expiring, or restoring an older counter can reissue IDs, and
Redis failover can roll back an increment when durability permits it. Configure
persistence and replication for the required uniqueness boundary.
GCRA rate limiter
GcraRateLimiter::new(key, quota, window) requires both quota and window. Its
single Lua call reads Redis TIME, prunes obsolete virtual arrivals from one
sorted set, and atomically admits or denies a request. A quota of 100 over one
second sustains 100 requests per second and permits an initial burst of 100.
use std::time::Duration;
use redis_tower::MultiplexedClient;
use redis_tower_primitives::GcraRateLimiter;
async fn example() -> Result<(), Box<dyn std::error::Error>> {
let mut client = MultiplexedClient::connect("127.0.0.1:6379").await?;
let limiter = GcraRateLimiter::new("{tenant:7}:rate", 100, Duration::from_secs(1))?;
let decision = limiter.check(&mut client).await?;
if !decision.is_allowed() {
tokio::time::sleep(decision.retry_after()).await;
}
Ok(())
}
The script touches one key and is cluster-safe. A shared Redis limiter enforces
cross-process quota, while
tower-resilience-ratelimiter
can put its RateLimiterLayer in backpressure mode for process-local admission
pressure; production services commonly use both.
Rate-limit failure mode
Redis failures are surfaced without a fail-open/fail-closed policy. Applications must choose that policy at their boundary. A connection loss is indeterminate: the request may have consumed a cell before the response disappeared, so an automatic retry can double-consume quota. Failover can lose recent admissions when Redis durability permits data loss.
Production tuning
redis-tower’s defaults favor predictable behavior and broad compatibility. Production tuning is the process of putting explicit bounds around your own latency, concurrency, payload, and failure budgets. There is no universally correct pool size or timeout: measure a representative workload, change one limit at a time, and keep enough observability to tell saturation from Redis latency or network failure.
This page describes the current public configuration surface. The values in examples are starting points to test, not recommendations to copy unchanged.
Start with the right client
Client choice has a larger effect than most numeric knobs:
| Workload | Start with | Why |
|---|---|---|
| Many tasks issuing short, independent commands | MultiplexedClient | One auto-pipelined connection; cheap clones; real queue backpressure |
| Long-running service that must reconnect | Factory-backed MultiplexedClient | Retains high concurrency while rebuilding the worker’s connection |
| Repeated standalone reads with tolerable bounded staleness | CachedMultiplexedClient | Cloneable auto-pipelined misses plus local hits and RESP3 invalidation tracking |
| Modest traffic where the simplest reconnecting handle matters most | ResilientRedisClient | Built-in reconnect and single-flight recovery, but commands serialize through one mutex |
| Blocking commands or expensive reply parsing | ConnectionPool<RedisConnection> | Multiple independent connections isolate head-of-line blocking |
| Stateful sequence, pub/sub, or MONITOR | Dedicated RedisConnection-based API | Exclusive protocol/session ownership |
| High-concurrency Redis Cluster | MultiplexedClusterClient | Per-node auto-pipeline workers, redirects, and topology refresh |
| Repeated master-routed Cluster reads with bounded staleness | CachedMultiplexedClusterClient | One slot-aware shared cache with complete per-master RESP3 invalidation coverage |
| Sentinel-managed primary with failover | MultiplexedSentinelClient::connect_with_reconnect | Re-discovers and verifies the primary after failure or READONLY |
Never send BLPOP, BRPOP, blocking XREAD / XREADGROUP, or another
blocking command through a shared MultiplexedClient. One blocked Redis
connection stalls its pipeline worker and every clone. Isolate blocking traffic
on dedicated or pooled connections, and ensure the pool has enough independent
slots for the maximum intended number of simultaneous blockers.
Likewise, keep PubSubConnection and MonitorStream dedicated. Redis changes
the connection’s mode for pub/sub and MONITOR, and MONITOR itself has a large
server-side throughput cost.
Plan for Smart Client Handoff maintenance
Planned-maintenance handoff is opt-in for a factory-backed
MultiplexedClient. Construct it with
MultiplexedClient::from_factory_with_maintenance and retain the returned
MaintenanceListenerHandle. The initial connection and every replacement must
be RESP3 and must accept
CLIENT MAINT_NOTIFICATIONS ON moving-endpoint-type none; construction or
reconnection fails if registration does not succeed.
For each valid MOVING notification, redis-tower waits until half of the
server-supplied TTL and then creates a replacement through the original
factory. That factory must reproduce all required authentication, TLS, URL,
and other per-connection session setup. The none endpoint mode means the
server does not select a destination: service discovery or endpoint changes
must happen inside the factory. Set its connect timeout and reconnect budget so
it can establish a usable connection within the remaining half of the TTL.
Once MOVING has been accepted, the worker lets any active batch make one
completion attempt and does not replay it for the handoff. Newly queued work is
held until replacement succeeds; if the reconnect budget is exhausted, that
work fails instead of being sent on an unverified connection. Account for this
pause when choosing queue capacity and end-to-end deadlines.
MIGRATING is observational only and does not reconnect. Use
from_factory_with_maintenance_and_events and its lifecycle event bus if the
application needs the notification metadata. This support does not relax
command timeouts and does not cover pools, Cluster, blocking connections, or
Pub/Sub.
Dropping the maintenance handle disables future handling without shutting down
the client and cancels a half-TTL wait that has not committed. Prefer
MaintenanceListenerHandle::shutdown().await during orderly shutdown: it
waits for worker acknowledgement and, if replacement has already begun, for a
connected or terminal result. Shut down the client separately.
Establish a baseline before tuning
Capture at least these workload characteristics under realistic concurrency:
- operations per second and the mix of command names;
- p50, p95, p99, and maximum command latency;
- number of concurrent application requests;
- request and response byte distributions, including worst-case collections;
- fraction of commands that block or perform expensive server work;
- auto-pipeline batch size and queue depth;
- client-cache hit/miss, invalidation, eviction, size, and tracking-health data;
- pool acquisition wait and in-flight commands, when using a pool;
- reconnect state, offline queue depth, lifecycle-event lag, timeout, cluster redirect, and topology refresh rates;
- Redis CPU, memory, evictions, connected clients, and slow-log entries.
Tune against the tail, not only average latency. A setting that improves mean throughput while growing the queue can make p99 latency and overload recovery worse.
Tune automatic pipelining
MultiplexedClient wraps AutoPipelineService. Its defaults are:
| Setting | Default | Effect |
|---|---|---|
max_batch_size | 100 frames | Upper bound on one flush |
batch_window | zero | Flush immediately after draining work already available |
queue_capacity | 1024 requests | Bound on pending worker requests |
shed_load_on_full | false | Apply backpressure instead of returning QueueFull |
response_timeout | None | No deadline for a whole batch response |
Configure the worker before wrapping the connection:
use std::time::Duration;
use redis_tower::{AutoPipelineConfig, MultiplexedClient, RedisConnection};
let connection = RedisConnection::connect("127.0.0.1:6379").await?;
let client = MultiplexedClient::from_connection_with_config(
connection,
AutoPipelineConfig {
max_batch_size: 128,
batch_window: Duration::from_micros(250),
queue_capacity: 2048,
shed_load_on_full: false,
response_timeout: Some(Duration::from_millis(500)),
..AutoPipelineConfig::default()
},
);
Batch size and window
- Increase
max_batch_sizewhen many commands arrive concurrently and Redis or network round trips dominate. Watch response size and p99 latency: larger batches occupy the connection longer and one slow response delays everything behind it. - Keep
batch_windowat zero for latency-sensitive or lightly concurrent traffic. A short non-zero window can improve batching for bursty writes, but it deliberately adds that wait to the first request in a batch. - Use the
redis_tower.pipeline.batch_sizehistogram to verify that a change actually creates batches larger than one. Raising a limit that workloads never reach only adds theoretical capacity.
Queue capacity and overload policy
The queue is a concurrency bound, not a throughput control. A larger queue can absorb short bursts, but it also permits more memory use and queueing latency. Start from a measured burst budget and alert well before sustained depth reaches capacity.
With shed_load_on_full: false, Tower readiness waits for capacity and paces
producers. This is the preferred default when upstream callers honor
backpressure. Set it to true only when the application has an explicit
load-shedding response for RedisError::QueueFull and would rather reject than
wait.
MultiplexedClient::queue_depth() gives an instantaneous snapshot. With the
metrics feature, spawn_queue_depth_exporter publishes
redis_tower.pipeline.queue_depth; give each pipeline a stable low-cardinality
name.
Response deadline
response_timeout covers the complete round trip for a flushed batch. When it
expires, the in-flight batch fails and a factory-backed worker discards and
rebuilds its connection. Set the deadline above the slowest command permitted
on that worker. A deliberately long blocking command and a 200 ms response
deadline are incompatible; separate those workloads instead of continually
raising the shared deadline.
Use pools for isolation, not by default
A pool adds independent connections and server client state. More connections are useful only when one connection is the bottleneck or commands must be isolated; they also cost sockets, Redis memory, TLS handshakes, authentication, health checks, and more opportunities for uneven load.
use std::time::Duration;
use redis_tower::{ConnectionConfig, ConnectionPool, RedisConnection};
use redis_tower::pool::{DispatchStrategy, PoolConfig};
use redis_tower::reconnect::AddrConnectionFactory;
let factory = AddrConnectionFactory::new("127.0.0.1:6379")
.with_connection_config(
ConnectionConfig::new()
.with_connect_timeout(Some(Duration::from_secs(3))),
);
let pool = ConnectionPool::<RedisConnection>::connect_with_factory(
PoolConfig::default()
.name("blocking-jobs")
.bounds(2, 8)
.dispatch(DispatchStrategy::LeastConnections)
.health_check_interval(Duration::from_secs(30))
.idle_timeout(Duration::from_secs(60))
.acquisition_timeout(Duration::from_millis(250)),
factory,
).await?;
// Background work is never implicit. Retain each handle for its lifetime.
let reaper = pool.spawn_idle_reaper(Duration::from_secs(15));
let prober = pool.spawn_health_prober(Default::default());
let _ = (reaper, prober);
Pool guidance:
RoundRobinis the cheapest and works well when command latency is uniform.LeastConnectionsis a better starting point for mixed short and long work; it avoids deliberately sending new work to the busiest slot.- Keep the default bounded acquisition timeout or choose a service-specific bound. Disabling it can turn saturation into an invisible hang.
- Configure a factory when health checks should replace a dead slot. A closure- built pool without a retained factory can detect failure but cannot restore capacity automatically.
- Dynamic sizing is opt-in.
bounds(min, max)starts withminconnections and creates more only when every live slot is contended. Anidle_timeoutis inert untilspawn_idle_reaperis called; dropping its handle stops shrinking immediately. - Active health probing is also explicit. The default PING probe, ROLE probe,
replication-lag-via-INFO probe, and custom
HealthProbeimplementations update metrics andPoolStats; they do not silently become a second circuit breaker or change dispatch policy. Run the byte-lag probe against primaries: it compares primary-side offsets for directly connected replicas and marks missing/offline replicas unhealthy. Replica-local INFO cannot establish the upstream primary’s current byte offset and is rejected conservatively. - Size from observed concurrency and Redis capacity. For blocking commands, the minimum useful size is usually the maximum number of blockers you intend to allow, plus deliberate headroom for health/control traffic. Enforce a higher- level concurrency limit rather than letting callers grow without bound.
- Use a stable, unique
PoolConfig::name; it becomes the pool metrics label.
Inspect pool.stats() for current/min/max size, idle and in-flight counts,
active-probe health, replication lag, and cumulative idle reaping. Acquisition
wait increasing while Redis command latency stays flat indicates pool
contention. Both increasing usually means Redis or the network is the
bottleneck, and adding connections may make it worse.
Set timeouts at distinct failure boundaries
One timeout cannot represent all phases of a Redis operation:
| Boundary | API | What it limits |
|---|---|---|
| TCP/Unix connect | ConnectionConfig::with_connect_timeout | Initial transport establishment |
| Reconnect attempt | ReconnectConfig::connect_timeout | One factory call during recovery |
| Auto-pipeline response | AutoPipelineConfig::response_timeout | A whole flushed batch’s response |
| End-to-end typed call | WithDeadline<Cmd> | One absolute budget carried through middleware, queueing, routing, retries, pool waiting, and I/O |
| Tower call | CommandTimeoutLayer | A static upper bound; .with_request_deadlines() lets an earlier WithDeadline shorten it |
| Pool acquisition | PoolConfig::acquisition_timeout | Waiting to reserve a pool slot |
Choose each from a remaining end-to-end latency budget. For example, pool wait plus command execution plus application processing must fit inside the request deadline; making all three independently equal to that deadline silently multiplies the worst case.
Carry one caller budget with WithDeadline and put CommandTimeoutLayer
outside ExecutorService. This ordering lets the timeout layer inspect the
typed command before ExecutorService adapts it, while the pool sees the same
absolute deadline during acquisition:
use std::time::Duration;
use redis_tower::{
CommandTimeoutLayer, ConnectionPool, ExecutorService, RedisConnection,
WithDeadline,
};
use redis_tower::commands::Get;
use tower_layer::Layer;
use tower_service::Service;
let pool = ConnectionPool::connect(4, || {
RedisConnection::connect("127.0.0.1:6379")
}).await?;
let mut service = CommandTimeoutLayer::new(Duration::from_secs(1))
.with_request_deadlines()
.layer(ExecutorService::new(pool));
let command = WithDeadline::after(Get::new("key"), Duration::from_millis(250));
let value = service.call(command).await?;
The earliest limit wins rather than starting a fresh budget at each stage:
- nested
WithDeadlineenvelopes retain the earliest absolute instant, and cloning or retrying them does not reset it; CommandTimeoutLayer::with_request_deadlinesuses the earlier of that instant and its static duration, returningRedisError::CommandTimeout;- a pool uses the earlier of the command deadline and its static acquisition
timeout. The command deadline returns
CommandTimeout; the pool’s own limit returnsPoolAcquisitionTimeout.
The plain CommandTimeoutLayer remains generic over arbitrary Tower request
types and applies only its configured duration. Opting in to request deadline
extraction adds the RequestDeadline bound; all typed commands implement it,
and raw RESP frames report no typed deadline.
CommandAdapter is the typed-to-frame boundary. Frame middleware inside the
adapter cannot inspect command metadata, although the adapter enforces the
absolute command deadline around its inner frame call. Likewise,
MultiplexedClient::execute applies it across readiness and dispatch. Put
metadata-aware middleware outside the adapter (normally outside
ExecutorService) when the middleware itself needs to observe the deadline.
An already-expired command is rejected before the inner service is called.
This prevents newly enqueueing a side effect after its caller budget is gone.
Cluster clients retain the same deadline through node lookup, routing,
redirects, and pinned-node execution; multiplexed Sentinel dispatch and the
resilient client’s offline queue retain it through their own waits as well.
If a deadline interrupts a direct or pool-member exchange after bytes reach
the wire, that socket is quarantined instead of allowing its late response to
poison a later request. On MultiplexedClient, the caller abandons its response
but the worker retains the in-flight batch, reads its replies for alignment,
and can keep a still-usable connection. In both cases, server execution remains
unknown to the timed-out caller.
The caller-level timeout still is not a substitute for worker-level
response_timeout, which detects a connection whose batch never returns. Use
both when a multiplexed production client needs bounded caller latency and
bounded connection recovery.
Know the request lifecycle and execution boundary
Cancellation is local; Redis execution is remote. Use these states when deciding whether a failed or abandoned operation may be attempted again:
| State | What redis-tower knows | Safe conclusion |
|---|---|---|
| Waiting for Tower readiness or worker queue admission | The request has not been accepted by that component | Dropping the future prevents that component from sending it |
Accepted by ConnectionPool, waiting for an available member | The pool counts the operation as accepted for graceful drain, but no member has received it | Dropping the future prevents wire dispatch and releases the accepted-operation count |
| Accepted but still in the auto-pipeline batch window | The worker owns the request, but prunes a closed response channel before building the wire batch | A request proven pruned is unsent |
| Partially or fully written | Redis may have received some or all request bytes | Execution is unknown unless a Redis reply later establishes an outcome |
| Waiting for the reply | Redis may already have applied the command | A caller timeout or dropped future does not roll it back |
| Complete reply decoded | The worker/connection knows the Redis outcome | Delivery to a now-cancelled caller can still be lost locally; do not replay merely because the caller disappeared |
| Reconnecting | The failed exchange is not implicitly reconstructed | Only an explicit typed retry/offline policy or a protocol redirect can send a command again |
| Shutdown requested | New admission is closed; behavior for accepted work depends on the owning handle | Await the documented shutdown/close method when draining matters |
The direct RedisConnection implementation quarantines its socket when a
typed call is cancelled after I/O begins, so a late response cannot satisfy a
later command. The auto-pipeline worker drops cancelled requests that are
still queued; after a batch starts it owns response alignment and continues to
read the batch while the transport is usable. A batch response timeout or
transport failure discards that connection and fails the affected callers. A
factory may create a replacement for later work, but it does not silently
replay that failed batch.
Client families add these explicit policies:
RetryClientreplays only when the command reportsidempotent() == true, the error is classified as retryable, and its retry/deadline budget remains.RedisError::is_retryable()says an attempt may be useful; it does not say a mutation is safe to repeat.ResilientRedisClientfails new work during reconnect by default. Its opt-in offline queue admits only idempotent typed commands, has bounded capacity and per-command replacement-wire attempts, removes cancelled queued work, and quarantines a socket when cancellation happens on the wire.- Cluster MOVED/ASK and bounded transient-error handling are protocol replies, not guesses after a lost reply. The Cluster connection never replays a transaction redirect; it updates topology for a newly built future attempt. A connection loss with no reply remains unknown execution.
- A
ConnectionPoolreserves one member for oneexecutecall and releases its in-flight accounting on cancellation. It exposes no checkout/lease API, so it cannot hold WATCH or other connection-local state across separate calls.ConnectionPool::closestops admission and waits for accepted operations to finish.
Connection replacement restores only state owned by its constructor/factory:
authentication, protocol version, database, TLS, decode limits, and any custom
setup performed there. The caching clients own and re-establish their tracking
sessions as described in Client-side caching.
PubSubConnection::reconnect_with explicitly replays confirmed subscriptions,
but messages published during the gap are lost. MonitorStream has no replay
or resume cursor. Ad hoc AUTH, SELECT, HELLO, or tracking commands do not
modify a factory and therefore are not automatically restored.
The auto-pipeline queued-cancellation contract and lost-reply contract distinguish unsent work from a lost non-idempotent reply. Treat any behavior not covered by the owning client’s source/tests as unknown rather than assuming another client family’s guarantee.
Do not retry every timeout automatically. Redis may have executed a write before the response was lost. The typed retry wrapper checks command idempotency and retryable error classification before replaying:
use std::time::Duration;
use redis_tower::RetryPolicy;
let client = client.retry(
RetryPolicy::default()
.max_retries(2)
.base_delay(Duration::from_millis(25))
.max_delay(Duration::from_millis(250)),
);
Keep jitter enabled in production so a Redis restart does not synchronize all
application instances into a reconnect or retry storm. RetryService also
checks the command’s absolute deadline while waiting for readiness, executing
each attempt, and sleeping between attempts. A retry layer cannot reset or
sleep past the caller’s WithDeadline, regardless of its position relative to
the timeout layer. RetryClient::execute also bounds its initial readiness
wait. With the lower-level Tower Service API, poll_ready necessarily occurs
before the request is supplied, so request metadata starts governing the retry
service once call begins; place a separate outer bound around a potentially
blocking caller-side readiness wait.
Make reconnection replay connection state
Configure logical client identity in ConnectionConfig, not with a one-time
CLIENT SETNAME on a borrowed connection:
async fn example() -> Result<(), redis_tower::RedisError> {
use redis_tower::{ConnectionConfig, ResilientRedisClient};
use redis_tower_cluster::MultiplexedClusterClient;
let standalone = ResilientRedisClient::connect_url_with_connection_config(
"redis://127.0.0.1:6379/0",
ConnectionConfig::new().with_client_name("redis-mcp"),
).await?;
let cluster = MultiplexedClusterClient::builder("127.0.0.1:7000")
.client_name("redis-mcp")
.connect().await?;
let _ = (standalone, cluster);
Ok(())
}
For a richer policy use ConnectionSetup::new().with_client_name(...),
optionally .with_client_no_evict(true) (Redis 7.0+) or
.with_client_no_touch(true) (Redis 7.2+), then attach it with
ConnectionConfig::with_setup or either Cluster builder’s connection_setup.
Builder calls replace the corresponding setting; connection_config replaces
the entire config, so put name/setup calls after it when overriding it.
Built-in factories retain this policy across initial and replacement sockets.
Both Cluster clients apply it to seed discovery, all connected masters and
replicas, MOVED/ASK targets, topology-discovered nodes, and replacements.
MultiplexedClusterClient::connect_to_node and its dedicated-session factories
also inherit it. Setup does not create replica connections when the selected
read preference only needs masters.
The order is optional best-effort library CLIENT SETINFO, authentication,
URL database selection, RESP negotiation, declared setup, then Cluster replica
READONLY. Explicit setup is not best effort: SETNAME, NO-EVICT, and
NO-TOUCH run sequentially, with each +OK checked before another is sent.
Unsupported commands, ACL denial, unexpected replies, or transport/protocol
failure close that socket and return RedisError::Connection. No ordinary
traffic is sent on a partially configured socket. Setup cancellation also
discards the socket. Already healthy Cluster nodes may continue serving while
a new node fails setup; that failed node is never admitted unconfigured.
Debug output redacts names, and setup errors show the failing command but omit
arguments and server reply text (which can echo values). Names still appear in
Redis CLIENT LIST; never include secrets. Grant +client|setname and only the
other configured subcommands in the ACL. There is no silent downgrade on older
servers. Bound full setup with a factory/reconnect timeout and configure a
finite reconnect budget for persistent ACL/configuration failures; the socket
connect timeout alone does not bound Redis setup replies.
The typed policy intentionally excludes arbitrary commands: AUTH/SELECT/HELLO,
transaction state, reply suppression, tracking, Pub/Sub, and blocking operations
have separate lifecycle owners. Custom factories remain responsible for setup
they add themselves and can call RedisConnection::apply_setup after completing
their own handshake. One-time commands on a borrowed socket are not recorded
for reconnect. Sentinel and topology-neutral client builders do not yet expose
this policy; this contract covers the standalone and Cluster APIs above.
Use a connection factory for long-running multiplexed clients. The factory is responsible for recreating all required session state on every connection:
- URL username/password and database selection;
- TLS roots and client certificates;
- RESP mode and decode limits;
- rotating credentials supplied by
CredentialConnectionFactory, or other initialization performed by a custom factory.
use std::time::Duration;
use redis_tower::{
AutoPipelineConfig, ConnectionConfig, KeepaliveConfig, MultiplexedClient,
RespLimits,
};
use redis_tower::auto_pipeline::AutoPipelineReconnectConfig;
use redis_tower::reconnect::{ReconnectConfig, UrlConnectionFactory};
let connection_config = ConnectionConfig::new()
.with_client_name("redis-mcp")
.with_connect_timeout(Some(Duration::from_secs(3)))
.with_keepalive(
KeepaliveConfig::new()
.with_idle(Duration::from_secs(30))
.with_interval(Duration::from_secs(10))
.with_probes(3),
)
.with_resp_limits(RespLimits {
max_frame_size: 16 * 1024 * 1024,
max_depth: 64,
});
let factory = UrlConnectionFactory::new(
"redis://default:secret@127.0.0.1:6379/0",
).with_connection_config(connection_config);
let reconnect = ReconnectConfig::default()
.base_delay(Duration::from_millis(100))
.max_delay(Duration::from_secs(5))
.connect_timeout(Duration::from_secs(3));
let client = MultiplexedClient::from_factory(
factory,
AutoPipelineConfig {
response_timeout: Some(Duration::from_secs(1)),
..AutoPipelineConfig::default()
},
AutoPipelineReconnectConfig::new(reconnect),
).await?;
Use CredentialConnectionFactory when authentication comes from a dynamic
provider rather than a static URL:
use redis_tower::{
AutoPipelineConfig, ConnectionConfig, CredentialConnectionFactory,
MultiplexedClient, ProtocolVersion, StaticCredentials,
};
use redis_tower::auto_pipeline::AutoPipelineReconnectConfig;
// Replace StaticCredentials with an application or cloud CredentialProvider.
let factory = CredentialConnectionFactory::new(
"127.0.0.1:6379",
StaticCredentials::password("token"),
)
.with_connection_config(
ConnectionConfig::new().with_protocol(ProtocolVersion::Resp3),
);
let client = MultiplexedClient::from_factory(
factory,
AutoPipelineConfig::default(),
AutoPipelineReconnectConfig::default(),
).await?;
The credential factory opens each transport in RESP2, fetches credentials,
runs AUTH, and then negotiates the protocol requested by
ConnectionConfig. The complete sequence runs on initial connection and every
reconnect, so each fresh socket consults the provider. The same factory also
implements PoolFactory, allowing lazy or replacement pool slots to use the
same setup.
An authentication rejection during connection establishment is handled
narrowly: NOAUTH or WRONGPASS calls the provider’s force_refresh() and
retries AUTH once. Providers shared by multiple clients or pool slots must
synchronize concurrent refreshes. A StreamingCredentialProvider can
proactively send replacement credentials to established standalone,
multiplexed, pooled, Cluster, and Sentinel data sockets. That update is an
explicit AUTH, never a replay of the command that exposed an authentication
error. If an already-established connection returns NOAUTH or WRONGPASS
for a user command, redis-tower returns the error to the caller.
The companion redis-tower-auth-aws and redis-tower-auth-azure crates
implement ElastiCache IAM and Microsoft Entra ID token acquisition, caching,
and proactive refresh. See Cloud and rotating credentials for
complete setup and failure semantics.
max_retries counts retries after the first reconnect attempt: zero still
allows one attempt, and n allows at most n + 1 attempts. The initial
connection is outside that retry count. base_delay is applied before the
first reconnect attempt, and connect_timeout bounds every factory call,
including initial construction and each reconnect.
TCP keepalive detects dead peers below the Redis protocol, but its effective timing is platform-dependent and usually much slower than an application deadline. It complements response and reconnect timeouts; it does not replace them.
Decide whether to queue while offline
ResilientRedisClient fails later work while reconnecting by default. For a
workload that benefits from waiting through a short outage, enable its bounded
offline queue explicitly:
use redis_tower::{OfflineQueueConfig, ResilientRedisClient};
let client = ResilientRedisClient::connect_url_with_offline_queue(
"redis://default:secret@127.0.0.1:6379/0",
OfflineQueueConfig::new(256).with_max_replay_attempts(3),
).await?;
The queue is shared across clones and replays admitted work in queue-ticket order after one reconnect campaign. Admission is serialized, but concurrently first-polled futures can receive tickets in either scheduler order. Admission is deliberately narrow:
- only typed commands marked idempotent can wait and replay;
- non-idempotent commands and raw frame-service calls fail offline;
- overflow returns
RedisError::QueueFullimmediately; - reconnect exhaustion returns
RedisError::ReconnectFailedto queued work; - canceling a queued future releases its slot; and
- canceling after a request reaches the wire quarantines that socket and reconnects before the next ticket runs, preventing a late response from being consumed by the wrong caller;
- each command has a finite replacement-wire replay budget (three by default), so repeated successful connects followed by immediate disconnects cannot keep the head ticket alive forever; and
- capacity zero preserves fail-fast behavior while reconnect continues in the background.
A WithDeadline budget includes admission, queue wait, reconnect, replay, and
wire I/O. Expiry removes the ticket, and an expiry after dispatch quarantines
the socket before the next ticket can run.
Size capacity from the number of operations the service can safely absorb over
the outage window, not from normal throughput. Alert on
offline_queue_depth() before it reaches capacity and use is_reconnecting()
as a diagnostic or readiness input. Queueing increases outage latency and
memory and can create a recovery burst, so fail-fast remains the safer choice
for many request/response services. The replay-attempt budget is distinct from
ReconnectConfig::max_retries, which limits retries after the first connection
attempt within one campaign. Configure a finite reconnect budget and
connect_timeout as well if each queued operation needs a bounded recovery
path through an unreachable endpoint.
For a frame-level Tower stack, tower::buffer::Buffer can sit in front of
ReconnectService. It cannot inspect Command::idempotent() after a command
has become a raw frame, so it does not provide the typed queue’s replay-safety
guarantee. Do not treat a generic frame buffer as authorization to replay
mutations.
Bound protocol memory
RespLimits constrains the maximum buffered frame size and nesting depth. The
defaults preserve normal Redis compatibility. Tighten them for shared,
untrusted, or memory-constrained environments, but keep max_frame_size above
the largest legitimate response (large values, collection reads, module
results, and administrative output are common surprises).
Install limits through ConnectionConfig before connecting. For reconnecting
clients put that config on the factory, so recovery cannot silently return to
unbounded/default decoding. Cluster and Sentinel builders expose
.resp_limits(...) and retain the limits across topology changes and failover.
Limit large collection work at the command level as well: prefer bounded scans and pagination over commands that materialize an entire unbounded keyspace or collection in one response.
Retry and circuit-breaker policy
Retries consume capacity precisely when a dependency is unhealthy. Keep retry
budgets small, exponential, and jittered, and reserve them for commands whose
typed implementation reports that replay is safe. A non-idempotent command is
not retried by RetryClient, even when its error is otherwise retryable.
The Redis-aware circuit breaker counts transport failures and command deadlines
that expire after dispatch, not readiness-only waits or ordinary Redis command
errors such as WRONGTYPE. Start with the default and tune only from
incident/load-test evidence:
use redis_tower::RedisCircuitBreakerConfig;
let client = client.with_circuit_breaker(RedisCircuitBreakerConfig {
failure_threshold: 5,
recovery_probe_interval: std::time::Duration::from_secs(5),
});
let health = client.circuit_breaker_handle();
Export health.state(), health.health_status(), or its metrics rather than
using a successful process check as Redis readiness. Keep liveness independent
from Redis so an outage does not restart every application instance in a loop;
use Redis health for readiness and traffic admission.
Cluster tuning
MultiplexedClusterClient creates an auto-pipeline worker per node. Its builder
accepts the same AutoPipelineConfig, a per-node reconnect config, a redirect
budget, read preference, credentials, TLS, and RESP limits.
- Leave
ReadPreference::Masterin place unless the application accepts replica staleness.ReplicaandPreferReplicachange consistency, not only performance. - Use
AdaptiveReplicaRoutingwhen replica reads should compose locality, latency, and health. Redis does not advertise availability zones throughCLUSTER SLOTS; configurelocal_zoneand eachreplica_zoneexplicitly, using the post-host_override/address_mapaddress. Unmapped replicas remain cross-AZ fallback candidates. - Adaptive routing samples new and recovered candidates round-robin, then uses
inverse-EWMA weighting. Lower alpha smooths transient latency spikes; higher
alpha reacts faster. Measure with production traffic before changing the
default
0.2. - Consecutive transport, protocol, reconnect, and deadline failures drive
ejection. Redis command errors are healthy responses and reset the failure
count. Recovery is evaluated lazily after
ejection_duration; there is no probe task to retain.minimum_healthy_replicasis a max-ejection floor: if every candidate would otherwise be excluded, the soonest-recovering entries are made eligible. The default floor of one favors availability and can continue trying a failing singleton replica; set it to zero withPreferReplicawhen master fallback is more important than retaining a replica target. max_redirectsbounds both latency and repeated work during resharding. Raising it can hide unstable topology while increasing tail latency; monitorMOVED,ASK, and topology refresh metrics first.- Use
host_overrideoraddress_mapwhen CLUSTER SLOTS advertises internal addresses that clients cannot reach. Validate every failover and resharding path, not only initial seed discovery. - Per-node metric labels are disabled by default. Enable them only when the
diagnostic value is worth the additional series; redis-tower bounds concrete
node-address labels and folds overflow into
_OTHER. - Call
shutdown()after stopping producers so all per-node workers can drain.
Sentinel tuning
Use MultiplexedSentinelClient::builder(...).connect_with_reconnect() for
production failover. Configure Sentinel-hop and Redis-node credentials/TLS
separately when the deployment does; the client re-queries Sentinel, verifies
ROLE, and reconnects after a transport failure or a READONLY response from
a demoted primary.
Test the complete failure path under load:
- stop or demote the current primary;
- verify in-flight errors are visible and bounded;
- verify Sentinel reaches agreement and the client discovers the new primary;
- verify credentials, TLS, and RESP limits still apply;
- verify writes resume without a retry storm or silent duplicate mutation.
Sentinel discovery availability depends on the set of addresses you provide. Use multiple independently placed Sentinel endpoints and monitor discovery failures separately from data-node failures.
To observe verified primary changes, pass a ConnectionEventBus through
MultiplexedSentinelClient::builder(...).connection_events(events) before
connect_with_reconnect(), or use
MultiplexedSentinelClient::connect_with_reconnect_and_events(...). The first
ROLE-verified primary establishes the baseline; only a later verified address
change publishes ConnectionEvent::Failover. The comparison uses the exact
endpoint strings Sentinel returns after ROLE verification; it is not durable
node identity. DNS/textual aliases for one server can therefore look like a
change, while replacing a node behind the same endpoint does not emit a
failover event.
Observability that drives tuning
Enable the metrics feature and install one application-level metrics
exporter. The built-in recorder can report:
db.client.operation.durationandredis_tower.commandsfor command latency/count/outcome;redis_tower.pipeline.batch_sizeandredis_tower.pipeline.queue_depthfor worker efficiency and saturation;redis_tower.cache.eventsfor bounded hit, miss, invalidation, and eviction counts;db.client.connection.wait_time,db.client.connection.pending_requests, connection count/max, and pool lifecycle counters, including active-probe outcomes, replication lag, and idle reaping. Treat replication lag as current only while its paired*_observedfreshness gauge is1;redis_tower.cluster.redirects, topology refresh count, and topology refresh duration.
Use stable pool and pipeline names. Do not put keys, user IDs, request IDs, or
unbounded host strings into metric labels. Pair client metrics with Redis
INFO, latency monitoring, slow log, CPU, memory, eviction, and network data;
client-side latency alone cannot distinguish its cause.
For cached clients, alert on unhealthy caching before optimizing hit rate.
While the invalidation receiver is being replaced, redis-tower clears and
disables the local cache, so a falling hit rate paired with healthy Redis
latency is an expected safety response rather than silent staleness. A lost
fixed data worker on standalone CachedMultiplexedClient also clears the cache
but requires constructing a new cached client. The Cluster cached client
instead rebuilds complete data-worker and receiver coverage across every
master before reopening its global gate, and requires a finite client_ttl as
a backstop for ownership changes it does not observe. is_caching_healthy()
distinguishes these states. See the
client-side caching guide for tracking modes and
failure semantics.
Tracing can be sampled more aggressively than metrics. Set a slow-command
threshold on TracingLayer to preserve useful tail diagnostics without making
every successful GET equally prominent.
Observe connection lifecycle without gating it
Attach a bounded ConnectionEventBus when constructing a reconnecting client.
Subscribe first if the initial connection result matters because the stream
does not replay old events:
use redis_tower::{
AutoPipelineConfig, ConnectionEventBus, ConnectionEventRecvError,
MultiplexedClient,
};
use redis_tower::auto_pipeline::AutoPipelineReconnectConfig;
use redis_tower::reconnect::{ReconnectConfig, UrlConnectionFactory};
let events = ConnectionEventBus::new(256);
let mut stream = events.subscribe();
let client = MultiplexedClient::from_factory_with_events(
UrlConnectionFactory::new("redis://127.0.0.1:6379"),
AutoPipelineConfig::default(),
AutoPipelineReconnectConfig::new(ReconnectConfig::default()),
events,
).await?;
tokio::spawn(async move {
loop {
match stream.recv().await {
Ok(event) => record_connection_event(event),
Err(ConnectionEventRecvError::Lagged { skipped }) => {
record_event_lag(skipped);
}
Err(ConnectionEventRecvError::Closed) => break,
Err(_) => break,
}
}
});
ConnectionEventBus::default() retains 64 events per subscriber; choose a
larger explicit capacity only from observed event bursts and consumer latency.
Publishing is synchronous and non-waiting, and every subscriber has an
independent cursor. A slow consumer receives an explicit Lagged { skipped }
error but never backpressures reconnect or failover. Treat the stream as an
observability feed rather than a durable audit log, and ensure consumer work
cannot grow an unbounded secondary queue.
Events distinguish initial connect failure, disconnect reason, each scheduled
and failed reconnect attempt, successful reconnection, exhausted retry budget,
intentional worker shutdown, and topology-confirmed endpoint failover.
Disconnected { reason: Shutdown } is a producer’s terminal transition and is
emitted separately after an earlier outage disconnect; when clients share one
bus, other producers may continue publishing afterward.
Human-readable connect/disconnect error strings can contain endpoint names,
server responses, and other deployment details; redact them before exporting
to systems with a broader audience. Aggregate events into stable counters and
durations, and never put their free-form error text or addresses into
unbounded metric labels. Sentinel publishes a single verified primary endpoint
change automatically. Redis Cluster has multiple slot-scoped primaries, so its
topology manager should call ConnectionEventBus::publish only when the
application can state an honest, specific failover transition; do not infer one
global failover from ordinary slot churn.
Graceful shutdown
Shutdown ordering prevents accepted work from disappearing in background queues:
- stop accepting new application requests and stop background producers;
- await tasks that own client clones, or retain a
MultiplexedClient::shutdown_handle()before placing clones in a router; - stop queue/pool metrics exporters that retain client clones;
- call
ConnectionPool::close()to reject new pool work and drain accepted operations; - consume the last multiplexed client with
shutdown().await(cluster and Sentinel clients also exposeshutdown); - flush tracing/metrics providers and exit.
Calling shutdown() while another multiplexed clone remains alive returns
without stopping the shared worker. Make ownership explicit in the application’s
shutdown coordinator, and test SIGTERM behavior while commands are queued and
in flight. For clone-heavy or trait-erased hosts, call the retained
AutoPipelineShutdownHandle::shutdown().await instead. It atomically stops new
admission across all clones, drains already accepted work while the connection
is usable, cancels reconnect/backoff work, and joins the worker. Surviving
client clones then return ConnectionClosed and can be dropped without the
unfinished-worker warning.
A repeatable tuning loop
- Define an end-to-end latency SLO and an overload response.
- Load test the correct client shape with representative command and payload distributions.
- Set connect, acquisition, response, and caller deadlines from that budget;
use
WithDeadlinewhen one absolute caller budget must cross pool waiting. - Choose backpressure, explicit
QueueFullshedding, and fail-fast versus bounded idempotent offline queueing; test saturation and recovery. - Adjust one of batch size, batch window, queue capacity, pool size, or dispatch strategy at a time.
- Compare throughput and latency distributions together with queue/pool/Redis utilization.
- Inject connection loss, a black-holed endpoint, Redis restart, cluster resharding, and Sentinel failover as applicable.
- Verify idempotent retry behavior and ensure mutations are never duplicated.
- Exercise graceful shutdown with live work.
- Record the chosen values, workload assumptions, and alert thresholds next to application configuration so future changes have a baseline.
Serverless and scale-to-zero deployments
Serverless processes are often created speculatively, frozen between requests,
or started while Redis is unavailable. Opening Redis during global
initialization adds cold-start latency and can prevent an otherwise healthy
process from starting. MultiplexedClient therefore has lazy constructors that
create its lightweight worker immediately but defer DNS and socket work until
the first command. Construct the client after entering a Tokio runtime (for
example inside an async main before starting the handler loop); the
synchronous constructor panics when no runtime is entered.
Lazy multiplexed client
For a Redis URL, including credentials, TLS, database selection, or a Unix socket, construct the client synchronously:
use redis_tower::MultiplexedClient;
use redis_tower::commands::Ping;
async fn invoke() -> Result<(), redis_tower::RedisError> {
let client = MultiplexedClient::connect_url_lazy(
"redis://default:secret@cache.internal:6379/0",
);
// The first network connection is made here, not above.
client.execute(Ping::new()).await?;
Ok(())
}
Use MultiplexedClient::connect_lazy("host:port") for an unauthenticated TCP
endpoint. For custom connection setup or reconnect policy, pass a
ConnectionFactory to MultiplexedClient::from_lazy_factory:
use std::time::Duration;
use redis_tower::{AutoPipelineConfig, MultiplexedClient};
use redis_tower::auto_pipeline::AutoPipelineReconnectConfig;
use redis_tower::reconnect::{ReconnectConfig, UrlConnectionFactory};
let factory = UrlConnectionFactory::new("redis://cache.internal:6379/0");
let client = MultiplexedClient::from_lazy_factory(
factory,
AutoPipelineConfig::default(),
AutoPipelineReconnectConfig::new(
ReconnectConfig::default()
.connect_timeout(Duration::from_secs(2))
.base_delay(Duration::from_millis(50)),
),
);
let _ = client;
Lazy construction cannot report a connection error because it performs no
connection attempt. The command that triggers a failed first attempt receives
RedisError::ConnectionClosed; a later command tries the factory again. After
the first successful connection, ordinary factory-backed reconnect behavior
applies when that connection is lost.
Lazy, scale-to-zero pool
When commands need independent connections, a factory-backed pool can also start empty. It grows on command demand and acquisition contention, up to the configured maximum:
use std::time::Duration;
use redis_tower::{ConnectionPool, PoolConfig};
use redis_tower::reconnect::UrlConnectionFactory;
let pool = ConnectionPool::connect_lazy(
PoolConfig::default()
.max_size(8)
.idle_timeout(Duration::from_secs(60)),
UrlConnectionFactory::new("redis://cache.internal:6379/0"),
);
assert_eq!(pool.size(), 0); // no DNS or socket work yet
// Reaping is explicit. Retain this handle for as long as the task should run;
// dropping it stops the task and lets the pool remain at its current size.
let reaper = pool.spawn_idle_reaper(Duration::from_secs(15));
let _ = reaper;
The lazy pool’s minimum is zero. An idle reaper may therefore return it to
zero after the configured timeout. Pool construction never starts the reaper
implicitly. Use connect_with_factory plus PoolConfig::bounds(min, max)
when a warm minimum is more important than scale-to-zero behavior.
Health and lifecycle semantics
A lazy client’s connection-health snapshot starts unhealthy. This means
“not connected yet,” not that construction failed. It becomes healthy just
before the first ConnectionEvent::Connected event. With
from_lazy_factory_with_events:
- construction emits no lifecycle event;
- a failed deferred attempt emits
ConnectionEvent::ConnectFailed; - the first successful deferred attempt emits
ConnectionEvent::Connected; - failures after that point use the normal disconnect and reconnect events.
is_connection_healthy() reads that local snapshot without opening a socket;
subscribe_connection_health() observes later changes. In contrast,
health_check() sends PING and therefore triggers the deferred connection.
Do not use a Redis PING as the platform’s process-liveness check: it forces a
cold instance to connect and defeats lazy initialization. Keep liveness local.
Use a Redis-backed readiness or dependency check only when the platform should
route traffic based on current Redis availability.
Invocation and shutdown pattern
Create one client per warm process and clone it into invocation handlers. Clones share one worker and connection, preserving auto-pipelining across concurrent invocations. Avoid constructing a client for every request.
Bound command duration using command deadlines or an outer timeout appropriate
to the function’s remaining execution budget. Configure
ReconnectConfig::connect_timeout; otherwise an unreachable endpoint may wait
for the operating system’s TCP timeout.
When the runtime offers a reliable shutdown hook, call shutdown() after all
client clones are gone so accepted work drains and the worker is joined. Many
serverless platforms freeze or terminate without such a hook, so correctness
must not depend on shutdown running.
Practical checklist
- Store credentials in the platform’s secret store and prefer a
redis://orrediss://URL so AUTH and SELECT are replayed after reconnect. - Keep the client outside the per-invocation handler and clone it cheaply.
- Set a finite connection timeout below the invocation deadline.
- Expect the first command on a cold process to include connection latency.
- Keep process liveness local; use Redis-backed readiness deliberately.
- Size concurrency so one auto-pipelined connection is appropriate, or use a bounded pool when individual blocking commands need independent sockets.
- Never send blocking commands through
MultiplexedClient; use a dedicated connection or a pool sized for the maximum simultaneous blockers. The pool dispatches commands; it does not expose a checkout for multi-call state.
See Production tuning for queue sizing, response timeouts, reconnect policy, and graceful shutdown details.
Pool health probing design
Status: accepted for issues #483 and #512.
Context
ConnectionPool already supports a lazy PING before using a connection that
has been idle longer than PoolConfig::health_check_interval. That check is
traffic-driven: it cannot report or repair a dead connection while the
application is idle.
An earlier experiment at commits 4025d1f and 869556d integrated
tower-resilience-healthcheck. The recovered design and review material show
that package was built as a generic, health-aware multi-resource selector. It
owned resource status, selection policy, thresholds, callbacks, and a
background task. The pool subsequently developed different invariants:
- each connection has a mutex and an in-flight reservation;
- a retained
PoolFactorycan replace or add slots; - dispatch, acquisition deadlines, draining, and metrics are pool-native;
- repository policy requires background work to be explicitly spawned and stopped by dropping an owned handle.
Putting the generic wrapper around the pool would duplicate selection state and would not let a probe safely coordinate with slot replacement or idle reaping. Putting each pool slot inside the wrapper would make two components compete to own dispatch and lifecycle state.
Decision
Active pool probing is implemented inside redis-tower rather than by adding
tower-resilience-healthcheck as a dependency.
The public surface has two layers:
HealthProbe<S>is a small async trait for one connection. Built-in PING, ROLE, and INFO-replication-lag probes cover the Redis-specific checks. The byte-lag probe targets a primary: it compares the primary offset with every directly connected replica offset reported by primary-side INFO. It treats no connected replicas, non-online replicas, and replica-local INFO as unhealthy rather than inventing a zero-byte lag.ConnectionPool::spawn_health_proberandConnectionPool::spawn_health_prober_withare the only APIs that create a prober task. Both returnHealthProberHandle; dropping the handle cancels the task, whileshutdowncancels and joins it.
Creating a pool never starts a task. The existing lazy health check remains
available for request-path validation and replacement. The active prober is
observational: it updates per-slot state, PoolStats, and MetricsRecorder,
but it does not silently change dispatch policy. Applications that need to
stop sending work after an unhealthy result should compose the pool with the
existing circuit breaker.
Dynamic pool shrinking follows the same lifecycle rule. Configuring an idle
timeout does not start work; spawn_idle_reaper returns an owned
IdleReaperHandle. Reaping only removes idle slots above the configured
minimum. Scale-up happens synchronously on acquisition contention, so it does
not require a background task.
Consequences
- Probe and reaper tasks share the pool’s slot synchronization and cannot invalidate an in-flight connection.
- Existing fixed-size configurations remain fixed by default (
min == max == size). Dynamic sizing is opt-in through explicit bounds and requires a retainedPoolFactory. - A lazy factory-backed pool can start at zero connections and creates its first slot on the first command.
- The library owns a small amount of Redis-specific parsing for ROLE and INFO, but avoids a second resource-selection model and an additional dependency.
- Health state is intentionally telemetry, not a second circuit breaker.
Command cookbook
This cookbook connects redis-tower’s typed command categories to the client or
session that should execute them. Command structs remain available from
redis_tower::commands::*; the same structs are also grouped into browsable
category modules such as commands::strings, commands::streams, and
commands::search.
Use the repository examples when you want a complete runnable program. The snippets here focus on the contract that is easy to miss when choosing a command, response type, or connection shape.
The Rust snippets require an external Redis server/topology, so mdBook marks
them ignore instead of trying to execute them. Their matching functions live
in examples/command_cookbook.rs, which CI
compiles with the advertised workspace crates and features.
Pick ownership before commands
| Work | Recommended owner | Important constraint |
|---|---|---|
| Concurrent independent requests | MultiplexedClient | One shared auto-pipelined socket; do not run blocking or stateful multi-call sequences on it |
| One exclusive sequence | RedisConnection | Mutable, single-owner socket; required for WATCH/read/build retries |
| Blocking requests spread across sockets | ConnectionPool | Dispatches each command to an independent member, but exposes no checkout/lease API |
| Cluster or Sentinel traffic | Their multiplexed clients | Routing/failover is topology-aware; Cluster multi-key atomic work must stay in one slot |
| Subscription stream | PubSubConnection / BinaryPubSubConnection | Consumes a dedicated connection and owns subscription state |
| MONITOR stream | MonitorStream | Permanently changes its dedicated socket until it is dropped |
See Production tuning for queueing, deadlines, reconnection, and shutdown. See Cloud and rotating credentials and Client-side caching for those specialized lifecycles.
Typed commands and response shapes
Command types declare their response at compile time. Nil, empty, integer, and server-error replies are not interchangeable:
async fn example() -> Result<(), Box<dyn std::error::Error>> {
use bytes::Bytes;
use redis_tower::{
MultiplexedClient,
commands::{Get, Incr, Set, SetOutcome, SetPreviousValue},
};
let client = MultiplexedClient::connect("127.0.0.1:6379").await?;
let missing_or_value: Option<Bytes> = client.execute(Get::new("profile:1")).await?;
let next: i64 = client.execute(Incr::new("visits")).await?;
// Finish with `with_outcome()` when conditional code needs to know whether
// Redis applied the write independently from the previous value.
let outcome: SetOutcome = client
.execute(Set::new("profile:1", "ready").nx().get().with_outcome())
.await?;
match &outcome.previous {
SetPreviousValue::NotRequested => unreachable!("GET was requested"),
SetPreviousValue::Missing => println!("the key did not exist"),
SetPreviousValue::Value(value) => println!("previous bytes: {value:?}"),
}
let _ = (missing_or_value, next, outcome);
Ok(())
}
Get distinguishes a missing key (None) from an empty value
(Some(Bytes::new())). Collection commands similarly preserve their documented
ordering and optional entries. Redis command errors remain RedisError::Redis;
they are not converted into an empty success value.
Set needs extra care because its options alter the wire reply. The base
command keeps its original Option<Bytes> response for compatibility: ordinary
OK and a rejected conditional write both become None. Conditional code
should call with_outcome() after all other builder methods. Its SetOutcome
separates SetStatus::{Applied, NotApplied} from
SetPreviousValue::{NotRequested, Missing, Value}; an empty value is
Value(Bytes::new()), not Missing.
| Builder mode | status | previous |
|---|---|---|
ordinary SET | Applied | NotRequested |
NX / XX without GET | Applied or NotApplied from OK / null | NotRequested |
ordinary SET GET | always Applied | Missing or Value |
NX GET | Applied when missing; otherwise NotApplied | Missing or Value |
XX GET | NotApplied when missing; otherwise Applied | Missing or Value |
The strings, hashes, lists, sets, and sorted-set pages summarize their main result shapes.
Binary keys and values
Redis keys, fields, members, and stored values are byte strings. Typed builders
accept byte slices, Vec<u8>, and Bytes through CommandArg; replies that
represent Redis byte strings remain Bytes.
async fn example() -> Result<(), Box<dyn std::error::Error>> {
use bytes::Bytes;
use redis_tower::{MultiplexedClient, commands::{Get, HSet, Set}};
let client = MultiplexedClient::connect("127.0.0.1:6379").await?;
let key = b"user:\xff".as_slice();
client.execute(Set::new(key, b"\x00\xfe\xff".as_slice())).await?;
client.execute(HSet::new(key, b"field".as_slice(), b"value\xff".as_slice())).await?;
let value: Option<Bytes> = client.execute(Get::new(key)).await?;
let _ = value;
Ok(())
}
Grammar remains typed or textual: cursors, stream IDs, JSON paths, Search queries, addresses, and SHA-1 digests are not arbitrary payload bytes. See Binary data and typed arguments for ownership and the exact per-family inventory.
Cursor iteration
SCAN is a cursor protocol, not a one-shot snapshot. Process every returned
page, including pages with zero results, and stop only at cursor 0. A changing
keyspace can produce duplicates.
async fn example() -> Result<(), Box<dyn std::error::Error>> {
use redis_tower::{MultiplexedClient, commands::Scan};
let client = MultiplexedClient::connect("127.0.0.1:6379").await?;
let mut cursor = "0".to_string();
loop {
let page = client
.execute(Scan::new().cursor(cursor).match_pattern("user:*").count(100))
.await?;
let finished = page.is_finished();
let next_cursor = page.cursor.clone();
for key in page.results {
println!("{key:?}");
}
if finished {
break;
}
cursor = next_cursor;
}
Ok(())
}
On Redis Cluster, a keyless SCAN routed normally visits only one node. Use
redis_tower_cluster::ScanClusterStream / ClusterScan for a cluster-wide
walk and account for resharding semantics documented on that type.
Pipelines, transactions, and Cluster slots
A Pipeline batches network I/O but is not atomic. A Transaction submits a
known MULTI/EXEC body atomically. Both keep typed results by index:
async fn example() -> Result<(), Box<dyn std::error::Error>> {
use bytes::Bytes;
use redis_tower::{Pipeline, RedisConnection, Transaction, TransactionResult};
use redis_tower::commands::{Get, Incr, Set};
let mut connection = RedisConnection::connect("127.0.0.1:6379").await?;
let pipeline = Pipeline::new()
.push(Set::new("{account:1}:name", "Ada"))
.push(Get::new("{account:1}:name"))
.execute(&mut connection)
.await?;
let name: &Option<Bytes> = pipeline.get(1)?;
let transaction = Transaction::new()
.watch(["{account:1}:counter"])
.push(Incr::new("{account:1}:counter"))
.execute(&mut connection)
.await?;
match transaction {
TransactionResult::Committed(mut replies) => {
let counter: i64 = replies.take(0)?;
println!("{counter}");
}
TransactionResult::Aborted => { /* rebuild if appropriate */ }
}
let _ = name;
Ok(())
}
A direct Transaction::watch is safe on supported shared clients because the
already-built WATCH/MULTI/EXEC exchange is submitted as one operation. A
read/compute/write retry loop spans separate calls and must hold a dedicated
RedisConnection for the complete WATCH/read/build/EXEC window. A
ConnectionPool does not expose a connection lease and cannot provide that
ownership.
Cluster atomic transactions and multi-key commands require every key to hash
to the same slot. Hash tags such as {account:1} make that relationship
explicit. A split helper can distribute independent reads, but splitting is
not atomic and is not a substitute for co-located keys.
Run the complete examples with:
cargo run -p redis-tower-examples --example pipeline
cargo run -p redis-tower-examples --example transaction
Streams, consumer groups, and blocking reads
The application owns acknowledgement. Create a group once, read on an isolated connection, finish processing, then acknowledge the exact entry ID:
async fn example() -> Result<(), Box<dyn std::error::Error>> {
use redis_tower::RedisConnection;
use redis_tower::commands::{XAck, XGroupCreate, XReadGroup};
let mut connection = RedisConnection::connect("127.0.0.1:6379").await?;
connection
.execute(XGroupCreate::new("events", "workers", "0").mkstream())
.await?;
let streams = connection
.execute(XReadGroup::new("workers", "worker-1", "events").count(10).block(1_000))
.await?;
for (_stream, entries) in streams {
for entry in entries {
// Persist the side effect before acknowledging this ID.
process(&entry).await?;
connection.execute(XAck::new("events", "workers", entry.id)).await?;
}
}
async fn process(_entry: &redis_tower::commands::StreamEntry) -> Result<(), Box<dyn std::error::Error>> { Ok(()) }
Ok(())
}
XREAD/XREADGROUP BLOCK and list blocking commands hold their socket until a
reply or Redis timeout. Do not put them on a MultiplexedClient, where they
would stop the shared worker. Prefer a finite Redis blocking timeout so the
task can observe shutdown; dropping a dedicated connection closes the socket.
A caller timeout after bytes reach Redis cannot prove that no entry was
delivered, so processing and acknowledgement must be idempotent where a lost
reply matters.
See examples/streams.rs for a runnable basic stream
flow and Production tuning for shutdown
ordering.
Raw, custom, and module replies
RawCommand is the binary-safe escape hatch. It returns Frame by default;
.query::<T>() applies a FromFrame decoder without changing the request:
async fn example() -> Result<(), Box<dyn std::error::Error>> {
use redis_tower::{MultiplexedClient, commands::RawCommand};
let client = MultiplexedClient::connect("127.0.0.1:6379").await?;
let count: i64 = client
.execute(RawCommand::new("SCARD").arg("members").query())
.await?;
let members: Vec<bytes::Bytes> = client
.execute(RawCommand::new("SMEMBERS").arg("members").query())
.await?;
let _ = (count, members);
Ok(())
}
Choose the decoder from the documented RESP shape: Option<T> for nil,
Vec<T> for ordered arrays, tuples/maps only when the protocol shape warrants
them, and raw Frame when a module varies by protocol version or options. Do
not normalize nil into empty bytes or unordered results into an invented
ordering.
The Cluster router knows the layouts of supported typed commands. An unknown custom command uses the legacy first-argument key convention for ordinary single-command routing; that is safe only when the first argument really is the complete routing key. Keyless node-local administration should use an explicit dedicated node connection. Unknown multi-key atomic layouts should not be guessed.
Feature modules have both Cargo and server prerequisites. Applications using
the redis-tower facade need its commands-* feature names; applications
depending directly on redis-tower-commands use the shorter names:
redis-tower feature | redis-tower-commands feature | Public category | Server capability |
|---|---|---|---|
commands-json | json | commands::json | RedisJSON |
commands-search | search | commands::search | Redis Search |
commands-bloom | bloom | commands::bloom | RedisBloom Bloom/Cuckoo |
commands-sketch | sketch | commands::sketch | RedisBloom CMS/Top-K |
commands-tdigest | tdigest | commands::tdigest | RedisBloom T-Digest |
commands-timeseries | timeseries | commands::timeseries | RedisTimeSeries |
commands-vector-sets | vector-sets | commands::vector_sets | Redis Vector Sets |
The facade’s default commands-stack feature and the command crate’s default
stack feature each enable all seven command families. Neither installs those
capabilities on the Redis server.
Pub/Sub and MONITOR
Publishing is an ordinary typed command. Receiving messages changes the socket into a stateful session, so construct the session from a fresh connection:
async fn example() -> Result<(), Box<dyn std::error::Error>> {
use redis_tower::{BinaryPubSubConnection, RedisConnection};
use tokio_stream::StreamExt;
let connection = RedisConnection::connect("127.0.0.1:6379").await?;
let mut subscriber = BinaryPubSubConnection::from_connection(connection)?;
subscriber.subscribe_bytes(&[b"events\xff".as_slice()]).await?;
if let Some(message) = subscriber.next().await {
let message = message?;
println!("{:?}: {:?}", message.channel, message.payload);
}
Ok(())
}
reconnect_with installs a replacement connection and replays subscriptions
that were previously confirmed. Messages published while disconnected or
before Redis processes the replayed subscription are not recovered. Messages
that arrive after server-side subscription processing but before the client
reads the confirmation are buffered and delivered. Cluster regular Pub/Sub is
tied to an explicitly selected node; sharded Pub/Sub follows slot ownership
through its separate API.
MonitorStream::new likewise consumes a fresh connection. There is no resume
cursor and no automatic recovery of commands observed during a gap. MONITOR is
an expensive debugging facility, not an application event stream.
Run the complete Pub/Sub example with:
cargo run -p redis-tower-examples --example pubsub
Standalone, Cluster, Sentinel, and Universal entry points
Choose the topology at construction; command types stay the same:
async fn example() -> Result<(), Box<dyn std::error::Error>> {
use redis_tower::MultiplexedClient;
use redis_tower_cluster::MultiplexedClusterClient;
use redis_tower_sentinel::MultiplexedSentinelClient;
use redis_tower_client::UniversalClient;
let standalone = MultiplexedClient::connect_url("redis://127.0.0.1:6379").await?;
let cluster = MultiplexedClusterClient::connect("127.0.0.1:7000").await?;
let sentinel = MultiplexedSentinelClient::connect(
&["127.0.0.1:26379"],
"mymaster",
).await?;
let universal = UniversalClient::connect_url("redis://127.0.0.1:6379").await?;
let _ = (standalone, cluster, sentinel, universal);
Ok(())
}
Use the explicit builders when Sentinel-hop and data-node credentials differ,
when Cluster TLS/address mapping needs customization, or when read routing and
reconnect policy matter. UniversalClient is the convenient application-level
enum; topology-specific APIs remain necessary for cluster-wide scan, dedicated
node sessions, sharded Pub/Sub, and advanced failover controls.
The runnable cluster and
sentinel examples document their local fixtures.
TLS/authentication and rotating credentials remain canonical in
Cloud and rotating credentials.
Administration and diagnostics
commands::server, commands::acl, commands::cluster, and
commands::diagnostics contain typed operator commands. Managed services often
restrict CONFIG, DEBUG, MODULE, ACL mutation, failover, and persistence
commands. Detect permission/server errors rather than assuming local Redis
configuration.
Connection-state commands deserve extra caution. Configure authentication,
RESP version, selected database, tracking, and decode limits through the
connection/factory APIs so every reconnect repeats the same setup. Issuing
AUTH, SELECT, HELLO, or tracking commands ad hoc on one shared socket does
not update its factory’s future connections.
For continuous operations, prefer the metrics and lifecycle-event APIs over polling expensive diagnostic output. Production tuning is the canonical guide for observability, health probes, reconnect state, and graceful shutdown.
Feature-gated and versioned families
Cargo feature availability, Redis server availability, and command version are
three separate checks. Building with commands-stack on the redis-tower
facade (or stack on redis-tower-commands) proves that the Rust builders are
present; it does not prove a deployment has RedisJSON, Search, Bloom, or
TimeSeries. Native Array commands require Redis 8.8+, and Vector Sets require a
supporting Redis 8 server.
Treat ERR unknown command or module-specific capability errors as deployment
configuration/version failures. Gate startup with an explicit capability probe
when a feature is required, and keep fallback behavior visible rather than
silently dropping an option or changing a reply shape.
Redis command capability ledger
This report separates four claims that cannot be reduced to one coverage percentage: command-name availability, typed semantics, RESP wire support, and session/topology behavior. A typed name does not prove every option, reply shape, protocol form, or routing mode.
The machine-readable source is conformance/command-capabilities.json. Unverified fields are explicit backlog, not implied support.
Pinned provenance
- Redis documentation version: 8.8
- Redis documentation revision:
ad12d9cd6d10 - Pinned metadata entries: 554
- Scoped command names: 506
- Redis documentation metadata license:
CC-BY-NC-SA-4.0(separate from the Rust crates’ MIT/Apache-2.0 license) - Ordinary generation and CI are offline; each vendored metadata file is SHA-256 verified from the detail manifest. The manifest records both the upstream byte digest and the digest after documented trailing-whitespace normalization.
Availability dispositions
| Disposition | Scoped names | Meaning |
|---|---|---|
dedicated_api | 6 | Supported by a stateful or higher-level API, not a stateless builder. |
intentionally_unsupported | 2 | Deliberately outside the client API contract. |
internal_protocol | 2 | Issued internally by routing/session machinery. |
raw_only | 8 | Available only through an explicitly routed raw command. |
typed_builder | 488 | Dedicated typed Command implementation. |
These counts describe availability only. They are not combined with semantic, wire, or operational verification into a headline score.
Scoped names without typed builders
| Command | Disposition | Public API | Rationale |
|---|---|---|---|
ASKING | internal_protocol | redis_tower_cluster::ClusterConnectionredis_tower_cluster::MultiplexedClusterClient | Cluster redirect handling sends ASKING and the redirected command as one connection-local exchange; a standalone builder would make unsafe separation easy. |
CLUSTER COUNT-FAILURE-REPORTS | raw_only | redis_tower_commands::RawCommand | Node-local cluster administration remains an explicitly routed raw operation. |
CLUSTER MIGRATION | raw_only | redis_tower_commands::RawCommand | Redis 8 cluster migration administration is not part of automatic client routing and must target a deliberate node. |
CLUSTER MYSHARDID | raw_only | redis_tower_commands::RawCommand | This node-local topology inspection command requires an explicitly selected cluster node. |
CLUSTER SLOT-STATS | raw_only | redis_tower_commands::RawCommand | This node-local Redis 8.2 administration command is available through a deliberately routed raw request. |
CLUSTER SYNCSLOTS | raw_only | redis_tower_commands::RawCommand | Cluster slot synchronization mutates node-local state and is intentionally not exposed as an ordinary routed builder. |
JSON.DEBUG HELP | raw_only | redis_tower_commands::RawCommand | JSON.DEBUG MEMORY has a typed builder; the module help variant is an operator-facing raw reply and remains raw-only. |
PSUBSCRIBE | dedicated_api | redis_tower::PubSubConnectionredis_tower::BinaryPubSubConnectionredis_tower_cluster::ClusterPubSubConnectionredis_tower_cluster::BinaryClusterPubSubConnection | Pattern subscription changes a dedicated socket into Pub/Sub mode and is owned by the streaming session API. |
PSYNC | intentionally_unsupported | — | Replication handshakes and replication-stream consumption are outside the request/response client contract. |
PUNSUBSCRIBE | dedicated_api | redis_tower::PubSubConnectionredis_tower::BinaryPubSubConnectionredis_tower_cluster::ClusterPubSubConnectionredis_tower_cluster::BinaryClusterPubSubConnection | Pattern unsubscription is part of the stateful Pub/Sub session and its acknowledgement stream. |
READONLY | internal_protocol | redis_tower_cluster::MultiplexedClusterClientBuilder | Replica-read connections issue READONLY during connection setup so reconnects restore the state before routing reads. |
READWRITE | raw_only | redis_tower_commands::RawCommand | The public cluster clients own their read-routing mode; ad hoc connection-state reversal is only available on an explicitly owned raw session. |
SSUBSCRIBE | dedicated_api | redis_tower::PubSubConnectionredis_tower::BinaryPubSubConnectionredis_tower_cluster::ShardedClusterPubSubConnectionredis_tower_cluster::BinaryShardedClusterPubSubConnection | Sharded Pub/Sub uses a dedicated standalone session or a slot-aware Cluster session that follows the channel slot’s owner. |
SUBSCRIBE | dedicated_api | redis_tower::PubSubConnectionredis_tower::BinaryPubSubConnectionredis_tower_cluster::ClusterPubSubConnectionredis_tower_cluster::BinaryClusterPubSubConnection | Subscription changes the connection protocol mode and is exposed through a dedicated streaming session rather than a stateless command builder. |
SUNSUBSCRIBE | dedicated_api | redis_tower::PubSubConnectionredis_tower::BinaryPubSubConnectionredis_tower_cluster::ShardedClusterPubSubConnectionredis_tower_cluster::BinaryShardedClusterPubSubConnection | Sharded unsubscription is owned by the dedicated standalone or slot-aware Cluster Pub/Sub session and its confirmation handling. |
SYNC | intentionally_unsupported | — | Legacy replication handshakes and replication-stream consumption are outside the request/response client contract. |
TRIMSLOTS | raw_only | redis_tower_commands::RawCommand | This node-local cluster maintenance operation is deliberately excluded from automatic key routing. |
UNSUBSCRIBE | dedicated_api | redis_tower::PubSubConnectionredis_tower::BinaryPubSubConnectionredis_tower_cluster::ClusterPubSubConnectionredis_tower_cluster::BinaryClusterPubSubConnection | Unsubscription and its confirmations are part of the stateful Pub/Sub session. |
Audited command semantics and operations
19/506 scoped names currently have a command-specific semantic or operational audit. The remaining rows are recorded as unverified in the machine ledger.
| Command | Semantics | Binary inputs | Options / response | Routing / session | Evidence |
|---|---|---|---|---|---|
ASKING | not_applicable | not_applicable | Owned by redirect machinery rather than an application command. | ASK target node; same connection and immediately followed by redirected command; One-shot connection state must not be separated from the redirected command. | topology.cluster-redirect |
AUTH | verified | partial | password or username/password; status; Prefer constructor credential configuration so reconnect restores authentication. | connection-local; connection setup; Ad hoc AUTH does not mutate a connection factory. | session.connection-setupsession.credential-provider |
BLPOP | verified | verified | one or more keys plus timeout; optional key/value pair; None on Redis timeout; Caller cancellation after write cannot prove that no element was delivered. | same slot in Cluster; exclusive connection recommended; a multiplexed response deadline can fail a slow command and retire its worker; Standalone value and Redis-timeout replies are verified. Cluster routing and non-stalling concurrent isolation are not command-specifically verified here. | command.blocking-dedicated |
CLIENT TRACKING | partial | not_applicable | owned by caching clients; raw ad hoc variants are not replayed; status; Caching clients enforce their own tracking configuration and invalidation lifecycle. | per data connection; connection-local tracking state; Tracking state is restored only by the caching client that owns it. | session.client-tracking |
EVAL | partial | verified | keys must be declared separately from arguments; raw Frame; The script defines the reply shape; applications must decode deliberately. | all declared keys in one Cluster slot; stateless command; Declared-key encoding is verified in standalone differential tests; Cluster routing remains unaudited here. | command.eval-declared-keys |
FT.SEARCH | partial | partial | FtSearch supports NOCONTENT, VERBATIM, WITHSCORES, LIMIT, RETURN and SORTBY; other documented options are not exposed by this builder; raw Frame; The separate high-level SearchClient::search API performs typed result deserialization. Both require the search Cargo feature and Redis Search server capability. | module/topology dependent; stateless command; Module availability is proven separately from compilation. | module.behavior-differential |
GET | verified | verified | none; Option<Bytes>; nil and empty values remain distinct; RESP2/RESP3 byte strings normalize to the same public shape. | single key slot; stateless command; Read-only and idempotent. | command.binary-and-scalartopology.cluster-core-routing |
HELLO | verified | partial | RESP version, optional auth and client name; server capability map; Prefer connection protocol configuration so reconnect repeats negotiation. | connection-local; connection setup; Protocol version is a socket property. | session.connection-setup |
HGETALL | verified | verified | none; Vec<(Bytes, Bytes)> parsed from RESP2 flat arrays or RESP3 maps; Binary fields and values are preserved; the public response is an ordered pair vector, not a map type. | single key slot; stateless command; Large hashes can produce large replies. | command.collection-stream-shapestopology.cluster-core-routing |
JSON.GET | partial | verified for keys | JSONPath and formatting grammar are textual; optional raw JSON bytes; Requires the JSON Cargo feature and RedisJSON server capability. | single key slot; stateless command; Module capability and standalone behavior are verified; Cluster module routing is not audited here. | module.behavior-differential |
MGET | verified | verified | one or more keys; ordered Vec<Option<Bytes>> preserving nil and empty entries; The result position corresponds to the input key position. | same slot for atomic server command; explicit split helpers are non-atomic; stateless command; Standalone response order is verified; cross-slot split-helper order is verified separately, but same-slot Cluster MGET remains unaudited here. | command.binary-and-scalartopology.cluster-core-routing |
MONITOR | verified | not_applicable | none; stream of monitor lines; MONITOR permanently changes the socket mode until it is dropped. | explicit server node; dedicated MonitorStream; No reconnect resume cursor; commands during a gap are lost. | session.monitor-dedicatedsession.credential-provider |
SCAN | partial | partial | cursor, MATCH and COUNT are live-tested; TYPE is encoded by the builder but is not live-tested here; cursor plus a possibly empty page of binary keys; Continue until cursor zero; changing keyspaces can repeat elements. | one selected node; use ClusterScan for all nodes; cursor sequence without connection affinity; A normal Cluster SCAN is not cluster-wide. | command.scan-cursor |
SELECT | verified | not_applicable | database index; status; Cluster does not support multiple logical databases. | connection-local; connection setup; Configure the database in the connection factory for reconnect replay. | session.connection-setupsession.credential-provider |
SET | partial | verified | EX/PX, NX/XX and GET are typed; EXAT/PXAT/KEEPTTL and Redis 8.4 IFEQ/IFNE/IFDEQ/IFDNE are unimplemented; Set::with_outcome returns independent Applied/NotApplied status and NotRequested/Missing/Value previous-value state; base Set preserves the legacy Option<Bytes> response; RESP2/RESP3 live and parser coverage exercises ordinary SET plus all applied/rejected NX/XX branches with and without GET, including missing, empty and binary previous values. Expiry combinations are not covered. | single key slot; stateless command; Non-idempotent option combinations are not replayed by typed retry policy. | command.set-optionscommand.binary-and-scalartopology.cluster-core-routing |
SSUBSCRIBE | verified | verified | one or more shard channels in one slot; subscription confirmations and message stream; Messages during a disconnect before server-side resubscription are not recovered. | standalone server or current Cluster owner of the shared slot; dedicated standalone or sharded Cluster Pub/Sub connection; Cluster cross-slot subscription sets are rejected before dispatch. | session.pubsub-reconnectsession.credential-provider |
SUBSCRIBE | verified | verified | one or more channels; subscription confirmations and message stream; Confirmed subscriptions can be replayed, but disconnected-gap messages are lost. | explicit standalone or Cluster node; dedicated Pub/Sub connection; Not a stateless multiplexed command. | session.pubsub-reconnectsession.credential-provider |
WATCH | verified | verified | one or more keys; status before EXEC commit/abort outcome; Use the Transaction abstraction for an already-built atomic exchange. | same Cluster slot; dedicated connection for read/compute/write retry loops; A pool has no multi-call connection lease. | command.pipeline-transactioncommand.binary-and-scalartopology.cluster-core-routing |
XREADGROUP | partial | partial | group, consumer, streams, IDs, COUNT and BLOCK; NOACK is not exposed by the builder; typed stream/entry vector for the tested non-blocking consumer-group lifecycle; The builder accepts byte-preserving CommandArg inputs, but the live XREADGROUP evidence uses text identifiers. Blocking timeout shape is not command-specifically verified here. | same slot in Cluster; exclusive connection recommended when BLOCK is used; Standalone consumer-group behavior is verified; blocking isolation and Cluster routing remain unaudited here. | command.stream-consumer-group |
Wire protocol contracts
| Contract | Status | Current behavior | Evidence |
|---|---|---|---|
configured-resource-limits | supported | Frame wire size, aggregate cardinality and nesting depth are preflighted before recursive frame materialization, independently of fragmentation. | protocol.boundariesprotocol.fragmentation |
fragmentation-equivalence | supported | Whole-buffer and arbitrarily fragmented decoding have the same semantic outcome and error category for the tested corpus. | protocol.fragmentation |
resp2-frames | supported | RESP2 scalars, nulls and aggregates are decoded within configured size/depth limits. | protocol.boundariesprotocol.fragmentation |
resp3-core-frames | supported | Supported RESP3 scalar and fixed aggregate forms are decoded and normalized by typed response parsers. | protocol.boundariescommand.collection-stream-shapes |
resp3-fixed-attributes | rejected | Fixed attributes fail closed without consuming bytes; attachment to the following value remains planned in issue #705. | protocol.boundaries |
resp3-streamed-forms | rejected | Streamed string/aggregate headers, chunks and terminators fail closed rather than becoming independent replies; bounded assembly remains planned in issue #705. | protocol.boundariesprotocol.fragmentation |
Logical behavior evidence
A source test, a configured workflow selector, and a dated successful run are separate evidence states. A later checkout must use its own attached checks rather than inheriting the observed result below.
Behavior command binary and scalar
Logical ID: command.binary-and-scalar.
Raw RESP2/RESP3 scalar, nil, empty, binary and numeric replies are compared with redis-rs. Separate typed live tests verify byte-preserving builders and public GET/MGET decoding, including ordered nil-versus-empty results, under forced RESP2 and RESP3.
- Dimensions:
typed-semantics,external-differential - Matrix: server
Redis 7.4.3 and 8.0.6 per-PR matrix; protocolsRESP2, RESP3; featurescore commands; topologiesstandalone - Source tests:
diff_mcp_scalar_nil_binary_and_numeric_boundaries,strings_preserve_invalid_utf8_and_protocol_looking_payloads,typed_binary_inputs_roundtrip_resp2,typed_binary_inputs_roundtrip_resp3,get_nonexistent,mget - Configured CI:
test-unit:,test-integration: - Last observed pass: 2026-09-25 at
07d10b1c0901
Behavior command blocking dedicated
Logical ID: command.blocking-dedicated.
Typed BLPOP returns a key/value pair when data is available and None after a Redis timeout on an exclusive connection. A separate multiplexed raw BLPOP test proves the configured response deadline returns CommandTimeout; the differential BLMOVE case is immediate because its source list is preloaded.
- Dimensions:
typed-semantics,session-ownership - Matrix: server
Redis 7.4.3 and 8.0.6 per-PR matrix; protocolsRESP2, RESP3; featurescore commands; topologiesstandalone - Source tests:
blpop_with_data,blpop_timeout,multiplexed_response_timeout_trips_on_slow_command,diff_mcp_blocking_command_uses_dedicated_sessions,lifecycle_blocking_scan_and_routing_arguments_are_byte_exact - Configured CI:
test-unit:,test-integration: - Last observed pass: 2026-09-25 at
6077899be738
Behavior command collection stream shapes
Logical ID: command.collection-stream-shapes.
Raw hash, collection and Stream replies are compared with redis-rs using independent keys and controlled normalization. Separate typed live and parser tests verify HGETALL’s public pair-vector shape, empty response and RESP3 map normalization.
- Dimensions:
typed-semantics,external-differential - Matrix: server
Redis 7.4.3 and 8.0.6 per-PR matrix; protocolsRESP2, RESP3; featurescore commands; topologiesstandalone - Source tests:
diff_mcp_hash_collection_and_stream_shapes,typed_binary_inputs_roundtrip_resp2,typed_binary_inputs_roundtrip_resp3,hgetall,hgetall_empty,resp3_hgetall_map - Configured CI:
test-unit:,test-integration: - Last observed pass: 2026-09-25 at
07d10b1c0901
Behavior command eval declared keys
Logical ID: command.eval-declared-keys.
Typed EVAL preserves binary declared keys and arguments and matches redis-rs for the tested nested reply.
- Dimensions:
typed-semantics,routing,external-differential - Matrix: server
Redis 7.4.3 and 8.0.6 per-PR matrix; protocolsRESP2, RESP3; featurescore commands; topologiesstandalone - Source tests:
diff_typed_remaining_families_preserve_opaque_bytes - Configured CI:
test-integration: - Last observed pass: 2026-09-25 at
7338c15d8442
Behavior command pipeline transaction
Logical ID: command.pipeline-transaction.
Pipeline ordering and transaction commit/abort outcomes are compared with redis-rs without reusing mutating datasets.
- Dimensions:
typed-semantics,session-ownership,external-differential - Matrix: server
Redis 7.4.3 and 8.0.6 per-PR matrix; protocolsRESP2, RESP3; featurescore commands; topologiesstandalone - Source tests:
diff_mcp_pipeline_and_transaction_outcomes - Configured CI:
test-integration: - Last observed pass: 2026-09-25 at
7338c15d8442
Behavior command scan cursor
Logical ID: command.scan-cursor.
Standalone SCAN cursor iteration exercises MATCH and COUNT until cursor zero; Cluster tests prove a plain keyless SCAN reaches one node while ClusterScan covers the full keyspace.
- Dimensions:
typed-semantics,cursor-lifecycle,routing - Matrix: server
Redis 7.4.3 and 8.0.6 standalone and Cluster per-PR matrix; protocolsdefault Auto negotiation; featurescore commands; topologiesstandalone, cluster - Source tests:
scan_basic,mux_cluster_scan_stream_covers_the_whole_keyspace - Configured CI:
test-integration: - Last observed pass: 2026-09-25 at
7338c15d8442
Behavior command set options
Logical ID: command.set-options.
SET outcome mode distinguishes applied and rejected NX/XX writes independently from unrequested, missing, empty and binary previous values. RESP2/RESP3 parser and live tests cover the complete supported outcome matrix; NX+GET and XX+GET branches are also compared with redis-rs. Expiry combinations remain outside this behavior row.
- Dimensions:
typed-semantics,external-differential - Matrix: server
Redis 7.4.3 and 8.0.6 per-PR matrix; protocolsRESP2, RESP3; featurescore commands; topologiesstandalone - Source tests:
set_outcome_without_get_distinguishes_condition_status,set_outcome_get_maps_nx_and_xx_semantics,cmd_set_nx,cmd_set_xx,typed_binary_inputs_roundtrip_resp2,typed_binary_inputs_roundtrip_resp3,diff_mcp_typed_set_builder_options - Configured CI:
test-unit:,test-integration: - Last observed pass: 2026-09-25 at
bdbcdf38471b
Behavior command stream consumer group
Logical ID: command.stream-consumer-group.
A typed XREADGROUP call reads new entries in a standalone consumer-group lifecycle, after which typed pending, acknowledgement, claim and autoclaim operations verify ownership transitions.
- Dimensions:
typed-semantics,session-ownership - Matrix: server
Redis 7.4.3 and 8.0.6 per-PR matrix; protocolsdefault Auto negotiation (RESP3 on the tested versions); featurescore commands; topologiesstandalone - Source tests:
stream_consumer_group_lifecycle,streams_consumer_groups - Configured CI:
test-integration: - Last observed pass: 2026-09-25 at
7338c15d8442
Behavior module behavior differential
Logical ID: module.behavior-differential.
Module capability preflight is followed by RedisJSON/Search/TimeSeries/probabilistic/Vector behavior and selected redis-rs reply comparisons.
- Dimensions:
typed-semantics,server-capability,external-differential - Matrix: server
Redis 8 module image; actual version discovered at runtime; protocolsRESP2/RESP3 for JSON differential, default negotiated protocol for other module assertions; featuresjson, search, bloom, sketch, tdigest, timeseries, vector-sets; topologiesstandalone - Source tests:
required_server_version_and_commands_are_present,differential_json_raw_replies_match_redis_rs,json_binary_key_roundtrip,search_create_index_and_query,timeseries_add_and_range,bloom_filter_add_and_exists,cuckoo_filter_add_exists_del,count_min_sketch_incrby_and_query,topk_reserve_add_and_query,tdigest_create_add_and_quantile,vector_set_basic_lifecycle - Configured CI:
modules: - Last observed pass: 2026-09-25 at
7338c15d8442
Behavior protocol boundaries
Logical ID: protocol.boundaries.
Independent wire fixtures prove frame limits, aggregate preflight, nesting limits and fail-closed handling for fixed attributes and streamed forms.
- Dimensions:
wire-protocol,resource-limits - Matrix: server
no server; independent wire fixtures; protocolsRESP2, RESP3; featuresprotocol; topologiesnot applicable - Source tests:
complete_oversized_frames_are_rejected_without_consuming_bytes,fixed_attributes_remain_unsupported_even_when_nested,all_streaming_tokens_fail_closed_at_top_level_and_inside_aggregates - Configured CI:
Run protocol boundary regressions - Last observed pass: 2026-09-25 at
7338c15d8442
Behavior protocol fragmentation
Logical ID: protocol.fragmentation.
Property and fuzz oracles compare whole-buffer decoding with arbitrary fragmentation plans and stable error categories.
- Dimensions:
wire-protocol,fragmentation,fuzzing - Matrix: server
no server; generated wire data; protocolsRESP2, RESP3; featuresprotocol, fuzz; topologiesnot applicable - Source tests:
pipelines_match_under_varied_fragmentation,decode_fragmented - Configured CI:
Smoke-test fuzz targets - Last observed pass: 2026-09-25 at
7338c15d8442
Behavior session client tracking
Logical ID: session.client-tracking.
Managed caching clients install tracking, invalidate external writes, reject unsafe connection-state commands and restore a lost redirect connection.
- Dimensions:
session-ownership,reconnect,server-capability - Matrix: server
isolated live Redis process in the per-PR standalone suite; protocolsRESP3; featuresclient-side caching; topologiesstandalone - Source tests:
default_tracking_caches_reads_and_observes_external_invalidation,managed_connection_state_commands_are_rejected_without_losing_tracking,tracking_receiver_loss_fails_closed_and_reinstalls_redirect - Configured CI:
test-integration: - Last observed pass: 2026-09-25 at
7338c15d8442
Behavior session connection setup
Logical ID: session.connection-setup.
Authentication, RESP negotiation and selected database state are exercised through connection setup and reconnect-aware paths.
- Dimensions:
session-ownership,reconnect - Matrix: server
Redis 7.4.3 and 8.0.6 per-PR matrix; protocolsdefault Auto negotiation, focused RESP3 HELLO path; featurescore, URL/auth; topologiesstandalone - Source tests:
reconnect_replays_auth,cover_hello_proto3,select_db - Configured CI:
test-integration: - Last observed pass: 2026-09-25 at
7338c15d8442
Behavior session credential provider
Logical ID: session.credential-provider.
One shared credential provider backs fresh standalone and Cluster sockets plus dedicated Pub/Sub and MONITOR construction. Provider URL setup preserves transport, selected database and RESP negotiation; lookup, forced-refresh, transport and session setup can share one deadline. A forced-refresh failure is redacted and stops setup before SELECT, HELLO or application traffic. Established ordinary clients can reauthenticate through owned tasks with explicitly documented owner-specific failure retirement, while Pub/Sub reconnects with confirmed binary subscriptions and MONITOR terminates/reopens rather than accepting unsafe generic in-band AUTH. Provider errors and refresh-task tracing omit third-party error text and retain authentication classification.
- Dimensions:
session-ownership,reconnect,security,routing - Matrix: server
Redis 7.4.3 and 8.0.6 per-PR matrix plus deterministic provider/unit fixtures; protocolsRESP2 setup, preserved RESP2/RESP3 URL negotiation; featurescore, URL/auth, Cluster, Pub/Sub, MONITOR, AWS and Entra providers; topologiesstandalone, cluster - Source tests:
staged_url_connection_orders_provider_auth_before_database_and_protocol,dropping_push_handle_drops_provider_stream,push_helper_tracing_does_not_render_provider_or_callback_errors,provider_url_factory_preserves_database_and_protocol_but_overrides_static_auth,provider_forced_refresh_failure_is_redacted_and_stops_setup,provider_factory_setup_timeout_bounds_a_stalled_provider,provider_failure_is_redacted_and_authentication_classified,shared_provider_handle_spans_concurrent_dedicated_connection_attempts,pubsub_rotation_reconnects_with_shared_provider_and_preserves_binary_subscriptions,monitor_rotation_terminates_and_reopens_through_shared_provider,node_factory_refetches_credentials_and_negotiates_after_auth,concurrent_force_refreshes_are_coalesced - Configured CI:
test-unit:,test-integration:,Run cluster integration tests - Last observed pass: 2026-09-25 at
5577d6820ca5
Behavior session monitor dedicated
Logical ID: session.monitor-dedicated.
A dedicated MONITOR session parses streamed events and preserves binary command arguments while another connection executes the command.
- Dimensions:
typed-semantics,session-ownership - Matrix: server
Redis 7.4.3 and 8.0.6 per-PR matrix; protocolsdefault negotiated protocol; featurescore; topologiesstandalone - Source tests:
monitor_stream_preserves_binary_command_arguments - Configured CI:
test-integration: - Last observed pass: 2026-09-25 at
7338c15d8442
Behavior session pubsub reconnect
Logical ID: session.pubsub-reconnect.
Dedicated regular and sharded Pub/Sub sessions preserve binary names, replay confirmed subscriptions and follow tested Cluster ownership changes.
- Dimensions:
session-ownership,routing,reconnect - Matrix: server
Redis 7.4.3 and 8.0.6 per-PR matrix; protocolsdefault Auto negotiation, focused standalone RESP3 push path; featurescore, cluster; topologiesstandalone, cluster - Source tests:
pubsub_resubscribes_after_reconnect,pubsub_resp3_push,binary_pubsub_names_round_trip_in_regular_and_sharded_sessions,sharded_pubsub_follows_topology_owner_change_and_resubscribes - Configured CI:
test-integration: - Last observed pass: 2026-09-25 at
7338c15d8442
Behavior topology cluster core routing
Logical ID: topology.cluster-core-routing.
Cluster live tests route binary single-key GET/SET, HGETALL, ordered non-atomic split MGET helpers and slot-pinned WATCH transactions through public APIs.
- Dimensions:
routing,typed-semantics,session-ownership,external-differential - Matrix: server
Redis 7.4.3 and 8.0.6 Cluster per-PR matrix; protocolsAuto (RESP3 on the tested Redis versions); featurescluster, core commands; topologiescluster - Source tests:
diff_redis_rs_cluster_public_entry_point,cluster_large_hgetall,mux_cluster_pipeline_and_split_helpers_preserve_order,cluster_transactions_are_slot_pinned_and_watch_safe - Configured CI:
Run cluster integration tests - Last observed pass: 2026-09-25 at
7338c15d8442
Behavior topology cluster redirect
Logical ID: topology.cluster-redirect.
Cluster clients keep ASKING with the redirected command, follow ASK then MOVED, and update slot ownership during a live reshard.
- Dimensions:
routing,session-ownership,topology - Matrix: server
Redis 7.4.3 and 8.0.6 Cluster per-PR matrix; protocolsAuto (RESP3 on the tested Redis versions); featurescluster; topologiescluster - Source tests:
mux_cluster_handles_ask_then_moved_during_live_reshard,cached_cluster_opt_in_follows_live_ask_without_conflicting_one_shot_flags - Configured CI:
Run cluster integration tests - Last observed pass: 2026-09-25 at
7338c15d8442
Offline validation and refresh
Validate references, dispositions, source builders, pinned file hashes, and generated output without network access:
python3 scripts/generate_command_capabilities.py --check
python3 scripts/generate_command_coverage.py --check
To review a newer Redis documentation snapshot, fetch an explicit commit into a separate directory and compare it before changing provenance:
python3 scripts/fetch_command_metadata.py --ref <redis-docs-commit> \
--output-dir /tmp/redis-command-metadata
python3 scripts/generate_command_capabilities.py \
--compare-metadata /tmp/redis-command-metadata
The comparison reports added/removed names and any command-definition change, including scope, routing keys/flags, arity, versions, summaries, history, and arguments/options. Updating the five vendored files, their hashes, and every new disposition is a reviewed repository change. The downstream MCP project’s Redis 8.10.1 application ledger has a different denominator and is not substituted for this client ledger.
Binary data and typed command arguments
Redis keys and most data arguments are byte strings. They are not required to
be UTF-8. redis-tower-commands uses CommandArg
for binary-safe typed inputs wherever Redis treats an argument as opaque data:
keys, stored values, fields, members, patterns, script arguments, channels, and
module payloads.
use bytes::Bytes;
use redis_tower::commands::{Get, HSet, Set, XAdd};
let string_key = b"user:\xff".as_slice();
let hash_key = b"profile:\xff".as_slice();
let stream_key = b"events:\xff".as_slice();
client
.execute(Set::new(string_key, vec![0x00, 0xfe, 0xff]))
.await?;
let value: Option<Bytes> = client.execute(Get::new(string_key)).await?;
client
.execute(HSet::new(
hash_key,
b"embedding".as_slice(),
vector_bytes,
))
.await?;
client
.execute(XAdd::new(stream_key).field(b"payload", payload))
.await?;
Ownership and compatibility
CommandArg owns shared Bytes storage so command values remain Send + 'static and can be cloned for Tower middleware. Conversions behave as follows:
| Input | Storage behavior |
|---|---|
Bytes | shares the existing allocation |
Vec<u8> | takes ownership without copying the payload |
String | takes ownership without copying the UTF-8 payload |
&[u8], byte arrays, &str, &String | copies once into owned storage |
CommandArg | moves the value; cloning shares storage |
Ordinary string calls such as Get::new("key") and Set::new(String::from("k"), "v") remain direct. Generic wrappers that want to accept both text and bytes
should use impl Into<CommandArg>. A wrapper intentionally restricted to text
can retain impl Into<String> and pass value.into() to a builder. As with the
old impl Into<String> API, avoid an unnecessary .into() on a literal at the
call site because an impl Trait parameter gives the conversion no unique
target; pass the literal directly or use CommandArg::from(...).
The standard inputs accepted through Into<String> remain accepted directly,
including Box<str>, char, and &mut str. A downstream newtype that only
implemented Into<String> is the direct source-compatibility exception: either
convert it to String before calling the builder or implement
From<YourType> for CommandArg. This intentional break must not ship in a
0.1.x patch release. It requires redis-tower-commands 0.2.0 and
redis-tower 0.2.0, with workspace and downstream dependency requirements
updated when that release is prepared.
Stream field names and stream names in
XREAD/XREADGROUP results are now Bytes, as are StreamMessage.stream and
its field names. The previous String representation used lossy UTF-8
conversion and could not round-trip an input that the builder now accepts.
Consumer and group names in stream metadata are also Bytes. Stream IDs remain
String because Redis defines their grammar as decimal numbers separated by
-.
Semantic inventory
The table is an argument-semantic inventory, not a search for Rust String
fields. “Text” means Redis assigns grammar or an encoding to the value; “opaque”
means Redis compares, stores, hashes, or routes the exact bytes.
| Family | Opaque inputs | Deliberately structured/text inputs | Typed status |
|---|---|---|---|
| Strings | keys and stored values, comparison values | integer/float increments, expiry numbers, digest syntax | CommandArg throughout |
| Hashes | keys, field names, field values | increments and expiry numbers | CommandArg throughout |
| Lists and non-blocking multi-list commands | keys, elements, pivots | indexes, counts, LEFT/RIGHT and BEFORE/AFTER tokens | CommandArg throughout |
| Sets | keys and members | counts and limits | CommandArg throughout |
| Sorted sets | keys and members; lex bounds contain a binary member after their Redis prefix | scores, rank indexes, aggregation and direction tokens | all builder byte strings use CommandArg; Redis validates bound grammar |
| Streams | stream keys, field names/values, group and consumer names | entry IDs/cursors, trim thresholds and option tokens | builder byte strings use CommandArg; stream keys and field names decode as Bytes |
| Keys and key lifecycle | keys, rename/copy destinations, patterns, serialized RESTORE payload | TTLs, database numbers, SORT grammar | opaque positions use CommandArg throughout |
| Blocking commands | list/zset keys | timeouts and direction tokens | keys use CommandArg throughout |
| Scan | collection keys and glob patterns | cursors, count, TYPE token | keys/patterns use CommandArg; results are Bytes |
| Scripting and functions | script bodies, KEYS and ARGV, function payloads | SHA-1 hex digests, function/library names and subcommand grammar | opaque positions use CommandArg; cached Script helpers also provide binary argument methods |
| Pub/Sub commands and sessions | channels, patterns, payloads | PUBSUB subcommands | typed commands and dedicated sessions are binary-safe |
| Geo and HyperLogLog | keys, members/elements | coordinates, units | opaque positions use CommandArg throughout |
| Bitmap, transaction, diagnostics, and server helpers | keys, WATCH keys, tracking prefixes and data-bearing arguments | numeric offsets plus administrative grammar | opaque positions use CommandArg; PING/ECHO echo responses are exact Bytes |
| ACL and cluster administration | ACL passwords and simulated command arguments; keys supplied to CLUSTER KEYSLOT | ACL usernames/rules/categories, node IDs, addresses, command grammar and numeric slots | deliberate split: opaque data uses CommandArg; operational identifiers remain text-first |
| Redis 8.8 arrays | keys, values, predicates | indexes and option grammar | already binary-safe |
| Bloom/Cuckoo, sketches, t-digest | keys and item payloads | capacities, probabilities and numeric observations | opaque positions use CommandArg throughout |
| JSON | keys | JSONPath and serialized JSON syntax | keys use CommandArg; structured JSON interfaces stay text/serde-oriented |
| Search | document keys, suggestion payloads, tag values and vector blobs | index/schema/query syntax and autocomplete suggestion text | opaque inputs and outputs preserve bytes; Redis Search can terminate NUL-containing suggestions and normalize malformed UTF-8 |
| Time series | keys and label names/values | timestamps, reducers and retention/configuration | opaque inputs and returned keys/labels preserve bytes |
| Vector sets | keys and element names; vector bytes | JSON attributes and filter/query expression grammar | keys/elements use CommandArg; returned elements are Bytes |
RawCommand::arg remains the universal escape hatch for commands or extension
syntax outside the typed surface. It accepts AsRef<[u8]>, can retain a typed
response with .query::<T>(), and does not perform lossy conversion.
Response decoding
Responses corresponding to opaque values preserve exact bytes. This includes
keys, members, stream names/fields, Search documents/suggestions/payloads/tag
values, TimeSeries keys/labels, Vector Set elements, TopK items, and echoed
PING/ECHO messages.
Redis-defined textual metadata remains String: scan cursors, stream IDs,
SHA-1 digests, JSON text, query plans, server diagnostic reports, ACL usernames
and rules, Cluster node descriptions and IDs, addresses, and command/config
names. Some of these existing metadata parsers use UTF-8 replacement for a
malformed server reply; that behavior cannot affect an opaque value because
those positions are classified separately above.
ACL usernames/rules/categories and Cluster node IDs/addresses intentionally
remain text-first. Redis exposes them as operational configuration grammar, not
application key/value payloads. In contrast, ACL passwords and ACL DRYRUN
arguments, and the key accepted by CLUSTER KEYSLOT, preserve exact bytes.
The client serializes Search suggestion strings as exact bulk-string bytes, but Redis Search implements them as text: an embedded NUL is a terminator and malformed UTF-8 can be normalized. Suggestion dictionary keys and payloads remain binary-safe. Suggestion responses preserve the exact bytes returned by the server rather than applying another client-side conversion.
Cluster routing
Typed commands serialize a CommandArg directly into RESP bulk strings. The
Cluster key extractor reads those same frame bytes, so slot hashing and hash-tag
extraction happen before any decoding and use the exact key bytes. Braces inside
invalid UTF-8 keys therefore have the same routing meaning as braces in an ASCII
key.
RESP codec support and validation
The protocol crate decodes one bounded RESP reply at a time. Its public
Frame type comes from resp-rs and can represent more wire forms than the
client currently accepts. This page is the disposition contract used by the
property tests, fuzz oracle, and reviewed seed corpus.
Frame disposition
| Wire form | Decode disposition | Generated and fixture coverage |
|---|---|---|
| Simple strings and errors | Supported; line contents cannot contain an unescaped CRLF | Generated leaves and independent pipeline fixtures |
| Integers | Supported across the full i64 range; overflow is rejected | Generated leaves plus malformed/overflow boundaries |
| Bulk strings and RESP2 null bulk strings | Supported with byte-exact binary payloads | Generated binary leaves and framing-looking payload fixtures |
| Blob errors | Supported with byte-exact payloads | Generated leaves and mixed aggregate fixtures |
| Finite doubles | Supported | Generated finite values |
inf, -inf, and nan doubles | Supported and normalized to canonical SpecialFloat frames | Generated canonical values and independent wire fixtures |
| Booleans and RESP3 null | Supported | Generated leaves |
| Big numbers | Supported as their wire bytes | Generated signed decimal values and independent fixtures |
| Verbatim strings | Supported when the payload has a three-byte format followed by : | Generated binary content under the txt format and malformed fixtures |
| Fixed arrays, null arrays, sets, maps, and pushes | Supported recursively within the configured size and depth limits | Generated mixed aggregates, raw fixtures, and exact boundary tests |
| Fixed attributes | Rejected without consuming input | Deterministic top-level and nested rejection fixtures |
| Streamed string/blob/verbatim headers, aggregate headers, chunks, and terminators | Rejected without consuming input | Every token at top level and in aggregates, across every split point |
| Malformed tags, scalars, lengths, terminators, or overflow | Rejected without consuming input | Deterministic regressions and arbitrary-byte fuzzing |
Frames over max_frame_size or max_depth | Rejected before materialization and without consuming input | Exact limits, every split point, declared cardinality, and nested maps |
Encoding a Frame is intentionally less restrictive because outbound frames
are application-built. In particular, the re-exported enum can serialize
attributes and streamed variants that this decoder rejects. Do not infer
receive support from the variants exposed by resp-rs.
Allocation and ownership
The codec preflights the first reply without materializing a frame. Once that
reply is complete and within its configured size/depth limits, decoding copies
exactly that bounded first frame into immutable Bytes for resp-rs; it does
not clone or copy the entire unread receive batch. Aggregate elements still
have decoded representation overhead, so max_frame_size is a wire-size bound,
not an exact heap budget. This is a bounded-copy path, not an end-to-end
zero-copy claim. The decode ownership and copy evidence
compares the historical whole-buffer clone, the production first-frame copy,
and a split/shared alternative, including retained-frame lifetime behavior.
The declared-cardinality regression uses a test-only scan observer: an array
or map declaring ten million entries visits exactly its one available header
and never spills the inline nesting stack. This deterministically verifies
constant scan work without attempting a dangerous allocation. A separate
test-only materialization counters wrap the public RespCodec::decode path:
modest incomplete array and map declarations leave both frame and byte counts
at zero, while a complete array is the positive control. A retained mixed
pipeline proves the byte count is exactly the sum of completed frame extents,
and byte-at-a-time delivery proves incomplete fragments are never copied for
parser input.
Fragmentation oracle
The decode_chunked fuzz target decodes each input twice under identical
limits: once with the entire wire available and once with a repeating sequence
of arbitrary one-to-64-byte chunks. It compares:
- every successful frame in order, represented by its canonical re-encoding;
- the bytes retained at the first incomplete or rejected frame; and
- the terminal disposition, including the exact parser error variant.
Canonical bytes avoid treating NaN != NaN as a protocol disagreement. A
frame-size error compares by disposition rather than the diagnostic size
field: a prefix can prove that the cap is exceeded before the complete extent
is known. If incremental decoding rejects a prefix, the harness appends the
not-yet-delivered bytes without retrying the decoder, matching a connection
that closes on a protocol error while preserving a whole-buffer-comparable
remainder.
The property suite separately builds up to sixteen valid replies, including nested mixed aggregates, then checks arbitrary partitions and truncation points. Raw wire fixtures are kept independent of the crate encoder so a paired encoder/decoder defect cannot be the only evidence.
Fuzz input and retained evidence
Both fuzz targets use this compact envelope:
[max_depth, max_size_hi, max_size_lo, plan_length, plan..., RESP wire...]
The plan is capped at 16 bytes and each value maps to a non-zero chunk size.
Inputs shorter than four bytes are treated as raw wire with conservative
limits. Reviewed seeds live as named hexadecimal fixtures under
fuzz/corpus-seeds/; generated corpus entries and crash artifacts remain
ignored working state. Every decode_chunked seed starts with a one-byte plan,
so its intended boundary behavior is exercised before mutation discovers new
partitions.
Pull requests run ten seconds per target. The weekly and manually dispatchable
Scheduled Fuzzing workflow accepts a one-to-3,600-second duration per target.
Each job retains its reviewed starting seeds, final corpus, crash artifacts,
log, toolchain versions, source SHA, resolved Cargo manifests and lockfiles,
their hashes, duration, exit code, and terminal status for 90 days. The manifest
is replaced with running before seed preparation and changes to passed only
after a zero process exit, the requested wall time, and libFuzzer’s normal
completion record. Graceful signal exits are explicitly rejected, so an
interruption cannot masquerade as a completed pass.
For local reproduction:
python3 scripts/prepare_fuzz_corpus.py \
fuzz/corpus-seeds/decode_chunked fuzz/corpus/decode_chunked
cargo +nightly fuzz run decode_chunked fuzz/corpus/decode_chunked -- \
-max_total_time=300
Historical mutation triage
The September 21 protocol report predated the current preflight limit scanner and scored a library-only mutation scope at 58.5%. Its survivors are evidence to classify, not a score that can be compared with a differently scoped run.
| Historical survivor family | Classification and current evidence |
|---|---|
| Delete or alter map depth traversal | Real assertion gap. nested_maps_consume_the_same_depth_budget_as_other_aggregates now rejects a two-level map at depth one and accepts it at depth two. Fixed attributes are separately unsupported. |
| Scan inside a blob instead of skipping its declared payload | Real framing and depth gap. Binary fixtures contain streaming and aggregate tokens, while unit and integration regressions prove those bytes remain opaque. |
| Alter CRLF scanning at a fragment boundary | Real fragmentation gap. Every strict prefix of independent frames is incomplete, exact split tests cover bounded frames, and the generated/fuzz oracles compare arbitrary partitions. |
| Change length/cardinality arithmetic or overflow handling | A real gap when the mutation changes rejection, consumption, or allocation bounds; deterministic tests cover oversized declarations, impossible cardinality, integer overflow, and malformed terminators. A mutation that changes only the early diagnostic lower-bound size while preserving its documented class is equivalent under this contract. |
| Timeout or mutation outside a reachable supported disposition | Tool limitation or excluded protocol feature, to be reported separately rather than counted as covered. Attributes and streaming remain intentional fail-closed exclusions. |
A future score comparison must record the exact source SHA, cargo-mutants version, test command, package scope, and raw outcomes. The scheduled mutation workflow already retains that evidence; this work does not relabel the old number as a current baseline.
RESP decode ownership and copy evidence
RespCodec deliberately copies one complete, bounded frame after preflight.
It does not copy the entire unread receive buffer, and it does not claim a
zero-copy decode path. This page records why that ownership model is the
current production choice and how to reproduce its evidence.
Decision
The historical decoder passed src.clone().freeze() to resp-rs for every
reply. BytesMut::clone() copies the full unread slice, so a batch of equal
frames caused the shrinking remainder to be copied repeatedly. The current
decoder first determines the complete frame extent, copies only those bytes,
parses that immutable frame, and advances the receive buffer only after parse
success. Total parser-input copying is therefore linear in decoded wire bytes.
A split_to(frame_len).freeze() design removes that copy and is faster in the
focused harness, but it couples response lifetime to receive-buffer lifetime.
A small response retained after the unread tail is drained can pin the entire
original allocation. It also complicates the codec’s promise that parse errors
leave input unchanged: restoring an already-split prefix requires rebuilding
the buffer on the exceptional path.
The bounded first-frame copy is the selected tradeoff because it:
- avoids the historical quadratic copied-byte mechanism;
- limits a retained response to frame-sized backing storage rather than the whole receive batch;
- preserves unchanged bytes on incomplete input and all errors; and
- requires no unsafe code in the production protocol crate.
The first-frame implementation arrived with the earlier bounded-preflight work. The evidence here validates and regression-tests that choice; it is not a claim that this evidence round alone produced an end-to-end client speedup.
Retained evidence
The checked raw evidence was recorded
on macOS arm64 at source b7256ba04acda830368632af1f2450f746029c81
with Rust 1.98.1, Cargo 1.98.1, bytes 1.12.1, resp-rs 0.1.8,
tokio-util 0.7.19, and Criterion 0.5.1. The source tree was clean when the
run began. Allocation values are requested layout bytes observed through an
instrumented System allocator, not RSS. “Allocated” is the sum of successful
initial layout sizes plus positive growth during reallocation; “live” is the
change in requested live layout bytes. Neither value includes allocator
metadata or size-class rounding. Durations are 30 raw within-process samples
of 20 iterations with allocation instrumentation active. They are directional
mechanism evidence, not Redis client throughput or a cross-host performance
claim.
The retained-batch rows below keep every decoded frame until the batch is
complete. “Copied wire” counts only bytes copied to create parser input;
“allocated” and “live” also include the requested layout sizes for receive
storage, Frame values, and aggregate vectors.
| Scenario and strategy | Copied wire | Allocated through decode | Live after decode | Median time |
|---|---|---|---|---|
| 512 five-byte replies; historical whole-buffer copy | 656,640 B | 708,352 B | 708,352 B | 54,307 ns |
| 512 five-byte replies; production first-frame copy | 2,560 B | 54,272 B | 54,272 B | 39,685 ns |
| 512 five-byte replies; split/shared comparison | 0 B | 39,464 B | 39,464 B | 12,722 ns |
| 128 mixed replies (71,296 wire bytes); historical whole-buffer copy | 4,630,640 B | 4,718,832 B | 3,576,560 B | 99,510 ns |
| 128 mixed replies; production first-frame copy | 71,296 B | 158,720 B | 158,544 B | 16,425 ns |
| 128 mixed replies; split/shared comparison | 0 B | 85,160 B | 85,160 B | 6,389 ns |
The lifetime counterexample is the reason the fastest microbenchmark strategy is not the production default. The input contains a 23-byte first frame and a 262,155-byte unread tail. After the unread buffer is dropped while the head frame remains alive:
| Strategy | Copied wire | Bytes pinned by the retained head | Median time |
|---|---|---|---|
| Historical whole-buffer copy | 262,178 B | 262,202 B | 8,819 ns |
| Production first-frame copy | 23 B | 47 B | 3,275 ns |
| Split/shared comparison | 0 B | 262,218 B | 3,637 ns |
These numbers should not be extrapolated to a complete Redis workload. The production path also performs the bounded preflight scan, while the comparison strategies receive known fixture extents. Later parser or buffer changes should rerun the same artifact rather than preserving these numbers as a permanent marketing baseline.
Regression and benchmark coverage
The protocol unit suite counts both completed materializations and their exact wire bytes. A mixed pipeline containing binary bulk data, a push, scalars, and an aggregate proves that production materializes the sum of frame sizes once, not the sum of shrinking unread suffixes. A byte-at-a-time case proves that no copy happens until a frame is complete. Existing boundary tests continue to cover limits, malformed input, unchanged error bytes, binary payloads, pushes, and response order.
Criterion now includes:
- dropped and batch-retained lifetimes for 100 simple replies;
- dropped and batch-retained mixed payloads from zero through 4 KiB; and
- a smaller mixed fixture with payloads through 256 bytes, delivered in 1-, 7-, 64-, and 1,024-byte fragments.
The raw allocation artifact repeats those four fragment sizes over the same smaller 112-frame fixture, retains the decoded batch, and records receive-buffer reallocations, net acquired requested layout bytes, and the change in requested live layout bytes. Every fragmentation plan decodes frames equivalent to the contiguous production path and reports exactly one frame’s wire bytes per completed materialization.
| Delivery chunk | Copied wire | Allocated through decode | Buffer reallocations | Median time |
|---|---|---|---|---|
| 1 B | 5,616 B | 20,720 B | 6 | 86,262 ns |
| 7 B | 5,616 B | 20,720 B | 6 | 24,997 ns |
| 64 B | 5,616 B | 20,720 B | 3 | 12,590 ns |
| 1,024 B | 5,616 B | 22,256 B | 1 | 11,136 ns |
To reproduce the checked allocation/copy artifact from a clean checkout, use an absolute output path because Cargo runs the custom benchmark from the package directory:
evidence_output="$PWD/conformance/codec-copy-evidence.json"
cargo bench -p redis-tower-protocol --bench codec_allocation -- \
--evidence --samples 30 --iterations 20 --check \
--output "$evidence_output"
The --check mode verifies equivalent decoded frames, exact deterministic
copy counts, lower production allocation than the historical mechanism, and
the retained-tail tradeoff. cargo test --benches runs a short silent smoke;
an explicit --evidence invocation is required for retained timing samples.
Run the Criterion timing suite separately with:
cargo bench -p redis-tower-protocol --bench codec
Client comparison
This page compares documented client contracts, not feature counts. A checkmark can hide the questions that matter in production: whether a connection is exclusive or multiplexed, which commands are replayed after reconnect, whether a cache is managed for you, and whether a “typed” API fixes the response type or lets the caller choose one.
The comparison was checked on 2026-09-25 against the versions and revisions below. Links point to upstream documentation or pinned source. Treat a claim as unknown when the cited material does not establish it; do not infer absence from an empty cell.
| Client | Baseline used here |
|---|---|
| redis-tower | 186719e; published workspace versions range from 0.1.1 to 0.1.3 |
| redis-rs | 1.7.0 (2a29a8f) |
| Fred | 10.1.0 (29d4790) |
| Lettuce | 4461844 plus the rolling Lettuce wiki, checked on the date above |
| go-redis | 2fc3ccd |
| StackExchange.Redis | 0a92ae4 |
| ioredis | b597303 |
| node-redis | d3eac3d |
| redis-py | ba6976b |
Rust clients
redis-tower, redis-rs, and Fred overlap heavily, but expose different contracts.
| Contract | redis-tower | redis-rs 1.7 | Fred 10.1 |
|---|---|---|---|
| Typed command API | Command values fix their response types; pinned Get returns Option<Bytes>. Raw commands can opt into a decoder. | AsyncTypedCommands provides typed convenience methods; Cmd::query_async::<T> retains caller-selected conversion. | Command traits expose typed arguments and caller-selected response types; see the versioned interfaces module. |
| Composition point | Frame services implement Tower Service; pinned middleware sources cover timeouts, circuit breaking, tracing, and metrics. | Unknown / not compared: the 1.7 API index documents commands and connections, but this review did not establish a general middleware contract. | Config and the versioned interfaces are client-specific extension points; no cross-client middleware equivalence is asserted here. |
| Shared connection | Pinned MultiplexedClient is cloneable and auto-pipelines concurrent commands over one connection. | MultiplexedConnection is cloneable and multiplexed. | Client is a cloneable handle over the driver’s connections. |
| Pool | Pinned ConnectionPool owns independent connections with fixed, dynamic, or lazy population and explicit health/lifecycle controls. | Optional r2d2 and bb8 integrations are listed in the pinned 1.7 feature definitions. A multiplexed connection is not itself a checkout pool. | Pool distributes commands across clients; stateful interfaces are intentionally omitted. |
| Topology | Standalone, Cluster, and Sentinel are separate clients; pinned UniversalClient provides one enum-backed entry point. | Standalone plus the cluster-async and sentinel features are defined in the pinned 1.7 manifest. | ServerConfig covers centralized, Cluster, Sentinel, and Unix deployments. |
| Reconnect and replay | Pinned ResilientRedisClient single-flights reconnects with bounded backoff. Ordinary in-flight commands are not silently replayed; stateful APIs document their own restoration rules. | ConnectionManager reconnects in the background; the command that observes the dropped connection errors and later commands wait for the replacement connection. | ReconnectPolicy controls reconnect delay. Fred’s pinned ConnectionConfig defaults to three command attempts, and reconnect_once flushes previously in-flight commands from the retry buffer. The redelivery metric counts requests sent again after close while awaiting a response; pinned command policy and attempt checks show that fail-fast, attempt exhaustion, or no reconnect policy stops redelivery. |
| Client-side caching | Pinned standalone and master-routed Cluster caching manages tracking, invalidations, bounds, metrics, and reconnect fail-closed behavior. | The experimental cache-aio feature integrates caching with multiplexed, connection-manager, and async Cluster connections. | TrackingInterface exposes RESP3 tracking and invalidation commands; it is not documented as a managed local cache. |
| Redis Stack and newer families | Pinned redis-tower-modules covers JSON, Search, TimeSeries, Bloom/Cuckoo, and Vector Sets. | The pinned 1.7 feature definitions include JSON, Bloom, and Vector Sets; Search is marked unfinished. | The versioned interfaces module documents JSON and Search interfaces. Other families remain unverified here. |
| Credentials | Pinned CredentialProvider supports reconnect/push rotation; sibling crates provide AWS IAM and Microsoft Entra providers. | The pinned 1.7 feature definitions include token-based authentication and Microsoft Entra support. | Versioned Config exposes a credential provider when its feature is enabled. |
| Observability | Pinned production guidance documents stable tracing fields, metrics-facade recording, pool/queue snapshots, and bounded opt-in per-node Cluster labels. | Unknown / not compared: no client-wide tracing/metrics contract was verified from the 1.7 API index; application wrappers are outside this comparison. | Pinned TracingConfig is feature-gated; versioned MetricsInterface exposes redelivery, queue, latency, and payload-size measurements. Export to an external metrics backend remains application work. |
| RESP behavior | Pinned redis-tower-protocol documents RESP2/RESP3 negotiation, push demultiplexing, and explicit rejection of unsupported attributes/streamed aggregates. | RESP2 and RESP3 are supported; URL selection is documented by versioned ConnectionInfo. | Pinned Config::version defaults to RESP2 and can require RESP3; TrackingInterface requires RESP3 for tracking. |
The important typed-API distinction is not “typed versus untyped.” All three clients provide typed conveniences. redis-tower command values own a specific decoder, while redis-rs and Fred also make caller-selected response conversion a common path.
Lessons from widely used clients
These clients are useful design references, not direct feature-score opponents.
| Client | Documented contract worth carrying into a review |
|---|---|
| Lettuce | Its execution reliability documentation treats reconnect replay, duplicates, ordering, and transaction behavior as separate guarantees. A generic “reconnects automatically” claim is insufficient. |
| go-redis | The pinned README documents automatic pooling, experimental streaming credentials, RESP3 client caching limits, and OpenTelemetry instrumentation. Its experimental auto-pipeline warns that retrying a whole batch can duplicate non-idempotent work. |
| StackExchange.Redis | ConnectionMultiplexer is designed to be shared and reused; cheap database handles and multiplexing are deliberately different from checking out a socket per request. |
| ioredis | The pinned README documents standalone, Sentinel, and Cluster connections, offline queues, retry behavior, and auto-pipelining. Queueing before connect and replaying after a disconnect are different decisions. |
| node-redis | Its pinned FAQ says already-sent commands reject when the socket closes because Redis may have executed them; unsent commands can remain queued for reconnect. That boundary is a useful retry model. |
| redis-py | The pinned unified response proposal separates wire-protocol shape from the public response contract. Differential tests must normalize client-facing values, not merely compare raw RESP frames. |
What this page does not claim
- It does not rank clients. API fit, deployed topology, language ecosystem, and operational familiarity usually matter more than a total capability count.
- It does not publish performance conclusions. The repository’s pinned benchmark publication protocol defines workloads, client versions and adapters, topology, connection count, timeouts, metadata, and reproducibility requirements. A result is evidence only for the measured environment and scenario.
- It does not turn undocumented behavior into “no.” Unknown behavior stays unknown until a stable upstream source or a reproducible experiment supports a narrower statement.
When updating this page, pin the source revision, record the check date, link the precise upstream contract, and describe reconnect, retry, replay, pooling, caching, and response typing independently.
Test conformance
This report maps redis-tower’s test surface to the claims it supports. It is intended to be citable at a particular commit: the compiled inventory is generated from the checkout, while the matrices below link each broader claim to its test code and workflow. A compiled test is not automatically a passing test, and a scheduled job is not evidence that a particular run was green; use the checks attached to the commit for that conclusion.
This page inventories compiled tests, configured workflow expansion, and observed execution evidence. Those are distinct from the logical client behaviors they exercise. The generated command capability ledger records those behaviors and separately tracks command-name availability, typed semantics, RESP wire support, and session/topology claims; neither page treats one dimension as proof of another.
Compiled inventory
The generated block is the only place this page records test counts. Keeping
the numbers mechanical prevents a prose headline from drifting as tests are
added, renamed, or split into new binaries. Refresh it with
python3 scripts/generate_test_conformance.py; CI checks that the committed block
matches the current source tree.
Compiled integration inventory
The scoreboard-scoped packages compile 756 integration tests across 45 test binaries. 212 tests are marked #[ignore] because they need explicit infrastructure; workflow selectors, rather than the annotation alone, determine whether they run.
| Surface | Package | Binaries | Compiled | #[ignore] | Pull request | Scheduled | No workflow selector |
|---|---|---|---|---|---|---|---|
| Standalone | redis-tower | 32 | 520 | 0 | 520 | 229 | 0 |
| Cluster | redis-tower-cluster | 3 | 112 | 106 | 112 | 0 | 0 |
| Sentinel | redis-tower-sentinel | 2 | 88 | 88 | 88 | 0 | 0 |
| Modules | redis-tower-modules | 7 | 20 | 18 | 20 | 18 | 0 |
| Distributed primitives | redis-tower-primitives | 1 | 16 | 0 | 16 | 0 | 0 |
| Total | 45 | 756 | 212 | 756 | 247 | 0 |
Counts are unique compiled tests. A test selected by both pull-request and scheduled workflows appears in both cadence columns, but only once in the compiled total.
Cross-backend command parity
The shared command corpus produces 210 compiled parity tests: 35 test cases × 6 client/topology expansions.
| Surface | Test binary / variant | Compiled cases |
|---|---|---|
| Standalone | integration / resp3 | 35 |
| Standalone | integration / root expansion | 35 |
| Cluster | cluster_integration / multiplexed | 35 |
| Cluster | cluster_integration / root expansion | 35 |
| Sentinel | sentinel_integration / multiplexed | 35 |
| Sentinel | sentinel_integration / root expansion | 35 |
Independent-client differential coverage
The independent redis-rs corpus contains 12 compiled test entry points. These are counted separately from shared-client topology expansions.
| Surface | Test binary | Compiled tests | Default |
|---|---|---|---|
| Standalone | differential_redis_rs | 9 | normal |
| Cluster | cluster_integration | 1 | ignored |
| Sentinel | sentinel_integration | 1 | ignored |
| Modules | differential_module_replies | 1 | ignored |
Server compatibility matrix
| Cadence | Target | Implementation source |
|---|---|---|
| Pull request | Redis 7.4.3 | source build |
| Pull request | Redis 8.0.6 | source build |
| Nightly | redis-7.2 | redis:7.2-alpine |
| Nightly | redis-7.4 | redis:7.4-alpine |
| Nightly | redis-8.0 | redis:8.0-alpine |
| Nightly | redis-8.2 | redis:8.2-alpine |
| Nightly | redis-8.4 | redis:8.4-alpine |
| Nightly | redis-8.6 | redis:8.6-alpine |
| Nightly | redis-8.8 | redis:8.8-alpine |
| Nightly | valkey-8.1 | valkey/valkey:8.1-alpine |
Fault and destructive-recovery inventory
| Surface | Test binary | Compiled test | Default | Workflow cadence |
|---|---|---|---|---|
| Standalone | integration | multiplexed_client_reconnects_after_server_restart | normal | pull request, nightly/scheduled |
| Standalone | test_resilience_integration | circuit_breaker_ignores_redis_command_errors | normal | pull request |
| Standalone | test_resilience_integration | circuit_breaker_opens_on_repeated_failures | normal | pull request |
| Standalone | test_resilience_integration | command_timeout_fires_on_slow_command | normal | pull request |
| Standalone | test_server | cover_replicaof_and_failover | normal | pull request |
| Cluster | cluster_integration | cached_cluster_fails_closed_and_recovers_after_master_connections_are_killed | ignored | pull request |
| Cluster | cluster_integration | cluster_pool_exhaustion_and_recovery | ignored | pull request |
| Cluster | cluster_integration | mux_cluster_handles_ask_then_moved_during_live_reshard | ignored | pull request |
| Cluster | cluster_integration | mux_cluster_keyless_commands_survive_default_master_failover | ignored | pull request |
| Cluster | cluster_integration | mux_cluster_replaces_killed_master_after_replica_promotion | ignored | pull request |
| Cluster | topology_recovery | keyless_commands_follow_the_default_master_through_failover | normal | pull request |
| Sentinel | sentinel_failover | sentinel_failover_sequence | ignored | pull request |
| Sentinel | sentinel_integration | sentinel_pool_exhaustion_and_recovery | ignored | pull request |
Reproducibility
The generator compiles only the client-facing integration-test packages with all features and --no-run, asks each emitted libtest binary for its full and ignored listings, and inspects checked-in workflow cargo test selectors. Regenerate or verify the marked section with:
python3 scripts/generate_test_conformance.py
python3 scripts/generate_test_conformance.py --check
The inventory describes what Cargo discovers under the generator’s documented
feature set. It does not claim that every test runs in a normal cargo test,
because infrastructure-backed suites are deliberately #[ignore]-gated and
invoked by the workflows described below. It also cannot infer a runtime early
return. The Redis 8.x command binary now discovers redis_version from INFO
and fails when REDIS_8X_REQUIRED=1 cannot reach a suitable server; an optional
REDIS_8X_VERSION is cross-checked against that discovery rather than treated
as proof by itself.
Independent-client differential corpus
The compiled differential entry points compare redis-tower with a test-only, exactly pinned redis-rs 1.7.0 adapter. They are independent-client evidence, not additional expansions of the shared redis-tower command macro, and their generated count is therefore reported separately above.
The standalone corpus runs on every per-PR Redis 7.4.3 and 8.0.6 leg under
both RESP2 and RESP3. Mutations use equivalent fresh namespaces, failures
retain the case/protocol/live server/client version and deterministic seed,
and normalization is limited to documented unordered or pair-shaped replies.
The module comparison remains #[ignore]-gated. It runs in the path-filtered
per-PR module gate and the broader module-enabled nightly matrix. See the reviewed
case ledger and lifecycle contract for exact inputs,
dispositions, negative controls, and reproduction commands.
What superseded the original fault comparison
Issue #416 proposed
running the former custom CircuitBreakerLayer and
tower-resilience-circuitbreaker side by side under identical faults. That
experiment was useful input while the implementation choice was still open.
PR #636 subsequently
made the choice: it replaced the custom circuit-breaker internals with the
Redis-aware adapter in
circuit_breaker.rs. There is
no longer a maintained custom breaker to use as the second side of the proposed
A/B matrix, so presenting that matrix as current conformance coverage would be
misleading.
The maintained contract is narrower and directly tested:
tower-resilience-circuitbreakerowns circuit state, half-open admission, metrics, and state transitions. The adapter preservesRedisError, maps an upstream open rejection toRedisError::CircuitOpen, and exposes an operational handle.- The Redis classifier counts connection failures, connect timeouts, command timeouts, and recursively classified reconnect exhaustion. Redis command replies and caller errors do not degrade circuit health.
- Unit tests in
circuit_breaker.rscover classification, open rejection, successful half-open recovery, and the distinction between a deadline exhausted while waiting for readiness and a timeout after dispatch. - Live tests in
test_resilience_integration.rscover refused TCP connections through open and half-open recovery, a real slow Redis command through the timeout layer, and aWRONGTYPEresponse that leaves the circuit closed.
This does not claim fault parity for every crate in the tower-resilience family. Redis-aware circuit breaking is the upstream-backed adapter in this repository; command timeouts and idempotency-aware retries remain redis-tower-owned policies. Upstream rate limiting and bulkhead layers can be composed by applications, but they are not part of this conformance matrix.
Continuous test tiers
| Tier | Coverage | Cadence | Evidence |
|---|---|---|---|
| Unit and rustdoc | All-feature library tests on stable and beta Rust | Every pull request and push to main | CI workflow |
| Standalone | Every redis-tower integration binary and the standalone UniversalClient path against source-built Redis 7.4.3 and 8.0.6 | Every pull request and push to main | CI workflow, standalone suite, universal-client suite |
| Universal topology entry points | Standalone URL auth, authenticated Cluster URL selection, Sentinel URL selection, command execution, and recovery after a real Sentinel failover | Standalone on both standard per-PR Redis legs; dedicated Cluster/Sentinel fixture in the transport gate | CI workflow, transport gate, UniversalClient topology suite |
| Independent differential | MCP-derived semantic cases through redis-tower and pinned redis-rs adapters, including RESP2/RESP3, binary/null/error, pipeline, transaction, blocking-session, topology, cancellation, and lost-reply contracts | Standalone, Cluster, and Sentinel per PR; module replies in the path-filtered module gate and nightly | case ledger, standalone corpus, module corpus |
| Cluster | Shared command corpus plus routing, redirect, caching, reshard, TLS, pool, and failover scenarios against Redis 7.4.3 and 8.0.6 | Every pull request and push to main; invoked explicitly with --ignored | CI workflow, cluster suite |
| Sentinel | Shared command corpus, discovery, routing, pool, reconnect, and isolated destructive failover scenarios against Redis 7.4.3 and 8.0.6 | Every pull request and push to main; invoked explicitly with --ignored | CI workflow, healthy suite, failover suite |
| Cluster fixture lifecycle | Partial-start cleanup, promotion/reshard/kill cleanup, and hard deadlines while nodes are frozen | Every pull request and push to main; invoked explicitly with --ignored | CI workflow, fixture tests |
| Server compatibility | Focused integration.rs suite across supported Redis minor lines and Valkey; Redis 8 command tests on each Redis 8 leg | Nightly and manual dispatch | nightly compatibility workflow |
| Verified TLS transports | Private-CA trust and hostname verification, percent-encoded password auth, RESP3/database setup, rejection of a wrong hostname and untrusted CA, Cluster roundtrips, and bracketed IPv6 URL transport | Isolated rustls-only and native-TLS-only live legs per PR; bracketed IPv6 required in the same transport gate | transport gate, standalone TLS suite, Cluster TLS suite |
| Redis modules | A fail-closed actual-version/COMMAND INFO preflight followed by JSON, Search, TimeSeries, probabilistic, Vector Set, and redis-rs differential assertions | Path-filtered per PR on Redis 8; broader Redis 8 and Redis Stack 7.x matrix nightly/manual | module gate, nightly modules workflow, capability preflight |
| Soak fault smoke | Short standalone SIGKILL/restart and cluster master-kill/promotion runs, including JSONL contract checks | Pull requests touching the harness/client paths, and manual dispatch | soak workflow, soak harness |
| RESP property and fuzz | Generated mixed RESP2/RESP3 frames, independent wire fixtures, whole-versus-fragmented semantic comparison, and arbitrary malformed input | Property and ten-second fuzz smoke per PR; bounded weekly/manual campaigns | protocol contract, fuzz targets, scheduled workflow |
| RESP decode copy/allocation | Exact test-only first-frame copied-byte counts; simple, mixed, retained-lifetime, and fragmented Criterion cases; instrumented comparison of historical, production, and split/shared ownership | Copied-byte unit contract and Criterion regression suite per PR; raw allocation artifact retained when ownership changes | ownership decision, raw evidence, benchmark sources |
The per-PR cluster and Sentinel commands intentionally include --ignored.
In this repository, that flag marks an infrastructure boundary, not an
optional quality tier. In the remaining tables, “per PR” also includes the
same CI tier on pushes to main.
Client and protocol parity
The reusable
command_tests! corpus
exercises strings, keys, hashes, lists, sets, sorted sets, bitmaps, geospatial
commands, HyperLogLog, and streams through the same typed-command API. It is
instantiated along these paths:
| Topology | Client path | Protocol policy exercised by the shared corpus | Automated cadence |
|---|---|---|---|
| Standalone | RedisConnection | Forced RESP2 | Per PR on Redis 7.4.3 and 8.0.6; nightly version matrix |
| Standalone | RedisConnection | Forced RESP3 | Per PR on Redis 7.4.3 and 8.0.6; nightly version matrix |
| Cluster | ClusterConnection | Default Auto policy across routed connections | Per PR on Redis 7.4.3 and 8.0.6 |
| Cluster | MultiplexedClusterClient | Default Auto policy across pipelined node connections | Per PR on Redis 7.4.3 and 8.0.6 |
| Sentinel | SentinelConnection | Default Sentinel connection path; no forced-protocol parity claim | Per PR on Redis 7.4.3 and 8.0.6 |
| Sentinel | MultiplexedSentinelClient | Default Sentinel connection path; no forced-protocol parity claim | Per PR on Redis 7.4.3 and 8.0.6 |
The generated parity total counts compiled expansions, not assertions that run unchanged on every server version. Six Vector Set cases probe capability and return early when the server does not support those commands.
Standalone also has focused negotiation coverage for forced RESP2, forced
RESP3, Auto, and the default connection path in
integration.rs. Cluster
connection builders retain an explicit protocol policy across discovery,
redirects, topology refresh, and reconnect, with focused coverage in
connection.rs and
multiplexed.rs. Sentinel
does not yet expose equivalent explicit protocol selection, so its rows above
must not be read as RESP2/RESP3 parity.
UniversalClient has focused public-entry tests for all three variants. The
Cluster case proves percent-decoded authentication; the Sentinel case kills the
original master and requires the already-created client to resume commands
after election. These are topology/lifecycle probes, not a replay of the full
shared command corpus, so this report does not claim full universal-client
command parity.
On Unix CI, the live
test_unix_url suite opens a
percent-encoded socket path through every supported Unix URL alias, applies
percent-decoded ACL credentials, selects a non-default database, negotiates
RESP3, and repeats setup through a fresh factory connection. Unix sockets are
not available on Windows, and the matrix does not claim otherwise.
Exclusive-session behavior is exercised through the constructors users call:
transactions and blocking commands own a RedisConnection, while Pub/Sub
converts an owned connection with PubSubConnection::from_connection and
explicitly replays subscriptions through reconnect_with. Those session types
do not promise transparent reconnect on their own; factory-backed multiplexed,
Cluster, and Sentinel handles provide that lifecycle. The corresponding live
tests run in the standalone, Cluster, and Sentinel per-PR suites, including
transaction abort/commit, bounded blocking, subscription replay, and topology
rediscovery. See the
standalone integration suite and
Cluster integration suite.
Server version matrix
| Server | Standalone | Cluster | Sentinel | Version-gated Redis 8 commands | Modules |
|---|---|---|---|---|---|
| Redis 7.4.3 | Per PR | Per PR | Per PR | Not applicable | No |
| Redis 8.0.6 source build | Per PR | Per PR | Per PR | Not activated on the module-free source build | No |
| Redis 8 module image | Focused Redis 8 command binary in the path-filtered gate | Not covered by this image leg | Not covered by this image leg | Per PR when relevant; actual version discovered from INFO | Per-PR capability preflight plus all module suites |
| Redis 7.2 and 7.4 container tags | Nightly | Not in the nightly matrix | Not in the nightly matrix | Not applicable | No |
| Redis 8.0, 8.2, 8.4, 8.6, and 8.8 container tags | Nightly | Not in the nightly matrix | Not in the nightly matrix | Nightly, with the corresponding version input | Redis 8 module image in the separate nightly module workflow |
| Valkey 8.1 container tag | Nightly | Not covered | Not covered | Not applicable | No |
| Redis Stack 7.4.0-v3 | No core-client matrix claim | Not covered | Not covered | Not applicable | Nightly JSON, Search, TimeSeries, and probabilistic suites |
The exact per-PR patch versions and nightly image tags are defined in
ci.yml,
nightly.yml, and
module-gate.yml, and
nightly-modules.yml. Moving
container tags intentionally test the current patch in their minor line; the
capability log records the actual INFO redis_version and required command set,
while the workflow log and image digest identify the image for a particular run.
The nightly compatibility legs run the focused integration.rs suite, not
every standalone test binary in the generated inventory.
Destructive and fault-injection inventory
| Injected condition | Required observation | Cadence | Evidence |
|---|---|---|---|
Redis command exceeds its deadline (DEBUG SLEEP or a blocking command) | A bounded CommandTimeout is surfaced before the slow command completes | Per PR | resilience tests, standalone integration |
| Connected TCP peer applies a write but withholds all or some pipeline replies | Explicit response timeout reports unknown execution, closes the old socket, and recovers on a replacement without replay or reply misalignment | Per PR, no Redis process | auto-pipeline tests |
| Refused TCP connections, then a healthy endpoint | Circuit opens at the configured threshold, rejects immediately, admits a half-open probe, and closes after recovery | Per PR | resilience tests |
| Managed standalone process stopped and restarted on the same port | The existing multiplexed client eventually resumes validated reads and writes against a fresh database | Per PR; deliberately bypassed in externally managed nightly service containers | standalone integration |
Standalone REPLICAOF role changes and coordinated FAILOVER | Replication completes, roles change, and acknowledged data survives promotion | Per PR | server integration |
| Cluster slot migration held between importing/migrating and final ownership | Client follows ASK and then MOVED, while its redirect metrics record the expected order | Per PR | cluster integration |
| Cached cluster data and invalidation connections killed on one master | Cache fails closed, clears unsafe entries, returns current data, and restores complete tracking coverage | Per PR | cluster integration |
| Master killed in a dedicated cluster and its acknowledged replica promoted | Client reaches a bounded first successful read, preserves data, and replaces the dead slot owner | Per PR | cluster integration |
| One Sentinel killed, then the monitored master killed | Discovery survives one Sentinel loss while quorum remains, Sentinel elects a different master, the existing client rediscovers it, and a fresh connection finds it | Per PR, in an isolated ordered test binary | Sentinel failover sequence |
| Pool contention with one available connection | Concurrent cluster and Sentinel callers serialize and recover rather than failing from exhaustion | Per PR | cluster suite, Sentinel suite |
| Cluster fixture startup expires, or all fixture nodes are frozen | Partial processes and artifacts are reaped; the caller’s deadline stays hard; teardown resumes stopped nodes and removes only owned state | Per PR | fixture lifecycle tests |
| Real standalone SIGKILL/restart under continuous validated load | Exact owned process dies, replacement starts on the same port, lifecycle event is observed, and useful work recovers within the bound | Path-filtered PR smoke; long run by explicit publication protocol | soak workflow, soak harness |
| Real cluster master SIGKILL under continuous validated load | Redis promotes the replicated slot owner and the client completes a validated probe within the bound | Path-filtered PR smoke; long run by explicit publication protocol | soak workflow, soak harness |
Destructive Sentinel phases use a separate port range and a single orchestrating test because killing shared processes would make independent tests order-dependent. Dedicated cluster failover and reshard fixtures use their own topologies for the same reason.
Ignored-test policy
#[ignore] is reserved for tests that need infrastructure or intentionally
mutate a shared topology: live cluster and Sentinel processes, module-enabled
servers, Docker, destructive fault injection, or signal-cleanup checks. Each
ignored test must state or inherit its prerequisite and must have one of these
execution paths:
- an explicit per-PR
cargo test ... -- --ignoredstep; - a scheduled workflow matched to the required server image; or
- a path-filtered/manual fault-harness workflow.
The mapping is visible in the CI, compatibility, per-PR module, nightly module, and soak workflows. A normal workspace test will therefore report ignored tests; that is expected. The conformance goal is no silently abandoned ignored tests, not a false claim that every environment can execute every test.
Reproduce the tiers
The process-backed commands require redis-server and redis-cli on PATH.
Cluster, Sentinel, and destructive suites should be run single-threaded to
match CI and avoid port/topology interference.
# Generated inventory and drift check
python3 scripts/generate_test_conformance.py
python3 scripts/generate_test_conformance.py --check
# Unit, rustdoc, standalone, and universal-client coverage
cargo test --lib --all-features
cargo test --doc --all-features
cargo test -p redis-tower --test '*' --all-features -- --test-threads=1
cargo test -p redis-tower-client --test '*' --all-features -- --test-threads=1
# Infrastructure-gated per-PR suites
cargo test -p redis-tower-client --test topologies --all-features -- \
--ignored --test-threads=1
cargo test -p redis-tower-cluster --test cluster_integration \
--all-features -- --ignored --test-threads=1
cargo test -p redis-tower-sentinel --test 'sentinel_*' \
--all-features -- --ignored --test-threads=1
cargo test -p redis-tower-test --lib live_ -- --ignored --test-threads=1
# The upstream-backed circuit and timeout contract only
cargo test -p redis-tower --test test_resilience_integration \
--all-features -- --test-threads=1
# Docker fixture used by nightly compatibility
REDIS_TEST_IMAGE=redis:8.8-alpine \
cargo test -p redis-chaos-tests --test docker_smoke -- --ignored
# Module suites; preflight the actual server before behavior
REDIS_URL=redis://127.0.0.1:6399 REDIS_8X_REQUIRED=1 \
cargo test -p redis-tower-modules --all-features --test capabilities \
required_server_version_and_commands_are_present -- --ignored --exact --nocapture
REDIS_URL=redis://127.0.0.1:6399 REDIS_8X_REQUIRED=1 \
cargo test -p redis-tower-modules --all-features -- --ignored
The exact short soak commands and the publication-grade long-run requirements
live in the soak harness guide. Reproduce a
nightly version leg by using the image, REDIS_URL, REDIS_8X_REQUIRED=1, and
the expected REDIS_8X_VERSION shown in
nightly.yml; that workflow is the
canonical matrix definition.
Known limits
- The old custom-vs-upstream circuit-breaker A/B from issue #416 is not runnable because the custom implementation was retired. The maintained adapter contract above is the replacement evidence.
- There is no automated true network partition, Docker pause, or Toxiproxy latency/jitter leg today. Current fault tests cover refused connects, connection kills, process SIGKILL/restart, role changes, resharding, and command stalls. Deterministic connected-peer response-stall tests verify explicit timeout, quarantine and no replay, but do not simulate packet-level partitions or node-to-node quorum loss.
- Valkey is exercised only through the standalone nightly suite. Cluster, Sentinel, module, and fault-injection parity are not claimed for Valkey.
- The nightly version matrix is standalone-only; per-PR cluster and Sentinel coverage is limited to the two pinned Redis patch versions in CI.
- Sentinel uses its default connection protocol path but does not expose the explicit protocol policy available to standalone and cluster builders.
- The per-PR module gate is path-filtered and uses one Redis 8 image. The Redis Stack 7.x feature subset and broader image cadence remain nightly.
- The short soak workflow proves harness and recovery behavior, not long-run performance, memory stability, or a publication result. Those claims require the sealed publication protocol and its retained artifacts.
- Live Redis integration, module, and transport handshakes currently run on Linux. Publishable all-feature library tests also run on macOS arm64, Windows x64, and Linux arm64. Managed services still need separate validation; protocol compatibility alone is not provider certification.
Differential testing against redis-rs
The differential corpus uses redis-rs as an independent implementation oracle, not as the Redis specification. Both clients run commands against the same server version but use separate key namespaces for every mutation. When the clients disagree, Redis documentation and server behavior decide the expected contract.
The test-only dependency is pinned to redis = 1.7.0; the per-PR job records
the resolved oracle package with cargo tree alongside the Redis server
version. It is not a production dependency.
Standalone command and conversion failures report the case, command or step,
adapter side, selected RESP protocol, live Redis version, both client versions,
and deterministic seed (0x52454449535f4d43). Connection failures use
server=unavailable because no server version can be queried safely.
A synthetic user/password regression drives both adapters through connection
failure and verifies that neither the target nor raw error text reaches the
captured panic message.
Initial MCP-derived case ledger
| Behavior | Input and isolated setup | Expected semantics | Protocol/server requirement | redis-tower adapter | redis-rs adapter |
|---|---|---|---|---|---|
| Scalar, null, empty, and binary | GET missing; binary-safe SET/GET/ECHO; MGET over present, missing, and empty values in per-client namespaces | Null stays distinct from empty bytes; arbitrary bytes round-trip without UTF-8 loss | RESP2 and RESP3; Redis 7.4.3 and 8.0.6 per PR | RedisConnection + RawCommand | async MultiplexedConnection + Cmd |
| Remaining typed binary families | typed RENAME, EVAL, ECHO, PUBLISH, GEOADD, PFADD, and BITOP with invalid UTF-8, NUL, CR/LF, and protocol-looking bytes | Typed serialization and server-visible results agree without decoding opaque arguments | RESP2 and RESP3 | typed command builders | Cmd with byte-slice arguments |
| Numeric boundaries and decoding | Store u64::MAX, -1, and invalid UTF-8 as bulk values | Both decode the maximum as u64; both reject negative-to-u64 and invalid UTF-8-to-String conversions | RESP2 and RESP3 | TypedRawCommand<u64/String> | FromRedisValue<u64/String> |
| Typed builder options | Exercise Set::nx().get() against an existing key and Set::xx().get() against a missing key; compare the returned old value and a follow-up GET | Builder options reach the wire: NX preserves the existing value and XX does not create a missing key | RESP2 and RESP3 | typed Set builder | Cmd with explicit NX/XX/GET arguments |
| Hashes and unordered sets | Invalid-UTF-8 keys, binary hash value plus multiple fields; set members inserted in a different lexical order | HGETALL compares as sorted pairs; SMEMBERS compares as a sorted collection | RESP2 flat pairs / arrays and RESP3 maps / sets | typed HSet / SAdd, raw response normalized after decode | Cmd request and raw response |
| Lists and sorted sets | Invalid-UTF-8 keys, ordered list including binary bytes; ZRANGE WITHSCORES | List order is preserved; zset member/score pairs survive RESP2/RESP3 shape differences | RESP2 and RESP3 | typed RPush / ZAdd, raw response | Cmd request and raw response |
| Streams and nested replies | Invalid-UTF-8 stream key, deterministic typed XADD 1-0, then XRANGE | Entry IDs, field order, binary values, and nested structure agree | Redis 5+; RESP2 and RESP3 | typed XAdd, raw response | Cmd request and raw response |
| Server and conversion errors | GET against a list; INCR at i64::MAX; typed range/UTF-8 failures | Server error codes agree (WRONGTYPE, ERR); client-side conversion failures remain failures without pretending their library-specific categories are identical | RESP2 and RESP3 | RedisError classification | redis::RedisError classification |
| Raw and administrative shapes | Lua returns null, empty aggregate, status, integer, and binary values; COMMAND INFO GET returns administrative nesting | Ordered nested values agree without blanket flattening or lossy conversion | RESP2 and RESP3 | RawCommand | Cmd |
| Pipeline outcome and alignment | SET, failing HSET, then GET, followed by PING | Per-command failure is retained, later reply stays paired with GET, and the next command remains aligned | RESP2 and RESP3 | Pipeline | Pipeline::ignore_errors |
| Transaction commit and abort | Atomic SET/INCR/GET; separate clients mutate a watched key before EXEC | Commit results agree; WATCH conflict returns an aborted transaction and does not apply its body | RESP2 and RESP3 | Transaction / exclusive connection | atomic Pipeline / exclusive connection |
| Blocking command ownership | Seed independent lists, then BLMOVE binary data with a finite timeout | Results agree while each blocking operation owns a dedicated, unshared session | Redis 6.2+; RESP2 and RESP3 | dedicated RedisConnection | unshared async connection |
| Public topology entry points | Binary SET/GET through live Cluster and Sentinel fixtures, with an independent redis-rs Cluster connection or direct connection to the Sentinel-discovered master | Routing/discovery does not change the binary result; mutations remain in independent namespaces | Per-PR Redis 7.4.3 and 8.0.6 Cluster/Sentinel fixture legs | ClusterConnection / SentinelConnection | async Cluster connection / direct async master connection |
| RedisJSON module replies | Independent JSON documents; compare JSON.GET and unordered JSON.OBJKEYS | JSON bytes agree and object-key order is the only normalization | Path-filtered per-PR and nightly module-enabled Redis 8; RESP2 and RESP3 | RawCommand | Cmd |
| Queued cancellation | Cancel single and multi requests before the configured batch window closes | Cancelled queue entries never reach the socket | Deterministic in-memory/TCP fixture; no Redis process | AutoPipelineService | Not applicable: redis-tower lifecycle contract |
| Lost non-idempotent reply | Fake server records an INCR-like request, applies it once, then closes before replying; replacement socket receives a probe | Caller gets an error, execution remains unknown, and reconnect never silently replays the write | Deterministic TCP fixture; no Redis process | factory-backed AutoPipelineService | Not applicable: redis-tower lifecycle contract |
| Connected peer withholds replies | Fake peer applies one INCR-like write while keeping TCP open; single request gets no reply, multi request gets only its first reply | Explicit response timeout surfaces CommandTimeout; old socket closes, replacement probe stays aligned, and the write is not replayed | Bounded deterministic TCP fixture; no Redis process or packet-level partition | factory-backed AutoPipelineService | Not applicable: redis-tower lifecycle contract |
The executable standalone cases live in
differential_redis_rs.rs.
The RedisJSON case lives in
differential_module_replies.rs
and is selected by the per-PR module gate and module-enabled nightly job.
Normalization policy
Normalization is deliberately narrow:
- Sets are sorted because Redis does not promise member order.
- RESP2 flat pair arrays, RESP3 pair arrays, and RESP3 maps become sorted pairs only for commands whose documented result is a map or pair collection.
- Status strings and bulk strings preserve their bytes;
OKis not converted through UTF-8 to compare successfully. - Null and empty values, ordered arrays, integer widths, binary bytes, and server error codes are never collapsed together.
- Client-side conversion error taxonomies may differ; the shared contract is rejection, while server-originated error codes must agree.
The negative controls prove that the comparator rejects lossy binary
conversion, reversed sorted-set rank order, and a removed NX option from the
typed Set path. A replay-count control and the live lost-reply fixture fail
if a non-idempotent request executes twice.
Cancellation and unknown execution
There are three materially different states:
- A request cancelled while still queued is unsent and has no server effect.
- A request whose complete reply was received has a known result.
- A request written before its connection loses the reply has unknown execution. Reconnection cannot determine whether Redis applied it.
redis-tower removes cancelled queue entries before flushing and quarantines a connection whose reply alignment is no longer knowable. It does not replay the lost non-idempotent request on the replacement socket. Applications that retry an unknown-execution write must supply their own idempotency mechanism.
An explicit AutoPipelineConfig::response_timeout also bounds a connected peer
that stops replying, including after only part of a pipeline’s responses arrive.
The deterministic TCP regressions keep that socket open until the timeout is
observed, then verify quarantine and useful replacement work without replay.
This is a client response-stall fixture, not a real network-partition test; the
default has no response deadline, so applications must configure one.
Reproduction
cargo test -p redis-tower --test differential_redis_rs \
--all-features -- --test-threads=1
# Requires REDIS_URL pointing at a RedisJSON-capable server.
cargo test -p redis-tower-modules --test differential_module_replies \
--all-features -- --ignored --test-threads=1
Engineering hygiene
The Engineering Hygiene workflow keeps expensive diligence visible without
putting every slow measurement on the ordinary Linux CI critical path.
Pull-request gates
Every pull request runs three classes of checks:
cargo-semver-checkscompares all publishable crate APIs with the pull request target revision under a patch-compatible policy. Intentional API breaks must therefore be explicit rather than arriving as collateral edits.- Publishable library tests run with every feature on macOS arm64, Windows x64, and Linux arm64. The matrix excludes Unix-only benchmark packages, and the reusable cluster fixture is compiled only on Unix because its process lifecycle depends on POSIX signals. The all-feature builds exercise native TLS on each host; Unix-socket tests run on the Unix hosts.
check_release_hygiene.pydiscovers every publishableredis-tower*crate from the workspace manifest. It requires#![deny(missing_docs)], complete docs.rs metadata, standard package metadata, and expected package contents.
Run the release audit locally with:
python3 scripts/check_release_hygiene.py --check-package-contents
Mutation score
The scheduled and manual workflow runs cargo-mutants against the library
tests of every publishable crate. Ordinary pull requests do not wait for this
suite. Each package retains its complete mutants.out directory for 90 days,
and the summary job emits per-package and combined JSON/Markdown reports.
The conservative headline score is:
caught / (caught + missed + timeout)
Unviable mutants are reported but excluded from the denominator. Timeouts are included because they need investigation rather than being silently treated as caught. The initial 70% floor is a tracked target, not a merge gate; establish a stable baseline before deciding whether to enforce it.
For a focused local run:
cargo mutants --package redis-tower-core --jobs 2
python3 scripts/mutation_score.py mutants.out --minimum-score 0.70
The scheduled workflow derives its package list from cargo metadata. It uses
deterministic round-robin shards for suites that exceed one runner’s budget or
need smaller retry units, then requires every shard before producing each
package artifact and all 13 package reports before the workspace headline score.
Mutation failure and recovery
Treat three outcomes differently:
| Outcome | Meaning | Recovery |
|---|---|---|
| A mutant is missed or times out | Valid test-quality evidence for that package | Inspect the retained shard outcome and add a focused assertion when the mutation is behaviorally meaningful. Do not relabel it as infrastructure failure. |
| A shard process or runner fails | The package result is incomplete | Keep the uploaded successful shard artifacts for diagnosis, then rerun the workflow on the same source SHA. Do not combine shards from different workflow runs. |
| A package or headline aggregation rejects missing reports | The fail-closed completeness guard worked | Find the absent shard/package in the job graph, inspect its logs, and rerun the workflow. Never lower --expected-reports to manufacture a score. |
Use the workflow’s Run workflow action for a clean recovery run. Record the source SHA and the replacement run URL in any issue or release evidence. The package and headline jobs intentionally aggregate only artifacts from their own run; partial results remain downloadable for 90 days but cannot silently enter a later score. The 70% target remains informational: a complete low score, an incomplete campaign, and a failing product test are three distinct states.
Protocol fuzzing
The RESP decoder has a separate weekly and manually dispatchable fuzz workflow. It runs both the arbitrary-input decoder and the whole-versus-fragmented semantic oracle for a bounded, configurable duration. Every campaign starts from the reviewed named seeds and retains the final corpus, crashes, source and dependency provenance, log, duration, and outcome for 90 days. See the protocol validation contract for the disposition table, input envelope, local commands, and historical mutation classification.
CI wall clock and flake signal
The scheduled report reads the latest 50 completed, non-cancelled runs of the
main CI workflow. It records p50, p95, and maximum wall-clock duration plus:
- a rerun/flake signal: runs whose GitHub
run_attemptis greater than one; - a failure signal: completed runs whose conclusion is not success.
The enforced operating budgets are a 15-minute p95 and a 10% rerun rate. Failure rate is tracked but not budgeted because a failing pull-request commit usually represents a real code/configuration error, not a flaky test. Cancelled runs are excluded because superseded jobs and GitHub queue incidents otherwise distort elapsed time.
The reporter accepts captured GitHub API JSON for local reproduction:
python3 scripts/ci_health.py --input workflow-runs.json \
--max-p95-minutes 15 --max-rerun-rate 0.10
Resource and build footprint
The existing Weekly Benchmarks workflow remains the source of comparative
resource evidence. Its isolated probes compare redis-tower, redis-rs, and Fred
for RSS per connection and CPU at a fixed offered rate, then compare cold-build
time and stripped binary size. JSON results and the exact Cargo.lock are
retained for 90 days. Hosted-runner output is trend/smoke evidence; release
claims still require an otherwise-idle dedicated host as described in the
benchmark publication protocol.
Release process
redis-tower uses the manual Release GitHub Actions workflow. Preparing a
release pull request and publishing immutable crates are separate maintainer
actions. Merging ordinary code never publishes a package.
Current published state
The complete 13-crate public workspace is available on crates.io. These versions were verified against the workspace manifests, crates.io search, and GitHub releases on 2026-10-22:
| Package group | Published version |
|---|---|
redis-tower-protocol, redis-tower, redis-tower-cluster, redis-tower-sentinel | 0.1.4 |
redis-tower-core, redis-tower-commands, redis-tower-sync, redis-tower-modules, redis-tower-client, redis-tower-primitives | 0.1.3 |
redis-tower-auth-aws, redis-tower-auth-azure, redis-tower-test | 0.1.2 |
This is a dated status snapshot, not a version source of truth. Before a release, compare the manifests, crates.io, tags, GitHub releases, and each crate’s changelog.
Prepare
-
Confirm that no open pull request already covers the release and that
mainis green. -
Run the local gates:
cargo fmt --all -- --check cargo clippy --workspace --all-targets --all-features -- -D warnings cargo test --lib --all-features cargo test --workspace --doc --all-features RUSTDOCFLAGS="-D warnings" cargo doc --workspace --all-features --no-deps mdbook build mdbook test python3 scripts/test_check_docs_links.py -v python3 scripts/check_docs_links.py python3 scripts/check_release_hygiene.py --check-package-contents -
Ask release-plz to prepare version and changelog changes:
gh workflow run Release --ref main -f command=release-pr -
Review the generated pull request. Check that every changed public package has the intended semver bump and changelog entry, workspace dependency requirements remain resolvable, and no unrelated package was bumped.
-
Merge only after the release pull request and required checks are complete.
A downstream cargo publish --dry-run cannot resolve a new workspace
dependency until that exact dependency version exists on crates.io. Dry-run the
first dependency tier (redis-tower-protocol) and rely on the full workspace
gates for downstream code before publication. The real workflow waits for each
published dependency to become visible before continuing.
Publish
Publish only from a clean, fully validated main:
gh workflow run Release --ref main -f command=release
The workspace dependency graph requires this order:
redis-tower-protocolredis-tower-coreredis-tower-commandsandredis-tower-testredis-tower- Cluster, Sentinel, modules, primitives, sync, and cloud-auth crates
redis-tower-client
GitHub serializes release workflow runs and skips publication in forks. The
repository must retain a CARGO_REGISTRY_TOKEN Actions secret; the first
publication of a crate cannot be bootstrapped with crates.io trusted publishing.
Do not retry a partially successful workflow blindly. Inspect the workflow, crates.io, tags, and releases first. Published versions are immutable: continue from the first missing tier, and bump only packages whose attempted version can no longer be published.
Verify
For every package reported as published:
- confirm the expected version exists and is not yanked on crates.io;
- confirm the docs.rs build completed with the expected features;
- confirm the GitHub tag and release exist;
- install the facade from crates.io in a fresh temporary project and run a
basic
PINGagainst a supported Redis version; - verify README badges and public documentation links resolve to the release.
Keep publication manual. It changes external immutable state and remains a deliberate action after review and validation.
Recovery history
The initial June 2026 publication attempt left six versions yanked and seven packages unpublished. Yanked versions cannot be reused, and release-plz could not infer a next version when every registry version was yanked.
PR #690 merged on 2026-08-28 with explicit patch bumps, changelogs for all public crates, staged publication order, and the manual operation-specific workflow. The first complete staged relaunch was published on 2026-09-18; follow-up releases on 2026-09-24 established the current version groups above. That incident explains the recovery safeguards but is no longer the active release plan.
See release-plz’s yanked-package guidance if a future release encounters the same registry state.