Architecture

Redis Sentinel: How to Set Up High Availability That Actually Fails Over

Redis Sentinel gives a non-clustered Redis automatic failover, but only if quorum, placement, clients and replication are set up with failure in mind. Here is a working configuration, what quorum really means, how writes get lost, and what changes with large datasets.

· 13 min read · AG Data Team

Redis Sentinel is the built-in high-availability system for Redis that is not running in Cluster mode. A group of Sentinel processes watches the master and its replicas, agrees when the master is down, promotes a replica and tells clients where the new master is. It works well, but most outages we see with it come from setups that were never tested with a real failure.

What Sentinel does, and what it does not

According to the Sentinel documentation, Sentinel does four jobs: monitoring, notifications, automatic failover and acting as a configuration provider, so clients ask Sentinel for the current master address instead of hard-coding it. It does not shard data or scale writes; for that you need Redis Cluster. And it does not make replication synchronous, which matters a lot further down.

A minimal configuration that is actually robust

# sentinel.conf, identical on three independent hosts
port 26379
sentinel monitor mymaster 10.0.1.10 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000
sentinel parallel-syncs mymaster 1
sentinel auth-pass mymaster <redis-password>

# start it (either form works)
redis-sentinel /etc/redis/sentinel.conf
redis-server /etc/redis/sentinel.conf --sentinel

Quorum is not majority

This is the most misunderstood part of Sentinel. The last argument of sentinel monitor is the quorum: the number of Sentinels that must agree the master is unreachable. It is only used to detect the failure.

The consequence: Sentinel never fails over in a partition that holds only a minority of Sentinels. Each authorized failover gets a unique configuration epoch, so the newest configuration wins when partitions heal. SENTINEL CKQUORUM checks that the current Sentinels can reach both quorum and majority; run it after every change.

redis-cli -p 26379 SENTINEL get-master-addr-by-name mymaster
redis-cli -p 26379 SENTINEL ckquorum mymaster
redis-cli -p 26379 SENTINEL replicas mymaster
redis-cli -p 26379 SENTINEL sentinels mymaster
redis-cli -p 26379 INFO sentinel     # sentinel_tilt:0 is what you want

How Sentinel loses acknowledged writes

The documentation is honest about this: because Redis replicates asynchronously, Sentinel cannot guarantee that acknowledged writes survive a failure. The classic case is a network partition that isolates the old master together with some clients. The majority side promotes a replica; the clients stuck with the old master keep writing to it; when the partition heals, the old master becomes a replica of the new one and those writes are discarded.

You cannot remove this window, but you can bound it:

# redis.conf on the master (and on replicas, which may become master)
min-replicas-to-write 1
min-replicas-max-lag 10

With these settings a master stops accepting writes if no replica has acknowledged within 10 seconds. The trade-off is explicit: if all replicas are down, your master refuses writes too. For a cache that is often the wrong choice; for data you care about it is usually the right one. Combined with sensible persistence (see our comparison of RDB and AOF), it turns "some writes lost" into "at most a few seconds of writes lost".

Clients must speak Sentinel. The docs state that Sentinel requires explicit client support, unless you redirect traffic another way, for example with a virtual IP. A client pinned to a fixed host keeps writing to the old master after failover and, once it is demoted, gets errors like the one in our READONLY replica error guide. Check that your library uses the Sentinel API and re-resolves the master on failover (Sentinel client guidelines).

Docker, NAT and hostnames

Operational details that bite later

Sentinel with large datasets

A failover drill on a 200 MB test instance proves very little about a 60 GB production dataset. What changes:

Key takeaways

Redis Valkey Sentinel High availability
Work with us

Redis failover that did not fail over?

We design, test and fix Redis and Valkey Sentinel setups, including failover drills on production-sized datasets and client configuration reviews.

Get in touch More from the blog