"Just restart it" is a 3 a.m. plan, not an availability strategy
A Redis primary dies at 3 a.m. The on-call human wakes to a page, SSHes in, confirms the box is gone, promotes a replica by hand, then chases down every service still pointed at the dead IP. On a good night that is fifteen minutes of downtime, and it depends on someone being awake. Redis Sentinel does the same job in a couple of seconds without paging anyone. The catch is that "automatic failover" is doing a lot of quiet work under that phrase, and misconfigure it and you get something worse than manual recovery: two primaries taking writes at once.
Here is what Sentinel actually monitors, how it decides a primary is dead, how it picks the replica that takes over, and the configuration mistake that quietly turns a safety system into a split-brain generator.
What Sentinel watches, and "I think it's down" vs "it's down"
A Sentinel is a separate process — the same redis-server binary started in sentinel mode, not a data node. Each Sentinel PINGs every monitored primary and replica once per second. If a primary stops answering for longer than down-after-milliseconds (5000 ms is common), that one Sentinel marks it SDOWN: subjectively down. Subjective is the operative word, because the broken thing might be that Sentinel's network path to the primary, not the primary itself.
SDOWN on its own triggers nothing. The Sentinel polls the others with SENTINEL is-master-down-by-addr to ask whether they see it too. Once at least quorum Sentinels independently report SDOWN, the primary flips to ODOWN — objectively down — and only then can a failover begin. The quorum is a vote on reality, built so that one bad link or one paused Sentinel can't yank a healthy primary offline.
Here is the first thing people get wrong. The quorum number governs detection only. It does not authorize the failover itself.
Why do you need three Sentinels in three failure domains?
Detecting the death is the easy half. Carrying out the failover requires one Sentinel to be elected leader with exclusive authority — because if two Sentinels both promoted replicas, you'd manufacture the split-brain you were trying to prevent. Election uses a variation of the Raft consensus algorithm. A Sentinel that sees ODOWN requests votes; each Sentinel grants its vote to the first asker in the current term; the winner needs both a majority of all configured Sentinels and at least quorum votes. Majority is (N / 2) + 1 of every Sentinel — not of the quorum.
With three Sentinels, majority is 2. Run only two and majority is still 2, so after a single failure the lone survivor can never reach a majority and failover never fires. That is why three is the floor, and why an odd count is the right call: it gives a clean majority on either side of a split with no ties.
Where the Sentinels live matters more than how many you run. Put all three on one rack, one hypervisor, or one availability zone and you have bought nothing — the failure that takes out your primary takes out its voters too, the surviving partition can't form a majority, and nothing promotes. Three Sentinels across three independent failure domains is the actual requirement. In Beebeeb's high-availability setup in Falkenstein, Germany, the Sentinels sit on separate nodes alongside a Patroni-managed Postgres quorum for exactly this reason: losing a node should cost you one vote, not the election.
Sentinel enforces the consequence honestly. A Sentinel stranded on the minority side of a split — unable to reach a majority of its peers — will not start a failover. It would rather sit read-only-degraded than promote a second primary. That restraint is the whole point.
How the leader picks the replacement
Once a leader is elected, it picks the replica to promote, and the ranking is deterministic. It drops any replica disconnected from the primary for too long. Among the rest it prefers the lowest replica-priority (set it to 0 to make a replica permanently ineligible — handy for a backup node you never want serving writes). Ties there go to the highest replication offset, because that replica holds the most recent data; a final tiebreak uses the lexicographically smallest run ID. The leader runs REPLICAOF NO ONE on the winner, repoints the other replicas at it, and the new primary is live.
What keeps a stale Sentinel from wrecking this is the epoch — Sentinel's term counter, a monotonic integer bumped on every failover attempt. A Sentinel that was partitioned at epoch 5 and rejoins to find the cluster already at epoch 7 knows its view is obsolete and stands down instead of launching a competing failover. Epochs are how Sentinel holds to one failover at a time even as Sentinels drop out and come back.
The mistake that breaks everything: hardcoding the master IP
You can wire up three perfectly-placed Sentinels and still eat an outage if your application connects to Redis by IP. Sentinel just promoted a replica at a new address; your client is still hammering the dead one. The failover worked and your app is down anyway.
A Sentinel-aware client never stores the primary's address. It stores the Sentinel addresses and asks them. On startup it walks its list of Sentinels, calls SENTINEL get-master-addr-by-name with a short per-Sentinel timeout, and connects to whatever comes back. When a failover lands, Sentinels broadcast a +switch-master message on a pub/sub channel; a subscribed client drops the old connection and reconnects to the new primary on its own. redis-py, ioredis, Jedis, Lettuce, the Rust crate — every major library does this natively, but only if you configure them in Sentinel mode and hand them the Sentinel list instead of a host. Point a plain client at the old primary's IP and you have built high availability that fails over to nothing.
Where Sentinel ends, and the honest limit
Sentinel is not Redis Cluster. It gives you one primary with replicas and automatic promotion; it does not shard your keyspace across nodes, so a dataset that outgrows a single primary's memory is a different tool's problem. Failover is also never instantaneous. Add up down-after-milliseconds, ODOWN agreement, the election, and client reconnection and you are looking at seconds of write unavailability, not zero. The honest claim is "recovers in seconds without a human," not "no interruption." For session state, rate limiters, and the cache tier behind an API, seconds beats fifteen minutes and a paged engineer every time.
The principle underneath it is the one the whole product runs on: don't trust a single point, and don't trust one observer's read of reality. Quorum to detect, majority to act, an epoch to settle the argument — the same redundancy thinking sits under Beebeeb's storage and database tier. If you want to see how that posture carries up from the cache layer into file versioning, encrypted sharing, and the rest of the stack, that's where it leads. Sentinel just happens to be the layer where getting the quorum math wrong shows up fastest.