What happens when the database node dies mid-write
A single PostgreSQL server is a single point of failure. Reboot it for a kernel patch, lose a disk, drop it off the network for ninety seconds, and every write in flight fails while every connected client hangs. Plain Postgres has no answer for this. It does streaming replication well, but it will not promote a standby for you, it will not tell your application where the new primary moved, and it will happily let two nodes both believe they are primary if you wire it up wrong. The pattern that closes that gap is Patroni plus a consensus store, and it is what runs underneath beebeeb's API.
The three pieces
A Patroni cluster is three layers, each with exactly one job. PostgreSQL on each node does the actual work — one primary accepting writes, one or more standbys replaying its write-ahead log. Patroni, a small Python daemon, runs next to each Postgres instance; it stores none of your data, it just starts, stops, promotes, and reconfigures the Postgres process beneath it according to cluster state. And a distributed configuration store (DCS) — etcd, Consul, or ZooKeeper — holds the single source of truth about who the leader is.
The DCS is the part people skip when they first reach for replication, and it is the part that makes the whole thing safe. We use etcd. etcd agrees on cluster state using the Raft consensus algorithm, and Raft needs a majority to decide anything, so a healthy etcd cluster runs an odd number of nodes — three at minimum — and keeps working as long as two of them can still talk to each other.
Leadership is a lease, not a vote you take once
Here is the mechanism, because the mechanism is the whole point. The primary holds a leader key in etcd with a short time-to-live, on the order of seconds. Patroni on the primary renews that key on every heartbeat. As long as it keeps renewing, it stays leader and Postgres keeps accepting writes.
So leadership is not a one-time election. It is a continuously-proven lease. If the primary crashes, freezes, or gets cut off from etcd, it stops renewing the key, the TTL expires, etcd deletes it. Now the key is up for grabs and the standbys race to claim it. Because etcd serializes every write through Raft, exactly one standby can win — the math does not permit two winners. That node gets promoted. The old primary, if it ever comes back, sees it no longer holds the lease and demotes itself to a standby.
That self-demotion is what prevents split-brain, the failure mode where two nodes both accept writes and your data quietly forks. A Patroni node that loses its connection to etcd does not assume it is still in charge — it assumes it has lost the lease and steps down, even if it can see nothing else wrong. In fact Patroni goes further: a primary that cannot renew its leader key stops PostgreSQL outright rather than risk a second node coming up behind it. Availability is sacrificed to consistency, on purpose. We would rather refuse a write for a few seconds than accept two conflicting ones.
Replication: how much loss can you tolerate?
Underneath leader election, the standbys stay current through PostgreSQL streaming replication, and the mode you pick is a real decision with a real cost.
By default Patroni configures asynchronous replication. The primary commits a transaction and tells the client "done" without waiting for any standby to confirm it received the change. That is fast — no extra round-trip on the write path — but if the primary dies before a committed transaction has reached a standby, that transaction is gone. Patroni bounds the blast radius with maximum_lag_on_failover: a standby that has fallen too far behind is not eligible for promotion, so you never fail over to a node missing more than a configured slice of recent history.
The stricter choice is synchronous replication. With synchronous mode on, the primary will not acknowledge a commit to the client until a standby confirms it durably has the change. The reward: Patroni will never promote a standby that is missing an acknowledged commit, which is zero data loss on failover. The cost is worth stating plainly — every write now pays for a network round-trip to the standby, so latency rises and write throughput falls. No setting gives you zero-loss failover and zero-latency writes both; you choose where on that line you sit. For a zero-knowledge backend where the database holds account state and file metadata, never the encrypted file bytes themselves, we run a synchronous standby so a node failure cannot silently drop an acknowledged change.
How clients find the new primary
Promotion is only half the problem. Your application was holding connections to a machine that is now a standby, or dead. It has to land on the new primary without anyone editing a config file at 3am.
Patroni exposes a small REST API on each node. The /primary endpoint returns HTTP 200 only on the node that currently holds the leader lock; every standby returns 503. We put HAProxy in front and point its health check at that endpoint. HAProxy polls every node continuously, and the only one answering 200 on /primary gets the write traffic. The instant the leader key moves, the old primary starts failing its check and the new one starts passing it. HAProxy reroutes, and the application reconnects to a working address it never had to know in advance.
The end-to-end sequence on a primary failure runs roughly like this: heartbeat stops, lease expires after a few seconds, a standby wins the key and is promoted, its REST endpoint flips to 200, HAProxy redirects. In practice automatic failover lands in the tens of seconds, not instant — the cluster is built to be sure a node is really gone before promoting, not to react to a single dropped packet.
What this does and does not buy you
It is easy to oversell HA, so here is the honest scope. Patroni gives you automatic failover: a node can die and the cluster recovers with no human in the loop. It does not give you a faster database. It does not remove the need for backups. And it does nothing about a bad migration that corrupts data on the primary — that corruption replicates to your standbys cleanly. HA and backups are different problems and you need both. A consensus store also adds operational surface of its own: etcd has to be healthy and quorate, and if you lose etcd quorum the cluster freezes writes rather than risk split-brain. That is the safe behaviour, but it is a new moving part that can break, and you have to watch it. We pair the live cluster with point-in-time backups taken off a standby on a fixed schedule, because failover protects against a node dying, not against the data itself being wrong.
Why a storage company writes about its database at all
Because the parts of an end-to-end encrypted product you can't see are the parts most likely to fail you. The encryption is the headline. The backend that survives a disk dying at 2am without dropping one acknowledged write is what decides whether your account, your shares, and your file metadata are still there next week. We describe the cryptography in detail on our security page, and where the platform is going — including the native desktop and mobile clients still in progress — on the public roadmap. The database is one layer of that, run the boring, durable way on purpose.
None of this changes the trust model: zero-knowledge means we cannot read your files on any tier, including the free one.