Building zero-knowledge cloud storage in Europe: the crypto, the mistakes, and what isn't done yet

This post is the full story of how Beebeeb works. It covers:

• how Beebeeb encrypts your files,
• how login works without the server ever seeing your password,
• what the 12-word recovery phrase actually is,
• the places where we got it wrong,
• and a list of the things that aren't finished.

Every claim about the crypto links to code you can read, because the clients are open source. The server isn't, and I'll explain why below.

It's a long read. Use the table of contents to jump around. Every major section starts with an "In short" line, so you can skim first and dig in where you care.

---

1. Why I built this

In short: I needed an encrypted offsite backup for a very large home NAS, with a recovery path that still works when every device I own is gone.

I have a self-built NAS at home. It runs Proxmox, with TrueNAS in a VM and an LSI 9207-8e passed through to a 60-bay expansion chassis. Not every bay is filled, but it's around 500 TB.

For years my offsite backup was a Google Drive plan. It only had room for part of the data. So I picked what I couldn't afford to lose and shipped that. Everything else had no offsite copy.

The other problem was trust. I didn't want Google able to read my family photos and tax papers. So I tried encrypting before upload myself.

That works until you picture the bad day. The house floods, the NAS is gone, and now you need the key. Where is it? On the NAS? In a password manager whose recovery depends on a device you no longer have?

Doing E2EE (end-to-end encryption: only you hold the keys) yourself moves the hard problem into key management. And key management is exactly what fails when everything else fails too.

Then there's a newer worry: harvest now, decrypt later. Someone stores encrypted data today and waits until a quantum computer can break it. The EU itself says these "store now, decrypt later" attacks are likely already happening. Family photos and tax papers need to stay private for decades, so that matters. While building Beebeeb I fell down that rabbit hole more than once. What I learned is spread through this post.

So I wanted one thing that combined the parts:

• client-side encryption that I didn't have to wire together myself,
• a recovery story that still works when every device of mine is gone,
• enough capacity for the whole NAS, not a curated slice (Beebeeb goes up to 99 TB self-service; past that we quote),
• in Europe, run by a European company.

My brother Bram and I started building it.

2. How it was built: two people and a lot of AI agents

In short: two people built Beebeeb with AI coding agents. A human checked every function and method, all crypto lives in one shared Rust library, and we test for the false claims agents like to make.

I'll be upfront because people will ask. Beebeeb is built with AI coding agents, mostly Claude, often several running in parallel on separate tasks.

Some dates and numbers:

• The first public commit in all three main repos (core, web, cli) is from 24 April 2026.
• The private server and workspace repos also start on 24 April 2026.
• We built for about five months before the public beta launched on 1 October 2026.
• Since then web has about 1,250 commits, cli about 420 and core about 170.

How we work with the agents:

• A human checked every function and method. The agents' Co-Authored-By trailers stay in the git history. We don't squash them out. Run git log and you'll see which commits were agent-assisted.
• One crypto implementation. The rule that saved us most often: no client re-implements crypto. Everything goes through one Rust crate (a crate is a Rust library), beebeeb-core. The web app uses it as WASM (WebAssembly, compiled code that runs in the browser). Mobile uses it via UniFFI (a tool that generates Swift and Kotlin bindings for Rust). The CLI, desktop and server use it as a native dependency. Agents are good at writing a second, slightly different version of something. A single core takes that option away.
• Agents make confident false claims, so we test for them. A real example: the initial import of core included UI mockups whose copy said the code had been externally audited. It named a security firm and a date. None of that was true. We removed it and wrote it up in the CHANGELOG. Then we added a CI "claims guard" job. It fails the build if a false audit or compliance claim appears anywhere in the tree. If you build with agents, assume they'll write marketing you never approved. Test for it like you'd test for a regression.

The toolbox:

• Claude, the main driver.
• Codex, hooked in through the Codex plugin for Claude. Claude could hand off work or get a second opinion without me switching tools.
• Local models running on an RTX 5090 at home: Qwen3.8-27B and whatever else fit on the card.

Here's the honest part: the crypto code was written by agents and reviewed by agents. Claude and Codex checked each other's work, like peer review.

When I ran out of tokens on one of them, I ran a second, separate Claude agent. Its only job was to review the one doing the building. That happened about halfway through most weeks.

My own role was mostly testing and validating behaviour, not line-by-line cryptographic review. I know I make mistakes as a human. And I don't pretend that agents reviewing agents is the same as a professional audit. That's exactly why the code is open, and why this post ends with an open invitation to review it.

3. Why all clients are open source but the server isn't

In short: the client is where the security claims live, so every client is open. The server is closed, but it's designed so it can't read your files even if it misbehaves.

From day one I wanted every client on GitHub. The reasoning is simple. In a zero-knowledge system, the client is where the security claims live:

• it generates the keys,
• it encrypts and decrypts,
• it decides what goes over the wire.

If the client is honest, a malicious server gets ciphertext. If the client is dishonest, nothing else you're told matters.

So these are AGPL-3.0 at github.com/beebeeb-io:

Repo	What it is
core	Rust crypto core, shared types, sync engine. Compiles to native, WASM, and UniFFI (Swift/Kotlin).
web	React web app; all crypto runs in the WASM build of core inside a Web Worker.
cli	bb, the Rust CLI: push/pull, two-way sync, share links, file requests, local WebDAV.
desktop	Tauri desktop app (OS WebView, Rust backend importing core).
mobile	React Native app, natively linked to core via UniFFI. Not released (see below).
rclone-backend	Experimental rclone backend.

The API server is closed. It:

• stores ciphertext,
• enforces quotas and rate limits,
• runs billing,
• handles OPAQUE and WebAuthn on the server side (both explained below).

Being honest about what that means: you can verify what your client sends, but not what the server does with it. The design goal is that this shouldn't matter for confidentiality.

A server that misbehaves can:

• deny service,
• mess with metadata it already has,
• or (see the chunking section) tamper in ways we haven't fully closed off.

It shouldn't be able to read files. You don't have to take my word for that. Put the open-source client behind a proxy and look at what leaves your machine.

---

4. The key hierarchy in one picture

In short: your 12-word recovery phrase is the root of every key. Your password only logs you in and protects the key on a device.

12-word BIP39 phrase (128 bits of entropy, generated on device)
        │
        │  Argon2id (256 MiB, t=4, p=2) → 32 bytes
        ▼
   Master key  ──────────────────────────────────────────┐
        │                                                │
        │  HKDF-SHA256(info = "beebeeb-file-key-v1" || file_id)
        ▼                                                │
   Per-file key ── AES-256-GCM ──► chunk frames,         │  wrapped locally by:
                                   encrypted filename,   │  • web: PBKDF2(password) → AES-GCM, in IndexedDB
                                   encrypted MIME type,  │  • web + passkey: WebAuthn PRF output
                                   encrypted thumbnails  │  • mobile/desktop/CLI: OS secure storage / config file
                                                         ┘

Password → OPAQUE (with Argon2id as KSF) → authenticates to the server. Never sent.

A few terms from the diagram:

• BIP39 is the standard for turning random bytes into a list of words, best known from crypto wallets.
• Argon2id is a password-hashing function. It is deliberately slow and memory-hungry, so guessing is expensive.
• HKDF (HMAC-based Key Derivation Function) turns one strong key into many separate keys, one per label.
• AES-256-GCM is the cipher that encrypts the data. "GCM" means it also detects tampering.
• PBKDF2 is an older password-stretching function that browsers support natively.
• WebAuthn PRF is a passkey feature that can output a secret value (explained in the passkeys section).
• OPAQUE is the login protocol (explained below).

The most important thing to notice: the password is not the root of your keys. The recovery phrase is.

The password does two jobs:

1. It authenticates you through OPAQUE.
2. In the browser, it wraps (encrypts) the master key at rest on that device.

Changing your password doesn't change your master key. So it doesn't mean re-encrypting anything.

5. Argon2id: where it's used and with what parameters

In short: Argon2id is used in three places, always with 256 MiB of memory, 4 iterations and 2 lanes.

Argon2id shows up in three places in core, all with the same parameters:

// beebeeb-core/src/kdf.rs
const MK_MEMORY_KIB: u32 = 256 * 1024; // 256 MiB
const MK_ITERATIONS: u32 = 4;
const MK_PARALLELISM: u32 = 2;

1. Recovery phrase → master key (recovery.rs). The 16 bytes of BIP39 entropy go in as the Argon2id "password". The salt is fixed and domain-separating:

// NB: trailing "x" is a historical artifact — do not change without migration,
// existing recovery keys depend on this exact salt.
const RECOVERY_SALT: &[u8] = b"beebeeb-recovery-v1x";

Yes, that's a global salt. And yes, the trailing x is a typo we now have to live with.

A global salt is normally a red flag. It lets an attacker precompute, and it makes multi-target attacks cheaper.

It's acceptable here because the input isn't a human-chosen password. It's 128 bits of CSPRNG output (a cryptographically secure random number generator). Precomputing over 2^128 inputs is no more feasible than attacking one input.

The multi-target point is real, though. I'll come back to it in the 12-vs-24 section. A per-user salt is planned for a v2 derivation.

2. OPAQUE key-stretching function (opaque_protocol.rs). Explained below.

3. A legacy password → master key path (kdf::derive_master_key).

• It's still exported for an old unlock flow. Signup doesn't use it.
• We kept it at full strength on purpose. Any prod-reachable export with weak parameters is a downgrade waiting to happen.
• A unit test pins the parameters. That way a "speed up the test suite" refactor can't quietly lower them.

Why these numbers?

• 256 MiB is the most memory we were comfortable requiring on every client. That includes a browser tab running WASM, and a phone.
• Speed: on a modern laptop the derivation takes roughly 0.5–1 s. We haven't benchmarked phones and browser WASM systematically yet. Numbers from your own device are welcome.
• Two lanes instead of four: WASM in the browser is effectively single-threaded for us. Extra lanes add cost for the defender without making the attacker's GPU work any harder.

A WASM-specific trap. WebAssembly linear memory only grows and never shrinks. One 256 MiB Argon2id run permanently inflates the crypto worker's memory.

So the web app checks the worker's memory every 16 operations. It recycles the worker when it passes 256 MiB. But never while a streaming upload holds a live encryptor handle inside that worker.

This is the kind of thing you only learn when someone keeps a tab open for a day.

6. OPAQUE: logging in without sending the password

In short: your password never leaves your device, not even inside TLS. We also got the first version wrong, and about 30 accounts are still on the old setup.

Most "secure" logins send your password to the server inside TLS. Then they trust that it gets hashed and never logged.

We use OPAQUE, an asymmetric PAKE (password-authenticated key exchange: a protocol where both sides prove they know the right secret without sending it). We use it through the opaque-ke crate:

// beebeeb-core/src/opaque_protocol.rs
impl CipherSuite for BeebeebCs {
    type OprfCs = opaque_ke::Ristretto255;
    type KeyExchange = opaque_ke::TripleDh<opaque_ke::Ristretto255, sha2::Sha512>;
    type Ksf = Argon2idKsf; // 256 MiB / 4 iter / 2 lanes
}

Simplified, a login goes like this:

1. The client blinds (masks) the password and runs an OPRF (oblivious pseudorandom function) with the server. The server holds a per-user OPRF key, but never sees the password.
2. The output is stretched with the KSF (key-stretching function, here Argon2id).
3. The stretched result opens an envelope stored on the server.
4. A 3DH (triple Diffie–Hellman) key exchange proves to both sides that the right party is on the other end.

The server stores a "password file". It never contains the password or a plain hash of it.

The mistake. OPAQUE's KSF is pluggable. Our first version used opaque_ke::ksf::Identity, which means no stretching.

The protocol is still secure on the wire with Identity. But imagine someone stole the password-file database and the server's OPRF secret. Then every password guess would cost about one Ristretto255 scalar multiplication (milliseconds). With Argon2id it costs an Argon2id evaluation (around a second at 256 MiB). That's a large difference for anyone grinding through a password list.

We switched the KSF to Argon2id. The code is blunt about the consequence:

/// OPAQUE cipher suite, **v0** — the LEGACY suite, retained only so the ~30
/// production accounts registered before the Argon2id switch can still complete
/// a fresh login (`opaque_ksf_version = 0`).

About 30 accounts had v0 password files. Changing the KSF invalidates existing OPAQUE registrations. So those accounts have two options:

• log in under the v0 suite and get re-registered under v1, or
• recover with their phrase.

If you build on OPAQUE, choose the KSF before your first user, not after.

On top of OPAQUE, the web app:

• enforces a 12-character minimum password,
• runs a k-anonymity breached-password check during signup (core/src/onboarding/breach.rs). A k-anonymity check is a lookup designed so the breach service can't tell which password you checked.

Hashing and matching happen in core. The server says whether to fail open or closed when the breach corpus can't be reached.

7. Passkeys and TOTP

In short: passkeys can both log you in and unlock your vault. TOTP codes protect your account, but they're not what keeps your files secret.

Passkeys (WebAuthn) sign you in with no shared secret to phish or leak. The server side uses webauthn-rs.

The interesting part isn't authentication. It's how a passkey login also unlocks the vault. A passkey proves who you are, but it doesn't give you key material.

Our answer is the WebAuthn PRF extension (pseudorandom function: the passkey can compute a secret value from an input, without revealing its own key). Here's how we use it:

1. We evaluate the passkey's PRF with a fixed app-specific salt (beebeeb.io/vault-wrap-key/v1).
2. We use the output as a wrap key.
3. We store the master key AES-GCM-wrapped under it.

The PRF output only exists inside the authenticator ceremony. So an XSS (cross-site scripting: malicious script injected into the page) can't read it out of storage.

Not every browser/authenticator pair supports PRF yet. For those:

• The fallback stores a random wrap key in localStorage, keyed by credential ID. The code comment says what that is: "Less secure (XSS-accessible) but functional on all browsers."
• If neither path can unlock the vault, you type your password once and the device gets provisioned.

We'd like to retire the localStorage fallback once PRF support is broad enough. That retirement is planned.

TOTP is standard RFC 6238 second-factor auth (the six-digit codes) with any authenticator app. It protects the account: sessions, shares, billing. It adds nothing to file confidentiality, which never depended on the server letting you in.

8. AES-256-GCM, chunked, on the client

In short: every file gets its own key, gets split into chunks, and each chunk is encrypted on your device. Filenames, file types and thumbnails are encrypted too.

Every file gets its own key, derived from the master key with HKDF:

// beebeeb-core/src/kdf.rs
pub fn derive_file_key(master_key: &MasterKey, file_id: &[u8]) -> FileKey {
    let hk = Hkdf::<Sha256>::new(None, master_key.as_bytes());
    let mut info = Vec::with_capacity(19 + file_id.len());
    info.extend_from_slice(b"beebeeb-file-key-v1");
    info.extend_from_slice(file_id);
    // ...expand to 32 bytes
}

The file is then split into chunks. Each chunk becomes one frame:

nonce (12 bytes, random) || AES-256-GCM ciphertext || tag (16 bytes)

A nonce is a "number used once": a random value that makes each encryption unique.

Some details that matter:

• Chunk size scales with the file. It ranges from 4 MiB up to a hard cap of 256 MiB, depending on client profile. It shrinks when you upload many files in parallel, so memory stays bounded. The cap is enforced on the client. Otherwise a compromised server could send chunk_size = 40 GB and OOM (run out of memory) the browser tab before a byte gets encrypted.
• Random 96-bit nonces are safe here because each key encrypts one file. The GCM random-nonce limit (about 2^32 messages per key) is per file. No file gets anywhere near 2^32 chunks.
• Quantum computers don't break this part. The best known quantum attack on a cipher like AES is Grover's algorithm, which roughly halves the effective key length. AES-256 then still has about 128 bits, which is still considered safe. Our file keys come from the recovery phrase via Argon2id and HKDF, not from public-key crypto, so the file encryption already holds up. (The phrase itself is a separate question, see 12 or 24 words.)
• Filenames and MIME types are encrypted with the per-file key, as a small JSON envelope. The server stores a hashed name, size, timestamp and owner.
• Thumbnails are generated on the client as WebP and encrypted with the file's own key.
• Keys are !Clone and zeroized on drop. MasterKey and FileKey are 32-byte structs that can't be copied by accident, and their memory is wiped when they're no longer used.

8.1 A war story about bytes

Here's a piece of decrypt_name I'm not proud of. It's the single function every client uses to decrypt filenames. Its doc comment lists the formats it has to accept:

1. plaintext (legacy entries from before name encryption),
2. the native serde EncryptedBlob (numeric byte arrays),
3. an older web format with base64 strings and no cipher_suite.

Each of those comes in two key variants. For a while:

• the CLI derived file keys from the 16-byte binary UUID,
• while the web app derived them from the 36-byte string UUID.

Same file, same master key, two different file keys. Every file decrypted correctly on the client that uploaded it, and failed on the other one.

We fixed it with a bb repair command and a fallback matrix that tries both.

The lesson is the reason core exists. An identifier crossing a language boundary is a key derivation input. And "the same UUID" can be different bytes.

9. The web app: Rust → WASM

In short: the web app does no crypto in JavaScript. It runs our Rust core as WASM. But like every web-delivered E2EE app, you still have to trust whoever serves it.

The web client is React, but it doesn't do any crypto in JavaScript. beebeeb-core is compiled with wasm-pack (the beebeeb-wasm crate) and runs in a Web Worker (a background thread in the browser). A WasmGuard component won't render any vault UI until the module has loaded.

Things we ran into:

• Build failures. wasm-pack's bundled wasm-opt rejected the bulk-memory and sign-ext ops that current Rust emits by default. So release builds failed at the optimization step. The fix is to enable those features explicitly in Cargo.toml. It's not hard, but it took far too long to figure out.
• An integrity check, with limits. The production build computes a SHA-384 hash of the .wasm. The worker checks it before instantiating. Let's be precise about what that gives you. The manifest comes from the same origin as the app. It catches a corrupted or swapped asset in the delivery path. It doesn't protect you from us. A malicious server can serve a malicious app and a matching manifest. That's the basic limitation of any web-delivered E2EE app, ours included. If that's in your threat model, use the CLI or desktop app built from source.
• A strict CSP (Content Security Policy, the browser rules for what a page may load): default-src 'self', wasm-unsafe-eval for WASM only, and no third-party origins in connect-src apart from our own status and error-reporting hosts.
• Key storage in the browser. The master key is wrapped at rest with PBKDF2-SHA256, 600,000 iterations, random 16-byte salt, via WebCrypto. The unlocked key is cleared after 30 minutes of inactivity by default (60 max). We use PBKDF2 here because it's native to WebCrypto. That way the wrapping key can stay non-extractable inside the browser, instead of passing through JavaScript or WASM memory.

10. The 12-word recovery phrase

In short: the 12 words are your master secret. With them you can rebuild every key on a new device. Without them and your password, nobody can get your files back, including us.

At signup, core generates a BIP39 mnemonic from the OS CSPRNG on your device:

// beebeeb-core/src/recovery.rs
let mnemonic = Mnemonic::generate_in_with(&mut OsRng, bip39::Language::English, RECOVERY_WORD_COUNT)?;
let entropy = Zeroizing::new(mnemonic.to_entropy());   // 16 bytes for 12 words
let mk = derive_key_from_entropy(&entropy)?;            // Argon2id → 32 bytes

The phrase is the master secret. It isn't an escrow of some other random key.

That's what makes the "flood took my NAS" scenario work. If you have the 12 words, you can rebuild every key on a brand-new device with no help from us.

It also means there's no backdoor. You keep access as long as you keep either the password (on a device that's already provisioned) or the phrase:

You lost…	Outcome
Password only	Re-derive master key from phrase, set new password. Nothing lost.
Phrase only	Account access via email reset; vault stays readable on already-unlocked devices. Generate a new phrase immediately.
Both	Files are unrecoverable. We can reset your login so you can stop billing and delete the account. We cannot decrypt anything.

Why BIP39 rather than a random base32 string?

• It has a checksum (4 bits for 12 words) that catches most typos.
• Its 2048-word list is built so the first four letters identify a word.
• People have years of practice writing it on paper for wallets.

We decided that familiarity was worth more than anything we'd invent.

11. When (and whether) to move to 24 words

In short: 12 words (128 bits) is plenty today. 24 words would add margin, but moving existing vaults means re-encrypting every file. We haven't decided yet.

This question came up repeatedly while we were designing it. Here's the reasoning as I actually see it.

The numbers.

• 12 BIP39 words encode 128 bits of entropy plus a 4-bit checksum.
• 24 words encode 256 bits plus 8.
• Our master key is 256 bits and file encryption is AES-256.

But the security of the whole system can't exceed the entropy of its root, which is 128 bits. AES-256 is not the weakest link. The phrase is.

Is 128 bits enough today? Yes, classically, by a very wide margin. Brute-forcing 2^128 is not a realistic attack. And each guess here also costs an Argon2id evaluation at 256 MiB.

An attacker does have an offline test. A breach of our storage gives them ciphertext with predictable structure: the encrypted filename JSON starts with {"name":. So they can check a candidate master key without talking to us. That's why we depend on the 128 bits and the Argon2id cost, and never on secrecy of the ciphertext.

What would justify 256? Three honest arguments, from weakest to strongest:

1. Multi-target attacks. The recovery salt is global. So one attacker computation can be checked against every user's ciphertext at once. With N users, the expected work to crack someone drops by a factor of N. With a million users that's 2^128 → 2^108 Argon2id evaluations. Still absurd, but it's a margin we gave away without needing to. A per-user salt would fix this more cheaply than 24 words would.
2. Quantum. Grover's algorithm (a quantum search algorithm) nominally turns a 2^128 search into about 2^64 quantum iterations. In practice, running Argon2id with 256 MiB of memory inside a quantum circuit is far beyond anything foreseeable. And Grover doesn't parallelize well. I don't think this is an urgent threat to the phrase. But "data you upload in 2026 should still be secret in 2050" is precisely the promise we make. The standard answer for long-lived secrets is 256-bit roots.
3. Defense in depth for high-value users. For a company keeping 50 TB of client records, typing 12 more words once is not much to ask.

A quick word on the quantum and AI headlines, because I get asked about both. What quantum computers really threaten is public-key crypto like RSA and elliptic curves, through Shor's algorithm. In May 2025 Craig Gidney of Google Quantum AI estimated that a 2048-bit RSA key could be factored in under a week with fewer than a million noisy qubits. The biggest machines today have a few thousand physical qubits (the largest array so far holds about 6,100). Those are estimates, not capabilities.

And AI? On 6 October 2026 OpenAI posted more than 700 AI-generated maths preprints, and mathematicians are still arguing about them. Impressive, but I know of no credible published result where AI derives a private key from a public key. None of this changes the trade-off above. It's still about the 128-bit root.

What 24 words cost:

• Usability. Twice as many words to write down, and twice as many chances to mis-transcribe. Most real-world data loss in systems like this comes from people losing the phrase, not from cryptanalysis. A phrase nobody backs up has zero bits of security for availability.
• Migration, and this is the real cost. Per-file keys are derived from the master key (HKDF), not randomly generated and wrapped. That keeps the design simple: no per-file key blobs to store or lose. But changing an existing user's root means a new master key. That means every file key changes, which means re-encrypting every file. For someone with 40 TB, that's a multi-day re-upload. The alternative is wrapping the old master key under a new 256-bit root. But that leaves the old 12-word phrase able to decrypt everything forever, so it adds no security for existing data.

What I think we should do:

• Keep 12 words as the default.
• Offer 24 words as an opt-in for new vaults, and as the default for business plans.
• Use a versioned derivation with a new salt and, while we're in there, a per-user salt.
• For existing vaults, only push a migration together with something else that already requires re-encryption, such as a v2 cipher suite.

In the code, recover_from_phrase already accepts any valid BIP39 length. It parses the mnemonic and feeds whatever entropy it gets into Argon2id. Generation is the part pinned to 12, by RECOVERY_WORD_COUNT. The real work is versioning, UX, and test vectors, not cryptography.

We haven't decided yet. Everything above is what we're weighing, not a roadmap. If you have strong opinions on 12 vs 24 words for this threat model, I'd like to hear them.

---

12. What I learned about running it

In short: Rust was our best decision. Production is four small servers in Falkenstein. And two real incidents taught me that failover and alerting only count if they actually reach the app and a human.

I came into this knowing much more about building software than about operating a storage service. Here's what I learned.

Rust. The type system did more for this project than anything else:

• MasterKey not being Clone,
• crypto paths returning Result and never panicking,
• the compiler stopping you from holding a key across an await by accident.

With AI writing much of the code, the language itself isn't the bottleneck anymore. So you might as well pick the best tool.

What sold me was packaging. One Rust core compiles to a native binary for every OS and to WASM for the browser. There's no runtime to install, unlike Java or Python. Go would have given us binaries too, but Rust was the best decision we made.

The cluster. Production is four small virtual servers in a single datacentre in Falkenstein: two app nodes and two database nodes, behind a managed TCP load balancer.

managed TCP load balancer (PROXY protocol, TLS passthrough)
        │                              │
 ┌──────┴──────┐                ┌──────┴──────┐
 │   app01     │                │   app02     │
 │ Caddy (TLS) │                │ Caddy (TLS) │
 │ API+workers │                │ API, status │
 │ web, admin  │                │ web, admin  │
 │ HAProxy     │                │ HAProxy     │
 │ etcd, Sentinel               │ Grafana &co │
 └──────┬──────┘                └──────┬──────┘
        └────────── private net ───────┘
 ┌──────┴──────┐                ┌──────┴──────┐
 │   db01      │                │   db02      │
 │ PG + Patroni│  sync repl.    │ PG + Patroni│
 │ Redis       │ ─────────────► │ Redis repl. │
 │ etcd, Sentinel               │ etcd, Sentinel
 └─────────────┘                └─────────────┘

Databases and failover.

• What's stored where: the database holds account state and file metadata, never file bytes. Those live in S3-compatible object storage.
• PostgreSQL runs under Patroni (a tool that manages PostgreSQL high availability) with a synchronous standby. So losing a node can't drop an acknowledged write.
• Leader election goes through a three-member etcd cluster: both DB nodes plus app01, because Raft (the consensus algorithm etcd uses) needs an odd number.
• HAProxy runs on each app node. It health-checks Patroni and gives the API a local port that always points at the current primary, plus one for the replica.
• Redis handles rate limiting, the status cache, and Caddy's certificate storage. It runs as a primary and a replica under three Sentinels (Redis's failover monitors) with a quorum of two, placed the same way as etcd.

The lesson I paid for: failover that the infrastructure handles isn't failover that the application handles.

In August, Sentinel promoted the other Redis node, exactly as designed. But both API processes had resolved the Redis primary once at boot. They kept talking to the demoted node.

Every rate-limit, ban-list and CLI device-auth check started erroring and, by design, failing open. That happened about 9,500 times in four hours, and nothing paged.

We fixed it with:

• a per-node refresher that follows Sentinel,
• alerts that fire when rate limiting fails open.

A topology change is now an info-level event. Rate limiting not being enforced is a page.

Load balancing.

• The provider's load balancer does plain TCP passthrough with the PROXY protocol to both app nodes, so the real client IP survives.
• Caddy on each node terminates TLS (Let's Encrypt, with certificates stored in Redis so both nodes share them).
• Caddy reverse-proxies to the API, the web app's nginx container, and the admin and status services on localhost.
• Background workers run on app01 only. So deploys roll app02 first, check health, then app01.

High availability did its job once, in a way I didn't enjoy.

1. In September our provider had a host outage that hard-stopped app01.
2. The load balancer kept serving from app02, and users didn't notice.
3. When app01 came back, HAProxy and sshd tried to bind to private and WireGuard addresses that didn't exist yet that early in boot.
4. They hit systemd's start limit and stayed down.
5. Caddy, whose certificate storage reaches Redis through HAProxy, got stuck behind them.
6. The alerts fired within a minute and went to a Telegram group at 00:17, where nobody was awake.

The fix was boring and permanent: ip_nonlocal_bind, restart-forever drop-ins, and a written runbook. The open item is real night-time paging.

Monitoring. All of it is self-hosted on our own nodes:

• Prometheus scrapes the API on both nodes, plus node_exporter, Caddy, HAProxy, Patroni, postgres_exporter and redis_exporter. All of it goes over the private network or WireGuard (a VPN). Metrics ports are never public. There are 31 alert rules. They're unit-tested with promtool test rules, against fixtures that should fire and fixtures that shouldn't.
• Alertmanager routes to Telegram receivers (critical, warning, info) and to a webhook into our API. As the incident above shows, that isn't real paging yet.
• Grafana (seven file-provisioned dashboards, including a "war room" view), Loki + Promtail for logs, Tempo for traces.
• The blackbox exporter probes api., app. and status.beebeeb.io every minute and checks TLS certificate expiry.
• Self-hosted GlitchTip in Falkenstein for client error reports. No Sentry SDK. The web app speaks the Sentry envelope protocol directly, from a small hand-written reporter. Why? SDK auto-instrumentation (breadcrumbs, URLs, fetch bodies) is a ready-made leak channel for filenames and share links. It's opt-in and off by default.
• An upload canary, running hourly from a small non-GitHub host. The real bb CLI logs in as a bot account, pushes 1 MB, pulls it back, compares SHA-256, and deletes it. Two critical alerts watch it. One is a dead-man alert that fires when the metric is absent, so a canary nobody installed is loud rather than quiet. It's off GitHub Actions because large chunk uploads from GitHub-hosted runners hung for about 25 minutes (4 out of 4 attempts). The same command worked from every other network we tried. It's live and runs every hour.
• Backups: nightly encrypted Postgres and GlitchTip backups, each with a "backup is stale" alert, and a restore drill.
• Status page: public, at status.beebeeb.io.

13. You can't fully avoid US tech in Europe

In short: European servers were easy. European payments weren't. Moving from Stripe to Mollie meant building our own billing platform, and card payments still run over US networks.

This part surprised me the most. We set out to be European end to end: Dutch company, servers in Falkenstein, Germany, no US cloud.

The infrastructure was the easy part. Today we run on Hetzner in Falkenstein. The setup is built to scale out, and we've been talking with BIT, a datacenter in Ede in the Netherlands. They're a great fit:

• mostly green energy,
• a water loop with the city. They take in 14 °C water from the city and return it at 25 °C to heat homes in Ede, so their waste heat lowers costs for local households,
• for backup power, 16-cylinder Volvo generators of 4,000 hp each (around 10 MW together), sometimes used to support the Dutch grid during shortages.

Payments weren't easy.

We started on Stripe, because everyone does. Its developer experience is excellent. It's also a US company.

Our pitch includes "no US CLOUD Act exposure". Having our payment processor sit under US jurisdiction didn't fit. It's the one sub-processor that touches customer billing data. So we moved to Mollie, a Dutch payment provider in Amsterdam.

We did it before launch, while the only accounts were mine and a few test friends'. That was the one lucky part: no live subscribers or card mandates had to be moved.

Everything else was harder than a vendor swap. Our own pre-migration assessment put it at roughly 48–58 engineer-days for full parity. It summed it up in one line I keep coming back to: this is not a vendor swap, it's building a billing platform. In practice it took about two weeks of full focus with AI agents. I thought it would be simple. It wasn't.

What Stripe gives you for free and Mollie doesn't:

• No hosted customer portal, no product/price catalog, no proration, no customer-facing invoices, no configurable dunning, no automatic VAT. ("Dunning" is the process of chasing failed payments.) We now own all of this ourselves:
  • the plan catalog in our database (Beebeeb became the source of truth instead of Stripe),
  • a proration engine for upgrades, downgrades, storage add-ons and switching between monthly and annual,
  • invoice and credit-note PDFs from our own invoices table,
  • a dunning state machine. Mollie cancels a subscription after repeated failed payments. We treat that as one input to our own grace → read-only → suspended timeline, rather than as the authority.
• Webhooks work completely differently. A Mollie webhook is a bare resource id: no event type, no payload, no signature. Every handler fetches the payment or subscription and branches on its status. Our Stripe HMAC verification became dead code. And our deduplication had to key on (id, status) rather than id. Deduplicating on the id alone silently drops a valid later status change for the same payment.
• No mature Rust SDK. The server's Mollie client is hand-written on reqwest, with idempotency keys on every create.
• Subscriptions start differently. Instead of an atomic "checkout completed" event, there's a first payment → mandate → subscription sequence. Trials start with a mandate: a card is checked with a €0 authorisation, and iDEAL with a €0.01 payment that's refunded. The first real charge happens on day 15.
• Native apps needed new flows. The desktop app opens Mollie's hosted checkout in the system browser. It needs a public "return to app" page to bring you back.

What broke along the way, from our tracker:

• an upgrade that charged without upgrading the plan,
• a storage add-on upgrade that didn't charge,
• cancelled accounts that couldn't resubscribe after the cutover,
• a phantom "paid" row left over from the Stripe era that granted paid quota for free.

Each was found and fixed before real money moved.

The most embarrassing one was a leftover Stripe customer-portal path. It was still reachable from an older CLI command, and it would have created a customer record at Stripe for whoever ran it. Nobody had triggered it. We cut it off by removing the Stripe keys from production, and the code path is being deleted.

And then the part we can't fix. Even with Mollie, a card payment runs over Visa or Mastercard, both American. iDEAL is Dutch, but outside the Netherlands most people pay by card.

Being "European end to end" is something you can get close to, not fully achieve. I'd rather say that plainly than put a flag emoji on the pricing page.

14. French crypto rules: a declaration, in French only

In short: to offer encryption in France, you have to declare it to ANSSI, the French national cybersecurity agency. Ours took four weeks, mostly because Beebeeb uses so many encryption building blocks. It was approved this morning.

Here's something most people building in Europe don't expect. France has its own rules for encryption products.

The rule comes from the French digital economy law of 2004, the LCEN (loi pour la confiance dans l'économie numérique), and the decree that implements it (décret 2007-663). In short:

• Using encryption in France is free.
• Supplying an encryption product in France, or bringing one in from another EU country, needs a declaration to ANSSI first. That applies when the encryption does more than just authentication or integrity checks.
• Some cases need an authorisation instead. Ours wasn't one of them. It's a declaration, not a permit, but you still can't skip it.

Beebeeb is a Dutch company, but anyone in France can sign up and run our clients. And our encryption obviously does more than authentication: confidentiality is the whole point. So the rule applies to us like it would to any EU E2EE provider with French users.

What we had to declare. The form wants a technical description of the cryptography: which algorithms, which key lengths, and what each one is used for. For most products that's a short list. For us it wasn't. Going by what's in core, these are the building blocks:

• AES-256-GCM for file chunks, filenames, MIME types and thumbnails,
• Argon2id for the recovery phrase, OPAQUE and the legacy password path,
• HKDF-SHA256 for per-file keys,
• OPAQUE (Ristretto255, triple Diffie–Hellman with SHA-512) for login,
• PBKDF2-SHA256 to wrap the master key in the browser,
• WebAuthn PRF to wrap the master key under a passkey,
• X25519 for sharing between accounts,
• SHA-384 and SHA-256 for integrity checks on the web app and releases.

And they show up across different clients: web, CLI, desktop, plus mobile, which isn't released yet. Each one had to be described accurately, and consistently with the others.

Why it took four weeks. Mostly because of that list. Every variant meant another description to write, another key length to state and another use to explain. A product with one cipher would have been much quicker.

The other surprise: I couldn't find an English version of the ANSSI forms or guidance at all. Not the form, not the explanations. Everything is in French. Honestly, I couldn't have done this without AI, both to translate and to understand what each question was really asking.

The good news: it was approved this morning, 16 October 2026.

Legal in general. France wasn't the only place where I had to learn the law. I read through everything myself. Friends who work in legal departments helped me understand it and pointed out what I'd missed. I learned a lot, and it's a part of building a company nobody warns you about.

My tip for other EU founders: if French users can sign up, check the ANSSI rules early. Don't wait until launch. Write down every algorithm, key length and purpose in your product before you start the form. And if your French isn't strong, plan time for translation, and get someone who knows the law to look at it.

---

15. What's not done yet

In short: no external audit, a known chunk-integrity gap (fix scheduled for this weekend), a closed server, and several apps and features that are still beta or missing.

This is the section I'd read first if I were you.

• No third-party audit. core has been reviewed in-house against an audit-readiness scope. It has not been audited externally. The accurate word is auditable, not audited. There's no paid audit planned right now, and that's a choice: I'd rather have the public look at it first. The clients and the crypto core are open. If you find an issue, open an issue or email me, and I'll fix it and credit you publicly. The first 10 people who report a confirmed security issue also get a free year of Pro as a thank-you. Consider this post that invitation.
• Chunks aren't cryptographically bound to their position or version. In plain terms: a hostile server can't read your file, but it could cut, shuffle or mix up its pieces without the client noticing. The details:
  • Each chunk frame is sealed independently, with a random nonce and no associated data (AAD: extra data that the encryption authenticates without encrypting). The chunk index, the chunk count, the last-chunk flag and the version aren't authenticated.
  • What is checked today: the server's storage layer refuses to serve a download unless chunk indices are contiguous from 0, every chunk except the last is exactly the chunk size, and each stored object's length matches the recorded size. The clients frame the stream using the chunk count and sizes the server reports. That catches storage corruption and stale metadata.
  • It doesn't stop a malicious or compromised server. It could drop trailing chunks and report a smaller size, reorder chunks, or splice chunks from an older version of the same file (same file key). Every chunk would still decrypt. Confidentiality holds. Whole-file integrity against the server doesn't.
  • The fix is a STREAM-style construction, with file_id || version || chunk_index || is_last as AAD under a new cipher_suite version (the format is already versioned as V1Aes256Gcm). Plus a migration that keeps reading the old format and writes only the new one.
  • This is tracked internally as a P1 crypto-format decision. The fix is scheduled for this weekend.
• Server is closed source. Clients are open. The server isn't, and we have no timeline to open it.
• Web E2EE has the usual weakness. Whoever serves the JavaScript can change it. Our SRI (Subresource Integrity) check catches delivery corruption, not a malicious operator. The CLI and desktop apps are open source and built in public GitHub Actions with pinned dependencies (--locked). Releases ship SHA-256 checksums that bb update verifies before installing. That proves you got the file we published, not that it matches the source. We haven't verified byte-identical reproducible builds yet. If you try and get a different hash, please tell us. The web app is the harder case.
• No mobile apps yet. iOS is in TestFlight. On Android the native crypto module is still a stub. On a phone, use the web app for now.
• Desktop is beta. macOS (Apple Silicon only, Developer ID signed and notarized) and Windows builds exist. Windows installers are signed by Initlabs B.V., but Windows SmartScreen needs time to build trust in a new certificate, so you may still see a warning for now. Intel Macs aren't built yet.
• WebDAV is local and limited. bb webdav serves your vault on 127.0.0.1 over plain HTTP, with a random per-run password. It's read-only by default. Windows Explorer won't connect (it refuses Basic auth over plain HTTP). Finder, rclone, Cyberduck and WinSCP work.
• The CLI stores your master key in its config file (~/.config/beebeeb/config.json on Linux). It's protected by your OS account and disk encryption, not by a second passphrase. Treat it like an SSH private key. Sessions expire after 30 days.
• Passkey vault unlock falls back to localStorage where WebAuthn PRF isn't supported. That fallback key is readable by XSS.
• One location. All ciphertext is in Falkenstein, Germany. We're building our own storage fleet in EU datacentres, and will only move to it after proper redundancy testing. More EU locations are planned but not live.
• Metadata isn't hidden. We see your email, billing details, file sizes, timestamps, and sharing relationships. Filenames and contents are encrypted. Access patterns aren't.
• No post-quantum key exchange for sharing yet. Sharing between accounts uses X25519. Hybrid ML-KEM (a post-quantum key exchange standard) is the obvious next step and isn't implemented. This is the one part of Beebeeb that a large quantum computer would actually threaten, because X25519 is public-key crypto. ML-KEM is NIST's lattice-based standard (FIPS 203, final since August 2024), and we'd run it as a hybrid next to X25519. It's not just us: banking relies on public-key crypto too, in TLS connections and card chips. The EU's coordinated PQC roadmap asks for high-risk use cases to move by the end of 2030, and as many systems as feasible by 2035. So we still have time, if we start.
• No published PGP key for security@beebeeb.io yet.

16. Try it, break it

In short: read the code, try the CLI, and tell us what's wrong.

• Code: github.com/beebeeb-io. Start with core/beebeeb-core/src/ (kdf.rs, recovery.rs, encrypt.rs, chunk_stream.rs, opaque_protocol.rs).
• Threat model and primitives: beebeeb.io/security
• CLI: brew install beebeeb-io/tap/bb, or download a release from GitHub and check the SHA-256
• Vulnerabilities: security@beebeeb.io. We acknowledge within 48 hours.

Plans start with a 14-day trial (card or iDEAL, nothing charged before day 15). They go up to 99 TB self-service, at €10.99 per extra TB.

Questions or criticism? Mail us or find me on X at @GuusLangelaar. The messages I find most useful are the ones that start with "you're wrong about…".

— Guus

17. Why we're doing this

In short: no investors, on purpose. We want to build a real European alternative.

Beebeeb has no funding and no investors. It runs on our own savings and income. That's deliberate.

We want to build a real European alternative, not another company that grows just big enough to be bought by US big tech.

If that sounds like something worth existing, try it, break it, and tell us what's wrong. We'd also love contributors. The clients and the Rust core are open on GitHub, and issues, PRs and reviews are all welcome.