uBix Vault, Beta 3 — Running It on My Own Cluster, With a Console

From a Binary to a Deployment

Last time, uBix Vault crossed from design notes into a working beta — encryption barrier, Shamir seal/unseal, dynamic database credentials, the crypto core built and tested from scratch. That post ended with a tagged v0.2.0-beta.1 and a binary I’d exercised end-to-end.

A binary you can run on your laptop and a system you can actually operate are different things. The two betas since — beta.2 and beta.3 — are entirely about closing that gap: packaging it for Kubernetes, making it observable, hardening it, and building a UI so it isn’t curl-only. Then I deployed it onto my own cluster.

Same disclaimer, because it’s a secrets manager and overselling maturity is how people get burned: it is a beta, single-node, not production-hardened, and it has not had an external security review. For production, use HashiCorp Vault or OpenBao. I run this on my own internal cluster, deliberately, with that understood.

It Runs on Kubernetes Now

There’s a Helm chart and a multi-arch image published to GHCR (my cluster is a mix of amd64 and arm64 nodes, and the vault pod has run on both). The chart is honest about what it is: a single-node StatefulSet that refuses replicaCount > 1 rather than pretending to be highly available, because there’s no Raft yet.

The parts I’m happiest with are the ones that encode a real operational opinion:

• The probes are split on purpose. A sealed vault answers 503 on its health endpoint. If you point a liveness probe at that, Kubernetes crash-loops the pod before it can ever be unsealed. So liveness is a plain TCP check (“is the process up?”) and readiness is the HTTP check (“is it unsealed and serving?”). Getting that wrong is a classic footgun; the chart gets it right by default.
• TLS without a manual step. The vault serves HTTPS using a wildcard certificate that Replikate — another uBix project — syncs into its namespace. The ingress re-encrypts to the pod, so the proxy never sees plaintext secrets.
• It auto-unseals. Restart the pod and it comes back unsealed on its own, no human feeding it key shares. I proved that the boring way: deleted the pod and watched it log auto-unsealed and go ready.

The Lockout I Almost Shipped

Here’s the bug I’m glad I caught while deploying rather than after.

Auto-unseal wraps the master key with a key-encryption key so the server can unseal itself. Great for uptime. But in that mode, init returned no key shares at all, and root-token regeneration was Shamir-only. Put those together and you get a trap: an auto-unsealed vault whose root token is lost is unrecoverable. No shares to regenerate it with, and the one recovery path didn’t apply.

The fix is the pattern HashiCorp uses: recovery keys. In auto-unseal mode, init now also generates k-of-n recovery keys — a separate random key, split into Shamir shares, with only its hash persisted. The KEK still unseals automatically; the recovery keys exist solely to regenerate a lost root token, verified in constant time against the stored hash. I built the whole flow end-to-end: init returns recovery keys, a threshold of them mints a fresh root token, wrong keys are rejected.

It’s the kind of gap you only trip over when you stop treating something as a demo and start treating it as infrastructure you depend on.

A Console, Built the Boring Way

Beta 3’s headline is a web console, served at /ui/ straight from the binary. It shows the vault’s seal state front and center (green unsealed, amber sealed, red uninitialized), and lets you manage KV v2 secrets — read, edit, version history, soft-delete/undelete/destroy — plus ACL policies and minting scoped tokens. You paste a token; it never leaves the browser tab.

The interesting decision was how to build it: vanilla HTML, CSS, and JavaScript, embedded in the Go binary. No framework, no build step, no CDN. That’s not nostalgia. It keeps the strict content-security-policy trivial (default-src 'none'; the page talks only to its own API), it keeps secret values rendered as text so a value can never inject markup, and it means the UI ships as part of the one artifact you already trust. A React bundle would have added a build toolchain and a dependency tree to a thing whose entire pitch is a small, auditable surface. Not worth it.

Making It Observable, and Harder to Brute-Force

Metrics rounded out beta 3, and a rate-limiting pass has just landed on top of it:

• Metrics. A Prometheus endpoint exposing seal state, uptime, and request counts — rendered by a ~100-line in-house exporter rather than pulling in a client library and its transitive graph. The chart can wire up a ServiceMonitor.
• Rate limiting (landing next). An in-house token-bucket limiter, keyed per client, to blunt brute-force against unseal shares and tokens. Health, metrics, and the console are exempt so probes and scrapers don’t get throttled; behind a proxy it keys on the forwarded client IP so one ingress doesn’t look like one client.

Both follow the same rule the crypto did: write the small, boring thing yourself before you add a dependency. Through all of this the project still has essentially one third-party dependency — the MySQL driver. For a security tool, a dependency graph you can actually read is a feature, not a limitation.

Where It Is, and What’s Next

uBix Vault is BSD 3-Clause, on GitHub at github.com/cwolsen7905/uBixVault, tagged v0.2.0-beta.3. It’s deployed on my own cluster and I use the console to poke at it, which is exactly the feedback loop I wanted.

The road to a real 1.0 is the honest limitations list, unchanged and written down where anyone can see it: Raft-based HA so it’s more than a single node, a pluggable cloud-KMS/HSM seal so the key-encryption key isn’t supplied directly, and — the one that actually gates production use — an external security review. Until those land, the disclaimer at the top stands.

But the shape of the thing has changed. Three posts ago it was a plan. Two ago it was a design. Last time it was a binary. Now it’s a service running on my own infrastructure, with a console, metrics, and a recovery story — and the parts I built myself are still the parts I understand best.