1,100+ tests · rocky 10 · selinux enforcing gossip :7300 raft :7302 api :7304 rocky 10 / systemd 257

Providers

OVHcloud, Scaleway, Hetzner and AWS agree on almost nothing.

built; Scaleway is wired, the other three are refused by name

The capability model is implemented and tested in soli-one-provider, and the OVH request signing, DNS reconciliation and Additional-IP logic in soli-one-ovh — which is a library only: no binary links it yet.

one cluster create drives Scaleway against a live account: it orders the machines, waits for their addresses, installs the package and forms the cluster. OVH has the signing, DNS and Additional-IP move built and no instance-create path; Hetzner and AWS are capability tables and nothing else.

Those three are refused before anything is ordered, by name, saying what is missing and what to do instead — a flag that accepts a provider it cannot act on is a flag that fails after the first machine is billed. AWS is refused for the more fundamental reason first: its service address is steered by a load balancer and never moves to an instance, so the cluster VIP has nowhere to live there at all.

Intersection, not union

With four backends the temptation is a trait that is the union of what they all offer. That produces a surface where most methods return Unsupported on most providers, every caller has to handle it, and the abstraction has bought nothing. The trait here is the intersection, and what does not generalise is modelled as a capability the caller asks about rather than a method that fails.

ProviderFailoverPrivate networkEgress
OVHcloud VPSAPI-routednoneunmetered
ScalewayAPI-routedyesunmetered
HetznerAPI-routed + announceyesmetered
AWSload balanceryesmetered

Failover is the one that must not be hidden

Behind a single move_address these four look identical at the call site and behave differently on the wire — and the wrong one fails silently: the address does not move, the old node keeps answering, and the health check keeps passing. So the mechanism is named, and the caller can refuse the ones it has not implemented.

built and tested for Scaleway; not wired in yet

Everything in this note lives in soli-one-vip, which no binary links yet: neither soli-oned nor one runs a VIP manager, so no running cluster binds or moves an address today. The crate and its tests are real; the wiring is the missing step.

soli-one-vip claimed an address with ip addr add plus a gratuitous ARP and nothing else. That is layer-2 failover, correct on a LAN, and offered by none of the four — asserted as a test. The decision logic is unchanged, and was never the problem: hold the address if and only if the lease is held. What was missing is the call that makes holding it mean anything.

RoutedBinder now does both halves. The order is not interchangeable: the provider attach is exclusive — it takes the address off the previous holder — so it goes first, and the window it opens is one where nobody answers. Configuring the interface first would open the other kind of window, two machines holding the address with one of them still being routed to, which is the single state this component exists to make unreachable. Release inverts it for the same reason: the local removal is what stops this machine answering, so it goes first and its failure is fatal, while a failed detach is survivable because the next holder's attach moves the address anyway.

A mechanism mismatch is a refusal, and it happens at startup. A provider whose failover this build does not implement produces an error when the binder is constructed, not a plan. Found at startup it is a configuration error someone reads; found at the first handover it is an outage that does not end, because every symptom points elsewhere — the new holder reports success, the old machine keeps answering, and the health check asks the address and reaches the old machine.

One half of the move cannot be read back. The kernel says whether the address is on the interface; nothing says whether the provider still routes it here without an API call on every pass. Folding that into “do we hold the address” would break the opposite decision: a node that had lost the lease would report holding nothing, conclude there was nothing to release, and keep the address. So the readable half stays the answer, and the unreadable half is re-asserted — idempotently — on the passes where the lease is held.

What is not proven: the Scaleway request shapes are written from the documented Instance API and have not been sent to it. The tests assert the request is the one intended, with no account and no key; they cannot assert Scaleway accepts it. OVH is the same mechanism and has no client yet; Hetzner needs the announcement half as well; AWS is refused, because a load balancer never moves the address.

Two capabilities change the business model, not just the code

  • No private network means every cluster port is reachable from the internet unless a peer allowlist is installed. That is why Security exists in the shape it does.
  • Metered egress means bandwidth_gb_cents cannot stay at zero, and “bandwidth included” stops being true. Moving to Hetzner or AWS without changing the pricing page would make it a false statement, so the capability check reports it explicitly.

What is deliberately absent

Ordering and destroying machines is not on the trait. It spends money and it is irreversible, so it belongs to an explicit, confirmed path with a plan attached — not to an interface that makes it look like any other call. Managed databases, object storage and load balancers are absent too: real differences, no current need, and adding them now would fix four vocabularies into a trait before anything has to use it.