Tutorial: bare machines to a deployed app

Three Rocky Linux 10 machines with nothing on them but SSH, and a laptop. Five steps later a Soli app runs on two of them, SoliDB and SoliKV clusters run on all three, and every machine's proxy serves the app.

Before you start

one on your laptop (the soli-one-cli package, or the tarball from Installation), SSH as root to each machine, and ONE_REPO_BASE set to the package channel. Machine names are whatever ssh reaches: an alias in ~/.ssh/config works.

export ONE_REPO_BASE=https://<your channel>

1. Make them one cluster

one cluster form --cluster-id demo rk-01 rk-02 rk-03

On each machine: the channel's key and .repo file, then dnf install soli-one soli-one-cli. On rk-01, one bootstrap creates the certificate authority and the keyring, and the agent starts. Then, for each other machine, a single-use token minted on rk-01 and one join redeeming it there. The first three machines are admitted as control nodes and later ones as workers. It ends by checking that the cluster sees every machine.

→ start the agent on rk-03
→ confirm the cluster reports 3 node(s)
cluster demo is up: 3 node(s), seeded by rk-01

From here, every command talks to the cluster through one machine:

export SOLI_ONE_HOST=rk-01
one nodes

2. Install the runtimes everywhere

one node install soli soli-proxy solidb solikv

The same packages on every alive node, one node at a time, from the signed channel. It stops at the first node where dnf fails and names it. The runtimes are published for Rocky 10 only: they are the projects' release binaries, built against glibc 2.39.

3. A proxy on every node

one proxy enable

Each node's proxy gets its admin API bound to that node's address behind a key of its own. Every agent is given all of the proxies' URLs and keys, because whichever node holds the scheduler lease pushes the routing table to every proxy. The agents are then restarted one at a time. Running it again keeps the keys; a proxy config you edited by hand is refused rather than overwritten.

4. SoliDB and SoliKV clusters

A spec per datastore, on the laptop:

# db.toml
service    = "solidb"
cluster_id = "db"
data_dir   = "/var/lib/solidb"
memory_mb  = 1024

[[member]]
node_id = "db-1"
machine = "rk-01"
address = "10.77.0.11:6746"

[[member]]
node_id = "db-2"
machine = "rk-02"
address = "10.77.0.12:6746"

[[member]]
node_id = "db-3"
machine = "rk-03"
address = "10.77.0.13:6746"

one datastore apply --spec db.toml --everywhere
one datastore apply --spec kv.toml --everywhere

--everywhere puts one keyfile and the spec on every member's machine, then runs apply there, the seed's machine first. Each member is confirmed before the next one starts: SoliDB by its replication view, SoliKV by its cluster view. SoliKV's keyspace is then split between the members, and the command checks that all 16384 slots are owned. A member runs as its own unprivileged user, with its data directory given to it by systemd.

5. Deploy an app from the laptop

one deploy ./hello --name hello --domain hello.test --replicas 2 \
    -- soli serve . --port '$PORT'

The directory travels to rk-01 as a tar stream over SSH and is stored there as an artifact. The workload is declared for the cluster, running in that artifact. The node the scheduler picks fetches it from whoever holds it, unpacks it, and runs the command inside it with $PORT set. hello.test is then pointed at the workload.

→ ./hello ingested as artifact 117bbdc1f2c3
→ hello declared for the cluster, 2 replica(s)
→ hello.test points at hello
→ placed on rk-03, rk-02

curl -k --resolve hello.test:443:10.77.0.11 https://hello.test/

Any node's proxy answers, and consecutive requests alternate between the two instances. Plain HTTP answers with a redirect to HTTPS. In the lab the certificate is self-signed, hence -k.

What this does not do yet