Cluster and configuration

One binary runs on all nodes of the cluster; the role of a node decides what it does: serves services, hosts actors, stores data, runs sagas. The cluster is declared once per program (§12.1):

cluster counters {
  role edge  { serves Gate; runs leader("traffic"); replicas 1.. }
  role store { journal; shards 8; replication 3; hosts Counter }
  assign by env("OXL_ROLE") else auto
}

The compiler checks consistency: every service with an impl is served by some role, every sharded actor is hosted, topics and sagas have a place to be stored. A node without cluster gets the implicit role all, which is why tests and oxl run run the whole code in one process without changes.

A local cluster: oxl dev#

cd examples/counter-cluster
oxl dev --nodes 4

oxl dev starts N processes of one binary, shows a table of nodes (state, role, cluster view 4/4) and writes the log of each node to <data-dir>/nX/stdout.log. Stop a node process and the others notice through SWIM; roles and shards move, and actors are activated on other nodes together with their state. oxl inspect opens the web interface of the cluster: nodes, roles, shards, sagas, live calls; oxl trace shows the call graph of a request across the nodes. The ready project is counter-cluster.

Configuration#

The static config is read at start: flag --set App.port=9000 → variable APP_PORT → config/<profile>.toml → profile block → default value. A wrong value gives a message naming the source and exit code 78 (§14.1).

config App {
  port: u16 = 8080
  name: str = "demo"
  token: secret<str>
  profile dev { token = "dev-token" }
  profile prod { port = 443 }
}

fn main() io {
  let app = App.get()
  print("{app.name} on {app.port}")
}

demo on 8080

The dynamic dynamic config lives in the cluster and changes without a restart (oxl config set Limits.rps 200): values are type-checked and versioned, and a Flag turns a feature on for a percentage of users (§14.2). The cluster storage, shards and replication are §12.8; security (always mTLS, authentication, rpc policies) is §13.