12.1 Cluster declaration

oxl

cluster shop {
  seeds k8s(label: "app=shop")             // dns("…"), static["10.0.0.1:7000", …]
  port 7000
  auth { jwt(issuer: AUTH_ISSUER) }

  defaults {
    timeout STD_TIMEOUT
    idempotency_ttl IDEMPOTENCY_TTL
    topic_retention TOPIC_RETENTION
    drain DRAIN_TIMEOUT
  }

  role api     { serves Users, Orders; hosts Cart; replicas 3..10; expose 443 tls(acme: PUBLIC_DOMAIN) }
  role gpu     { serves Thumbs; needs label("gpu=true") }
  role storage { journal; shards 64; replication 3 }
  role saga    { runs sagas }

  assign by env("OXL_ROLE") else label("oxl/role") else auto
}

Clause	Meaning
`seeds`	how to find the nodes of the cluster
`port`	the port of the inter-node protocol (7000 by default)
`auth`	user authentication providers (§13.2)
`defaults`	defaults for rpc and the runtime: `timeout`, `idempotency_ttl`, `topic_retention`, `drain`, `db_tx_retries`, `db_tx_retry_budget` (an unknown key is an error listing the valid ones; D-8064; D-7780)
`trace_sample r`	the fraction of requests (`0.0..=1.0`) traced by default; the process flag `--trace-sample` / the variable `OXL_TRACE_SAMPLE` overrides it
`role name { … }`	a node role
`serves S, …`	the role serves services (topics only need a role with journal)
`hosts A, …`	the role hosts sharded actors
`runs sagas` / `runs leader("name")`	the role executes sagas / leader tasks
`journal`, `shards n`, `replication k`	the role holds the built-in storage
`replicas a..b`	the desired number of nodes of the role
`needs …`	node requirements (labels, resources)
`expose port tls(…)`	an external gRPC + HTTP/JSON listener; routes(r) adds custom HTTP routes (II.4)
`assign by …`	the order of determining a node's role

Implementation-defined. Seed sources (D-8055; DIV-023; D-4259). An optional feature: seeds k8s(label: …). This implementation does not resolve k8s(…); static[…] and the local cluster oxl dev work.

Implementation-defined. Placement by resources (D-8055; DIV-016; D-4601, D-4899). This implementation does not account for the resources needs cpu/mem/gpu; needs label(…) takes part in role assignment; hosts does not control the placement of shards (only the compiler checks it: an actor is hosted by at least one role).

A program has at most one cluster. Without it the node gets the implicit role all, which serves everything, with the built-in storage in single-node mode.
Build-time checks (compile errors): every service with an impl is served by at least one role; every sharded actor is hosted; topics and sagas have a role with journal and runs sagas; every role in a pipeline's on role exists.
A node MAY have several roles: OXL_ROLE=api,saga.

actor-not-hosted, cluster-auth, cluster-decl, cluster-needs, cluster-seeds, placement-conflict, saga-without-role, service-not-served, topic-without-journal, unknown-role

12.1 Cluster declaration

Related diagnostics

See also