OxL Reference

III.8 State web interface (inspector)

Enabled by the flag --inspect-ui[=addr] on any node or by the command oxl inspect --cluster <address> locally. Since the cluster registry is replicated to all nodes, the interface of any node shows the whole cluster.

SectionWhat it shows
Nodesall nodes: roles, zone, binary version, address, status, CPU, memory, number of tasks, load
Rolesthe roles from cluster: desired and actual replicas, needs, served services and actors
Configthe static config of each node with the source of every value (flag, env, file, profile, default); secrets hidden
Dynamic configcurrent values with overrides by level, change history, rollback
Servicesrpcs, their nodes, rate, errors, latencies p50/p95/p99, policies
Actors and shardsthe shard map by node, active actors, actor state (for @reflect)
Sagas and pipelinesrunning, stuck, current step, compensations, DLQ
Streams and topicssubscribers, lag, offset, broadcast hubs
Traces and logssearch by traceId, the call graph of a request, node logs
Permissionsthe "role → method" graph, policies, who-can
  • Access is per the inspect policy (a principal with the role Admin or inspect; in this implementation also a principal with the role debug and a cluster certificate with a role — an edition 1.0 correction, D-8056; DIV-041; D-6401); the default address is 127.0.0.1 only.
  • Actions from the interface (changing the dynamic config, draining a node, manual resolution of a saga, enabling the debugger) require separate permissions and are written to oxl.audit.
  • The same API is available as the gRPC service oxl.Inspect for automation.

Implementation-defined. Completeness of the inspector (D-8056; DIV-040; D-6408, D-6409). This implementation: the actor state is shown by field numbers (names — for @reflect); policy rules with conditions are not visible in the "Permissions" section.

Implementation-defined. Contents of a program by profile (D-8063; D-7702). A prod program that serves no rpc and does not join a cluster contains no inspector, no remote debugger, no telemetry exporter and no node server: the flags --inspect-ui, --debug-listen, --otlp-endpoint are not valid for it. Programs of the dev and test profiles and any program with services or a cluster contain all of them.

Source: docs/spec/oxl-standard.en.md