III.8 State web interface (inspector)
Enabled by the flag --inspect-ui[=addr] on any node or by the command oxl inspect --cluster <address> locally. Since the cluster registry is replicated to all nodes, the interface of any node shows the whole cluster.
| Section | What it shows |
|---|---|
| Nodes | all nodes: roles, zone, binary version, address, status, CPU, memory, number of tasks, load |
| Roles | the roles from cluster: desired and actual replicas, needs, served services and actors |
| Config | the static config of each node with the source of every value (flag, env, file, profile, default); secrets hidden |
| Dynamic config | current values with overrides by level, change history, rollback |
| Services | rpcs, their nodes, rate, errors, latencies p50/p95/p99, policies |
| Actors and shards | the shard map by node, active actors, actor state (for @reflect) |
| Sagas and pipelines | running, stuck, current step, compensations, DLQ |
| Streams and topics | subscribers, lag, offset, broadcast hubs |
| Traces and logs | search by traceId, the call graph of a request, node logs |
| Permissions | the "role → method" graph, policies, who-can |
- Access is per the
inspectpolicy (a principal with the roleAdminorinspect; in this implementation also a principal with the roledebugand a cluster certificate with a role — an edition 1.0 correction, D-8056; DIV-041; D-6401); the default address is127.0.0.1only. - Actions from the interface (changing the dynamic config, draining a node, manual resolution of a saga, enabling the debugger) require separate permissions and are written to
oxl.audit. - The same API is available as the gRPC service
oxl.Inspectfor automation.
Implementation-defined. Completeness of the inspector (D-8056; DIV-040; D-6408, D-6409). This implementation: the actor state is shown by field numbers (names — for
@reflect); policy rules with conditions are not visible in the "Permissions" section.
Implementation-defined. Contents of a program by profile (D-8063; D-7702). A
prodprogram that serves no rpc and does not join a cluster contains no inspector, no remote debugger, no telemetry exporter and no node server: the flags--inspect-ui,--debug-listen,--otlp-endpointare not valid for it. Programs of thedevandtestprofiles and any program with services or a cluster contain all of them.
Source: docs/spec/oxl-standard.en.md