OxL Reference
EnglishРусский

10.6 Pipelines

A pipeline is a linear chain of steps, each of which runs on its own node; data is passed from node to node directly, the caller receives only the result of respond.

oxl
pipeline uploadPhoto(img: Image) -> PhotoId {
  validate
  -> Store.put(it)              on storage
  -> respond it.id
  -> resize(it, THUMB_SIZES)    on gpu
  -> moderate                   on ml  durable
  -> SearchIndex.add(it)
} on_error(e, step) { Dlq.push(e, step) }

A step is expression (on role)? (durable | best_effort)?, where the expression is:

  1. a function name — applied to it;
  2. an expression with it, or a lambda;
  3. respond e — sends e to the caller; the value of e is passed to the next step.

The it of the first step is the pipeline parameter (or a tuple of parameters).

Rules:

  • on role — the step is executed on a node with this role (chosen by lowest load); an rpc call in a step is routed as usual. Without on — on the node of the previous step.
  • The request carries a plan (the remaining steps and the caller's address); a node executes its step and passes the result to the next node.
  • respond occurs at most once. Without respond the result is the value of the last step. The type of the respond value MUST match R.
  • An error before respond is returned to the caller; the type of the call is R | E | NetError. An error after respond is passed to on_error(e, step); without on_error — to the log and the node's DLQ.
  • Steps after respond run in the node's background scope (§8.5) and do not delay the response.
  • durable makes this and all subsequent steps reliable: before each step the state is written to the journal; after a node failure the step is repeated on another node (at-least-once). The analyzer warns if a durable step calls an rpc without @idempotent. The default is best_effort.
  • A pipeline is called like a suspendable function and can be the body of an rpc: rpc upload(img) => uploadPhoto(img).
  • A pipeline with a stream<T> parameter applies the steps to each element (parallelism — @parallel(n), 1 by default, order is preserved); respond is forbidden in it.

pipeline-idempotency, respond-twice, respond-type, unknown-role

See also

Source: docs/spec/oxl-standard.en.md