lobster

Lobster

An OpenClaw-native workflow shell: typed (JSON-first) pipelines and approval gates.

Example

A named workflow fetches a pull request with gh, compares it with the saved snapshot, and returns a notification only when it changes:

node bin/lobster.js run --mode tool "workflows.run --name github.pr.monitor.notify --args-json '{\"repo\":\"openclaw/lobster\",\"pr\":167}'"

For repeatable automation, put shell commands, native pipeline steps, and explicit approval/input gates in a workflow file. OpenClaw presents the gate to the user and resumes with their decision.

Goals

• Typed pipelines (objects/arrays), not text pipes.
• Local-first execution.
• No new auth surface: Lobster must not own OAuth/tokens.
• Composable macros that OpenClaw (or any agent) can invoke in one step to save tokens.

Quick start

Requires Node.js 22 or newer and the pnpm version pinned in package.json. From this folder:

• pnpm install --frozen-lockfile
• pnpm build
• pnpm test
• pnpm lint
• node ./bin/lobster.js --help
• node ./bin/lobster.js doctor
• node ./bin/lobster.js "exec --json --shell 'echo [1,2,3]' | where '0>=0' | json"

Notes

• pnpm test runs tsc and then executes tests against dist/.
• bin/lobster.js runs the compiled entrypoint in dist/; build after changing source files. Invalid explicit workflow file paths are reported as input errors (exit code 2). In --mode tool, these failures use the same JSON error envelope as other parsing errors.

Process output limits

Captured process stdout and stderr remain unlimited by default. Set LOBSTER_MAX_OUTPUT_BYTES to a positive integer to opt into a byte limit per stream, per process. Unset or empty means unlimited; zero, negative, fractional, and non-numeric values are rejected before a process starts. This applies to CLI exec, workflow shell steps, SDK exec/shell calls, GitHub recipes, and Gog commands. The existing openclaw.agent 10 MiB limit remains in place; an opted-in limit can lower it.

LOBSTER_MAX_OUTPUT_BYTES=1048576 lobster run --mode tool 'exec my-command'

Use workflow or step env to scope the limit:

steps:
  - id: bounded
    env:
      LOBSTER_MAX_OUTPUT_BYTES: "1048576"
    run: my-command

SDK callers set the same variable in new Lobster({ env: { ...process.env, LOBSTER_MAX_OUTPUT_BYTES: "1048576" } }). Values exactly at the limit succeed; overflow terminates the process tree and returns an error without partial output. POSIX capped calls own a process group so background producers cannot keep pipes open after overflow. Calls without an AbortSignal relay terminal interrupts to that group and retain the host's existing signal handlers or conventional interrupt exit status.

Commands

• exec: run OS commands
• exec --json : parse subprocess stdout as JSON before the next stage
• exec --stdin raw|json|jsonl: feed pipeline input into subprocess stdin
• Explicit quoted empty arguments are retained by both pipeline exec and SDK exec. Invocation shims preserve argument punctuation without shell expansion.
• where, pick, head: data shaping
• head --n N stops reading upstream as soon as N items have been yielded; --n 0 does not read upstream input.
• pick and map preserve field names such as __proto__ as ordinary own data fields.
• json, table: renderers
• approve: approval gate (TTY prompt or --emit for OpenClaw integration)

SDK cancellation

Pass an AbortSignal to new Lobster({ signal }) to cancel SDK exec() stages, including exec(command, { shell: true }). The signal is retained by clone() and forwarded when a pipeline resumes. Aborting terminates the running child process tree before run() or resume() returns an error result containing the abort reason. A signal that is already aborted prevents the command from starting.

import { Lobster, exec } from '@clawdbot/lobster';

const controller = new AbortController();
const pending = new Lobster({ signal: controller.signal })
  .pipe(exec('node long-task.js', { json: false }))
  .run();
// When the host needs to stop the command:
controller.abort(new Error('Host cancelled the command'));
const result = await pending;

Custom stages receive the signal as ctx.signal and must cooperate with it to cancel their own work.

GitHub recipe cancellation

The SDK GitHub recipes accept an AbortSignal in both prMonitor({ repo, pr, signal }) and prMonitorNotify({ repo, pr, signal }). Aborting stops the running GitHub CLI child before the recipe returns an error result containing the abort reason. A cancelled fetch does not update the saved PR snapshot or emit a notification.

For a manually composed pipeline, pass the signal to new Lobster({ signal }).pipe(ghPrView({ repo, pr })). Custom stages receive ctx.signal and must cooperate with it to cancel their own work.

Workflow files

Lobster workflow files are meant to read like small scripts:

• run: or command: for deterministic shell/CLI steps
• pipeline: for native Lobster stages like llm.invoke
• approval: for hard workflow gates between steps
• stdin: $step.stdout or stdin: $step.json to pass data forward

lobster run path/to/workflow.lobster
lobster run --file path/to/workflow.lobster --args-json '{"tag":"family"}'

Example file:

name: jacket-advice
args:
  location:
    default: Phoenix
steps:
  - id: fetch
    run: weather --json ${location}

  - id: confirm
    approval: Want jacket advice from the LLM?
    stdin: $fetch.json

  - id: advice
    pipeline: >
      llm.invoke --prompt "Given this weather data, should I wear a jacket?
      Be concise and return JSON."
    stdin: $fetch.json
    when: $confirm.approved

Notes:

• run: and command: are equivalent; run: is the preferred spelling for new files.
• pipeline: shares the same args/env/results model as shell steps, so later steps can still reference $step.stdout or $step.json.
• If you need a human checkpoint before an LLM call, use a dedicated approval: step in the workflow file rather than approve inside the nested pipeline.
• cwd, env, stdin, when, and condition work for both shell and pipeline steps.
• Use retry, timeout_ms, and on_error per step to control transient-failure behavior and recovery.
• On for_each, these policies cover the entire loop: each attempt gets one timeout budget (including batch pauses), and retry restarts at the first item. Previously completed children may run again, so only enable loop retries for work that is safe to repeat. A cost_limit stop or an attempted OpenClaw dispatch prevents retries; paid LLM usage is checked even when a later pipeline command fails. on_error: continue records $loop.error and proceeds to the next workflow step; skip_rest skips the remaining workflow steps.
• Approval steps can optionally enforce identity constraints:
  • approval.required_approver (or requiredApprover) requires an exact approver id.
  • approval.require_different_approver (or requireDifferentApprover) requires approver id to differ from initiator.
  • approval.initiated_by (or initiatedBy) sets the initiator id for comparison.
  • LOBSTER_APPROVAL_INITIATED_BY can provide a default initiator id at run time.
  • LOBSTER_APPROVAL_APPROVED_BY is used at resume/approval time for identity checks.

Retries after OpenClaw dispatch

openclaw.invoke (including clawd.invoke) and openclaw.agent suppress automatic workflow-step retries once dispatch is attempted. This applies to timeouts, HTTP or process failures, and errors in later commands of the same pipeline step, even when retry.max is greater than one. A failed local request cannot establish whether a remote action already completed. Read-only and dry-run requests use the same conservative rule: Lobster does not know the remote tool’s retry guarantees, and the OpenClaw gateway currently ignores dryRun. The flag does not prevent tool execution. Check the remote outcome before manually running the step again. llm.invoke retains its existing retry policy.

Command-level input requests

Pipeline commands can call ctx.requestInput({ prompt, responseSchema, defaults, subject, suspendedState }) to pause in tool mode, workflows, or the SDK and resume the same command after a structured response. CLI/tool resume tokens store only a state key; the persisted state validates the suspended request metadata before returning the submitted response to the command. SDK same-command resumes store the command frame in the configured SDK state directory.

Commands are re-run on resume, so they must be idempotent until requestInput returns. Array-backed command input is snapshotted with bounds for replay; lazy stream input is not buffered and requires a compact JSON suspendedState supplied by the command. On resume, call ctx.requestInput.getSuspendedState() before reading lazy input to restore that command-owned continuation state.

Visualizing workflows

Use lobster graph to inspect workflow structure before execution.

lobster graph --help
lobster graph --file path/to/workflow.lobster
lobster graph --file path/to/workflow.lobster --format mermaid
lobster graph --file path/to/workflow.lobster --format dot
lobster graph --file path/to/workflow.lobster --format ascii
lobster graph --file path/to/workflow.lobster --args-json '{"location":"Seattle"}'

What gets visualized:

• each workflow step as a node (run, pipeline, approval, etc.)
• data-flow edges from stdin: $step.stdout / $step.json references
• conditional dependencies from when: / condition: expressions
• approval gates as diamond-shaped nodes in mermaid and dot output

Format notes:

• mermaid (default): emits flowchart TD text for GitHub/Markdown rendering
• dot: emits Graphviz DOT syntax
• ascii: emits a terminal-friendly node/edge list

Calling LLMs from workflows

Use llm.invoke from a native pipeline: step for model-backed work:

llm.invoke --prompt 'Summarize this diff'
llm.invoke --provider openclaw --prompt 'Summarize this diff'
llm.invoke --provider pi --prompt 'Summarize this diff'

Provider resolution order:

• --provider
• LOBSTER_LLM_PROVIDER
• auto-detect from environment

Built-in providers today:

• openclaw via OPENCLAW_URL / OPENCLAW_TOKEN
• pi via LOBSTER_PI_LLM_ADAPTER_URL (typically supplied by the Pi extension)
• http via LOBSTER_LLM_ADAPTER_URL

A host embedding Lobster can supply its own adapters through ctx.llmAdapters. Step timeout_ms and workflow cancellation reach an adapter as ctx.signal: Lobster stops waiting as soon as that signal aborts, so the step fails or retries on time either way, but it cannot cancel work an adapter has already started. An injected adapter should observe ctx.signal and abort its own request — otherwise a timed-out step can leave a model call running, and billed, in the background.

Workflow _meta.cost and cost_limit use a static pricing table plus optional overrides from LOBSTER_LLM_PRICING_JSON, for example {"my-model":{"input":1.0,"output":2.0}} in USD per million tokens. Unknown or missing model IDs still record token counts with zero estimated cost, but Lobster warns on stderr so stale or missing pricing does not fail silently.

A cached or replayed answer is not billed again: a model call is counted once, in the run that made it, however many later steps re-emit its answer. This holds while the answer stays inside Lobster: through pipelines, renderers, projections, run state, workflow: steps and a resume. It does not survive a stage that hands the items to an external process and reads them back — exec --stdin json --json ... — because what comes back is whatever that process printed, and Lobster cannot tell a faithful copy of a replay from