OpenClaw EnterpriseDOCSGitHub

SSH compute lifecycle

Overview

The worker realizes an admitted embedded OpenClaw AgentRevision on a configured Linux host. The SSH Compute Driver sends a controller-owned helper to the host, which stages the Agent revision and activates the Agent's systemd gateway after OCC commits its selected version. This trace covers preparation, activation, retirement, and Agent and Namespace deletion. It stops when control returns to the worker; gateway request execution is outside this flow. The SSH reference owns configuration and supported boundaries.

Entry Points

Flow

graph TD
  subgraph OCC["OCC worker"]
    A["Authorize claimed Agent revision and bind Agent"] --> B["Validate embedded Agent revision"]
    B --> C["Send prepare operation over SSH"]
  end
  subgraph Stage["Host preparation under shared-root flock"]
    C --> D["Verify ownership and private Agent account"]
    D --> E["Stage immutable snapshot; preserve running gateway"]
  end
  E --> F{"OCC selected version commit"}
  F -->|fails| G["Previous gateway continues serving"]
  F -->|succeeds| H["Run beforeWorkloadStart hooks"]
  subgraph Activate["Host activation under shared-root flock"]
    H --> I["Render unit and switch current pointer"]
    I --> J["Restart systemd gateway and poll readiness"]
    J -->|ready| K["Record served Agent revision"]
  end
  J -->|fails| L["Compensate lifecycle bindings; retry finalization"]
  K --> M["Retire previous snapshot and complete work"]

Execution Trace

1. Select the driver and bind resource ownership

installation-config.ts:loadInstallationConfiguration, worker.ts:processRevision, ssh/index.ts:preflight, bindAgent, ensureNamespace

The exact packageless ID compute-ssh selects occ/ssh; other packageless Compute IDs select Kubernetes. Schema and semantic validation reject invalid host settings and Sandbox composition. Preflight checks the two local SSH files and probes each host's required executables and account-management tools.

Namespace preparation creates or verifies its ownership marker. OCC then runs selected Namespace setup hooks. After Agent revision authorization and provider validation, the worker calls bindAgent; the driver copies the authoritative Namespace, Agent, and ServicePrincipal binding into its in-memory maps. Agent revision operations require matching bindings and the selected Compute identity. Admission and dispatch require the explicit runtime binding. The snapshot contains only the method: neither source authorization nor credential resolution runs for it, while Agent/Configuration authorization and worker reauthorization remain required. SshComputeDriver.validateHarnessAuth rejects dedicated Harnesses (a fixed 409), then any other method through requireHarnessAuthMethod (compute/harness-auth-method.ts), which names harnessAuth.method (400). Preparation also rejects OCC Secret delivery and Sandbox use. SSH dispatches selected beforeWorkloadStart hooks during activation, before starting the candidate workload.

2. Cross SSH and serialize host changes

ssh/index.ts:execute, ssh/executor.ts:SystemSshCommandExecutor.execute, ssh/remote-helper.cjs:run, acquireLock

The driver sends its helper on stdin and one base64 JSON argument containing the operation, resource data, configuration hash, and trusted runtime settings. The system SSH client uses an explicit identity and known-hosts file, strict host-key checks, batch mode, and a 180-second operation timeout. The helper also enforces a 170-second deadline and detects closed output pipes with heartbeat writes. Cancellation terminates the local client; a systemd job already submitted can still complete. Once stdout exceeds 64 Ki UTF-16 code units, the executor stops retaining subsequent chunks and reports SSH helper output exceeded its limit. The crossing chunk can exceed the threshold. The local client still receives SIGTERM, then SIGKILL after one second if needed; the operation settles on its close event. One second after the client exits, the executor closes its output pipes, so a descendant that inherited them, such as a ProxyCommand, cannot hold the operation open for more than about a second after the client exits.

Host mutations take <root>/.compute-lock through a child holding kernel flock, with a 30-second acquisition timeout. The shared root defines the inventory and serialization boundary. Namespace, Agent, Agent revision, and unit markers are checked before use; ownership failures are permanent. The lock releases when its holder exits. Probe does not take the lock. Deletion of an already-missing Namespace still holds it while retrying cleanup of remaining owned accounts.

3. Create the private account and stage the snapshot

ssh/remote-helper.cjs:prepare, ensureRuntimeIdentity, snapshot

Before host work, sshGatewayConfigurationDocument admits only the supported authentication fields and modes. Omitted auth mode renders explicit password mode with a managed environment SecretRef; explicit trusted proxy retains its configured trust. sshGatewayAuthentication reads an authored password with the shared reader (gatewayPasswordRequirement). An unset password, ${OPENCLAW_GATEWAY_PASSWORD} and a default-provider SecretRef render as the managed reference, as before. Any other readable environment SecretRef to that variable renders as written. When the managed reference is unreadable under the Configuration's secrets, admission refuses it. The refusal offers the reference on the default env alias only if the reader accepts that reference. Verify, stop and retire check only the password's shape and render it the same way, so the snapshot hash still matches.

The helper creates or verifies a deterministic per-Agent system user and private group, then creates the Agent directory and reserves the lowest free port across the shared root. Root-owned account markers bind the UID/GID to the exact Driver, Namespace, Agent, and ServicePrincipal. Existing unowned accounts or conflicting markers fail closed. Agent-owned home/ and state/ use mode 0700 and persist across versions.

The helper creates gateway-password.env only when needed and missing, preserving its value across later versions. It does not migrate historical credential files.

Snapshots store the Driver-rendered configuration as root-owned, private-group-readable openclaw.json. The helper validates existing bytes against their hash and rejects a different Agent revision ID with the same version number. An older candidate returns not-ready. Preparation leaves the running unit and current pointer untouched, so failed OCC publication does not cut over the gateway.

4. Commit activation, start the gateway, and retire the prior Agent revision

worker.ts:observeRevision, finalizeRevision, ssh/index.ts:activateRevision, retireRevision, ssh/remote-helper.cjs:activate, renderUnit, waitReady

SSH uses the default activation order. After successful preparation, OCC compare-and-sets the Agent's selected version in its database. The worker then calls activateRevision, which invokes selected Configuration and IAM beforeWorkloadStart hooks before sending the activation operation. Accepted opaque launch placeholders enter the systemd unit's environment. Hook failure prevents launch; failure after hook preparation invokes workload-stop compensation with a separate five-second cancellation signal if the original signal was cancelled. Incomplete finalization remains retryable.

The helper renders the unit for the Agent's private Unix account, enables its absolute path, replaces current, restarts the gateway, and polls loopback /readyz for up to 120 seconds. It writes served.json only after readiness succeeds. A matching current pointer alone never establishes successful activation. The unit loads the per-Agent managed gateway-password.env for password access and optionally loads the operator-owned env file. Explicit trusted proxy without a password omits the managed file. The unit has no legacy credential or environment migration path. The Driver never reads or writes the operator credential file and never submits a model probe. Readiness confirms gateway startup only; an invalid key can leave the gateway ready while model requests fail. Host credential changes may affect an existing version after restart without a new Agent revision.

Retirement runs beforeWorkloadStop hooks and removes the specified snapshot. If that snapshot is current, stopOwnedUnit verifies the configured file's ownership, links its absolute path into systemd's load path, then stops/disables the unit and removes current and served.json. Persistent state, the private account, and other snapshots remain. deactivateRevision only verifies ownership; the worker's dedicated-only deactivation path is outside SSH's supported topology. Relinking permits cleanup after disable removed a custom directory's load-path link.

5. Delete the Namespace host state

ssh/index.ts:deleteNamespace, ssh/remote-helper.cjs:removeNamespace

Deletion validates every Agent, Agent revision, and unit in the deletion set before stopping gateways. The helper stops and disables owned units through stopOwnedUnit, removes their unit files, reloads systemd, and removes the Namespace tree, including persistent state, and its owned runtime accounts. The driver clears its in-memory bindings only after the host operation succeeds.

Agent deletion (ssh/index.ts:deleteAgentRuntimeCredentials, remote-helper.cjs:removeAgent) does the same for one Agent after its Agent revisions retire, which frees its gateway port.

Debugging and Verification

Search documentation