Deploy and verify production Agents

Deploy an Agent into a ready Namespace and verify model execution. Complete control-plane installation and its authenticated API check. Run commands from the repository root, retaining credentials and Kubernetes context. For Secret-backed authentication, you or an Installation administrator must grant the Agent access to the model Secret before deployment.

Use Bash, Python 3, kubectl, and the OCC CLI for the commands below. For browser deployment, prepare the Namespace and follow the console deployment workflow.

Prepare each Namespace

Use a driver-managed Kubernetes namespace

Fresh bootstrap creates a platform Namespace named default. Run occ namespace list and export its server-assigned ID:

occ namespace listexport NAMESPACE_ID='<ID shown for default>'

The worker creates a Kubernetes namespace labeled openclaw.dev/namespace=$NAMESPACE_ID, separate from Kubernetes' built-in default namespace. Discover and export its name for tenant RoleBindings:

TENANT_NAMESPACE="$(kubectl --kubeconfig "$KUBECONFIG_FILE" --context "$CONTEXT" \  get namespaces -l "openclaw.dev/namespace=$NAMESPACE_ID" -o json | \  python3 -c 'import json,sys; items=json.load(sys.stdin)["items"]; print(items[0]["metadata"]["name"]) if len(items)==1 else sys.exit("Expected one backing Namespace; check worker provisioning")')" && export TENANT_NAMESPACE

If no backing namespace is found, check the worker logs and repeat discovery after creation. Complete the tenant RoleBindings below, then wait until GET /namespaces/$NAMESPACE_ID reports ready before creating Configurations. If it reports failed, inspect audit evidence and worker logs. Exhausted lease recovery stops provisioning permanently.

Grant tenant RoleBindings

Single-cluster Compute uses the same tenant namespace for runtime roles and canonical storage. Use the Helm release and Kubernetes namespace from the control-plane install. The chart prefixes the three tenant ClusterRoles with the release name and creates fixed-name ServiceAccounts in the install namespace.

# Control-plane installation exampleRELEASE=oceCONTROL_PLANE_NAMESPACE=openclaw-system# For a different install, use its values. Example:# RELEASE=openclaw-enterprise# CONTROL_PLANE_NAMESPACE=oce-system tenant_kubectl() {  kubectl --kubeconfig "$KUBECONFIG_FILE" --context "$CONTEXT" -n "$TENANT_NAMESPACE" "$@"}WORKER_SA="$CONTROL_PLANE_NAMESPACE:openclaw-enterprise-worker"API_SA="$CONTROL_PLANE_NAMESPACE:openclaw-enterprise-api"

For OpenShell Sandbox, first create and bind the tenant-local workspace provisioner Role. Use the Role and its RoleBinding from OpenShell Namespace preparation with the worker ServiceAccount above.

Grant the API credential and configuration roles, then the worker runtime role.

tenant_kubectl create rolebinding openclaw-enterprise-api-secrets \  --clusterrole="$RELEASE-openclaw-tenant-api" --serviceaccount="$API_SA"tenant_kubectl create rolebinding openclaw-enterprise-api-configuration \  --clusterrole="$RELEASE-openclaw-tenant-configuration" --serviceaccount="$API_SA"tenant_kubectl create rolebinding openclaw-enterprise-worker \  --clusterrole="$RELEASE-openclaw-tenant-worker" --serviceaccount="$WORKER_SA"

The Secret role also grants Deployment list access for credential preflight and Pod read/proxy access for diagnostics. With agentRuntimeLogs.enabled (default), it grants pods/log get and events get,list for Agent logs. Custom roles need the same rules; missing log grants return 503 RUNTIME_LOGS_CLUSTER_RBAC. Missing credential grants return 503 RUNTIME_CREDENTIALS_CLUSTER_RBAC; API logs agent_runtime_credentials.cluster_denied identify the denied call and namespace. OCC IAM grants remain required. Workload ServiceAccounts receive no Secret API access. Wait for Namespace ready after these grants. Namespace workload managers are trusted for both roles; use disjoint trusted Gateway and Harness node pools. For the experimental two-cluster profile, grant these storage roles in the separately discovered control target and worker/observer roles in the execution target; see two-cluster testing.

Prepare each Agent

Prepare Agent deployment after the Namespace is ready. The operator shell must have OCC_URL, OCC_SERVICE_KEY_FILE, OCC_NAMESPACE, NAMESPACE_ID, TENANT_NAMESPACE, KUBECONFIG_FILE, and CONTEXT set. TENANT_NAMESPACE is the Kubernetes namespace created by the driver during Namespace preparation.

Configure the Agent runtime

Both examples enable Control UI. Their explicit loopback origins allow the first version to start without trusting an arbitrary browser host. After deployment, finish Control UI access by adding this Agent's exact HTTPS origin. Loopback origins alone do not enable the OCE native admin link.

Complete private routing first. Configure verified Envoy source CIDRs in the trusted Installation YAML before deploying either example. Kubernetes Compute renders native trusted-proxy authentication for Console workspace access; the examples omit Driver-owned settings. Follow the native authentication requirements for proxy identity and NetworkPolicy isolation; do not trust arbitrary client addresses. Dedicated Codex also requires the matching runtime images.

Choose one runtime mode and write the matching Namespace-owned kind: "agent" Configuration. Use embedded for built-in OpenClaw:

export AGENT_EXECUTION_MODE='embedded'cat > configuration.json <<'JSON'{  "kind": "agent",  "values": {    "gateway": {      "mode": "local",      "bind": "lan",      "controlUi": {        "enabled": true,        "allowedOrigins": [          "http://127.0.0.1:18789",          "http://localhost:18789"        ]      },      "auth": {        "password": {          "source": "env",          "provider": "default",          "id": "OPENCLAW_GATEWAY_PASSWORD"        }      },      "http": {        "endpoints": {          "chatCompletions": {            "enabled": true          }        }      }    },    "agents": {      "defaults": {        "model": "openai/gpt-6-astra",        "skipBootstrap": true,        "models": {          "openai/gpt-6-astra": {            "agentRuntime": {              "id": "openclaw"            }          }        }      }    },    "models": {      "providers": {        "openai": {          "baseUrl": "https://api.openai.com/v1",          "api": "openai-responses",          "models": [            {              "id": "gpt-6-astra",              "name": "gpt-6-astra"            }          ]        }      }    }  }}JSON

Or use dedicated for the Codex runtime and its app-server placeholders. The Configuration keeps transport placeholders separate from harness authentication and does not contain OPENAI_API_KEY:

export AGENT_EXECUTION_MODE='dedicated'cat > configuration.json <<'JSON'{  "kind": "agent",  "values": {    "gateway": {      "mode": "local",      "bind": "lan",      "controlUi": {        "enabled": true,        "allowedOrigins": [          "http://127.0.0.1:18789",          "http://localhost:18789"        ]      },      "auth": {        "password": {          "source": "env",          "provider": "default",          "id": "OPENCLAW_GATEWAY_PASSWORD"        }      },      "http": {        "endpoints": {          "chatCompletions": {            "enabled": true          }        }      }    },    "agents": {      "defaults": {        "model": "codex/gpt-6-astra",        "skipBootstrap": true,        "models": {          "codex/gpt-6-astra": {            "agentRuntime": {              "id": "codex"            }          }        }      }    },    "models": {      "providers": {        "codex": {          "baseUrl": "http://127.0.0.1:9",          "api": "openai-responses",          "models": [            {              "id": "gpt-6-astra",              "name": "gpt-6-astra"            }          ]        }      }    },    "plugins": {      "allow": [        "codex"      ],      "entries": {        "codex": {          "enabled": true,          "config": {            "appServer": {              "mode": "guardian",              "approvalPolicy": "on-request",              "sandbox": "read-only",              "transport": "websocket",              "url": "${APP_SERVER_URL}",              "authToken": "${APP_SERVER_TOKEN}"            }          }        }      }    }  }}JSON

When using native model fallbacks, keep them on the primary model provider and give each model a compatible Harness policy. OCC validates the full selection and preserves its fallback order; mixed Harnesses are rejected. See native runtime selection.

If you customize configuration.json and plan to verify a local model reply, check these settings before saving it. The OpenClaw TUI uses the values.gateway.auth.password SecretRef shown in both examples. The loopback HTTP check also needs values.gateway.http.endpoints.chatCompletions.enabled set to boolean true. Agents using neither check can omit these settings.

Post the Configuration and capture the server-generated ID:

export OCC_NAMESPACE="$NAMESPACE_ID"CONFIGURATION_RESPONSE="$(occ configuration create --file configuration.json --output json)" &&CONFIGURATION_ID="$(printf '%s' "$CONFIGURATION_RESPONSE" | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')" &&export CONFIGURATION_ID

Create the Agent with the captured Configuration ID and the matching execution mode. Mismatched Harness and mode pairs fail before deployment. Create a Namespace-owned OCC Secret containing the credential; set HARNESS_SECRET_ID to its data.id. Choose api_key, or codex_pat for Dedicated Codex with an externally issued Codex service-account token. Neither requires a Backend. The caller needs exact Secret operate. For OCE-managed accounts, use the account/Backend binding in Harness authentication.

: "${AGENT_EXECUTION_MODE:?choose embedded or dedicated above}": "${HARNESS_SECRET_ID:?set the OCC Secret ID containing the credential}"export HARNESS_SECRET_IDexport HARNESS_AUTH_METHOD='api_key' # Or codex_pat for Dedicated Codex.printf '{"name":"production-agent","configurationId":"%s","executionMode":"%s","harnessAuth":{"method":"%s","source":{"kind":"secret","namespaceId":"%s","id":"%s"}}}\n' \  "$CONFIGURATION_ID" "$AGENT_EXECUTION_MODE" "$HARNESS_AUTH_METHOD" "$NAMESPACE_ID" "$HARNESS_SECRET_ID" > agent.jsonAGENT_RESPONSE="$(occ agent create --file agent.json --output json)" &&AGENT_ID="$(printf '%s' "$AGENT_RESPONSE" | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')" &&AGENT_SERVICE_PRINCIPAL_ID="$(printf '%s' "$AGENT_RESPONSE" | python3 -c 'import json,sys; print(json.load(sys.stdin)["servicePrincipalId"])')" &&export AGENT_ID AGENT_SERVICE_PRINCIPAL_ID

Check saved settings for local model verification

If you used either example unchanged, continue to the grant below. To use a local check with a reused or customized Configuration, check the settings that the saved Agent references before granting model Secret access or deploying. The caller needs read on the Agent and its Namespace Configuration. Set AGENT_ID to the saved Agent's ID and pass tui or http.

OCC_NAMESPACE="${NAMESPACE_ID:?}" python3 - "${AGENT_ID:?}" tui <<'PYTHON'import json, subprocess, sys agent_id, route = sys.argv[1:]if route not in ("tui", "http"):    sys.exit("Choose tui or http.") def read(kind, resource_id):    return json.loads(subprocess.check_output(        ["occ", kind, "get", resource_id, "--output", "json"],        stderr=subprocess.DEVNULL,    )) try:    configuration_id = read("agent", agent_id)["configurationId"]    if not isinstance(configuration_id, str) or not configuration_id:        sys.exit("The saved Agent has no readable Configuration ID.")    configuration = read("configuration", configuration_id)except (OSError, subprocess.CalledProcessError, KeyError, TypeError, ValueError):    sys.exit("Could not read the saved settings. Check the CLI setup, IDs, and read access.") try:    gateway = configuration["values"]["gateway"]    password = {"source": "env", "provider": "default", "id": "OPENCLAW_GATEWAY_PASSWORD"}    if configuration["kind"] != "agent" or gateway["auth"]["password"] != password:        sys.exit("Use the kind and gateway password SecretRef from the example.")    if route == "http" and gateway["http"]["endpoints"]["chatCompletions"]["enabled"] is not True:        sys.exit("Enable chat completions for the HTTP check.")except (KeyError, TypeError):    sys.exit("Add the missing settings for the chosen check.")print("Saved Configuration matches the chosen check.")PYTHON

A match confirms the saved fields. Complete the required grants and deployment, then verify a model reply.

Grant the Agent access to its model Secret

The deploying caller and the Agent's service principal both need operate on the exact Secret. To grant the Agent access, use a credential with Installation administer, read on this Namespace, and read on the exact Secret. The fresh bootstrap service key has these permissions with native IAM unless a Restriction denies access. If you use a more limited credential, give NAMESPACE_ID, AGENT_ID, and HARNESS_SECRET_ID to an Installation administrator. They can look up the service principal and then run the commands below. Kubernetes RoleBindings do not grant OCC access.

export NAMESPACE_ID AGENT_ID HARNESS_SECRET_IDexport OCC_NAMESPACE="$NAMESPACE_ID"AGENT_LOOKUP="$(occ agent get "$AGENT_ID" --output json)" &&AGENT_SERVICE_PRINCIPAL_ID="$(printf '%s' "$AGENT_LOOKUP" | python3 -c 'import json,sys; print(json.load(sys.stdin)["servicePrincipalId"])')" &&export AGENT_SERVICE_PRINCIPAL_ID

Create the Role once per Namespace, or use occ iam role list --output json to find a Role with exactly the permission shown and export its ID as IAM_ROLE_ID:

cat > model-secret-role.json <<'JSON'{"name":"Use a model Secret","permissions":[{"action":"operate","resourceKind":"secret"}]}JSONIAM_ROLE_RESPONSE="$(occ iam role create --file model-secret-role.json --output json)" &&IAM_ROLE_ID="$(printf '%s' "$IAM_ROLE_RESPONSE" | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')" &&export IAM_ROLE_ID

Bind the Role to this Agent and this Secret, then read the stored binding:

: "${NAMESPACE_ID:?}" "${AGENT_SERVICE_PRINCIPAL_ID:?}" "${IAM_ROLE_ID:?}" "${HARNESS_SECRET_ID:?}"python3 -c 'import json,os; print(json.dumps({"subjectKind":"identity","subjectId":os.environ["AGENT_SERVICE_PRINCIPAL_ID"],"roleId":os.environ["IAM_ROLE_ID"],"resourceKind":"secret","resourceId":os.environ["HARNESS_SECRET_ID"]}))' > model-secret-binding.json &&IAM_BINDING_RESPONSE="$(occ iam access-binding create --file model-secret-binding.json --output json)" &&IAM_BINDING_ID="$(printf '%s' "$IAM_BINDING_RESPONSE" | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])')" &&export IAM_BINDING_ID IAM_BINDING="$(occ iam access-binding get "${IAM_BINDING_ID:?}" --output json)" &&printf '%s' "$IAM_BINDING" | python3 -c 'import json, os, sysbinding = json.load(sys.stdin)expected = {    "namespaceId": os.environ["NAMESPACE_ID"],    "subjectKind": "identity",    "subjectId": os.environ["AGENT_SERVICE_PRINCIPAL_ID"],    "roleId": os.environ["IAM_ROLE_ID"],    "resourceKind": "secret",    "resourceId": os.environ["HARNESS_SECRET_ID"],}if any(binding.get(key) != value for key, value in expected.items()):    sys.exit("Binding does not match this Agent and Secret.")print("Exact Agent Secret binding is stored")'

Continue only after the confirmation. If a create response is lost, inspect occ iam role list --output json or occ iam access-binding list --output json before retrying; duplicate bindings can coexist. The stored binding does not override a matching Restriction. See Namespace IAM policy.

Prepare transport credentials and deploy

Use a separate operator shell with the production OCC_URL, Namespace, resource IDs, and Kubernetes context. Set OCC_SERVICE_KEY_FILE to the private service API key response file for a non-Agent service principal in this Namespace. An Installation administrator who covers its grants can issue the key, which uses the principal's current permissions. Leave the bootstrap copy in the Installation shell for cleanup.

Kubernetes Compute creates missing transport credentials during the first deployment. The caller needs read and operate on this Agent for that step. See initial runtime credentials to provision credentials separately or investigate a conflict.

Model authentication comes from the saved harnessAuth binding and is projected only into the model-executing workload. Slack tokens use separate Namespace Secrets; follow Slack setup. Keep credential values out of Helm values, Installation YAML, Configurations, shell history, and this repository.

Deployment requires deploy on this Agent and read on its saved Configuration. Both the caller and the Agent's service principal need operate on the model Secret and each Configuration secretBindings Secret.

For this new, stopped Agent, review the saved Configuration and note its generation. Keep the caller identity, Namespace ID, Agent ID, Configuration ID, generation, and nonsecret request key in your private operator record.

occ configuration get "$CONFIGURATION_ID" --output jsonread -r -p 'Reviewed Configuration generation: ' CONFIGURATION_GENERATIONFIRST_DEPLOYMENT_KEY=production-agent-first

Submit the guarded first deployment once. OCC checks that the Agent has no revision history and still uses the Configuration and generation you reviewed.

REVISION_RESPONSE="$(occ agent deploy-first "$AGENT_ID" --configuration "$CONFIGURATION_ID" \  --generation "$CONFIGURATION_GENERATION" --key "$FIRST_DEPLOYMENT_KEY" --output json)" &&REVISION_ID="$(printf '%s' "$REVISION_RESPONSE" | python3 -c 'import json,sys; print(json.load(sys.stdin)["revisionId"])')" &&export REVISION_ID &&occ agent deployment-status "$AGENT_ID" "$REVISION_ID"

If the reply is lost, recover the original first-deployment receipt with the same caller and recorded inputs. A missing receipt leaves the outcome unknown. Do not submit another deployment.

A receipt identifies the admitted revision. Repeat the status check until it reports succeeded or failed. If status access is denied, ask an Installation administrator for read on that agent_revision. After success, verify workspace access and a model response below.

Verify workspace access

After deployment, complete Verify routing and file access: require the Gateway and Agent HTTPRoute to be accepted, TLS certificates ready, and a successful read through the OCC workspace-file API. In the Console, open the Agent and select Reload AGENTS.md. An existing file should load without Workspace access is unavailable. The caller needs exact-Agent read permission; saving also requires operate.

An empty editor after an error is not evidence of an empty workspace. A missing file is a separate result: the file API can create or replace a file, but cannot delete it. Verify access before creating a missing file. See workspace-file errors. Next, verify a model response from the same version.

Verify production workloads

Wait for GET /namespaces/$NAMESPACE_ID/agents/$AGENT_ID to report the expected activeRevisionId, then require a real model response from that Agent. Use its optional loopback password to attach with the OpenClaw TUI, or verify rejection of an unauthenticated request and a real model response over an operator's local Kubernetes connection.

If model access fails, check the Pod's network profile: model egress requires an explicit grant. See what each check establishes.

Then check Agent network isolation by probing from inside the workload Pods.

Attach with the OpenClaw TUI

Follow TUI verification to select exactly one Ready gateway Pod for REVISION_ID, send a fresh nonce, and verify its reply. Keep the same operator environment and return here to clean up temporary credentials.

End the operator session

After model verification, remove only temporary delivery copies created by these guides. The command leaves a caller-supplied OCC_SERVICE_KEY_FILE and the protected original OCC_BOOTSTRAP_KEY_FILE untouched:

case "${OCC_SERVICE_KEY_DIRECTORY:-}" in  /tmp/occ-service-key.[[:alnum:]][[:alnum:]][[:alnum:]][[:alnum:]][[:alnum:]][[:alnum:]][[:alnum:]][[:alnum:]])    if [ -n "${OCC_BOOTSTRAP_KEY_FILE:-}" ] &&       [ "${OCC_SERVICE_KEY_FILE:-}" = "$OCC_SERVICE_KEY_DIRECTORY/occ-service-key.json" ] &&       [ "$OCC_SERVICE_KEY_FILE" != "$OCC_BOOTSTRAP_KEY_FILE" ] &&       [ -d "$OCC_SERVICE_KEY_DIRECTORY" ] && [ ! -L "$OCC_SERVICE_KEY_DIRECTORY" ]; then      rm -f -- "$OCC_SERVICE_KEY_FILE" && rmdir -- "$OCC_SERVICE_KEY_DIRECTORY"    else      printf '%s\n' 'Service-key path was not the documented temporary copy; nothing was deleted.' >&2    fi ;;  '') ;;  *) printf '%s\n' 'Service-key directory was not created by this guide; nothing was deleted.' >&2 ;;esacunset OCC_SERVICE_KEY_FILE OCC_SERVICE_KEY_DIRECTORY case "${SECRET_DIRECTORY:-}" in  /tmp/occ-agent-transport.[[:alnum:]][[:alnum:]][[:alnum:]][[:alnum:]][[:alnum:]][[:alnum:]][[:alnum:]][[:alnum:]])    if [ -d "$SECRET_DIRECTORY" ] && [ ! -L "$SECRET_DIRECTORY" ]; then      rm -f -- "$SECRET_DIRECTORY/app-server-token" "$SECRET_DIRECTORY/gateway-password" &&      rmdir -- "$SECRET_DIRECTORY"    fi ;;  '') ;;  *) printf '%s\n' 'Transport directory was not created by this guide; nothing was deleted.' >&2 ;;esacunset SECRET_DIRECTORY

This does not revoke the service key or delete the Kubernetes Secrets. Keep the original in protected bootstrap storage; bootstrap will not reissue it.

Open Control UI

After model verification, optionally enable native admin access and configure this Agent's exact origin. You need a browser session with exact-Agent administer permission. Add the returned origin to gateway.controlUi.allowedOrigins and deploy again; then open Native admin UI and verify the native UI loads.

Related

• Private routing for workspace files.
• Troubleshoot the platform.
• Stop or remove a production deployment.
• Production TUI flow.