OpenClaw EnterpriseDOCSGitHub

Local Kubernetes development

Run OpenClaw Enterprise (OCE) in a disposable, loopback-only k3d cluster. The Kubernetes-only profile runs PostgreSQL, the OpenClaw Control Plane (OCC), and Agents.

Start the profile

Install Linux tools: Node.js satisfying engines.node in package.json, repository-pinned pnpm, the Go version from go.mod, k3d, kubectl, Helm, and Docker or Podman. Kubernetes-only mode uses the engine for k3d and images, not OCE services.

K3s requires the cpuset cgroup controller, which systemd does not delegate to a rootless session. On Podman, run as root or use a rootful Podman machine; native rootless Podman cannot start this profile even with cpuset delegated. On macOS:

bash
podman machine stoppodman machine set --rootfulpodman machine start

Rootful describes the virtual machine; run podman as your normal host user. Rootful and rootless keep separate container storage, so the first start after switching rebuilds the images.

On Linux without root, use a rootful machine, which needs /dev/kvm, gvproxy, and virtiofsd. List a helper directory outside Podman's defaults, plus those defaults, in [engine] helper_binaries_dir of a CONTAINERS_CONF_OVERRIDE file exported for startup and cleanup:

bash
podman machine init oce-dev --rootful --cpus 8 --memory 16384podman machine start oce-devexport CONTAINER_CONNECTION=oce-dev-root

The machine shares only $HOME by default, so set OCC_DEVELOPMENT_STATE_DIRECTORY beneath it.

For failed to find cpuset cgroup (v2), including Docker inside a containerized development host, follow local cgroup troubleshooting.

Startup resolves the engine's host API socket. Do not export DOCKER_HOST or CONTAINER_HOST from the path podman info reports: on a machine-backed installation that path exists only inside the virtual machine.

Build the CLI and start the Kubernetes-only profile, which needs no selection:

bash
go build -trimpath -o bin/occ ./cmd/occ./scripts/dev-up

The launcher installs PostgreSQL and OCE, without OpenShell, on pinned K3s in oce-system and writes the administrator password and service key to private state. It prepares routing prerequisites and matching Helm/Installation settings before bootstrap.

Before bootstrapping, startup checks the Codex 0.164.0 sandbox with its exact imported runtime image. If the node's RuntimeDefault blocks it, the launcher derives a reviewed profile from the node's actual policy and installs it only on the owned k3d node. Only dedicated Codex containers select it. Startup verifies workspace-write and outside-write boundaries and missing-profile failure. Private codex-seccomp-provenance.json records hashes and node provenance. Unsupported policy, runtime, or verification fails startup and rolls back the owned cluster. Ubuntu 24.04: see troubleshooting. Recreate the installation after runtime, kernel, or image changes.

To enable GitHub repository credentials during a fresh start, prepare the approved local repository inputs before running the launcher.

Select another profile

Set OCC_DEVELOPMENT_CONTROL_PLANE=compose for Compose OCC, or OCC_DEVELOPMENT_COMPUTE_DRIVER=docker for the Docker Compute preview, which cannot deploy Agents. See the profile table.

Sign in through Keycloak

Set OCC_DEVELOPMENT_SIGN_IN=keycloak for Keycloak sign-in. The engine must bind free host port 127.0.0.1:443 and make it reachable from the host; other published ports must differ. k3d publishes ports at creation; changes require dev-down and fresh dev-up.

The persistent realm attaches alice to the existing administrator through OIDC, makes passwords recovery-only, and keeps Agent native administration enabled. OpenShell is unsupported; password-only remains the default.

Import browser-ca.crt and gateway-ca.crt into your browser; add the printed /etc/hosts entry. In Console, choose Continue with Keycloak; use alice with keycloak-alice-password. Keep the administrator password for recovery.

dev-down destroys the realm volume and cluster. If cleanup retains state, repair the reported access problem and retry with the same profile and directory. Remove host entries and imported CAs separately. The Keycloak lifecycle covers Helm, recovery, installation-only drift checks and certificate copying.

Run OCC in Compose with Kubernetes compute

To select the hybrid profile on a fresh state directory:

bash
export OCC_DEVELOPMENT_COMPUTE_DRIVER=kubernetesexport OCC_DEVELOPMENT_CONTROL_PLANE=composeexport OCC_DEVELOPMENT_SANDBOX_DRIVER=none./scripts/dev-up

Expect Control plane: Compose and Compute Driver: Kubernetes. PostgreSQL, OCC, and its worker run in Compose; Agents run in k3d. This profile includes both default Presets, the Codex Plugin Driver, and the same Codex seccomp preparation, but not the Kubernetes-only browser endpoint, private workspace routing, or repository service. Dedicated Codex requires hybrid private routing, configured before creating Agent Namespaces; that page also covers Compose repository and Slack service connections.

If the K3s channel lookup fails or times out, follow local image-lookup troubleshooting.

Start the OpenShell fail-closed profile

bash
go build -trimpath -o bin/occ ./cmd/occexport OCC_DEVELOPMENT_SANDBOX_DRIVER=openshell./scripts/dev-up

The CLI creates one k3d cluster with pinned OpenShell v0.1.8, Agent Sandbox, cert-manager, and Envoy Gateway assets. PostgreSQL, the central OpenShell Gateway, and OCE run in oce-system; Agent Sandbox runs in agent-sandbox-system. Tenant Workspaces, Sandboxes, and Agent Pods use OCC-owned oce-* Namespaces. Startup exposes loopback, waits for Namespace/Workspace readiness, and privately writes kubeconfig and an administrator key.

Agent Sandbox reuses verified native content from cached children or indexes; cache misses pull on the host. Custom manifests retain image/pull policy.

Preload failures report the host import error before controller creation. Rollout failures capture controller status, image IDs and Warning reasons when available, excluding messages, environment values and logs.

Set OCC_DEVELOPMENT_CONTROL_PLANE=compose to run PostgreSQL, OCC, and its Kubernetes worker in Compose. Startup installs private Envoy routing in k3d, mounts its service key and public CA only into the controller and worker-kubernetes, and records the node hostname and Envoy NodePort in the Installation. It enables OCC native administration, publishes loopback HTTPS for console and Agent hosts, and prints the browser CA path. OCC_DEVELOPMENT_BROWSER_PORT must differ from Compose and k3d API ports. Skip manual hybrid routing.

Runtime changes need explicit source verification.

Image preparation requires host registry access. For reviewed local charts and manifests, set OCC_DEVELOPMENT_OPENSHELL_HELM_CHART, OCC_DEVELOPMENT_OPENSHELL_WORKSPACE_HELM_CHART, and OCC_DEVELOPMENT_OPENSHELL_AGENT_SANDBOX_MANIFEST to absolute paths. To test a candidate release, select its OpenShell images by digest.

See startup network policy for binary/endpoint rules.

Set OCC_DEVELOPMENT_CONTAINER_ENGINE=podman or docker to select the host engine. Kubernetes-only startup rejects Compose arguments.

Startup prints its private state directory containing kubeconfig and credentials. Its default is openclaw-development under the temporary directory (macOS: /private/var/folders/<id>/T). Set absolute OCC_DEVELOPMENT_STATE_DIRECTORY for startup and cleanup to override it; parent paths cannot contain symlinks (macOS: use /private/tmp/..., not /tmp/...). OCC_DEVELOPMENT_STARTUP_TIMEOUT_SECONDS bounds cluster readiness and later waits. Failures roll back owned resources; retained state includes cleanup instructions. Failed exclusive key writes remove partial outputs, never existing files, and report removal errors alongside write errors.

Startup refuses existing state or clusters. For source edits, rebuild services; cleanup discards the Installation. State stays mode 0700; generated mounts are container-readable but inaccessible to other host users. The default kubeconfig and context stay unchanged.

For separate stacks, select distinct state directories, cluster names, bridge subnets and published ports. Compose also needs a distinct OCC_POSTGRES_PORT; changing the API port alone leaves PostgreSQL on port 55432. Each Agent Gateway requests 2560 MiB with a 4 GiB limit, and each dedicated Codex Harness requests 768 MiB with a 6 GiB limit, so a dedicated Codex Agent reserves 3.25 GiB, or 5 GiB with initial workspace files; size the local engine VM for OCC plus the Agents you run (measurements). Keep each stack's resources under the helper's lifecycle until cleanup.

Require both proxies before enabling Slack

For a Slack-enabled local installation, complete both Slack proxy paths before creating or deploying the Agent. The launcher neither provisions these proxies nor configures their URLs.

Provision the reviewed proxy on the owned k3d container network (no host-published listener needed) or at another private address reachable from the API and dedicated gateway Pods. Do not use host loopback: 127.0.0.1 inside a Pod is that Pod. Restrict its source access to this cluster's actual traffic, accounting for node source NAT.

Back up both generated files in the private state directory, then update them:

Apply the edits to the existing release as shown below. Use the chart source matching the installed controller, retain its image references and other protected inputs, and plan a maintenance window if Agents are running. Confirm the release and namespace; the values below are launcher defaults. The checksum rolls the API and worker even when only the Installation document changed. After an in-place image upgrade, helm-values.json still names the bring-up images: make both edits in that page's recovered INSTALLATION and VALUES and apply them with apply other Installation changes instead.

bash
set -euo pipefailumask 077export OCC_SLACK_STATE='<existing private state directory>'export OCC_SLACK_CONTEXT='<context printed by scripts/dev-up>'export OCC_SLACK_NAMESPACE='oce-system'export OCC_SLACK_RELEASE='openclaw-enterprise' node --input-type=module <<'NODE'import { createHash } from 'node:crypto';import { readFileSync, writeFileSync } from 'node:fs';import { join } from 'node:path';const root = process.env.OCC_SLACK_STATE;const path = join(root, 'helm-values.json');const values = JSON.parse(readFileSync(path, 'utf8'));values.controlPlane ??= {};values.controlPlane.installationChecksum = createHash('sha256')  .update(readFileSync(join(root, 'installation.yaml'))).digest('hex');writeFileSync(path, JSON.stringify(values, null, 2) + '\n');const document = readFileSync(join(root, 'installation.yaml')).toString('base64');writeFileSync(join(root, 'installation-secret-patch.json'),  JSON.stringify({ data: { 'installation.yaml': document } }));NODE kubectl --kubeconfig "$OCC_SLACK_STATE/kubeconfig" --context "$OCC_SLACK_CONTEXT" \  -n "$OCC_SLACK_NAMESPACE" patch secret occ-installation-startup \  --type merge --patch-file "$OCC_SLACK_STATE/installation-secret-patch.json"rm "$OCC_SLACK_STATE/installation-secret-patch.json" helm upgrade "$OCC_SLACK_RELEASE" deploy/helm/openclaw-enterprise \  --kubeconfig "$OCC_SLACK_STATE/kubeconfig" --kube-context "$OCC_SLACK_CONTEXT" \  --namespace "$OCC_SLACK_NAMESPACE" -f "$OCC_SLACK_STATE/helm-values.json" \  --wait --timeout 5m

Do not rerun dev-up, recreate bootstrap storage, or delete the cluster to apply this configuration. After OCC recovers, deploy a new version for each affected running Slack Agent so its gateway receives the new proxy and network policy. Verify directory search and gateway Socket Mode using the Slack checks.

Verify the local boundary

Use the printed API and kubeconfig paths:

bash
export KUBECONFIG="<Kubeconfig path printed by scripts/dev-up>"kubectl get pods -A export OCC_URL="<API URL printed by scripts/dev-up>"export OCC_SERVICE_KEY_FILE="<Service key file printed by scripts/dev-up>"./bin/occ installation get

In Kubernetes-only mode, the API is reachable only through the loopback k3d publication, whose Service selects a dedicated in-cluster proxy admitted by exact Namespace and Pod labels in the OCE Helm NetworkPolicy. The OCE API itself remains a ClusterIP Service, and the worker authenticates to Kubernetes in-cluster. With OpenShell, the API (which registers credential sources), the worker, and dedicated Agent Gateways reach OpenShell Gateway through a narrow development NetworkPolicy in oce-system.

The launcher sets network.pluginStatusProxySourceCidrs to the k3d node's Pod bridge address, enabling plugin status and diagnostics and letting a dedicated Codex Gateway start once on a first deploy. The two-minute bridge-route wait also bounds each container-engine lookup.

The OpenShell profile declares an openshell Backend for the Gateway endpoint and selects both the OpenShell Sandbox and the OpenShell Credential Gateway, bound to the runtime image's native Codex executable. Agents in this profile must authenticate through a credential source; to register a key and bind it to an Agent, follow Use a credential source on the local OpenShell profile.

The profile supports managed Codex plugins.

The Kubernetes-only OpenShell Gateway requires TLS, mTLS and authentication. Static-inference enforcement is separate and off by default; it is not required for model access or protected Git. The Compose-backed profile still permits unauthenticated Gateway users inside its disposable, loopback-owned cluster. Do not carry that setting into a shared cluster or container network.

Because this cluster is disposable and single-profile, the helper binds the chart's tenant worker, configuration, and Secret ClusterRoles to the OCE service accounts cluster-wide. This is not a production RBAC pattern: production and shared clusters must use the tenant-local RoleBindings in the production deployment guide.

To verify a fresh Kubernetes-only installation without OpenShell, use the real local installation test.

To prove OpenShell setup and cleanup in a separate fresh cluster, stop the reusable environment, install the test dependencies from the trusted repository root, and run:

bash
pnpm install --frozen-lockfileOCC_TEST_DEV_UP_OPENSHELL_REAL=1 \  node --test tests/integration/dev-up-openshell-k3d-real.test.mjs

The selected real test must pass without a skip. It verifies the Helm-installed OCE control plane and PostgreSQL Pods, bootstrap and new Namespace Workspace reconciliation, immutable image registration, and owned-cluster cleanup, but creates no Agent and performs no model turn; for that credentialed proof, follow the Kubernetes model-turn procedure.

Configure workspace storage on single-node k3d

Dedicated Agents use separate RWO workspace and Gateway claims on local-path storage. Deployment stops the previous version, causing downtime; see storage ownership and recovery.

Check the Agent namespace with the printed kubeconfig and context:

bash
kubectl --kubeconfig '<profile-kubeconfig>' --context '<profile-context>' \  -n '<agent-kubernetes-namespace>' get pvc,pods

Expect a Bound RWO claim, then a running Harness Pod. With WaitForFirstConsumer, claims remain pending until Pod creation. RWX claims are unsupported. Do not change a PVC's access mode in place.

Preserve storage across restarts

The node's /var/lib/rancher/k3s volume holds workspace files and K3s state. Restarts keep it; deleting the cluster, volume, or profile destroys it. Local-path storage is node-bound; cross-node workloads need a portable StorageClass.

PostgreSQL is a StatefulSet, and a launcher NetworkPolicy allows egress by its Pod label and the k3d subnet, so the profile survives k3d cluster stop/start.

The local-path reclaim policy is Delete: deleting a claim may delete its backing directory. Nothing here backs up Agent workspaces. Long-lived demos need a durable private state directory, not /tmp.

Rebuild after a source edit

Controller builds record Git HEAD; read it with Debug mode in Console Settings. See controller image builds for standalone builds and source metadata limits.

The environment registers immutable image digests, so running containers do not pick up source edits. Recreate it after controller, runtime, migration, Helm, or OpenShell integration changes; startup rebuilds both images from the current checkout unless an existing controller or runtime image was selected explicitly:

bash
./scripts/dev-down./scripts/dev-up

This discards the owned Installation; it is not an in-place upgrade. If repository access was enabled, check the credential cleanup limitation before removing the cluster. For a persistent Helm-installed k3d environment, complete the upgrade migration checklist, then the local k3d image upgrade procedure.

Resolve node DNS failures

On Linux Docker, the launcher gives its k3d node the host's first non-loopback IPv4 upstream resolver, because k3d's default refuses queries on iptables-nft hosts. If the node still cannot resolve image registries, set a DNS server reachable from the node network for a fresh startup:

bash
OCC_DEVELOPMENT_K3D_DNS_RESOLVER='<reachable-dns-ip>' ./scripts/dev-up

The host resolver is unchanged; the setting reference covers its node effect and the k3d value.

Stop and clean up

If repository access was enabled, first follow the credential cleanup guidance; the launcher does not verify credential disposal before removing the cluster. Then remove only the cluster and private state the launcher recorded:

bash
./scripts/dev-down

Cleanup uses the recorded engine endpoint and cluster name. If deletion fails, it keeps the state directory so a retry never discovers or deletes an unrelated cluster.

Limits

Both local k3d profiles use K3s legacy iptables mode. Kubernetes-only startup without OpenShell checks policy traffic before configuring gateway proxy trust and again against the initial Gateway Namespace. These startup checks cover only the tested single-node traffic and do not monitor later policy failures.

Startup readiness does not prove Agent deployment, model execution, provider authentication, or dedicated Codex WebSocket execution; OpenShell startup proves only its infrastructure and fail-closed boundary. The Kubernetes testing guide covers those credentialed real-cluster checks.

Gateway placement boundary

Dedicated Gateways and Harnesses share one tenant namespace but retain separate Pods, ServiceAccounts, credentials and storage. The local profile schedules Gateways on its single k3d server, which does not prove production node isolation. Production must configure runtime.gatewayNodeSelector and runtime.nodeSelector for disjoint trusted and data-plane pools. See production Namespace preparation for the scoped RoleBindings.

Search documentation