OpenClaw EnterpriseDOCSGitHub

Compose development startup

Overview

scripts/dev-up runs PostgreSQL, migration, bootstrap, API, and worker in Compose or an owned k3d cluster, by default the Kubernetes-only profile (internal/occdev/profile.go:ResolveProfile). Startup requires authenticated Installation and bootstrap Namespace readiness, plus Workspace readiness for OpenShell.

Entry Points

Flow

graph TD
  A["<b>Start development</b><br/>scripts/dev-up"] --> B{"<b>Compute Driver</b><br/>Docker or Kubernetes"}
  B -- "Docker" --> C["<b>Render Compose</b><br/>Validate publications"]
  B -- "Kubernetes" --> D{"<b>Control plane</b><br/>Compose or Kubernetes"}
  D -- "Kubernetes (default)" --> E["<b>Owned k3d stack</b><br/>PostgreSQL and OCE"]
  D -- "Compose" --> F["<b>Hybrid stack</b><br/>Compose OCC and k3d Compute"]
  C --> G["<b>Prove Installation</b><br/>Authenticated service key"]
  F --> F2{"<b>OpenShell selected?</b>"}
  F2 -- "No" --> R["<b>Resolve Pod proxy source</b><br/>Bound bridge-route queries"]
  F2 -- "Yes" --> F3["<b>Expose browser endpoint</b><br/>Loopback HTTPS console and Agent hosts"]
  F3 --> F4["<b>Admit exact browser origin</b><br/>Trusted bridge peer only"]
  F4 --> R
  F3 --> H["<b>Own Workspace</b><br/>OpenShell operator mode"]
  E --> R
  R -- "Validated cni0 source" --> G
  R -- "Deadline or query failure" --> X["<b>Fail startup</b><br/>Skip Installation writing"]
  E -- "OpenShell" --> H
  G -- "Password-only" --> I["<b>Record cleanup</b><br/>Exact engine and resources"]
  G -- "Canceled" --> J["<b>Fail startup</b><br/>Clean owned resources"]
  H --> I
  E -- "Optional Keycloak profile" --> K["Provision Keycloak, TLS and Pod DNS"]
  K --> G
  G -- "Keycloak ready" --> L["Second Helm pass: OIDC and recovery-only passwords"]
  L --> M["Attach Alice to the existing administrator"]
  M --> I

Execution trace

1. scripts/dev-up: host preflight and runtime image selection

scripts/dev-up:require_command, internal/occdev/up.go:Up, internal/occdev/compose.go:AnalyzeCompose, deploy/runtime/Dockerfile

bin/occ (pnpm cli:build) is required; arguments after -- go to Compose. For Kubernetes Compute, see step 12. Without a Sandbox Driver, Compose Kubernetes startup requires Node before creating resources.

Engine and Compose JSON probes select Docker or Podman independently of a docker alias. Podman requires yq v4 and a Compose provider. Docker Compute requires standalone podman-compose, which scripts/dev-up pins and occ dev down matches so the stack is stopped with the provider that started it. The Kubernetes-profile Compose command that occ dev up and occ dev down drive prefers docker-compose — it passes --project-directory, which standalone podman-compose does not support — and falls back to podman-compose when docker-compose is absent. Rootful Podman on Linux must expose its API socket (for example, systemctl enable --now podman.socket); startup resolves that socket and pins it as DOCKER_HOST for the provider. Explicit -f FILE, -fFILE, -f=FILE, --file FILE, or --file=FILE overrides COMPOSE_FILE and the default file. Startup and cleanup forward attached short options as --file=FILE so Podman does not mistake an h in the path for help. Empty or missing file values fail before configuration rendering.

Docker Compose renders JSON; Podman YAML is converted to JSON in private temporary storage. ./bin/occ dev analyze-compose enforces loopback controller and database publications and resolves Docker runtime image selection. The final compose.podman.yaml override mounts Podman's reported API socket at /var/run/docker.sock in the worker. The Docker Compute worker disables SELinux process labeling because socket relabeling is unsafe; other services retain confinement. Migration, bootstrap, API, and worker share one image. dev-up builds through migration, uses --no-build; Docker uses up --build. Configuration and credentials are never printed. command.go and k3d_import.go report monotonic durations through timing.go. See local command diagnostics; concurrent durations can overlap.

For unselected runtime images, dev-up builds missing openclaw-enterprise-runtime:quickstart from deploy/runtime/Dockerfile at repository root. Custom images must exist; partial selection fails. Tags survive recipe edits; rebuild and verify after package changes. The recipe owns plugin and gateway/Codex compatibility; dev-up neither installs plugins nor proves model turns.

2. compose.yaml:services.postgres and services.migrate

compose.yaml:services.postgres, compose.yaml:services.migrate

Compose starts PostgreSQL with the local occ_postgres_data volume. Local-only administrator credentials initialize the database; checked-in SQL creates the less-privileged occ_migrator and occ_app roles.

Through Compose, dev-up runs migration after PostgreSQL, using OCC_MIGRATION_DATABASE_URL and Drizzle. API and worker never use the migrator or PostgreSQL administrator URL.

3. Initialize before starting the API or worker

compose.yaml:services.bootstrap, scripts/bootstrap-installation.mjs

After migration exits 0, bootstrap creates administrators, Installation, native IAM, audit evidence, service key, and a provisioning default Namespace. The worker provisions its Docker boundary. Existing Installation state is preserved; missing, expired, or revoked keys never trigger reissue.

Only the initializer mounts occ_bootstrap_data; the API and worker load committed state after initializer success. The bootstrap flow owns credentials, concurrent attempts, and failure recovery.

4. The API admits only local development traffic

apps/controller/src/api-startup.ts:startApi, apps/controller/src/composition/development-postgres.ts:composePostgresDevelopment, apps/controller/src/drivers/configuration/filesystem/index.ts:FilesystemConfigurationDriver

The API starts in NODE_ENV=development, binds inside the Compose network, and publishes its host port only on 127.0.0.1. OCC_AUTH_SECRET signs user sessions and OCC_AUTH_BASE_URL fixes the cookie origin. The API and initializer accept HTTP(S) origins on localhost, 127.0.0.1, or [::1] in development.

Development accepts the explicitly configured Compose bridge CIDR as local control-plane traffic, while non-loopback clients, forwarded headers, caller-supplied identity headers, bearer credentials, and trusted-proxy claims remain rejected.

After controller and worker readiness, dev-up copies initializer-owned service-key JSON from the stopped bootstrap container. Docker uses compose cp; podman-compose lacks cp, so Podman selects one bootstrap container by Compose labels and runs podman cp by ID. --key-output requires an absent file in a private operator-owned directory; otherwise startup creates a private temporary directory. It never overwrites files, prints data.key, or bootstraps replacement keys.

./bin/occ installation get uses the copied key. The returned Installation ID must match the key's meta.installationId. apps/controller/src/auth/index.ts:ControllerAdmissionVerifier.verify resolves the Installation-scoped service administrator; IAM authorizes each operation. The service-key flow owns admission and 401 rejection without cookie fallback.

When OCC_CONFIG_PATH is absent, PostgreSQL-backed development selects the filesystem Configuration Driver from OCC_DEVELOPMENT_CONFIGURATION_ROOT. Compose sets that root to /app/.development/configurations and backs it with the occ_configuration_data named volume, mounted only into the controller. PostgreSQL remains the OCC metadata system of record.

5. The worker selects Docker Compute and claims queued operations

apps/controller/src/worker.mjs:configuration, apps/controller/src/drivers/compute/docker/index.ts:DockerComputeDriver.preflight, apps/controller/src/drivers/compute/docker/index.ts:DockerComputeDriver.request

After the controller is healthy, the worker starts with the same application-role OCC_DATABASE_URL. Without OCC_CONFIG_PATH, development selects compute-docker-development with implementation docker-local; otherwise it uses the trusted Driver set in that file.

Before its database pool, preflight runs engine ping, version, and image checks. Response-stream errors reject interrupted requests under the existing 10-second deadline and claim-owned cancellation, with no automatic retry; complete responses keep status and JSON validation.

The worker loads the singleton Installation, validates IAM policy, and polls PostgreSQL work. Readiness proves durable claims, not Agent or TUI existence. Each operation reauthorizes its original actor before Compute. Only the worker receives Docker-compatible engine access.

Kubernetes-only startup

internal/occdev/openshell_k3d.go:upK3d, internal/occdev/gateway_k3d.go:installDevelopmentRoutingControllers, internal/occdev/repository_k3d.go:enableDevelopmentRepository.

Before tools or state creation, upK3d requires a Namespace DNS label of at most 63 characters. Cleanup accepts longer historical names, preserves ownership checks, and deletes only the recorded cluster through its recorded endpoint.

Both k3d profiles use legacy iptables and honor an explicit IPv4 node resolver without changing host DNS. Linux Docker selects the first glibc nameserver. internal/occdev/node_dns_k3d.go:checkDevelopmentNodeDNS fails startup on refused node DNS. Kubernetes-only startup imports matching OCE images.

Without OpenShell, startup verifies pinned cert-manager and Envoy manifests, waits for the k3s Gateway API CRDs, then installs Envoy before rollback on failure. internal/occdev/gateway_k3d.go:waitDevelopmentCRDEstablished polls each CRD until Established, a kubectl error, or timeout. Before gateway proxy trust, internal/occdev/network_k3d.go:verifyDevelopmentNetworkPolicy checks allowed and denied Pod traffic with credential-free Pods and a temporary policy, then rechecks Driver policies after bootstrap creates the initial gateway Namespace. Probe Pods use short grace periods and UID-preconditioned deletes. Cleanup waits for selector-matching Pods before removing their egress policy; probe or cleanup failure fails startup. Checks are point-in-time and single-node.

Before Installation, scripts/prepare-development-codex-seccomp.ts probes RuntimeDefault in a credential-free Pod. Confirmed denial triggers verified, content-addressed Localhost profile derivation. Failure triggers cluster rollback. On readiness failure, writable stderr receives OCE_CODEX_SECCOMP_READINESS_FAILURE before cleanup with operation runtime-default-pod-readiness and a recognized waiting reason or null.

Startup waits for the Gateway, certificate, and proxy Pods before reporting success. Envoy source addresses must fall inside the selected node's Pod CIDR; the tenant ingress policy must still admit only the Gateway's exact proxy peer.

Loopback HTTPS uses a per-installation CA whose leaf covers only its console and Agent hosts. API and browser NodePorts publish only on host loopback; the CA key stays private. internal/occdev/openshell_k3d.go:upK3d prints required browser trust instructions beside the URL and public CA path. Stack readiness does not establish browser trust.

12. Select Kubernetes development and preserve cleanup ownership

internal/occdev/command.go:selectEngine, internal/occdev/command.go:pinEndpoint, internal/occdev/command.go:podmanEndpoint, internal/occdev/up.go:Up.

Without a Sandbox Driver, startup resolves Docker's context or podman machine inspect. Private state pins the socket, Compose project, and occ-dev-* cluster for cleanup.

Startup refuses existing resources. internal/occdev/compose.go:AnalyzeCompose validates publications; internal/occdev/up.go:validateResourceOwnership rejects external or unscoped networks and volumes. Exclusive 0700 state creation precedes resources. setKubernetesBridgeGateway preserves an explicit gateway or selects the subnet's first usable address. A private Compose snapshot pins startup and cleanup against later .env edits.

internal/occdev/compose.go:prepareComposeBootstrapSQL snapshots default SQL as 0644 for PostgreSQL. Only the default read-only bind, including checkout aliases, is redirected; custom and writable mounts and original permissions stay unchanged. State remains 0700. Cleanup removes the copy. Pre-start failures remove incomplete state despite OCC_DEVELOPMENT_KEEP_ON_FAILURE; retention requires a recoverable snapshot and resource startup.

Kubernetes-only mode enters internal/occdev/openshell_k3d.go:upK3d before Compose rendering.

internal/occdev/repository_k3d.go validates inputs. After upK3d completes authenticated bootstrap and Namespace readiness, repository setup binds the server-assigned Namespace ID, generates CA and exact-host certificates, and upgrades Helm with Kubernetes inputs, Repo Driver and worker sidecar. Discovery only checks approved references and profiles. See required inputs.

OCC_DEVELOPMENT_CONTROL_PLANE defaults to compose for Docker Compute, else kubernetes; unsupported values fail before resource creation.

13. Bootstrap OCC, create k3d, and prepare runtime configuration

internal/occdev/up.go:Up, internal/occdev/up.go:waitCompleted, internal/occdev/kubernetes.go:writeKubeconfigs, internal/occdev/kubernetes.go:importRuntime, internal/occdev/kubernetes.go:writeInstallation, internal/occdev/openshell.go:prepareOpenShell.

With Docker and a selected runtime, Compose overlaps bootstrap and cluster setup after creating PostgreSQL. For OpenShell, internal/occdev/gateway_k3d.go:installDevelopmentRoutingControllers starts after kubeconfig setup, alongside runtime import and OpenShell installation. Both finish before routing resolves PostgreSQL’s address and starts OCC. Failure cancels and drains siblings before rollback. Cold builds and Podman install routing afterward.

Both concurrently install cert-manager and Envoy, verifying checksums, CRDs and rollout before GatewayClass acceptance or routing configuration. Envoy waits for k3s-owned Gateway API CRDs. Without OpenShell, OCC_DEVELOPMENT_K3S_IMAGE selects the node image; its default +v1.35 resolves the latest 1.35 patch. An explicit image skips lookup. OpenShell uses its pinned image in both control-plane modes. Cluster creation binds the API to loopback and preserves the default kubeconfig and current context.

Host kubeconfig stays private. Non-root API/worker containers receive read-only configuration and kubeconfig targeting the TLS-verified internal load balancer, without the engine socket.

Before importing a source-built runtime, importRuntime passes the optional build CA file to the engine as a build secret. Selecting an existing runtime image skips this build.

internal/occdev/kubernetes.go:engineImageReference resolves engine-recorded names, including Podman localhost/ and Docker Hub, defaults omitted tags to :latest, and rejects missing or ambiguous matches. Digest-pinned sources are usually multi-arch lists that Docker's containerd image store (Docker 29) can neither save with --platform nor export completely, so internal/occdev/k3d_import.go:stageDevelopmentPlatformChild re-points the staging tag at the registry's platform child so the importer can save a complete single-platform archive. Unlisted sources and a failed child pull keep the pinned reference and filter. importDevelopmentImage resolves cluster digests. Both paths call internal/occdev/status_proxy_k3d.go:developmentStatusProxySource before writeInstallation; node inventory retains caller context outside polling.

internal/occdev/up.go:poll bounds route queries to two minutes through exec.CommandContext; deadline or cancellation stops blocked queries. developmentStatusProxyCidr requires one non-network IPv4 cni0 source in the Pod CIDR and returns /32. Errors stop Installation writing and trigger cleanup.

internal/occdev/kubernetes.go:writeInstallation then selects Kubernetes Compute, Configuration, and Secret Drivers with native IAM; without OpenShell, it adds both bundled Presets and the Codex Plugin Driver after the shared Codex sandbox check. Its runtime sets the transport Secret prefix, gateway storage class, and memory limits of 4 GiB per gateway and 6 GiB per Harness.

Compose installs OpenShell in k3d operator workspace mode; see the OpenShell provisioning flow.

internal/occdev/compose_routing.go:prepareComposePrivateFiles builds selected services before --no-build startup. Both Compose Kubernetes profiles assign browser TLS files to the proxy UID/GID; OpenShell routing files use the API/worker UID/GID. Ownership follows each reader's image and Compose user setting, preserving mode 0600.

The hybrid snapshot adds compose.native-admin.yaml. Its CA signs only the installation's console and wildcard Agent hosts. browser-proxy terminates loopback HTTPS; Agent ClusterIP Services remain behind Envoy.

apps/controller/src/composition/development-postgres.ts:composePostgresDevelopment passes OCC_AUTH_BASE_URL into development admission. apps/controller/src/index.ts:createFastifyApp accepts that exact Host and optional exact Origin only from loopback or trusted development CIDRs. Sibling hosts, another Origin, forwarded headers, and untrusted peers fail before identity resolution.

14. Prove readiness and clean up the owned Kubernetes profile

internal/occdev/up.go:waitReady, internal/occdev/up.go:copyAndVerifyKey, internal/occdev/openshell_k3d.go:copyAndVerifyKubernetesKey, internal/occdev/down.go:Down, internal/occdev/down.go:cleanup, internal/occdev/state.go:readState.

After API/worker readiness, both bootstrap readers use private temporary keys and cancellation-bound occclient requests. Installation IDs must match bootstrap before exclusive key export; cancellation triggers owned cleanup. OpenShell also waits for its bootstrap Namespace and operator-mode Workspace. Namespace readiness and repository discovery use occclient.Client.WithContext for polling deadlines, preserving the original client. Both Compose Kubernetes profiles verify the HTTPS session route using the generated CA and console name.

Both Kubernetes profiles pass OCC_DEVELOPMENT_STARTUP_TIMEOUT_SECONDS to k3d cluster create --timeout; node readiness timeout fails startup.

internal/occdev/state.go:exclusiveWrite removes new outputs, including partial external keys, when chmod, writing or closing fails. Existing targets stay untouched; errors include failed removal.

Failure triggers cleanup. Shutdown validates the marker, state and Compose snapshot, then stops API/worker and deletes owned k3d and Compose resources through the recorded engine. Cleanup continues after errors, retaining state on failure; success removes state and the helper-owned key. External --key-output files remain operator-owned, except startup removes a newly written key if later OpenShell readiness fails.

15. Start and remove the optional Keycloak profile

internal/occdev/keycloak_k3d.go:installDevelopmentKeycloak, internal/occdev/keycloak_k3d.go:signInDevelopmentKeycloak, internal/occdev/down.go:cleanup.

With OCC_DEVELOPMENT_SIGN_IN=keycloak, Kubernetes-only sandbox none provisions Keycloak before authenticated readiness, then enables OIDC and attaches Alice to the administrator. See the Keycloak lifecycle for validation, DNS/TLS, both Helm passes, recovery and destructive teardown.

Debugging and Verification

internal/occdev/openshell_k3d.go:upK3d reports selected controller-image-import failures after cleanup. internal/occdev/startup_failure.go preserves routing details; concurrently retains sibling cleanup failures. Helpers and CI retain one allowlisted final failure record, including older formats. Ambiguity, success and cleanup yield none; stdout is unauthenticated. Field meanings.

Search documentation