Prepare a production handoff

Decide whether a Kubernetes installation is ready for a business workflow after deploying and verifying production Agents. Record who operates it, access decisions, and evidence in your organization's private operations record. The deployment guides cover installation and runtime checks; your organization is responsible for service commitments, regulatory compliance, and recovery targets for the workload.

Record the initialization Job for handoff

Use the CLI from the chart's source revision. Exclude other install, upgrade and Job writers until the installer exits. Choose a new receipt directory under a private, durable parent for this target. Use a filesystem that enforces owner-only permissions and supports hard links and directory sync. Keep the receipt after install.

Record the chart source commit and the matching CLI release or build commit and binary checksum in a separate private handoff record. The receipt's version is its file-format version, not the OCE release; an unversioned local CLI build can print dev. Keep this source identity with the receipt without editing its files.

The CLI syncs intent.json (target, attempt ID, time and input hashes), writes the receipt notice to stdout, then syncs dispatch.json before starting Helm once. If the notice write returns an error, the command exits unsuccessfully and names the preserved receipt directory in its error. That error confirms that this attempt created no dispatch.json and started no Helm install. Keep the receipt, repair the stdout destination, and confirm the target is still unused before selecting a new attempt with a new receipt directory.

While Helm waits, it can save the first recognized, attempt-marked Job UID in job.json. helm-result.json records Helm's separate outcome. After dispatch, a missing result means unknown. A failed or disconnected Helm client can leave the Job running and the database changed. Do not rerun install or switch to upgrade.

The installer removes private temporary copies of the kubeconfig, chart and values on normal exit. After a forced kill, confirm its Helm process has exited before removing that installation's occ-install-* temporary directory. A forced kill during handoff can also leave an occ-handoff-* directory with the reader's private kubeconfig. Share only the receipt and this source identity record with the authorized reader.

If install reports a template validation failure, run offline helm template with the same local chart, values, release, namespace and recorded installation.attemptId. Use the same Helm version. Keep rendered manifests and stderr in owner-only local outputs outside the receipt; they may contain secrets. For the shared DNS suffix refusal, the host in auth.baseUrl and agentNativeAdmin.domain must share a non-public DNS suffix. Correct a copy of the inputs and render again. Neither a render result nor a missing Job establishes cluster effects or authorizes retry; preserve the receipt and reconcile the original attempt below.

Resume an installation in progress

Use the matching CLI, kubectl, your own trusted kubeconfig, and permission to get the named Job (normally <release>-initialization). The CLI may also read API discovery and check whether the Namespace exists after a missing Job. Neither Namespace access nor permission to list Jobs or Pods is required. If the CLI release is unavailable, run go run ./cmd/occ from the matching checkout.

Run plan for the Job result and recovery steps, or use doctor to inspect the Job. The example uses the release name and Kubernetes namespace from the custom Helm guide. Substitute the values recorded for your installation.

bash

target=(--kubeconfig "$KUBECONFIG_FILE" --context "$CONTEXT"        --kube-namespace openclaw-system --release oce)# For an installation started by occ, use its original receipt directory.target+=(--receipt "$INSTALL_RECEIPT_DIRECTORY")# For an older installation without a receipt, instead use# --expect-job-uid "$RECORDED_JOB_UID" only if captured during that attempt.occ installation plan "${target[@]}"

With a receipt, the commands compare the recorded cluster connection, namespace and release before reading the Job. A different context name or credential can select the same connection. Keep the same server URL (including path), TLS and kubeconfig proxy settings. Neither command changes the receipt or cluster.

Completed. For the original checkpoint, match the first-install marker, captured UID, chart steps and terminal completion. Without a captured UID, a copy of the marker on a replacement can look identical. The commands report the current Job but leave the original outcome unknown. For a legacy install, supply only a UID recorded while the installer excluded replacement. Without a receipt or UID, exit zero describes the current Job only. Ask the release operator to verify Helm status and API and worker readiness. Job completion does not establish current database state or authorize replay.
Recorded Job replaced or missing. Recover the original installation record. A later Job with the same name and a different UID cannot establish what happened to that attempt, even if its chart labels have changed.
Not found, no UID recorded. Confirm with the database, identity, and volume owners whether this is a first installation. A missing Job is not permission to bootstrap again.
Failed. With a matching Job UID, plan shows a Pod lookup for an operator who can already list Pods in that namespace; it does not list Pods itself. That operator sets POD_READER_KUBECONFIG to their own trusted kubeconfig and POD_READER_CONTEXT to its matching context in their shell, then runs the displayed command. Match the Pod's controlling owner UID to the recorded Job UID before interpreting the migration and bootstrap exit codes; a label alone is insufficient. <none> means no current terminated exit code was reported; neither <none> nor an empty result proves that no Pod ran or no effects occurred. The selected columns limit what is printed, not what Kubernetes authorizes or returns. If needed, ask for a short, redacted diagnostic, not raw logs or credentials. Ask the database and storage owners for current state and output provenance; treat unverified state or provenance as unknown. Preserve the attempt and do not retry.
Unknown or incomplete. An unreadable Job remains unknown, even if kubectl reports a login problem or refusal; it is not evidence that the Job is absent. Confirm the target. For a reported login problem, use a separately authorized read-only session. For a reported refusal, confirm the intended reader with the access owner without restoring temporary privilege or requesting broader access. FailureTarget alone does not establish terminal failure. An interrupted initialization may already have changed state.

The Job UID identifies the initialization, not a later API operation. If an API attempt had an uncertain outcome, reconcile that original attempt before another write; do not replay the uncertain operation. Confirm removal of temporary access with its owner, then use the workload checks below.

Choose the first workflow

Start with one task and a small group. For example, let an Agent read a support queue and draft replies before enabling customer-facing sends. Name the business owner, acceptable output quality, response time, cost, and the person who handles exceptions. Use test records and a private test channel for the initial checks.

Confirm that the organization can operate the host infrastructure, database, credentials, and integrations before relying on the workflow.

Assign operating ownership

Record a primary and backup contact, the exact scope they manage, and how to reach them when the platform is unavailable. The same person can own several areas; each responsibility still needs an explicit handoff.

Area	Decision to record	Owning guidance
Platform	Who maintains OCC, PostgreSQL, the cluster, storage, images, and deployment inputs.	Production installation
Business workflow	Who approves actions, checks result quality, handles exceptions, and can suspend the work.	Agent deployment
Access and credentials	Who admits users, grants resource access, renews credentials, and handles compromise or staff departures.	Authorization, credential lifecycle
Data and evidence	Which data may leave each workload, where logs and backups live, and their access and retention rules.	Runtime security, observability
Usage and cost	Who owns provider billing, expected request volume, budget alerts, and the response to quota exhaustion.	Model credentials

Kubernetes resource quotas constrain the configured infrastructure resources; they do not impose a model-spending budget. Use the provider's available spending or quota controls and document their limits. Include model calls, integrations, storage, and operator time when measuring the pilot's cost.

Review access and data destinations

Keep the platform and runtime boundaries distinct. OCC authorizes operations on its exact resources and manages separate Agent gateways. An OCC role does not define which CRM records or email recipients a connected business tool may use. Enforce those limits in the destination system or integration with a dedicated business identity and narrowly scoped access.

For each integration, record the allowed reads, writes, external destinations, required reviewer, and revocation owner. Check that the selected runtime can represent the requested tool policy. A saved plugin selection is desired state, not proof that its policy is supported or active; see Agent plugins.

Inventory model endpoints, channel providers, tools, MCP servers, log exporters, and backup destinations. Include fallback models and secondary processing such as search or media services. Self-hosting the workload does not establish that data stays on its host or in one region. The current Kubernetes model-egress exception allows public TCP/443 beyond a provider-specific allowlist; review networking and isolation before making a destination-isolation claim. Treat retrieved messages, documents, and web pages as untrusted inputs when deciding which actions may run.

Verify the deployed workflow

Keep a short evidence record identifying the Namespace, Agent, version, verification time, expected result, and observed result. Store credential references rather than values.

Complete the production workload checks: the expected version, denied and allowed gateway access, and a real model turn through the selected runtime. A deployment response of 202 means work was admitted; activeRevisionId alone is not serving-health evidence.
Submit a representative business request through the intended channel. Verify the source records read, the generated result, and any approved change in the destination system. Check the result against the pilot's quality, time, and cost expectations.
Exercise denied access and, where supported, rejected approvals. Confirm that the destination remains unchanged. Demonstrate the human handoff for an ambiguous request or unavailable integration.
Before enabling retries for writes, test a timeout or repeated request against test records. Verify the integration's duplicate-prevention behavior; OCC deployment reconciliation does not establish exactly-once CRM updates or message delivery.
Repeat the relevant checks after a version, credential, plugin, or access policy change. Confirm that unrelated Agents retain their intended behavior.

Connect alerts to a response

Verify each private API/worker metrics target using Pod discovery, and use platform observability to connect operational logs and prove backend receipt. Default local output does not prove remote delivery. Then assign alert thresholds and a response owner for the workload. Keep an alert route that does not depend on the affected Agent replying.

Signal	First operator response
Agent remains unready or work stops advancing	Check the independent worker, current permissions, runtime readiness, and the selected Driver's diagnostics in controller troubleshooting.
Channel or business action fails despite a reachable gateway	Verify that integration's credentials, provider quota, and destination result; repeat the affected workflow check.
Credential is near expiry or provider quota is exhausted	Notify the credential or billing owner and use the credential-specific renewal path.
Export fails, records are refused, or the Collector queue fills	Check backend receipt, exporter connectivity, and Collector metrics.

Operational logs, Collector metrics, PostgreSQL audit records, and destination transaction records answer different questions. The Collector does not export the audit ledger or provide application metrics, distributed traces, dashboards, or business-success alerts. Filtered operational logs are best-effort; their absence does not prove that an action did not occur. Confirm consequential business effects in their destination system.

Prepare recovery and update acceptance

Agree on acceptable downtime and data loss before relying on the workflow. Controller lease recovery handles interrupted reconciliation; it is not a backup or a complete Installation restore. There is currently no general OCC backup/restore command or explicit Agent rollback endpoint.

Record the protection and recovery owner for each required asset:

External PostgreSQL, including platform state, IAM, work, and audit records.
Protected Installation YAML, Helm values, image digests, selected Driver versions, and required infrastructure configuration.
Startup, authentication, provider, and runtime Secrets; protected bootstrap output; and any private routing certificates and keys.
Each Agent's private gateway state claim and, for dedicated execution, its Harness workspace claim. Review the exact persistent and ephemeral storage boundary.
External Configuration and Secret stores required by the selected Drivers.

Use database- and storage-owner backup procedures that capture consistent state; do not treat copies of live SQLite files or PVC persistence as a verified backup. Coordinate the restore points across platform state, runtime state, configuration, and credentials, and record any consistency limits. Protect backups like the live data and retain a copy outside the failure domain being recovered.

Have the infrastructure owner review and rehearse the installation-specific restore procedure on an isolated target. Block production connectivity before starting a restored API, worker, or runtime: restored work may reconcile or repeat external actions. Substitute test credentials and disable copied production channels and scheduled work before allowing test egress. Verify state, access, and the business workflow; measure recovery time and the age of recovered data. Before an actual cutover, fence old writers and workloads. RWO storage and normal version routing do not provide independent process fencing during a node partition.

For an update, retain the reviewed deployment inputs and a verified recovery point, rehearse the candidate in a separate environment, and repeat the workload checks before widening use. Define the failure response with the infrastructure owner; reverting a Helm release or image does not restore database, Secret, or workspace state. A saved version fixes the requested configuration and references. Referenced Secret values and resolved plugin releases can change. See version snapshots and plugin lifecycle.

Check conversation continuity

Before the upgrade, choose a test conversation on an existing Agent and note a harmless exchange. After recovery, reopen that conversation on the same Agent. Find the earlier exchange, then send a follow-up and check for a new reply. Repeat the check if you return to the old build. A new chat or a restored workspace file can work even when the original history is missing.

For dedicated Codex, the Gateway claim holds the conversation and the Harness claim holds codex-sessions. To check native thread continuity, record the thread ID before the change and verify the follow-up used that same thread. Keep IDs and results private; omit message bodies and credentials.

Re-creating a split-layout tenant assigns new Agent IDs; the separate archive keeps old conversations for reference.

Complete the handoff

The business and platform owners should be able to locate the accepted workflow, deployed version, access and data decisions, credential renewal dates, alert responses, and recovery evidence without the original installer. Record unresolved limits and the decision to accept them or defer the workload. Revisit the handoff when a new integration, permission, runtime, or data category changes those assumptions.