Gateway routing with Envoy
The OpenClaw Control Plane (OCC) uses Envoy Gateway to read and write Agent workspace files on Kubernetes. A private HTTPS listener routes each Agent URL to that Agent's native OpenClaw gateway:
OCC API -- WSS + service key --> Envoy -- WebSocket --> Agent gateway ServiceFor installation commands, use Configure private Agent workspace routing. For caller permissions and file operations, see the workspace-files API.
Resources and ownership
| Owner | Resources or responsibility |
|---|---|
| Installation operator | Envoy Gateway and cert-manager controllers, Gateway API CRDs, an existing GatewayClass, enforced NetworkPolicies, and the service-key Secret. |
| Helm chart | Shared Gateway, ClusterIP EnvoyProxy, SecurityPolicy, certificates and optional CA issuers, API/worker credential mounts, and controller/Envoy NetworkPolicies. |
| Kubernetes Compute Driver | Tenant attachment labels, Agent HTTPRoutes and node-route SecurityPolicies, gateway Services, and tenant NetworkPolicies through worker reconciliation. |
| OCC API | Caller authorization, endpoint derivation through Compute, and native file RPCs using the mounted service key. |
| OCC worker | Native node enrollment through Compute, using the mounted service key and revision-owned enrollment Secrets. |
The proxy Pods inherit controlPlane.nodeSelector from Helm values, keeping
credential verification on the trusted OCC pool. Install the separately managed
Envoy Gateway and cert-manager controllers on trusted nodes as well.
The shared Gateway and certificate resources are in the Helm release namespace.
Envoy's proxy Service and Pods are in envoyNamespace. Each Agent's HTTPRoute
and gateway Service are in its Gateway runtime namespace for dedicated execution,
or its tenant data-plane namespace for embedded execution. The installer needs
permission to create the shared resources, including the NetworkPolicy in the
Envoy namespace. The worker needs tenant HTTPRoute and SecurityPolicy permissions; the API does
not need to write routes or execute commands in gateway Pods.
Endpoint and route
The private endpoint is:
wss://<hostname>/namespaces/<namespaceId>/agents/<agentId>With no explicit hostname, Helm and Compute independently derive:
serviceName = occ-gateway-<first 12 hex characters of SHA-256(gatewayNamespace/gatewayName)>hostname = <serviceName>.<envoyNamespace>.svcThe chart sets Envoy's Service name to this value. Standard Linux Pod DNS search
resolves it without a separate DNS record or a fixed cluster.local suffix.
An explicit hostname must match in Helm and Compute and resolve to the Envoy
Service. The listener certificate uses that same hostname.
getGatewayEndpoint(revision) computes the URL without a Kubernetes lookup;
it does not check whether the gateway is ready. When preparing and activating a
revision, Compute creates or updates an HTTPRoute attached to the shared
Gateway's https listener. The route matches the exact hostname and forwards
to the Agent Service in the same namespace. The exact Agent path rewrites to
/; a bounded prefix rule preserves suffixes for native UI traffic. See the
Kubernetes route rules.
TLS ends at Envoy. From there, plaintext HTTP and WebSocket traffic to the
gateway is restricted by NetworkPolicy.
The route and Service stay stable when a revision is activated or a gateway Pod is replaced. Retiring an older revision preserves the newer gateway's route; final gateway cleanup removes it. Compute repairs the route while reconciling a revision. There is no separate periodic repair.
Native node endpoint
Runtime-enabled dedicated revisions also receive an exact /node route and an
exact /node/__openclaw__/worker route under the same Agent URL, hostname and
https listener. Both use the same backend Service; the worker route rewrites
to OpenClaw's /__openclaw__/worker ingress. The route removes x-occ-identity, x-api-key,
forwarded identity and scope headers, and Tailscale identity headers while
setting x-real-ip from Envoy's downstream socket. This preserves trusted
proxy attribution without granting the OCC administrative identity.
Compute attaches a tenant-local SecurityPolicy with no authentication fields to this node HTTPRoute. Envoy Gateway v1.6.7 replaces the entire inherited Gateway policy at this more specific scope, so the node route does not require the OCC service key. Native OpenClaw verifies the signed device identity and node-only bootstrap or device token. Invalid credentials and attempts to use node credentials as an operator fail at the native Gateway. Worker callbacks authenticate their first WebSocket frame with the Gateway-minted, session-bound worker admission credential; the Harness never receives the OCC service key.
Preparation creates or repairs these resources under the serving Gateway's revision. Preparing a replacement preserves that ownership until activation replaces the Deployment. Stop and retirement remove the exact revision's node route before its policy, checking ownership and deletion UIDs; newer revisions remain. The Compute enrollment path also admits Harness egress to this installation's Envoy Pods on the configured HTTPS target port. File, Memory and Skills access use the enrolled node. Dedicated Gateway does not mount the Harness workspace or generated-image directories; sessions stay in Gateway private storage. Native runtime and Envoy integration verification remain incomplete. See the enrollment trace. Real Envoy node-authentication verification is described in routing tests.
Service key and native identity
The Installation operator must create the service key, even when the certificate
authority (CA) is created automatically. Generate 32 random bytes encoded as
hex without a trailing newline and store the result in a dedicated Opaque
Secret under the key occ. Set gatewayRouting.apiKeySecretName to its name.
The Secret belongs in the Helm release namespace; see the setup commands.
Helm mounts that Secret into the OCC API and worker at
/etc/openclaw/gateway-api-key/key and sets OCC_GATEWAY_API_KEY_PATH.
The Gateway-level SecurityPolicy references the same Secret. OCC reads the key
for each operation and sends it as x-api-key. Envoy validates and strips that
header before forwarding.
The tenant-worker role grants Secret get/create/update/delete for admitted Gateway credential delivery and Compute-owned node enrollment. Operators bind this role only in approved data-plane and Gateway namespaces; the chart creates no cluster-wide binding for it. The worker is part of the trusted control plane. Harnesses receive a node-only setup code and public CA bundle, never this administrative service key.
The HTTPRoute sets x-occ-identity: occ-workspace-files and sets x-real-ip
from Envoy's direct downstream connection. It removes x-forwarded-for,
forwarded, and x-openclaw-scopes. Kubernetes Compute renders native
trusted-proxy auth from the operator's network.gatewayTrustedProxyCidrs, enables
allowRealIpFallback, and grants the fixed identity operator.admin. Agent
Configuration cannot override that trust boundary. A direct loopback connection
can still use the Driver-managed gateway password if the native Configuration
explicitly selects its environment SecretRef.
This password is separate from the Envoy service key. See the complete
operator proxy trust setup.
This key grants native administrative access across the Installation's routed gateways. OCC separately checks the caller's exact Agent permission. Keep the key separate from Better Auth, provider, and native Agent credentials. The trusted OCC API and worker receive it; Agent Gateway and Harness Pods do not.
TLS and certificate lifecycle
When issuerRef.name is empty, Helm creates a namespaced SelfSigned Issuer,
a root CA Certificate, and a CA Issuer for the listener certificate.
cert-manager generates and stores the key material.
| Certificate | Requested lifetime | Renew before expiry | Additional settings |
|---|---|---|---|
| Root CA | 87,600 hours | 720 hours | ECDSA P-256; isCA: true; key rotation policy Never. |
| Listener | 2,160 hours | 720 hours | DNS name equals the routing hostname. |
The API Pod receives only the root Secret's public tls.crt, mounted as
ca.crt, and loads it through NODE_EXTRA_CA_CERTS. It waits for the Secret
before starting. The CA signing key is never mounted into OCC. Normal CA and
hostname verification remain enabled; OCC does not pin the listener leaf.
To use an existing issuer, set issuerRef.name and its kind/group. Set
caSecretName and caSecretKey together if OCC needs an additional public
CA bundle; otherwise it uses Node's existing trust store. Explicit CA trust
requires an explicit issuer. Keep root, listener, service-key, and other
credential Secrets distinct.
OCC rereads the service-key file for new operations, allowing projected Secret updates without an API restart after Envoy also observes the update. Rotation is not coordinated atomically between those consumers. Node loads additional CA trust at process startup: changing the trust bundle requires restarting both the API and worker. Workspace nodes receive a public CA snapshot at launch; their Harness workloads also need replacement when that trust changes. Listener renewal under the existing CA does not require changing OCC trust.
Routing configuration
Helm's gatewayRouting settings configure shared infrastructure:
| Setting | Default or requirement |
|---|---|
enabled |
false; enable to render routing resources and API mounts. |
gatewayClassName |
Required existing Envoy GatewayClass. |
gatewayName |
<release>-agent-gateways. |
envoyNamespace |
envoy-gateway-system. |
hostname |
Empty derives the Service DNS hostname. |
apiKeySecretName |
Required operator-created Secret with entry occ. |
issuerRef.name |
Empty creates the private CA and issuers. |
issuerRef.kind / group |
ClusterIssuer / cert-manager.io for an explicit issuer. |
caSecretName / caSecretKey |
Empty; optional public trust bundle with an explicit issuer. |
tlsSecretName |
<gatewayName>-tls, truncated to 63 characters with trailing hyphens removed. |
tenantGatewayPort |
8080; must equal Compute's network.gatewayPort. |
envoyHttpsTargetPort |
10443; NetworkPolicy port for the Envoy listener Pod. |
envoyGatewayPodLabels |
Chart defaults select the Envoy Gateway controller for control-plane egress. |
The Installation's drivers.compute.configuration.gatewayRouting separately
requires gatewayName, gatewayNamespace, and envoyNamespace; hostname is
optional. endpointPort defaults to 443. Set it only when the external load
balancer exposes the Gateway listener on another port; Helm does not configure
that external mapping. envoyHttpsTargetPort defaults to 10443 and must match Helm's value,
so the Harness egress rule permits the listener's actual Pod port.
Match the Helm values and use the release namespace for
gatewayNamespace. Helm does not rewrite the Installation Secret. Remove
network.gatewayClients when enabling routing: Compute derives the Envoy peer
and rejects explicit clients in this mode. Restart API and worker after changing
their Installation startup configuration. Adding an Agent requires no endpoint
map or controller restart.
Network enforcement and failures
Envoy ingress permits the selected OCC API/worker Pods and Harness Pods in attached tenant namespaces. It also permits OpenShell supervisor Pods from those namespaces because the supervisor opens policy-enforced Harness connections. The namespace attachment label limits both sources to this Gateway. Envoy egress permits tenant gateway traffic, configured DNS, and the Envoy Gateway control-plane connection. Tenant gateway ingress permits the selected Envoy Pods. The Gateway accepts HTTPRoutes only from namespaces bearing its attachment label.
These restrictions require a Kubernetes network plugin that enforces NetworkPolicy. Only trusted actors can be allowed to change routes, policies, attachment labels, or native Configuration. For proxied connections, the native real-IP fallback requires OCC to connect from a nonloopback address; a loopback port-forward alone cannot supply the caller's address.
| Symptom | Check |
|---|---|
| Helm render fails | Required GatewayClass/service-key settings and distinct Secret names. |
| API Pod waits or startup fails | Service-key/root-CA Secret availability and valid key-file contents. |
| Agent reconciliation fails | Routing/native-auth configuration, installed CRDs, and worker RBAC. |
Workspace API returns 503 DEPENDENCY_UNAVAILABLE |
Endpoint support, active gateway, DNS, TLS trust, key agreement, route attachment, and NetworkPolicies. |
A write returns 503 UNKNOWN_OUTCOME |
Read the file before deciding whether to resubmit; OCC does not replay uncertain writes. |
The Kubernetes Driver implements the optional ComputeDriver.getGatewayEndpoint
method. It returns no endpoint when routing is not configured. The Docker Driver
does not implement this method; exposing its gateway port on localhost does
not enable workspace-file access through OCC.
Native admin UI routing
Agent native admin UI access reuses the same private Envoy routing primitive as
workspace files. The public browser hosts are operator-owned wildcard names
served by the OCC API process, using agentNativeAdmin.domain; Envoy and Agent
gateway Services remain private ClusterIP resources. The Agent hosts share the
ordinary OCE session cookie through the configured agentNativeAdmin.sharedCookieDomain,
so every matching console and Agent subdomain must be a trusted OCE ingress
endpoint. OCC authenticates and authorizes the human session before proxying,
then strips browser cookies and credentials before forwarding to Envoy.
The private Compute endpoint remains:
wss://<private-host>/namespaces/<namespaceId>/agents/<agentId>OCC converts that endpoint to https: for native UI HTTP traffic while keeping
the same private authority and exact Agent base path. Workspace-file traffic
continues to use the original WSS endpoint. Native-host requests are
intercepted before the normal API not-found path, resolved to the exact Agent
represented by the host, and checked against the current active revision before
the API proxies HTTP or WebSocket traffic through the private route.
Public preview routing
Optional gatewayRouting.sandbox in Kubernetes Compute adds a stable per-Agent
HTTPS origin for dedicated execution under the operator's preview domain.
Embedded OpenClaw retains its native preview configuration. Compute owns the native
sandboxOrigin and sandboxPort values and rejects conflicting Agent settings.
The sandbox backend port is network.gatewayPort + 1, so the main port must be
below 65535. The selected runtime must support the dedicated sandbox listener.
The Agent's -sandbox HTTPRoute attaches only to the shared Gateway's separate
sandbox listener. It accepts GET and HEAD and forwards to the sandbox port,
never the administrative Gateway port. Cookies, authorization, API keys and
native identity headers are removed. A route-specific SecurityPolicy permits
public shell and renderer assets without granting the OCC administrative identity.
The runtime owns shell CSP, resource allowlisting and iframe isolation; private
HTML content still arrives through the authenticated native UI.
Sandbox routes and policies follow serving revision ownership. Replacing a Pod keeps the origin stable; stopping or deleting its serving revision removes the route before its policy. Retiring an older revision preserves newer resources. An Agent-owned ingress policy and the Envoy egress policy admit the additional backend port only when configured. Agent deployment reconciles preview ingress even when the tenant namespace already exists. Helm requires explicit ingress peers on the separate listener, a wildcard certificate, and a domain outside the shared session cookie scope. See HTML preview setup.
