Agent plugin testing

Overview

Run Agent plugin tests for API, Driver, persistence, and runtime behavior from the repository root. The plugin reference describes supported behavior.

Local and integration suites

The discovery fixture uses simulated provider responses. These checks do not prove a live provider or native runtime.

Kubernetes status transport

Prepare a disposable Kubernetes fixture, a worker node, and a migrated openclaw_k8s_* database. Set OCC_TEST_KUBERNETES_KUBECONFIG, OCC_TEST_KUBERNETES_CONTEXT, OCC_TEST_KUBERNETES_IMAGE, OCC_TEST_DATABASE_URL, and OCC_TEST_KUBERNETES_PLUGIN_STATUS_PROXY_CIDRS to the comma-separated API-server proxy source CIDRs from the networking reference.

node --test tests/integration/kubernetes-plugin-status-real.test.mjs

This checks status transport and restart behavior with controlled producers. It does not prove native plugin compatibility.

Native startup metadata reads

After credential authorization under the contribution policy, inject CODEX_ACCESS_TOKEN without printing it. Set OCC_TEST_KUBERNETES_AGENT_IMAGE to an available immutable image containing Codex 0.164.0, and start Docker.

OCC_TEST_CODEX_STARTUP_READS_REAL=1 node --test \  --test-name-pattern='native Codex app-server' \  tests/integration/codex-plugin-startup-reads-real.test.mjs

The test removes its container. It checks the authenticated metadata protocol, not a model turn or connector call.

Offline native local plugins

With a qualified image containing stock Codex 0.164.0, Node, and ws, run:

OCC_TEST_RUNTIME_IMAGE=<qualified-image> node --test \  tests/integration/runtime-image-codex-local-plugins.test.ts

The images-runtime-startup-3 lane runs two cases in separate containers, with --network none, UID 1000, fresh homes, and read-only mounts of two synthetic skill packages. The test stages only its TypeScript clients, setup data, packages, and generated OCE launcher in a temporary read-only context. Node loads the clients from files; the normal tests project typechecks them. No account credentials, host home, or model turn are used.

The protected control runs the generated OCE launcher with empty selections and its original disabled plugin features. The production release signer supplies synthetic enrollment. A wrong bearer cannot upgrade; the correct external client initializes and reads the native session. Both packages are registered locally on the read-only mount. Native detail requests must fail with plugins disabled, and neither skill may load. This client performs no configuration write or installation. The Harness receives only the bearer verifier and public release key.

The second case tests stock-native compatibility for the proposed plugin preparation path. It selects one trusted package before starting a separate stdio app-server, installs it, and checks the plugin ID and installed skill file. The excluded package must stay uninstalled and disabled; disabling or omitting the selected entry must remove its loaded skill. This is not a protected OCE selection or a serving-client installation contract. A component-only image can prove these cases without qualifying an OCE runtime image or deployment.

No image input means a reported skip. OpenShell's nonempty-selection refusal and the remaining protected plugin acceptance requirements are unchanged.

Native runtime prerequisites

Prepare the Kubernetes runtime setup, including immutable gateway and Agent images, an authorized model, a disposable cluster, and the proxy CIDRs above. For one scenario, set its database URL below or use OCC_TEST_DATABASE_URL. Set one scenario flag:

Dedicated Codex cases also require the real Envoy Gateway and cert-manager controllers from Gateway routing setup. Provide OCC_TEST_GATEWAY_CA_CERT_PATH and OCC_TEST_GATEWAY_CA_KEY_PATH for that fixture. The test creates an owned Gateway; Kubernetes Compute creates each Agent route. Set OCC_DOCKER_RUNTIME_IMAGE to the qualified runtime for the fixture's TLS-preserving host relay. This fixture runs its controller on the host; the OpenShell Linear case instead starts the shipped control plane and uses its configured OCE Gateway route directly.

Set the chosen flag to 1, then run:

node --test tests/integration/plugin-driver-real.test.mjs

OCC_TEST_PLUGIN_DRIVER_REAL=1 selects all three and requires all three scenario-specific URLs with distinct databases; separate host aliases for one database do not count. Unselected cases skip. The OpenClaw case needs OPENAI_API_KEY; the repository's pinned runtime supports plugins install --no-enable. Codex cases need an existing authorized account's CODEX_ACCESS_TOKEN, an immutable OCC_TEST_KUBERNETES_AGENT_IMAGE or OCC_TEST_KUBERNETES_CODEX_IMAGE, and the Codex sandbox profile on each test node, selected with OCC_TEST_KUBERNETES_CODEX_SECCOMP_PROFILE. Choose a supported model with OCC_TEST_OPENAI_MODEL; the default is gpt-6-astra. Provide sufficient cluster memory for version overlap.

For Linear, connect the account and set OCC_TEST_CODEX_LINEAR_PROMPT, OCC_TEST_CODEX_LINEAR_TOOL_NAME, and OCC_TEST_CODEX_LINEAR_RESULT_EXPECT for a harmless read. For the failure case, connect the successful plugin account and select a real failing plugin; use OCC_TEST_CODEX_SUCCESS_PLUGIN_ID and newline-separated OCC_TEST_CODEX_FAILURE_PLUGIN_IDS to override the defaults. Calendar tool names differ between accounts: set OCC_TEST_CODEX_CALENDAR_TOOL_NAME, OCC_TEST_CODEX_CALENDAR_RESULT_EXPECT, and, if needed, OCC_TEST_CODEX_CALENDAR_PROMPT for an available harmless read. Never print credentials or account identifiers.

Per-call approval acceptance

The dedicated Codex failure case verifies a successful harmless Calendar read after a plugin failure. To verify human and automatic approval separately, use normal Agent turns and record the approval decision and tool execution for each turn.

Tool override acceptance

To verify a tool override, enable a harmless read and confirm execution in the session transcript. Disable the same tool and confirm that the Agent cannot execute it.

Current proof notes

The nested policy contract has not been verified in a real Kubernetes Agent deployment. Before claiming approval enforcement, verify effective native configuration, session approval and permissions, reviewer availability, and a normal Agent turn. Check write_actions with both a read and a non-read action; a permissive session can bypass MCP review. Native effective-policy introspection, installation composition, and later session or model changes remain unverified. Historical installation results in PR #228 do not prove these policies or production Helm and shared-cluster admission guardrails.

Record the commit, selected cases, nonsecret image references, runtime versions, model, pass/fail/skip counts, and sanitized evidence for any claimed proof.