OpenClaw EnterpriseDOCSGitHub

GitHub Actions testing flow

Overview

Ordinary CI lanes reject missing or skipped coverage.

Entry Points

Flow

graph TD
  A["PR or other CI event"] --> S["Select impact mode"]
  A --> AQ["Select advisory scope"]
  AQ -->|eligible; focused source verified| AJ["Selected advisory jobs: setup, tests, cleanup, results"]
  A --> N["Suite Audit and Static Checks: every mode"]
  S -->|full PR| Q["Report affected packages (advisory)"]
  S -->|full or tests| B["All ordinary or selected CI lanes"]
  B --> F["Prepare owned resources"]
  F -->|prepared| H["Run tests and validate cases"]
  F -->|preparation fails| J["Owned-resource cleanup"]
  H --> J
  H --> K["Sanitized lane results"]
  J --> G["CI Required: verify mode and job states"]
  S --> G
  N --> G
  K --> G
  G -->|full or tests| L["Aggregate same-source lane results"]
  G -->|docs| M["Documentation coverage result"]
  L --> R["Full CI coverage result"]
  C["Scheduled tick or approved manual dispatch"] --> P["Source and environment preflight"]
  P -->|scheduled tick skips admission| D["Finish without suite jobs"]
  P -->|eligible| V["Acquire execution lock and revalidate"]
  V -->|scheduled recheck skips admission| D
  V -->|eligible| E["Protected test jobs"]
  V -->|all or dev-up-k3d| U["Credential-free installation job"]
  U --> T
  P -->|missing protection| X["Failed check"]
  V -->|missing protection| X
  E --> T["Owned preparation, tests, cleanup and aggregation"]

Execution Trace

1. Select one source commit and coverage group

.github/workflows/ci.yml:jobs, .github/workflows/full-integration.yml:jobs, .github/workflows/full-integration-execution.yml:jobs, scripts/ci/full-integration-preflight.ts:validateFullIntegrationPreflight, and scripts/ci/test-suites.ts:loadTestSuites

loadTestSuites reads ordered test, input, and preparation definitions. scripts/ci/run-tests.ts:main rejects noncanonical manifests before complete attempts. runner-inputs.ts:parseArgs rejects unknown options and duplicate --test-name. selectLane validates paths and names. runFile uses an anchored, escaped pattern for one required standalone case. runner-aggregate.ts:validateLaneEvidence rejects focused results.

CI Impact selects coverage. Suite Audit and Static Checks verifies checkout identity. Ordinary CI uses the event checkout without external service credentials. Kubernetes runners require kernel NetworkPolicy enforcement.

Pull requests compare the verified merge to its first parent using that parent’s policy. The second parent must match the event head. Main pushes use before policy for the before→verified after diff. Missing history, forced pushes, and older policies retain full CI. Decoding preserves leading UTF-8 BOMs. scripts/ci/impact.ts:documentationPath admits allowlisted Markdown, docs/docs.json and specs/rfcs/**/*.png. Generated API references, other images, executables and symlinks require full CI.

Registered test changes may include docs and their own lane-manifest entries. CI runs the owning lanes from either tree, adds checks-baseline-2 for TypeScript, and uses checks-baseline-1 if only the fixture job would run. Test-only main pushes, other changes, and remaining event types select full CI. Missing policy or unverifiable source selects full or fails closed.

CI Required rechecks mode and lanes: impact, static checks and selected lanes must succeed; other jobs must skip. Full/tests modes aggregate same-source results; docs emits no test artifacts. Impact summaries expose mode, fixed reason and accepted lanes, excluding paths and raw output. Selector errors fail.

affected-packages verifies full-mode PR checkouts and reports dependents without selecting tests or gating CI. Base-loaded policy cannot prevent PR workflow changes.

.github/workflows/qa-advisory.yml:eligibility selects fixed, nonempty scopes. Focused jobs verify SHA and ordered PR parents. QA materializes only the selected credential.

Full Integration pins github.sha. Scheduled admission rechecks main, cadence and receipts under lock. scripts/ci/full-integration-schedule-history.ts:readUnmarkedClock requires explicit targeting to ignore live shared work. scripts/ci/full-integration-preflight.ts:fetchGithubJson distinguishes malformed JSON from request failures; diagnostics omit raw content. Manual dispatch verifies dispatcher, team and environment. Slack serializes channel/app access; installation and Skipper Git serialize per branch. Retries require approved dispatch.

First-attempt all reuse requires completed execution (scripts/ci/full-integration-reuse.ts:validateResults). Success selects artifacts, skipping lanes. Failure uses preflight’s fresh fallback. QA awaits Slack despite failure, stops on cancellation, and requires selected results.

Migration and application suites own separate PostgreSQL servers. Kubernetes fixture lanes each own a cluster and PostgreSQL server. The shared action requires bridge netfilter before creating k3d nodes. Blacksmith runs repository credential HTTP, PostgreSQL, Unix-control, and credential-material checks. Fixture lanes verify NetworkPolicy enforcement. scripts/ci/run-tests.ts:runLane runs files sequentially except audited parallelFiles. State and cleanup stay runner-local.

Only the native IAM barrier test receives its migrated PostgreSQL database and both application and migrator connection details. The CLI excludes them from GITHUB_ENV. The unregistered supplier stays in this disposable database. Production writers still require separate barrier verification.

runtime-artifact supplies CI image-lanes, Full Integration's image-lanes-same-sha and Docker Model Runtime, and First Agent Smoke. Aggregates check producer and consumers.

run-ci-lane prepares and cleans resources, runs tests, and uploads sanitized results. Callers set source, credentials, environment and timeout; tests stop five minutes early.

Dependency caches for PR merge-refs and main exclude results and credential-bearing state. Protected jobs never promote PR artifacts.

2. Prepare resources under the job owner

scripts/ci/prepare.mjs:main and scripts/ci/prepare.mjs:ensureK3dCluster

See resource preparation.

scripts/ci/prepare.mjs:prepareLane pre-pulls logging-collector’s configured Collector and Node images and the shared Prometheus and Grafana pins. ensureDockerSourceImage reuses verified local digests or verifies downloads from scripts/ci/image-pull.ts:pullImage before publishing inputs. Registry 5xx/rate limits retry within the shared budget; missing manifests/authorization refusals fail immediately. Unpinned Node overrides retain test-owned pulls.

Full Integration requires compatible cache exports before OpenShell source builds. These retain their builder and fixed epoch; cache credentials stay in preparation. Mutable scopes make cache hits best effort.

scripts/ci/k3d-preparation-diagnostics.ts records Kubernetes host snapshots. Failed reads, bounded by time and size, save <state-file>.diagnostics.json outside the cluster directory. Fixture lanes use --no-rollback and registered teardown. Staged sign-in also disables rollback, but unacknowledged creation requires reconciliation instead of name-based deletion/retry. Observation failures preserve the original error. nodeLogExcerpt orders streams and retry summaries by UTC timestamp and ASCII comparison (CI guide).

scripts/ci/run-tests.ts:runFile watches Compute-managed Pods and namespace events in ready k3d clusters before tests delete them. On pass or failure, scripts/ci/k3d-diagnostics.ts:projectAgentNamespaceActivity records agentNamespaces: up to 200 Pod and 200 event records per file and cluster, retaining 40 entries. Messages are redacted and truncated; Pod specs are omitted. Raw watches use separate files in the cluster directory and are removed on completion. Cluster-wide watches can include sibling namespaces under fileConcurrency. Failed starts log Agent namespace activity unavailable. Each producing lane uploads the artifact.

Before Agent Pods exist, tests/helpers/kubernetes-real.ts:waitForDeploymentAvailable captures Deployment logs and Pod/events, retaining the last eight containerLogs records. Without runner capture, it emits a projected diagnostic. unavailable means a failed observation; missing records do not prove healthy startup.

Dedicated Codex preparation and the operator's offline profile generator share scripts/lib/codex-seccomp-profile.ts:deriveCodexBwrapProfile. Preparation requires an actual workspace write and denied write to a container-writable outside path before publishing the selected Localhost profile to the live suite. Native runtime-image tests trust a dynamic Codex Docker seccomp profile only when OPENCLAW_ENTERPRISE_CI_STATE records the exact prepared cluster.codexDockerSeccompProfile path and SHA. A self-hashed profile without that state is not CI proof; the standalone fallback remains the pinned reviewed manual profile. Production node provisioning remains outside CI ownership; see Codex sandbox setup.

scripts/ci/staging-qualification.ts:prepare-signin binds verified publications to the reviewed v3 inputs and catalog; staging qualification owns credential isolation and historical-pair limits.

3. Execute and account for actual cases

scripts/ci/run-tests.ts:runFile starts Node via runner-process.ts:runNode. runner-reporter-output.ts:readFileEvidence accounts for the JSON Lines output from reporter.ts:jsonLinesReporter. runner-lifecycle.ts:runCompleteLifecycle forks the same entry for each phase. The selected checkout supplies the manifest, tests, preparation, and cleanup.

runner-inputs.ts:normalizeFile validates file.env. runFile merges filtered inherited values, declared inputs, lane.env, preparation defaults, file.env, and generated values in that order. Later values win. Before OpenShell preparation, validateLaneInputsBeforeSideEffects reads the manifest and runFile forwards model selectors with file.env precedence. Unfocused openshell, openshell-policy, and openshell-installers require both credentials. scripts/k3d scopes preparation with --selected-file.

The reporter retains names, locations, outcomes, failure messages (600 characters) and top frames (240). Redaction removes repository paths, substitutes nonpublic job/child values (8+ characters) with [env:NAME], and masks token/key shapes, URL passwords and password= as [redacted]. Generated values require a matching shape.

HTTP diagnostics retain numeric actual/expected statuses, allowlisted OCC codes and available ChatGPT operation/status. Denied traffic retains an allowlisted category and plugin failures an allowlisted readiness/rollout stage. Rollouts retain allowed Pod phases, readiness/scheduling flags, restart counts, exit codes and allowlisted reasons.

skipper-release.ts:runSkipperOperation projects captured Skipper errors. After failed installed proof, the workflow invokes skipper-failure-receipt.ts to publish the wrapper's checkpoints.

scripts/ci/openshell-progress.mjs:emitOpenShellProgress emits fixed lifecycle categories; scripts/ci/run-tests.ts:runFile adds lane and file labels. Installation ordinals and millisecond offsets enter file receipts, not live logs. Readers bound partial records; parent byte limits apply first. Markers and separators stay out of failure tails. Timing and case accounting remain separate.

Skips and TODOs fail required-case checks. Coverage requires actual cases and final reporter output. Results retain failure, timeout, and cleanup outcomes.

4. Clean up and report results

scripts/ci/cleanup.ts:main and scripts/ci/run-tests.ts:main

.github/actions/run-ci-lane/action.yml replaces only the retried lane’s sanitized result. Separate attempt-<run attempt>-<artifact-prefix>-<lane> artifacts preserve failed --failed attempts outside aggregation.

scripts/ci/memory-sampler.sh samples memory pressure, RSS, process IDs, containers and disk every 15 seconds, excluding arguments and environment. k3d-fixture-configuration samples during tests and logs live summaries for runner-loss diagnosis. Runtime startup and browser lanes sample from before preparation through cleanup, then print summaries.

Each attempt uploads memory-<artifact-prefix>-<lane>-attempt-<run attempt> (about 1 MiB maximum). Sampler and upload errors do not fail the lane. Only live summaries survive runner loss.

For images-packaging, scripts/ci/export-image-reconciliation.ts retains prepared controller and runtime tag cleanup records when available. Planned records do not prove creation; tags confer no authentication or deletion authority. Missing state or empty inventories cannot prove cleanup. The runtime-images test tag, other resources and private environment values are excluded. Export, upload or runner failure can prevent retention.

Fixture bootstrap diagnostics upload separately as diagnostics-<artifact-prefix>-<lane>-attempt-<N>. They survive cluster/private-state cleanup but cannot satisfy required test results.

Per-file cleanup releases databases; job cleanup removes owned resources, including Collector objects and logging data. Sign-in transitions and proof readers reject already-expired issuance observations. Cleanup errors fail except QA image warnings. cleanup-staged-signin.ts passes the prepared state digest and custody to scripts/ci/cleanup.ts:cleanupState: locked validation precedes exact server/network deletion and volume checks. A persisted mutation marker forbids fixture fallback without completed cleanup; verified pre-mutation refusals retain fallback authority. Named-volume limits remain.

scripts/ci/cleanup.ts:execFile awaits exit and drained output. On POSIX, scripts/ci/cleanup-command.ts kills its group on caller IPC closure, including caller SIGKILL. Completion drains output, kills grouped descendants and returns command status. At timeout, TERM then KILL still apply after direct-child exit. Escaped groups are unsignaled; retained streams close after grace. Windows bounds streams and terminates only the direct child.

scripts/ci/runner-process.ts:killDescendants stops cleanup groups before sweeping descendants. tests/helpers/postgres-worker-revision-fixture.mjs:disposeStep records timeouts and continues. Root hooks allow two seconds for reports (immediate on cleanup failure), with a 60-second skipped-hook backstop. Failed resources remain recorded.

scripts/ci/runner-aggregate.ts:aggregateGroup checks SHA, ownership, jobs and execution. Reused lanes retain separate owners; fresh lanes retain matrix parents. scripts/ci/cleanup.ts:main records final cleanup; missing state cannot clear failure. Installed Skipper records its own. Execution evidence requires every lane's required cleanup to pass. Later delivery failures keep checks red; missing artifacts cannot qualify. ssh-host needs a disposable host. Runner loss can prevent teardown and erase private RUNNER_TEMP state; external reconciliation awaits an approved resource ledger.

Debugging and Verification

Search documentation