# Ci Monitor

> Use when changing CI readiness, forge check collection, reruns, CI timeouts, CI step findings and…

Use when changing CI readiness, forge check collection, reruns, CI timeouts, CI step findings and fix rounds, or PR lifecycle monitoring.

- Page: https://ss33.ai/skills/kunchenguid/ci-monitor-61750049945
- Publisher: kunchenguid
- Category: [Software Development](https://ss33.ai/skills/category/software-development-t44z8z) › [DevOps & cloud](https://ss33.ai/skills/category/software-development-t44z8z/devops-and-cloud-unkt50)
- License: MIT
- Source: https://github.com/kunchenguid/no-mistakes/blob/12ad62fcc83caaeb094a58ef0ef0037d405888be/.agents/skills/ci-monitor/SKILL.md

## Skill

SKILL.md: https://ss33.ai/api/v1/ss33/download/kunchenguid/ci-monitor-61750049945

**CI Monitor Lifecycle**

- The green observation only clears the failure signal by default: the step keeps polling until the PR is merged, closed, or its idle timeout elapses, which is what every run has always done. Under the opt-in `ci.release_after_checks_pass` (`sctx.Config.CI.ReleaseAfterChecksPass`, resolved from the operator's global `ci` block and overridable by a trusted repository config) `CIStep.Execute` returns readiness in `StepOutcome.CIReadyNoCI` instead of polling again; `Executor.executeStep` commits it with CI completion through `db.CompleteCIStep`, which also stamps `runs.ci_released_at`, then emits readiness after the step-completed event. Release readiness must never appear with a running CI step, which would make AXI promise a live monitor. Readiness-write and terminal-verdict read failures fail the run. The run ends there and releases its worktree. `Executor.terminalRunStatus` records `types.RunChecksPassed` only when that durable release marker is set (with `pr_state` open and the CI step `completed`); it deliberately does not infer the verdict from `ci_ready_at`, because a monitoring run also records readiness and can end with an open PR after a timeout gate is approved. `db.RecoverStaleRunsExcept` finalizes the durable state before generic stale-run failure if the daemon crashed before run completion, preserving the CI duration (`TestRecoverStaleRunsCompletedCI`, `TestRecoverOnStartup_FinalizesCommittedCI`); `axi` maps the status to `outcome: checks-passed` (`outcomeFor`) and renders the released contract, `types.RunStatus.Terminal()` includes it, and a merge observed afterwards writes only `pr_state` - never the run's recorded outcome, and never the CI step's status or duration (which is why a released run's recorded duration covers CI only). Regressions: `TestCIStep_DefaultKeepsMonitoringAfterGreenChecks` (the default), `TestCIStep_GreenChecksReleaseWithoutWaitingForTheMerge`, `TestCIStep_TrustedNoCIReleasesWithoutWaiting`, `TestExecutor_GreenCIRunRecordsChecksPassedAndReleases`, `TestExecutor_MergedCIRunStillRecordsCompleted`, `TestRenderDriveResult_TerminalChecksPassed`, `TestDriveRun_ReleaseAndWatchGuidance`, `TestExecutor_CIReleaseReadinessFollowsStepCompletion`, `TestExecutor_CIReadinessWriteFailureFailsRun`, `TestExecutor_TerminalVerdictReadErrors`, `TestReleasedChecksPassedGuidance_SyncedAcrossSurfaces`, `TestCIReleaseAfterChecksPassDefaultsOffAndResolvesFromTrustedConfig`.
- The CI step produces findings, not fixes: a settled observation returns one finding per issue with an `action` (`ciObservationFindings`, `internal/pipeline/steps/ci_findings.go`) and the executor's shared auto-fix loop drives fix rounds (`repairFromFindings` in `ci_fix.go` consumes `sctx.PreviousFindings`), so `auto_fix.ci` is counted by the executor from round history and a review-bot check (`scm.ReviewBots`, identified by `scm.Check.App`) parks as `ask-user` comment findings without spending a round. A published repair keeps monitoring inside the fix round and must call `sctx.MarkRunning()` so status returns to `running` for the readiness consumers. "CI Step Findings Model" below owns the contract and regression list.
- `ci_timeout` is an idle timeout, not an absolute deadline: only `timeoutAnchor` re-arms when the upstream default-branch tip advances, `started` stays fixed for poll pacing, and re-arm only ever extends the deadline (fail-safe on transient base-tip failures). It bounds the wait for checks to finish, and the merge watch after that unless `ci.release_after_checks_pass` ends the step at the verdict. Value semantics (`0` unset, negative unlimited sentinel, keyword parsing) live in `config.go`; keep `config.DefaultCITimeout` and `defaultConfigYAML` in sync (`TestDefaultConfigYAML_MatchesGoDefaults`). User-facing semantics are owned by `docs/src/content/docs/reference/global-config.md`.
- GitHub readiness is the union of the exact current PR head commit's check rollup and every Actions workflow run returned by the Actions API for that same SHA. A workflow rejected before creating jobs/check-runs is absent from the commit rollup but still present in that API; run discovery errors and unknown run states fail closed instead of certifying a green rollup. Regressions: `TestGetChecksIncludesFailedWorkflowRunMissingFromPRRollup`, `TestGetChecksBindsRollupAcrossABAHeadMovement`, `TestCIStep_FailedHeadWorkflowRunPreventsChecksPassed`.
- GitHub's raw commit `statusCheckRollup` returns every check run a commit ever had, including a same-named run a later run at that same head has already superseded (e.g. a required-check job re-triggered by `synchronize` after a pipeline auto-fix push). `Host.GetChecks` (`internal/scm/github/github.go`) collapses same-name reruns of one workflow to the newest `startedAt` (using Actions run identity to break timestamp ties, comparing against every retained duplicate, and preserving unordered records, including ambiguous pending replacements, before a `completedAt` fallback) while preserving independent workflows, unidentified external checks, and same-named commit status contexts, restoring the semantics `gh pr checks` already applies and this repo's own gate `verify.py` already assumes (last-wins). Collapse must run AFTER `appendUnrepresentedWorkflowRuns`, never before: that call dedupes the Actions-run union against checks by run ID over the FULL uncollapsed rollup, so collapsing first drops a superseded run's ID out of the "represented" set and the union re-adds the same stale run under its own workflow run name. Without this collapse, a monitor that keeps `synchronize` in its gate trigger set can loop forever: each auto-fix push legitimately fails the gate at the old head, a later same-head rebind turns it green, but the stale FAILURE stays visible and re-triggers another auto-fix round. Regressions: `TestGetChecksCollapsesSupersededSameNameCheckToLatestAtOneHead`, `TestGetChecksCollapseOrderingDoesNotLetWorkflowRunUnionResurrectSupersededCheck`, `TestGetChecksPreservesIndependentSameNameWorkflows`, `TestGetChecksPreservesIndependentSameNameExternalCheckRuns`, `TestGetChecksCollapseComparesNewestRunWithEverySameNameCandidate`, `TestGetChecksKeepsQueuedReplacementWithEqualStartTime`, `TestGetChecksPreservesUnorderedExternalPendingReplacement`, `TestGetChecksPreservesSameNameStatusContextAndCheckRun`.
- A published CI repair rebinds an existing pipeline attestation in the PR body to the new head (`restampPublishedAttestation` after `publishRepair`) only when the host has an existing attestation and a `PRContentReader`. All supported providers implement raw reads; Bitbucket owned templates carry the same marker as visible text. Owned restamps enforce provider budgets before the adapter can clamp (especially Azure). Hosts without readers skip with a warning. A restamp that cannot settle after retries is unsettled, not success. The CI fixer prompt may conclude that a stale/attestation/infrastructure red is not a code defect; a genuine failing test or build must still be fixed. Regressions: `TestCIStep_PublishRepairRebindsAttestationAcrossRepairPushes`, `TestCIStep_PublishRepairDoesNotMintAttestation`, `TestCIStep_PublishRepairSkipsRestampWithoutReader`, `TestRestampPRAttestation_MissingReaderIsSkipped`, `TestRebindPipelineAttestationHead_VerifyPyRoundTrip`, `TestCIStep_FixPromptPrefersSimplificationOverMachinery`, `TestCIStep_FixPromptDistinguishesCodeDefectFromExternalFailure`.
- CI readiness never treats an unproven empty forge check list as green. Ready requires observed all-green checks, or trusted default-branch `no_ci: true` with zero registered checks (`internal/pipeline/steps/ci.go` decides whether the declaration applies; `internal/cimonitor` owns the agent-facing log vocabulary and Ready/DeclaredNoCI parse). Delayed registration, pending checks, failures, errors, unknowns, and stale-head evidence stay not-ready; registered checks on a declared no-CI repo are still honored. Regressions: `TestChecksPassed_PR607RealLogSequence`, `TestCIStep_EmptyChecksWithoutNoCIStaysNotReadyPastOldGracePeriod`, `TestCIStep_EmptyChecksWithTrustedNoCIBecomesReady`, `TestCIStep_DelayedCheckRegistrationStaysNotReadyUntilGreen`, `TestCIStep_DeclaredNoCIWithUnexpectedChecksHonorsThem`, `TestEffectiveRepoConfig_NoCITrustedOnly`.
- Persistent provider check-read failures are not an invisible spin: 6 consecutive `GetChecks` errors while the PR is still open park the CI step at an ask-user gate whose provider-neutral finding names the provider CLI/credentials support and includes the underlying error (with a GitHub-specific clause requiring `gh` >= 2.50 for the `gh pr checks --json` call), and the streak resets on any successful read. Regression: `TestCIStep_PersistentCheckReadFailureParksAtAskUser`.
- Reap an orphaned monitor from outside its worktree with `no-mistakes axi abort --run <id>`; it needs only `NM_HOME` and never starts a stopped daemon. A known run succeeds only with durable terminal truth, a recorded nonterminal run fails unconfirmed, and only an unknown id is an idempotent no-op. Bare `axi abort` stays worktree/branch-scoped.
- A merged or closed PR observation transactionally completes an active run and its CI step; PR lifecycle state is monotonic, so duplicate or delayed observations cannot reactivate or regress a terminal run. Startup reconciles legacy `pending` or `running` rows that already hold terminal PR state before parked-run planning and generic crash recovery. Regressions: `TestUpdateRunPRStateFinalizesActiveTerminalOutcomes`, `TestUpdateRunPRStateIgnoresDuplicateAndDelayedRegressions`, `TestReconcileTerminalPRRunsFinalizesLegacyActiveRows`, `TestRecoverOnStartup_FinalizesLegacyTerminalPRRun`, e2e `TestTerminalPRRunDisappearsFromActiveListing`.
- A provider-reported `cancelled` check is never a job verdict, so the deterministic rerun runs strictly before any CI fix round: cancellation and a GitHub pre-run infrastructure failure (a job that failed in setup/action resolution before any repository step ran, flagged `Check.PreRunFailure` by the opt-in `scm.PreRunFailureDetector` and re-bucketed to cancel so it never masks a real test/lint failure, which cleared setup and failed a later step) are the outcomes that earn a rerun, either outcome after its budget parks as `ask-user` instead of entering the `auto_fix.ci` loop, and any genuine or unrecognized failure or merge conflict in the same poll suppresses reruns so real failures still escalate on their first observation. The budget is per check name per run and spent on request rather than on success, and a rerun is never issued once the published branch head no longer equals `runs.head_sha`, because it would certify a commit this run never delivered. Each outstanding rerun records its verified pipeline head and the same-name provider links visible when it was requested; it retires durably when the run head advances or a new conclusive non-cancel link appears. Retirement keeps the spent budget and never changes check buckets. A delayed same-named green sibling can satisfy the link trigger on the same head, matching the default branch's existing name-keyed masking; removing that limitation requires provider truth outside this policy. Classification, the deliberate `TIMED_OUT`/`STALE` exclusions, rollup-lag grace, and retirement live in `internal/pipeline/steps/ci_transient.go`; provider support is the optional `scm.CheckRerunner` (GitHub only), pre-run infrastructure detection is the optional `scm.PreRunFailureDetector` (GitHub only, folded in by `markPreRunInfraFailures`), and user-facing semantics are owned by `docs/src/content/docs/reference/repo-config.md`. Regressions: `TestCIStep_CancelledCheckIsRerunBeforeEscalating`, `TestCIStep_CancelledCheckStaysUnresolvedAfterItsBudget`, `TestCIStep_LaggingRerunRollupKeepsWaitingForTheRepublishedCheck`, `TestCIStep_SameHeadGreenRerunEmitsChecksPassed`, `TestCIStep_DelayedSameNameCheckRetainsLegacyNameBehavior`, `TestCIStep_ResolvedRerunDoesNotParkALaterGreenHead`, `TestRetireResolvedReruns`, `TestRetireResolvedRerunsRetriesAfterPersistenceFailure`, `TestCIStep_MovedPublishedHeadTerminatesInsteadOfRerunning`, `TestCIStep_MovedPublishedHeadClearsCIReadiness`, `TestClassifyCheckFailure`, `TestMarkPreRunInfraFailures_RetriesInfraButNotGenuine`, `TestMarkPreRunInfraFailures_OptInGated`, `TestPreRunFailures_FlagsSetupFailureNotGenuine`, `TestPreRunFailures_FailsClosedOnUnreadableRun`.
- **Terminal is not pending.** Readiness must reject every non pass/fail/skip bucket (`hasUnresolvedChecks`), but only checks that can still finish on their own (`hasExecutingPendingChecks`) may defer escalation. A GitHub run held for maintainer approval (`scm.Check.AwaitingApproval`: `action_required` with no jobs) is pending for readiness but never defers another check's failure (`internal/pipeline/steps/ci_awaiting_approval_test.go`). A `cancel` bucket - GitHub `CANCELLED`, GitLab `canceled`, Bitbucket `STOPPED`, and how GitHub reports a job killed by its own `timeout-minutes` - is a published conclusion that nothing will replace, so with no rerun outstanding it is reported as an `ask-user` transient finding (`unresolvedTransientFindings`, selected by `cancelledWithoutRerun`) instead of waiting. Conflating the two is the #628 regression that hung real runs for their whole `ci_timeout`; an unrecognized bucket is deliberately still treated as waiting, because unknown is not evidence of terminal. Regressions: `TestCIStep_CancelledCheckAmongPassingChecksEscalatesInsteadOfPollingForever`, `TestCIStep_ZeroRerunBudgetEscalatesCancelledCheckWithoutMakingItReady`, `TestCIStep_BitbucketStoppedCheckParksForADecision`.
- A green GitHub PR reporting `mergeStateStatus: BEHIND` (`scm.MergeableBehind`, parsed by `parseMergeableState` in `internal/scm/github/github.go`; GitHub reports it only when the base requires up-to-date branches) reads as MERGEABLE and reports `checks-passed` unless trusted `ci.rebase_behind` opts in (off by default: each rebase is another full pass). Opted in, it is never `checks-passed`: once a second consecutive poll sees it green and BEHIND at the same resolved base tip (`behindBaseTip` in `ci.go`), it becomes one `auto-fix` `ci-behind-base` finding that `parseCIFixTargets` routes into the merge-conflict rebase repair (`ciFixTargets.BehindBase`), so it revalidates from Review like a conflict. Other providers never report BEHIND. Regressions: `TestGetMergeableStateReadsMergeStateStatusBehind`, `internal/pipeline/steps/ci_behind_base_test.go` (default off: `TestCIStep_GreenBehindPRReportsChecksPassedWhenRebaseBehindIsOff`), `TestCIRebaseBehind_OffByDefaultAndTrustedOnly`.
- A rebase repair (merge conflict or `BehindBase`) normally revalidates from Review because ancestry cannot prove continuity. Trusted opt-in `ci.carry_review_on_replay` lets `recordRepair` try `recordReplayedRepair` first: `pureReplayGap` (`ci_replay.go`) must prove the repaired head is the reviewed chain replayed onto exactly `CIStep.replayBase` (the resolved rebase target, bound only for that repair's recording) - equal-length linear chains, each reviewed commit `git merge-tree`-replaying conflict-free onto its new parent to the repaired commit's exact tree, and `git patch-id --verbatim` over the full binary diff equal. Any shape mismatch or git error fails closed to today's Review restart; the flag is ignored under `ci.revalidate_repairs: true` and when a gate is anchored `after: review` (Test restart would skip it). On success the approval moves via `db.UpdateRunHeadSHAWithCarriedReview`, a compare-and-swap on the approval the proof used (never resurrects a revoked one), and the outcome is `RestartFrom: types.StepTest` (`ciRepairResult.restartFrom`). Nothing publishes early. That outcome does not survive a parked CI gate (a Fix response replaces it, a daemon restart drops it), so the same write sets the durable `runs.ci_test_restart_pending`, which only `UpdateRunPublication` clears (CI never publishes while it is set, so only Push can). While set, `recordRepair` restarts the unchanged head at Test and sends a commit on top to `recordLocalRepair`, a fix round whose fixer failed records its worktree head through `recordRepair` instead of monitoring (a commit it left on top revalidates from Review, the unchanged head restarts at Test), and `Resume` restarts a recovered CI gate's approval at Test. Regressions: `internal/pipeline/steps/ci_carry_review_on_replay_test.go`, `TestCICarryReviewOnReplay_OffByDefaultAndTrustedOnly`, `TestExecutor_RestartFromTestKeepsTheCompletedReview`, `TestExecutor_RecoveredCIGateApprovalRestartsAPendingCarryAtTest`.
- CI readiness is read from the provider's live PR head check rollup on every poll, so it always describes the head the forge currently has for that PR; no recorded SHA gates it, and a run whose row still names a pre-advance commit must still recognize green at the head the pipeline last pushed. Regression: `TestCIStep_GreenChecksAtAdvancedHeadAreRecognizedWhileRunTracksOlderHead`.

**CI Step Findings Model**

- `CIStep.Execute` has the review step's shape: a fix round (`sctx.Fixing`) first repairs the findings the executor selected (`repairFromFindings`, `internal/pipeline/steps/ci_fix.go`), then the monitor runs; every settled observation (all checks finished, every authorized transient rerun spent, at least one issue) returns one finding per issue with its action (`ciObservationFindings`, `ci_findings.go`) and the executor's shared loop decides what follows. The step never counts or limits fix rounds; `auto_fix.ci` is enforced by the executor from the round history, which survives `ResetStepsFrom` and daemon restarts. `step_results.ci_fix_attempts` is retired (column kept, never read).
- Classification reads provider structure only: a job verdict (fail bucket) and a merge conflict are `auto-fix` errors; an unresolved transient is `ask-user` (`unresolvedTransientFindings`); a check whose `scm.Check.App` (GitHub `checkSuite.app.slug`) is in `scm.ReviewBots` (Greptile: `greptile-apps`) becomes one `ask-user` warning per unresolved bot comment, bounded (`maxReviewBotCommentFindings`), so it never spends a round. An empty `App` is never a bot. The comment half uses provider-scoped `scm.ReviewBotForLogin`: GitHub matches `greptile-apps` and `greptile-apps[bot]`, while GitLab matches only `greptileai`; only GitHub can attribute a *check* to a bot, so a GitLab run reaches the comment read through the green/not-yet-registered path below and `on_failure` reads nothing there. `scm.Capabilities.ReviewComments` is the opt-in (`GetReviewComments`), true on GitHub (unresolved review threads) and GitLab (unresolved discussion notes: `resolvable && !resolved`, not a system note, bot-authored, diff position for the anchor); a GitLab host without the project path answers `scm.ErrUnsupported` rather than erroring, so it degrades instead of parking. Under opt-in `ci.review_bot_comments: always` (default `on_failure`), a bot check that completed green, or a registered bot with no check on the head yet (its earlier threads can still be unresolved), goes through the same path (`greenReviewBotFindings` -> `ciObservationFindings` with `greenBots`): its unresolved comments are the same ask-user findings and block `checks-passed`, a green bot with none produces nothing, an unreadable comment list withholds readiness and parks after `consecutiveCheckErrorLimit`, and `VerifyApprovalOverride` records an approval over them as an override. The fix agent, not metadata, decides whether an unknown red is about the code: its `code_change_needed=false` conclusion parks the selected findings as `ask-user` (`ciRepairParkOutcome`).
- `types.Finding.Check` (display name), `types.Finding.CheckID` (provider identity), and the `FindingCategoryCI*` constants are the routing keys the fix half reads (`parseCIFixTargets`); `Finding.UnmarshalJSON` and `findingWire` copy fields explicitly, so a new field must be added there or it drops on every parse. Findings ride the IPC event stream, so per-comment findings stay bounded.
- A published repair keeps monitoring inside the same fix round and must call `sctx.MarkRunning()` (executor-owned: durable status back to `running` plus a `step_started` event), or the step stays `fixing` and neither the TUI nor `axi`'s `checks-passed` (`ciReadyToMerge` requires `running`) can surface. The post-repair stale-evidence guard is `lastRepairStillUnverified`: selected checks are snapshotted before a resumed repair, and targets that remain red with neither a newer completion/execution discriminator nor an observed pending state keep the monitor waiting instead of re-escalating. Findings deferred by a fix round survive pending observations and terminal monitor errors until a fresh settled issue set or an established clean result supersedes them.
- Regressions: `TestCIStep_GreptileOnlyRedParksAsAskUserAndSpendsNoAutoFixRound`, `TestCIStep_MixedGreptileAndTestFailureRoutesOnlyTheTestToAutoFix`, `TestCIStep_PublishedRepairResumesMonitoringAndWaitsForTheRerun`, `TestCIResumedRepairSnapshotsSelectedCheckCompletion`, `TestTerminalFailureFreshnessUsesExecutionIDWithoutCompletionTime`, `TestCIStep_PersistentCheckReadFailureParksAtAskUser`, `TestCIStep_UserFixOfTransientFindingIsHonored`, `TestCIObservationFindings_ClassifiesEachIssueByProviderStructure`, `TestExecutor_MarkRunningReturnsAFixingStepToRunning`, `TestExecutor_RestartsValidationFromRequestedStep`, `TestGetChecksCarriesCheckSuiteAppSlug`, `TestParseFindingsJSON_RoundTripsCICheckFields`, `TestCIStep_GreenReviewBotCommentsParkUnderAlways`, `TestCIStep_GreenReviewBotCommentsIgnoredByDefault`, `TestCIStep_GreenReviewBotUnreadableCommentsParkUnderAlways`, `TestCIStep_JobFailureWithGreenReviewBotFollowsThePolicy`, `TestCIStep_VerifyApprovalOverride_GreenReviewBotComments`, `TestCIStep_UnregisteredReviewBotCommentsFollowThePolicy`, `TestReviewBotCommentsOnGitLabFollowThePolicy`, `TestCIStep_GitLabReviewBotDiscussionCommentsParkUnderAlways`, `TestGetReviewCommentsReadsUnresolvedBotDiscussions`, `TestReviewBotRegistry_ResolvesAppAndLoginToTheSameBot`. Step tests drive the loop with `stepstest.ExecuteWithAutoFix` (the executor's auto-fix loop in miniature); a test that calls `Execute` once sees only the observation.