fix(auto-merge): add opt-in retention of queued evaluations - #1151
Conversation
✅MegaLinter analysis: Success
See detailed reports in MegaLinter artifacts Your project could benefit from a custom flavor, which would allow you to run only the linters you need, and thus improve runtime performances. (Skip this info by defining
|
User evaluation at Every slot then completed successfully, in order 1 → 3 → 2, and the exact-attempt observer passed. This verifies preservation and serialization of the burst, without assuming slot/event-dispatch order. No PR approval, revocation, branch write, or App credential was needed for that scheduler observation. The existing behavioral fixtures separately verify authorization/revocation logic; this run does not claim to have modified a real PR through those privileged paths. The new observer is required by the aggregate CI check. A retry must rerun the full burst in one attempt; its failure diagnostic explains that recovery. Intentional actor-enforced lifecycle/evidence-removal cancellation and GitHub's 100-pending limit remain documented constraints. |
@coderabbitai review Please review this exact head. Pre-submission correctness, security, and simplification review is complete, including an independent review. Its one test-retry finding was fixed: an incomplete queue observation tells operators to rerun all jobs to recreate the same-attempt burst. Validation: the regression guard failed on the original queues and now passes; deleting each of the three queue settings independently fails the guard. Existing author, revocation, lifecycle, review-gate, credential-boundary, merge-group isolation, harden-runner, OIDC, and required-check wiring tests pass. GitHub's official workflow parser 0.3.61 accepts both changed workflows, and its expression evaluator passed 1,260 flag/event/action combinations with cancellation unchanged. The observer predicate passed eight complete/incomplete/paginated/failed-slot probes. ShellCheck and YAML formatting pass; Zizmor reports only the same 67 existing low-severity self-repository suggestions. Local actionlint lacks the new queue key and existing job.workflow_* context fields, so schema validation used the official GitHub parser and the accepted live workflows instead; no lint configuration was weakened. The live user evaluation records two simultaneous pending jobs followed by all three slots and the observer completing successfully. All 111 applicable checks now pass, with 74 skipped and nothing pending or failing. The fixture and observer have no repository write permissions or App credentials. Intentional actor-enforced workflow cancellation remains active, and the public documentation states the 100-pending limit and ordering constraint. |
CodeRabbit's correctly formed request at 09:46:17Z has no reaction, acknowledgement, or substantive review after the bounded no-reaction window (rechecked at 09:50Z). Review objects and provider checks are empty; its 09:47:49Z status says @codex review Please review the current queue repair. CI is fully green and the preceding comments record independent pre-submission review, unchanged cancellation across 1,260 cases, and the successful live three-job burst. No production credentials or authorization gates were widened. |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
✅ Action performedReview finished.
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 234e0e5fda
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Organization UI (inherited) Review profile: ASSERTIVE Plan: Team Run ID: 📒 Files selected for processing (4)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📜 Recent review details🧰 Additional context used📓 Path-based instructions (1)Pin third-party actions (non-`actions/*`, non-`github/*`, non-`devantler-tech/*`) to commit SHAs with a `# v` comment — enforced by `zizmor.yml`.📄 CodeRabbit inference engine (AGENTS.md) Files:
🔇 Additional comments (4)
📝 WalkthroughWalkthroughThe auto-merge workflow now uses conditional workflow-level queue settings and Merge Risk: ⚪ Minimal · up to This change preserves queued auto-merge evaluations during event bursts while retaining intentional cancellation for lifecycle and evidence-removal events. Queue behavior and required CI coverage are in place, with no current merge-blocking risk identified. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
devantler
changed the title
User evaluation at The final-head burst again reached one running job with two simultaneously pending jobs. Slots 1, 2, and 3 then completed successfully and without overlap (10:21:16–10:21:40Z, 10:21:46–10:22:07Z, and 10:22:10–10:22:33Z). The exact-attempt observer passed at 10:22:56Z. The fixture uses no repository write permissions or App credentials. Public reusable calls with the input omitted, explicitly false, and enabled all passed their eligibility checks; privileged mutation jobs correctly skipped for this human-authored PR. This is scheduler and public-call validation, not a claim of modifying real PRs through privileged paths. Independent integration review at this exact head found no actionable issues: all 99 required jobs map uniquely to 99 result entries, the newly merged cooldown coverage remains wired, and the production queue workflow is byte-identical to the independently reviewed opt-in repair. Existing authorization and intentional cancellation remain unchanged. Caller rollout and later flag removal remain open under #725 and #1155. |
@coderabbitai review Please review this exact current-base head. The previous Codex finding is fixed and its thread resolved: Pre-submission correctness, security, and simplification review is complete. The official GitHub workflow parser 0.3.61 accepts both changed workflows. Its expression evaluator passed 3,780 flag/event cases and 144 direct-event cases. The default-off regression failed before the fix, and removing any of the three queue declarations independently fails the guard. Author/revocation behavior, required-check isolation and parity, merge-group isolation, harden-runner-first, OIDC, credential boundaries, ShellCheck, and changed-YAML formatting pass. Local actionlint lacks the new queue key and existing job.workflow_* context fields, so schema validation uses the official parser and accepted GitHub workflows without weakening lint configuration. The branch integrates current main |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '=== HEAD AND BASE ==='
git status --short
git rev-parse HEAD
git merge-base HEAD origin/main 2>/dev/null || true
echo '=== CHANGED FILES ==='
git diff --name-status "$(git merge-base HEAD origin/main)" HEAD
echo '=== DIFF ==='
git diff --find-renames --find-copies --unified=80 "$(git merge-base HEAD origin/main)" HEAD -- \
.github/workflows/enable-auto-merge.yaml \
.github/workflows/ci.yaml \
.github/tests/test-enable-auto-merge-author-gate.sh \
README.md
echo '=== OUTLINES ==='
ast-grep outline .github/workflows/enable-auto-merge.yaml --items all 2>/dev/null || true
ast-grep outline .github/workflows/ci.yaml --items all 2>/dev/null || true
ast-grep outline .github/tests/test-enable-auto-merge-author-gate.sh --items all 2>/dev/null || true
echo '=== INPUT, CONCURRENCY, AND CALLER REFERENCES ==='
rg -n -C 5 --glob '*.yaml' --glob '*.yml' --glob '*.sh' \
'queue-pending-evaluations|^\s*concurrency:|^\s*queue:|enable-auto-merge\.ya?ml|uses:\s*\./\.github/workflows/enable-auto-merge' \
.github README.mdLength of output: 117964 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '=== FOCUSED SOURCE WITH LINE NUMBERS ==='
sed -n '1,190p' .github/workflows/enable-auto-merge.yaml
sed -n '330,435p' .github/workflows/enable-auto-merge.yaml
sed -n '2800,2965p' .github/workflows/ci.yaml
sed -n '4080,4260p' .github/workflows/ci.yaml
sed -n '1,75p' .github/tests/test-enable-auto-merge-author-gate.sh
sed -n '205,285p' .github/tests/test-enable-auto-merge-author-gate.sh
tail -n 80 .github/tests/test-enable-auto-merge-author-gate.sh
echo '=== ALL REPOSITORY REFERENCES ==='
rg -n -C 3 --hidden -g '!/.git/**' \
'queue-pending-evaluations|test-enable-auto-merge-queue-results|test-enable-auto-merge-queued|Enable Auto-Merge - Queue Slot|enable-auto-merge\.yaml' .
echo '=== REQUIRED CHECK WIRING PARITY ==='
python3 - <<'PY'
from pathlib import Path
import re
p = Path(".github/workflows/ci.yaml")
text = p.read_text()
start = text.index(" ci-required-checks:\n")
block = text[start:]
needs_block = re.search(r'(?ms)^ needs:\n(.*?)(?=^ runs-on:)', block).group(1)
env_block = re.search(r'(?ms)^ JOB_RESULTS: >-\n(.*?)(?=^ run:)', block).group(1)
needs = re.findall(r'^\s*-\s+([A-Za-z0-9_-]+)\s*$', needs_block, re.M)
results = re.findall(r'\$\{\{\s*needs\.([A-Za-z0-9_-]+)\.result\s*\}\}', env_block)
print("needs:", len(needs), "unique:", len(set(needs)))
print("results:", len(results), "unique:", len(set(results)))
print("missing_result_expressions:", sorted(set(needs) - set(results)))
print("orphan_result_expressions:", sorted(set(results) - set(needs)))
print("duplicate_needs:", sorted(x for x in set(needs) if needs.count(x) > 1))
print("duplicate_results:", sorted(x for x in set(results) if results.count(x) > 1))
PY
echo '=== YAML SHAPE CHECKS ==='
yq -o=json '{input: .on.workflow_call.inputs."queue-pending-evaluations", workflowConcurrency: .concurrency, disarmConcurrency: .jobs."disarm-untrusted-update".concurrency, autoMergeConcurrency: .jobs."auto-merge".concurrency}' .github/workflows/enable-auto-merge.yaml
yq -o=json '{queuedCall: .jobs."test-enable-auto-merge-queued", queueFixture: .jobs."test-enable-auto-merge-queue", queueObserver: .jobs."test-enable-auto-merge-queue-results"}' .github/workflows/ci.yamlLength of output: 80161 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '=== AGGREGATE GATE PARITY ==='
python3 - <<'PY'
from pathlib import Path
import re
text = Path(".github/workflows/ci.yaml").read_text()
block = text[text.index(" ci-required-checks:\n"):]
needs = re.findall(r'^ - ([A-Za-z0-9_-]+)\s*$', block.split(" runs-on:", 1)[0], re.M)
job_results_start = block.index(" JOB_RESULTS: >-\n") + len(" JOB_RESULTS: >-\n")
job_results_end = block.index(" run: |", job_results_start)
results = re.findall(
r'\$\{\{\s*needs\.([A-Za-z0-9_-]+)\.result\s*\}\}',
block[job_results_start:job_results_end],
)
print(f"needs={len(needs)}, unique_needs={len(set(needs))}")
print(f"results={len(results)}, unique_results={len(set(results))}")
print("missing:", sorted(set(needs) - set(results)))
print("orphan:", sorted(set(results) - set(needs)))
print("duplicate_needs:", sorted(x for x in set(needs) if needs.count(x) > 1))
print("duplicate_results:", sorted(x for x in set(results) if results.count(x) > 1))
PY
echo '=== WORKFLOW AND FIXTURE YAML SHAPES ==='
yq -o=json '{input: .on.workflow_call.inputs."queue-pending-evaluations", workflowConcurrency: .concurrency, disarmConcurrency: .jobs."disarm-untrusted-update".concurrency, autoMergeConcurrency: .jobs."auto-merge".concurrency}' .github/workflows/enable-auto-merge.yaml
yq -o=json '{queuedCall: .jobs."test-enable-auto-merge-queued", queueFixture: .jobs."test-enable-auto-merge-queue", queueObserver: .jobs."test-enable-auto-merge-queue-results"}' .github/workflows/ci.yaml
echo '=== TEST EXIT PATH AND DOCUMENTED INPUTS ==='
tail -n 20 .github/tests/test-enable-auto-merge-author-gate.sh
python3 - <<'PY'
from pathlib import Path
import re
workflow = Path(".github/workflows/enable-auto-merge.yaml").read_text()
readme = Path("README.md").read_text()
# Confine the README extraction to the Enable Auto-Merge section.
section = readme[readme.index("### 🔀 Enable Auto-Merge"):readme.index("### 🧹 Lint")]
workflow_inputs = set(re.findall(r'^ ([a-z][a-z0-9-]+):\n type:', workflow, re.M))
documented = set(re.findall(r'^\|\s*`([^`]+)`\s*\|\s*(?:Input|Secret)', section, re.M))
print("workflow_inputs:", sorted(workflow_inputs))
print("documented_inputs:", sorted(documented & workflow_inputs))
print("undocumented_workflow_inputs:", sorted(workflow_inputs - documented))
PY
echo '=== EXACT CHANGED-LINE LOCATIONS ==='
git diff --unified=0 c8e0575f3489bdc3890be9e4f0bc9bb202d2df1a HEAD -- .github/workflows/enable-auto-merge.yaml .github/workflows/ci.yaml .github/tests/test-enable-auto-merge-author-gate.sh README.mdLength of output: 3131
I found no actionable issues. The default-false input preserves the existing queue behavior. Opted-in non-cancelling workflow runs and both mutation jobs use The queue fixture has no write permissions. The observer is scoped to the current run attempt and requires all three queue slots to succeed. The required-check gate has complete parity: 99 unique The sandbox did not provide You are interacting with an AI system. |
Readiness at
Complete paginated comments, reviews, inline comments, and review threads contain no unresolved actionable findings; the earlier default-on finding is fixed, replied to, and resolved. Head/base, ownership, and activity are unchanged; the draft is clean and no auto-merge request is armed. The capability remains default-off, with caller rollout and flag removal owned by #1155. Promoting this delivery and proceeding through the normal head-pinned squash merge. |
1 participant
Bursts of auto-merge events can discard waiting evaluations, including a queued revocation. Add the temporary
queue-pending-evaluationsopt-in to retain pending work in both mutation jobs and non-cancelling workflow runs. It defaults to false, preserving existing callers and direct/required runs. Actor-enforced lifecycle and evidence-removal events keep their existing cancellation behavior.Tests cover omitted, explicit-false, and enabled public calls, plus a credential-free three-job burst with an exact-attempt completion check. Documentation states the 100-pending limit, ordering behavior, and temporary flag lifecycle. Authorization, actor gates, and production permissions are preserved.
Validation: failing-before/passing-after queue and default-off guards, three removal controls, author/revocation/lifecycle/review-gate fixtures, credential boundaries, and required-check wiring. Both workflows pass GitHub's official parser; 3,780 expression combinations plus 144 empty-input cases preserve cancellation and legacy defaults. Current-head live CI must pass before promotion.
Fixes #1154
Part of #725. Production rollout, direct/required activation, and flag removal remain tracked in #1155, with review due 2026-09-12.