ci(release): make downstream release triggers idempotent and race-free - #1005
Merged
Conversation
The release pipeline chains non-idempotent cross-repo triggers, so a re-run for an already-published tag failed in new ways instead of resuming cleanly: - release-cli-desktop re-pushed docker-model-cli-desktop-module:<tag> and got "403 denied" because the tag was already published on a previous attempt. - Both trigger jobs identified the downstream run by querying "the latest run" right after dispatch, which races with concurrent/stale runs — a job could watch (and pass on) a run that wasn't the one it triggered. Changes: - release-cli-desktop: log in to Docker Hub and skip the trigger when the desktop module tag already exists (idempotent re-run, no more 403). - Both trigger jobs: capture the newest run ID before dispatch and wait for the first run with a strictly greater ID (run IDs are monotonic) instead of trusting "the latest run". - github-release: skip creation when the release already exists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
|
Note Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported. |
Contributor
There was a problem hiding this comment.
Hey - I've found 1 issue, and left some high level feedback:
- The trigger-and-wait logic for downstream workflows is now duplicated in two places; consider extracting this into a shared script or composite action so the monotonic ID handling and retry behavior stay consistent and easier to maintain.
- In the new run ID polling loops, you might want to surface more context in the failure case (e.g., last observed BEFORE_ID/CANDIDATE or total wait duration) to make diagnosing races or misconfigurations easier when the run ID cannot be determined.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The trigger-and-wait logic for downstream workflows is now duplicated in two places; consider extracting this into a shared script or composite action so the monotonic ID handling and retry behavior stay consistent and easier to maintain.
- In the new run ID polling loops, you might want to surface more context in the failure case (e.g., last observed BEFORE_ID/CANDIDATE or total wait duration) to make diagnosing races or misconfigurations easier when the run ID cannot be determined.
## Individual Comments
### Comment 1
<location path=".github/workflows/release.yml" line_range="538-541" />
<code_context>
- echo "⚠️ Could not extract run URL from gh output, querying latest run..."
- sleep 5
- RUN_ID=$(gh run list \
+ RUN_ID=""
+ for _ in $(seq 1 30); do
+ sleep 3
+ CANDIDATE=$(gh run list \
--repo docker/inference-engine-llama.cpp \
--workflow release-cli-dd.yml \
</code_context>
<issue_to_address>
**issue (bug_risk):** Guard against empty or non-numeric CANDIDATE values before numeric comparison
If `gh run list` fails or returns an unexpected payload, `CANDIDATE` may be empty or non-numeric. In that case `[ "$CANDIDATE" -gt "$BEFORE_ID" ]` will error (`integer expression expected`) and fail the job. Please add a guard that checks for non-empty, numeric values before the `-gt` comparison (e.g., skip the iteration when `CANDIDATE` is empty or not `[0-9]+`).
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Comment on lines
+538
to
+541
| RUN_ID="" | ||
| for _ in $(seq 1 30); do | ||
| sleep 3 | ||
| CANDIDATE=$(gh run list \ |
Contributor
There was a problem hiding this comment.
issue (bug_risk): Guard against empty or non-numeric CANDIDATE values before numeric comparison
If gh run list fails or returns an unexpected payload, CANDIDATE may be empty or non-numeric. In that case [ "$CANDIDATE" -gt "$BEFORE_ID" ] will error (integer expression expected) and fail the job. Please add a guard that checks for non-empty, numeric values before the -gt comparison (e.g., skip the iteration when CANDIDATE is empty or not [0-9]+).
ericcurtin
approved these changes
Jul 9, 2026
pull Bot
pushed a commit
to TheTechOddBug/model-runner
that referenced
this pull request
Jul 10, 2026
Follow-up to docker#1005 (Sourcery review). The monotonic run-ID polling loops compared `$CANDIDATE` numerically without validating it. If `gh run list` fails transiently or returns an unexpected payload, CANDIDATE can be empty or non-numeric and `[ "$CANDIDATE" -gt "$BEFORE_ID" ]` errors with "integer expression expected" — the exact kind of flake this logic was meant to absorb. - Tolerate transient `gh` failures mid-poll (`2>/dev/null || echo ""`) and skip any iteration where CANDIDATE is empty/non-numeric instead of erroring. - Validate BEFORE_ID up front and fail with a clear message (before dispatch) if it can't be read. - Include before/last-seen IDs and wait duration in the failure message to make race/misconfig diagnosis easier. Applied to both the desktop (release-cli-dd.yml) and packaging (release-model.yml) trigger jobs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
While releasing v1.2.6 the pipeline failed repeatedly across three different root causes, and each full-pipeline retry re-triggered steps that had already completed for the tag, producing new failures instead of resuming cleanly:
verify-docker-cefailed because theprepareresolver'sv*glob also matched the off-trackvdmr-v0.1.1tag. Already fixed by ci(release): keep image release on its own v* track #1004 (strictvX.Y.Zfiltering).ubuntu2510linux/arm/v7,mk-build-depsexit 29). Retry succeeded.403 deniedre-pushingdocker-model-cli-desktop-module:v1.2.6, because the tag was already published on an earlier attempt.The recurring, structural problem:
release.ymlorchestrates non-idempotent cross-repo triggers, and both trigger jobs identify the downstream run by querying "the latest run" right after dispatch — which races with concurrent/stale runs (a job can watch, and pass on, a run it didn't trigger).Changes
release-cli-desktop: log in to Docker Hub and skip the trigger when the desktop module tag already exists — a re-run for an already-published tag no longer 403s.release-cli-desktop,release-cli-docker-ce-trigger): capture the newest run ID before dispatch and wait for the first run with a strictly greater ID (run IDs are monotonic), instead of trustinggh run list --limit 1.github-release: skip creation when the release already exists.Fixes 2 and 3 originate in external repos (
docker/packaging,docker/inference-engine-llama.cpp); this PR makes the orchestrator tolerant of retries so those transient/duplicate conditions no longer break the whole release.🤖 Generated with Claude Code