✨ feat(policy): evaluate repository retention plans#609
Open
gaborbernat wants to merge 3 commits into
Open
Conversation
gaborbernat
added
documentation
gaborbernat
enabled auto-merge (squash)
July 18, 2026 03:17
gaborbernat
disabled auto-merge
July 18, 2026 03:18
gaborbernat
enabled auto-merge (squash)
July 18, 2026 03:18
Merging this PR will not alter performance
Comparing 
Footnotes
|
gaborbernat
force-pushed
the
feat/evaluate-repo-retention-plans-481
branch
from
July 18, 2026 04:00
3f13b9c to
5e98e34
Compare
Add an ecosystem-neutral retention planner. A policy holds ordered keep and expire rule groups; the engine evaluates one project's candidates and returns a deterministic decision per artifact without reading storage. A keep rule wins over an expire rule, matching Google Artifact Registry cleanup precedence, and each decision names the rule that decided it. Candidates carry a caller-assigned version rank, so the engine orders keep-latest and output without naming a package format. Decisions sort by rank, artifact, then digest, so repeating an evaluation yields byte-identical output. A criterion bench plans a generated million-record repository to hold evaluation linearithmic. Refs tox-dev#481
Adapt an index's hosted upload records into neutral retention candidates and stream their decisions. Uploads scan in key order, so a project's files group without a second pass; the adapter ranks each project's releases newest-first under PEP 440 and collapses equal spellings to one rank. The scan reads only indexed metadata, so an interrupted evaluation writes nothing. The result carries the policy version and the metadata frontier a later apply step needs to reject stale input. Refs tox-dev#481
Describe the rule syntax, keep-over-expire precedence, PEP 440 version ordering, byte estimates, snapshot and policy identity, the output schema, and the side-effect-free contract. Refs tox-dev#481
gaborbernat
force-pushed
the
feat/evaluate-repo-retention-plans-481
branch
from
July 18, 2026 04:25
5e98e34 to
2ba75cd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Operators need to know which artifacts a retention policy would keep and which it would remove before anything deletes them. This adds the evaluation half of that feature, reading one metadata snapshot, applying the configured rules, and returning an ordered plan. It computes and reports decisions only. Applying a plan, exposing it over HTTP or the CLI (#482), and reclaiming bytes stay out of scope, and this planner is built for #482 to consume without rebuilding it.
The engine lives in
peryx-policyand stays ecosystem-neutral. A policy holds two ordered rule groups,keepandexpire, with selectors for age, source, project prefix, keep-latest count, cache, trash, and orphan state. A keep rule wins over a matching expire rule, the precedence Google Artifact Registry cleanup policies define, and every decision names the rule that decided it. 📋 Candidates carry a caller-assigned version rank, so the engine orders keep-latest and its output without naming a package format; decisions sort by rank, then artifact, then digest, so repeating an evaluation over the same snapshot and policy produces byte-identical output. Each removal records its estimated bytes and the project's surviving versions, and the plan carries a policy version and a metadata frontier a later apply step reads to reject stale input.The
peryx-ecosystem-pypiadapter feeds that engine. It scans an index's hosted upload records in key order, groups each project's files, ranks their releases newest-first under PEP 440 (pre-release, local, and invalid legacy versions all get a documented position), and streams one project's decisions before reading the next. 🔒 The scan opens read transactions only and never enumerates backend blobs, so an interrupted evaluation writes no metadata or blob state. Evaluation stays linearithmic in the number of records, so a million-file repository plans in well under a second.The ecosystem boundary sits at PyPI hosted uploads, the surface #481 scopes through PEP 440. The neutral engine is ready for an OCI adapter, but OCI carries no per-artifact publish time or yank state, so its age and visibility signals need a separate design; cached-page eviction and blob reclamation are their own concerns (#571-#573). This planner covers #481; the OCI adapter and the API and CLI surface remain follow-ups.
Closes #481