DepUnlock

DepUnlock scans frozen research packages on HPC clusters, identifies version conflicts and security risks, recommends safe upgrades that preserve core research logic.

Comment

Inspiration

Researchers on HPC clusters run legacy operating systems like CentOS 7 with old glibc, old GCC, and strict module systems. Their Python dependencies get frozen for years because upgrading is risky. One wrong version bump can break C extensions, fail to link against system MPI, or silently change numerical results. We wanted to build an agent that understands these constraints and does the full upgrade workflow automatically, not just a scanner that prints a report.

What it does

DepUnlock is a GitLab Duo agent that resolves dependency conflicts on HPC clusters. Given a project and the user's HPC environment (GCC version, module system, OS, glibc), it autonomously:

  1. Scans all dependency files (requirements.txt, environment.yml, Pipfile, setup.py, pyproject.toml)
  2. Greps source code for deprecated API patterns (np.float, DataFrame.append, yaml.load)
  3. Creates GitLab issues for critical and high-risk findings, each with before/after code fix snippets
  4. Creates a branch, edits dependency files to bump versions, edits source files to fix deprecated APIs
  5. Opens a merge request with version bump and code fix tables
  6. Writes a changelog, posts it as a collapsible MR comment, and requests a Duo AI code review

It uses 14 GitLab Duo tools in a single autonomous turn. It determines safe version ceilings based on the user's glibc (manylinux compatibility), GCC (ABI compatibility), and Python version (wheel availability).

## How we built it

  • GitLab Duo Custom Agent (agents/agent.yml) with 18 registered tools and a system prompt that enforces fully autonomous execution
  • GitLab Duo Custom Flow (flows/flow.yml) with 5 sequential AgentComponents: scanner, analyzer, issue creator, patch/MR creator, changelog/review
  • Python backend (src/) with modules for scanning, analysis, recommendation, patching, and changelog generation, each exposing methods that produce GitLab-tool-compatible output (to_gitlab_issues, to_edit_instructions, generate_commit_message, generate_mr_note)
  • PlasmaFlow demo project (samples/) with intentionally outdated dependencies and deprecated API usage for demonstrating the agent
  • CI/CD pipeline (.gitlab-ci.yml) with test, analyze, issue creation, and GitLab Pages stages
  • 48 unit tests covering all new methods

## Challenges we ran into

  • The agent kept writing long analysis tables as text output instead of calling tools. We had to restructure every prompt to say "call tools immediately, do not write analysis" and add explicit "DO NOT STOP HERE" markers between steps.
  • The v1 flow spec uses context:component_name.final_answer for inter-component references, not component:output. Getting the input wiring wrong caused silent failures with no error messages.
  • The agent would pause after scanning and ask "shall I proceed?" We had to add rules like "NEVER pause, NEVER ask" and "if HPC info is missing, assume conservative defaults and proceed."
  • Hardcoding version targets made the agent brittle. We replaced static version tables with instructions on how to reason about glibc/manylinux tags, GCC ABI, and wheel availability.
  • The scanner component kept making dozens of individual gitlab_search calls for each package name instead of batch-reading dependency files. We had to remove grep from the scanner toolset and restructure the prompt to enforce a two-call pattern: find_files once, read_files once.

Accomplishments that we're proud of

  • The agent edits source code, not just dependency files. It greps for np.float, df.append(), and yaml.load(), then applies the fixes in the same MR alongside version bumps. Most dependency tools only touch version numbers.
  • It respects HPC constraints. It will not recommend numpy 2.0 on a CentOS 7 cluster because manylinux2014 wheels are the ceiling for glibc 2.17.
  • 14 distinct GitLab Duo tools orchestrated in a single autonomous agent turn with zero user interaction after the initial prompt.
  • Every issue includes a code fix snippet with the exact file, line number, and before/after change so researchers know what to do.
  • The flow passes structured data between 5 components using the v1 registry spec with proper from/as input wiring.

What we learned

  • Prompt engineering for action-taking agents is fundamentally different from chat prompts. The agent's default behavior is to explain what it would do. Getting it to actually call tools requires aggressive, repeated instruction.
  • The GitLab Duo tool registry has 89 tools. Knowing which ones exist and how they map to Python class names was essential for building the flow.
  • Inter-component data flow in the v1 spec is powerful but unforgiving. Wrong input references fail silently.
  • LLMs will always prefer generating text over calling tools unless the prompt structure makes tool calls the path of least resistance.
  • HPC dependency management is genuinely hard. The intersection of glibc constraints, compiler ABI, module systems, and Python wheel compatibility creates a constraint satisfaction problem that benefits from domain-specific AI guidance.

What's next for DepUnlock

  • Support more ecosystems: Spack specs, EasyBuild configs, CMake find_package, and Fortran/C mixed-language projects
  • Integrate with the CI pipeline so the agent triggers automatically on merge request events or scheduled pipelines via flow triggers
  • Add container-aware upgrades that suggest Singularity/Apptainer definitions when the host OS is too old for any safe upgrade path
  • Build a version compatibility database backed by actual wheel metadata from PyPI and conda-forge, replacing LLM reasoning with ground truth
  • Support multi-cluster environments where researchers deploy the same code across different HPC systems with different OS and module stacks

Built With

  • duo
  • gitlab
Share this project:

Updates