Use official llama.cpp images for Linux

[ilopezluna]

Use official upstream images from ghcr.io/ggml-org/llama.cpp for all Linux Docker images instead of building llama.cpp ourselves.

Scope: Linux only. macOS and Windows (Docker Desktop) are unchanged.

Key changes

• Dockerfile pulls from upstream images (server-vulkan, server-cuda13, server-rocm)
• New resolve-llama-upstream-image.sh script maps variants to upstream GHCR tags with validation
• resolveLlamaServerBin() discovers llama-server (upstream) or falls back to com.docker.llama-server (Desktop)
• Build-time ldd check catches missing shared library dependencies early
• Added libgomp1 to apt-install.sh (required by upstream CPU backend plugins)

Follow-ups

• Add MUSA and OpenVINO variants (comment)
• Use upstream releases for macOS/Windows (comment)

[gemini-code-assist]

Code Review

This pull request refactors the llama.cpp integration to vendor binaries from official upstream images, updates the vLLM version to 0.19.1, and adds automated image resolution scripts. Feedback identifies a need for explicit error handling in the Makefile when image resolution fails and suggests using BuildKit bind mounts in the Dockerfile to avoid image layer bloat and redundant file copies.

[ericcurtin]

Nice

[ericcurtin]

Musa is available btw:

server-rocm-b8895 "2026-04-23T06:19:54.284808434Z"
server-cuda-b8895 "2026-04-23T06:33:03.536404803Z"
server-cuda12-b8895 "2026-04-23T06:33:03.536404803Z"
server-vulkan-b8895 "2026-04-23T05:27:20.537711822Z"
server-musa-b8895 "2026-04-23T05:43:19.123167181Z"
server-intel-b8895 "2026-04-23T05:33:58.173828869Z"
server-openvino-b8895 "2026-04-23T05:20:15.646954065Z"
server-cuda13-b8895 "2026-04-23T06:26:18.288770943Z"
server-b8895 "2026-04-23T05:22:57.043953717Z"

[ericcurtin]

By the way what makes sense to me is these:

server-rocm-b8895
server-cuda13-b8895
server-vulkan-b8895
server-musa-b8895
server-openvino-b8895

We don't need "intel", "intel" is legacy at this point. We should add "openvino" for intel though.

[ericcurtin]

We can avoid building for Windows and macOS also, the releases are here:

https://github.com/ggml-org/llama.cpp/releases

[sourcery-ai]

Hey - I've found 2 issues, and left some high level feedback:

• The scripts/resolve-llama-upstream-image.sh helper assumes curl/jq are present and uses curl -si | tail -1 to extract the JSON body, which is brittle if headers/body formatting changes or stderr gets mixed in; consider splitting headers/body on the first blank line and emitting a clear error when curl or jq is missing to make failures easier to diagnose locally.
• Resolving LLAMA_UPSTREAM_IMAGE via a $(shell ...) in the Makefile happens at parse time and will call out to bash/curl/jq even for targets that don't build images; you might want to defer this resolution to the Docker build targets (or gate it with a variable) to avoid surprising failures for users running other make targets.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The `scripts/resolve-llama-upstream-image.sh` helper assumes `curl`/`jq` are present and uses `curl -si | tail -1` to extract the JSON body, which is brittle if headers/body formatting changes or stderr gets mixed in; consider splitting headers/body on the first blank line and emitting a clear error when `curl` or `jq` is missing to make failures easier to diagnose locally.
- Resolving `LLAMA_UPSTREAM_IMAGE` via a `$(shell ...)` in the Makefile happens at parse time and will call out to `bash/curl/jq` even for targets that don't build images; you might want to defer this resolution to the Docker build targets (or gate it with a variable) to avoid surprising failures for users running other make targets.

## Individual Comments

### Comment 1
<location path="scripts/resolve-llama-upstream-image.sh" line_range="87-96" />
<code_context>
+
+# ghcr_fetch_all_tags retrieves every tag in the repository (paginated).
+# Prints one tag per line on stdout.
+ghcr_fetch_all_tags() {
+  local token="$1"
+  local url="https://${REGISTRY}/v2/${REPO}/tags/list?n=10000"
+  local page=0
+
+  while [ -n "$url" ] && [ "$page" -lt 50 ]; do
+    page=$((page + 1))
+    local response
+    response=$(curl -si -H "Authorization: Bearer $token" "$url" 2>&1)
+
+    # Body is the last line of the curl -si output.
+    local body
+    body=$(echo "$response" | tail -1)
+    echo "$body" | jq -r '.tags[]'
+
+    # Follow the Link header for the next page, if any.
+    local link
+    link=$(echo "$response" | grep -i '^link:' | sed 's/.*<\(.*\)>.*/\1/' || true)
+    if [ -n "$link" ]; then
+      url="https://${REGISTRY}${link}"
+    else
</code_context>
<issue_to_address>
**issue (bug_risk):** Body parsing in ghcr_fetch_all_tags only captures a single line, which will break if the JSON response spans multiple lines.

`ghcr_fetch_all_tags` currently relies on `curl -si` and `tail -1` to extract the body, which only works if the JSON is a single line at the end of the response. If the registry returns multi-line JSON, `tail -1` won’t be valid JSON and `jq` will fail, breaking tag retrieval. Please adjust this to reliably separate headers and body (e.g., use `curl -sS` for body-only, or split `curl -si` output at the first blank line and pass the full body to `jq`).
</issue_to_address>

### Comment 2
<location path="README.md" line_range="279-282" />
<code_context>
   --build-arg BASE_IMAGE=nvidia/cuda:13.0.2-runtime-ubuntu24.04 \
   --build-arg LLAMA_SERVER_VARIANT=cuda \
-  --build-arg VLLM_VERSION=0.17.0 \
+  --build-arg VLLM_VERSION=0.19.1 \
   -t docker/model-runner:vllm .
 ```
</code_context>
<issue_to_address>
**issue (bug_risk):** Example uses VLLM_VERSION=0.19.1 but tags the image as vllm-0.17.0, which is likely inconsistent.

In the example, the build arg is updated to `VLLM_VERSION=0.19.1` but the image tag still suggests `vllm-0.17.0`. Please align the tag with the version (or vice versa) to avoid confusion.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[ericcurtin]

It would be better if the final images became full "FROM ggml" images. Copying around binaries means risk, incompatible OS version, dependant libraries that don't get copied, etc. Full "FROM ggml" resolves many of these issues.