Inspiration
Cloud AI code-review tools mean your code has to leave your machine. For teams under strict confidentiality policies finance, healthcare, defense, or any company that simply doesn't want its proprietary code training someone else's model that's not a preference, it's a hard blocker. Apple Silicon's Arm64 architecture, combined with Apple's own MLX framework, made it possible to run a genuinely useful code review model entirely on a laptop's own chip. We wanted to build something that proves that's not just technically possible, but fast and efficient enough to actually use.
What it does
DiffGuard reviews your local, uncommitted git changes entirely on-device:
Reads your real git diff never anything committed or pushed, just your working changes.
Builds a lightweight local RAG index over the rest of your codebase, so the review understands existing conventions rather than judging the diff in isolation.
Runs a code-specialized small model (Qwen2.5-Coder-3B) through MLX — Apple's Arm-native, Metal-accelerated ML framework.
Verifies every finding against the actual diff text before showing it to you, flagging anything that doesn't check out as unverified rather than presenting it as confirmed.
Prints a severity-ranked report straight to your terminal.
Nothing is ever sent anywhere. You could disconnect your Wi-Fi after the one-time model download and DiffGuard would keep working, we verified this directly by testing with Wi-Fi off.
Measured against an identical CPU-only baseline (same model, same 4-bit quantization, only the Arm-native Metal acceleration path changes): ~1.5x faster generation, ~4.6x faster model load, ~9% less power draw, ~19% better power-efficiency per token, and the 4-bit quantized model itself is 71.9% smaller than the unquantized original.
How we built it
The core is a small Python pipeline: a git-diff reader, a local RAG indexer (sentence-transformers embeddings over the codebase), and a review step that calls the model through mlx-lm. We measured the actual Arm-specific optimization three separate ways rather than just one: generation speed and load time (MLX vs. a CPU-only llama.cpp baseline, same model, same 4-bit quantization), real system power draw via macOS's powermetrics, and model size on disk (4-bit MLX vs. the unquantized fp16 original, checked via Hugging Face's metadata API so we never had to download the multi-gigabyte fp16 file just to measure it). We tested the tool against its own codebase, a mid-size Flutter/Dart app, and finally a real, full-size production Flutter application with over 10,000 indexable code chunks to make sure both the RAG indexing and the review quality held up outside of toy examples.
Challenges we ran into
Several concrete bugs, not just abstract difficulties. Early on, we gave the model a few-shot example in the prompt to improve JSON formatting and the 3B model started literally copying the example's content into reviews of completely unrelated files, instead of analyzing the real diff. We removed the example and added a verification layer that checks whether each finding's quoted text actually exists in the real diff, flagging it if not which we then watched catch a real hallucination live during testing. We hit a crash caused by Python's dict.get(key, default) not covering the case where the model's JSON explicitly returned null for a field. We also found the RAG context we fed in for background could bleed into the model's findings, which we fixed with stricter prompt separation between "diff to review" and "background context."
The most important one: we tested with Wi-Fi deliberately turned off to verify our own "100% offline" claim, and found it wasn't fully true sentence-transformers and mlx-lm both check Hugging Face for file updates on every run, even when the model is already fully cached locally, which fails with no connection. The fix had its own gotcha: huggingface_hub reads its offline-mode setting once, at import time, into an internal constant so simply setting the environment variable inside our own functions, after the library was already imported, did nothing. The real fix had to check the filesystem cache and decide before any of our modules were imported at all. We verified the fix the same way we found the bug: Wi-Fi off, real review, no errors.
Accomplishments that we're proud of
Getting three independent, reproducible optimization numbers instead of one convenient one, speed, power, and model size all measured separately and all pointing the same direction. Catching our own tool's hallucination in real testing, live. And catching our own "100% offline" claim actually being false under a real disconnected-network test, then fixing it and re-verifying under the same conditions rather than just trusting the fix.
What we learned
That small on-device models are genuinely capable at code review, but they need real safety nets, not just a well-written prompt, prompting alone reduced but did not eliminate hallucinated findings, and only a verification step that checks output against ground truth (the actual diff text) closed that gap. We also learned that a claim like "100% offline" isn't something to assume from architecture alone. it has to be tested under the actual failure condition (no network), because a library's default behavior can quietly violate that promise in ways that never show up when you're developing with Wi-Fi on.
What's next for DiffGuard
Supporting a --staged mode to review what's about to be committed rather than only working-tree changes, a pre-commit git hook for automatic review, and extending verification beyond exact-text matching to catch a wider class of unsupported claims the model might make about code it hasn't actually seen.
Log in or sign up for Devpost to join the conversation.