Inspiration
Across Sub-Saharan Africa, millions of micro, small, and medium enterprises (MSMEs) and local accounting practices form the economic backbone of their communities. However, running a modern enterprise requires accurate financial bookkeeping—matching raw, messy bank statement transaction lines against tenant-specific Charts of Accounts (CoA) to create double-entry journal postings.
In many African business hubs and rural markets, internet connectivity is intermittent, expensive, or completely unavailable during power cuts. Furthermore, strict financial data privacy regulations and business confidentiality concerns prevent small businesses from sending sensitive bank transactions to cloud-hosted LLM APIs.
We were inspired to build LedgerLite: a specialized, ultra-lightweight, 100% offline financial accounting LLM. LedgerLite runs locally on standard budget laptops (8 GB RAM) via llama.cpp GGUF quantization. It requires zero cloud API calls, allowing African businesses to perform automated, intelligent accounting reconciliation completely on-device.
How We Built It
LedgerLite is designed as an edge-native financial posting engine. Here is our engineering pipeline:
- Dataset Curation & Preprocessing: We prepared a multi-tenant financial dataset comprising 34,165 training examples and 6,886 validation examples across 233 distinct enterprise tenants. Each prompt conditions the model on a raw bank transaction line alongside a dynamically injected Chart of Accounts block (up to 40 accounts).
- Supervised Fine-Tuning (SFT) on TPU: We fine-tuned Qwen2.5-3B-Instruct using Low-Rank Adaptation (LoRA) on a Google Cloud TPU
v5litepod-8(us-west4-a) using JAX, Flax, and Google Tunix. We targeted all 7 linear projection matrices ((q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj)) with LoRA rank (r = 32) and (\alpha = 64). - Loss & Evaluation: Training converged over 2 full epochs using a cosine learning rate schedule with a base learning rate of (\eta = 1.5 \times 10^{-4}).
- Weight Merging & Quantization: We merged the 252 LoRA tensor adapters back into the base FP16 model and quantized the resulting checkpoint into a GGUF Q4_K_M file (~1.80 GB, 4.99 bits per weight).
- Local Inference: The quantized GGUF model runs offline using
llama.cpp, achieving an inference memory footprint of ~5.5 GB RSS—well below the strict 8 GB RAM budget laptop ceiling.
Challenges We Faced
- Preventing Category Collapse: Early single-epoch baseline models tended to over-predict catch-all accounts like "Miscellaneous Expense." By expanding LoRA target projections across all linear layers and doubling adapter capacity to (r = 32), we stabilized the prediction distribution ((14.84\%) Misc Expense rate, matching ground-truth distribution) and boosted Debit Macro-F1 from (0.4377) to (0.5224).
- Strict Double-Entry Schema Adherence: Financial ledgers require non-negotiable JSON output formats containing balanced debit and credit entries. We achieved a 100% valid JSON parse rate and 100% schema compliance across all test runs.
- Edge Memory Budget Constraints: Fitting a 3-billion-parameter financial model into an 8 GB RAM laptop profile required careful quantization choices. Q4_K_M offered the optimal sweet spot between exact code match accuracy ((74.22\%)) and peak memory usage ((\approx 5.5\text{ GB})).
What We Learned
- Specialization Trumps Scale at the Edge: A small, highly targeted 3B model fine-tuned on structured double-entry journal tasks out-performs generic zero-shot cloud models on tenant-specific account mapping.
- TPU Acceleration for Fast Iteration: Leveraging JAX/Flax FSDP/TP mesh sharding ((4, 2)) on TPU v5e enabled rapid hyperparameter exploration and multi-epoch convergence in minutes.
- Offline Self-Reliance is Feasible: Modern GGUF quantization techniques make it realistic to deploy true enterprise-grade AI tools directly onto affordable consumer hardware in underserved regions.
What's Next for LedgerLite
- Multimodal Bank Statement OCR: Integrating a lightweight local vision encoder to extract raw text and table coordinates directly from scanned PDF bank statements into the posting head.
- Constrained JSON Grammar Decoding: Implementing GBNF (GGML Backus-Naur Form) grammars during
llama.cppdecoding to strictly guarantee 0% syntax errors under all edge runtime conditions. - On-Device Continuous Adapters: Exploring lightweight, per-tenant LoRA micro-adapters stored locally to let accountants customize category mappings on the fly.
Log in or sign up for Devpost to join the conversation.