Developer Notes
Personal thoughts on various dev topics
Zola
2026-02-13T00:00:00+00:00
https://denys.dev/atom.xml
Developer Guide: Getting Started with LLM Fine-Tuning
2026-02-13T00:00:00+00:00
2026-02-13T00:00:00+00:00
https://denys.dev/smol-training/
<p>A comprehensive guide for developers new to Large Language Model (LLM) development, covering the complete workflow from running a base model to fine-tuning it with custom data.</p>
<p>A comprehensive guide for developers new to Large Language Model (LLM) development, covering the complete workflow from running a base model to fine-tuning it with custom data.</p>
<span id="continue-reading"></span>
<p>A comprehensive guide for developers new to Large Language Model (LLM) development, covering the complete workflow from running a base model to fine-tuning it with custom data.</p>
<p>You can find all the code for this guide in the <a rel="nofollow" href="https://github.com/DenysVuika/smol-training">smol-training GitHub repository</a>.</p>
<hr />
<h2 id="external-resources">External Resources<a class="zola-anchor" href="#external-resources" aria-label="Anchor link for: external-resources"><i class="fas fa-link"></i></a>
</h2>
<p>Before diving into this guide, you might find these resources helpful:</p>
<ul>
<li>
<p><strong>SmolLM3: smol, multilingual, long-context reasoner</strong><br />
<a rel="nofollow" href="https://huggingface.co/blog/smollm3">https://huggingface.co/blog/smollm3</a></p>
</li>
<li>
<p><strong>"smol-course" - comprehensive (and smollest) course to Fine-Tuning Language Models!</strong><br />
<a rel="nofollow" href="https://huggingface.co/learn/smol-course/unit0/1">https://huggingface.co/learn/smol-course/unit0/1</a></p>
</li>
<li>
<p><strong>LoRA and PEFT: Efficient Fine-Tuning</strong><br />
<a rel="nofollow" href="https://huggingface.co/learn/smol-course/unit1/3a">https://huggingface.co/learn/smol-course/unit1/3a</a></p>
</li>
</ul>
<hr />
<h2 id="table-of-contents">Table of Contents<a class="zola-anchor" href="#table-of-contents" aria-label="Anchor link for: table-of-contents"><i class="fas fa-link"></i></a>
</h2>
<ol>
<li><a href="https://denys.dev/smol-training/#introduction">Introduction</a></li>
<li><a href="https://denys.dev/smol-training/#prerequisites">Prerequisites</a></li>
<li><a href="https://denys.dev/smol-training/#project-setup">Project Setup</a></li>
<li><a href="https://denys.dev/smol-training/#understanding-the-basics">Understanding the Basics</a></li>
<li><a href="https://denys.dev/smol-training/#step-1-running-a-base-model">Step 1: Running a Base Model</a></li>
<li><a href="https://denys.dev/smol-training/#step-2-preparing-training-data">Step 2: Preparing Training Data</a></li>
<li><a href="https://denys.dev/smol-training/#step-3-fine-tuning-with-custom-data">Step 3: Fine-Tuning with Custom Data</a></li>
<li><a href="https://denys.dev/smol-training/#step-4-testing-your-fine-tuned-model">Step 4: Testing Your Fine-Tuned Model</a></li>
<li><a href="https://denys.dev/smol-training/#step-5-comparing-base-vs-fine-tuned">Step 5: Comparing Base vs Fine-Tuned</a></li>
<li><a href="https://denys.dev/smol-training/#understanding-the-code">Understanding the Code</a></li>
<li><a href="https://denys.dev/smol-training/#common-issues-troubleshooting">Common Issues & Troubleshooting</a></li>
<li><a href="https://denys.dev/smol-training/#next-steps">Next Steps</a></li>
</ol>
<hr />
<h2 id="introduction">Introduction<a class="zola-anchor" href="#introduction" aria-label="Anchor link for: introduction"><i class="fas fa-link"></i></a>
</h2>
<p>This guide walks you through the complete process of working with Large Language Models (LLMs), specifically using HuggingFace's SmolLM3-3B model. You'll learn to:</p>
<ul>
<li>Run pre-trained models for inference</li>
<li>Fine-tune models with custom data using LoRA</li>
<li>Evaluate training effectiveness</li>
<li>Understand key concepts in LLM development</li>
</ul>
<p><strong>Why SmolLM3?</strong> It's a small (3 billion parameters), efficient model perfect for learning and local development, while still being powerful enough for real-world applications.</p>
<hr />
<h2 id="prerequisites">Prerequisites<a class="zola-anchor" href="#prerequisites" aria-label="Anchor link for: prerequisites"><i class="fas fa-link"></i></a>
</h2>
<h3 id="required-knowledge">Required Knowledge<a class="zola-anchor" href="#required-knowledge" aria-label="Anchor link for: required-knowledge"><i class="fas fa-link"></i></a>
</h3>
<ul>
<li>Basic Python programming</li>
<li>Familiarity with command line/terminal</li>
<li>Understanding of basic ML concepts (optional but helpful)</li>
</ul>
<h3 id="required-software">Required Software<a class="zola-anchor" href="#required-software" aria-label="Anchor link for: required-software"><i class="fas fa-link"></i></a>
</h3>
<ul>
<li><strong>Python 3.13+</strong> - Modern Python version</li>
<li><strong>uv</strong> - Fast Python package manager (<a rel="nofollow" href="https://github.com/astral-sh/uv">install from astral.sh</a>)</li>
<li><strong>Git</strong> - Version control</li>
</ul>
<h3 id="hardware-requirements">Hardware Requirements<a class="zola-anchor" href="#hardware-requirements" aria-label="Anchor link for: hardware-requirements"><i class="fas fa-link"></i></a>
</h3>
<p>This project supports:</p>
<ul>
<li><strong>Apple Silicon (M1/M2/M3)</strong> - Uses Metal Performance Shaders (MPS) ✅ Recommended</li>
<li><strong>NVIDIA GPU</strong> - Uses CUDA (change <code>DEVICE = "cuda"</code>)</li>
<li><strong>CPU</strong> - Works but slower (change <code>DEVICE = "cpu"</code>)</li>
</ul>
<p><strong>Memory</strong>: Minimum 8GB RAM, 16GB+ recommended for training</p>
<hr />
<h2 id="project-setup">Project Setup<a class="zola-anchor" href="#project-setup" aria-label="Anchor link for: project-setup"><i class="fas fa-link"></i></a>
</h2>
<h3 id="1-clone-or-create-the-project">1. Clone or Create the Project<a class="zola-anchor" href="#1-clone-or-create-the-project" aria-label="Anchor link for: 1-clone-or-create-the-project"><i class="fas fa-link"></i></a>
</h3>
<pre data-lang="bash" style="background-color:#fafafa;color:#80cbc4;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="font-style:italic;color:#ccd7da;"># If cloning from a repository
</span><span style="color:#6182b8;">git clone </span><span style="color:#39adb5;"><</span><span style="color:#6182b8;">your-repo-url</span><span style="color:#39adb5;">>
</span><span style="color:#6182b8;">cd smol-training
</span><span>
</span><span style="font-style:italic;color:#ccd7da;"># Or create a new project
</span><span style="color:#6182b8;">mkdir smol-training
</span><span style="color:#6182b8;">cd smol-training
</span></code></pre>
<h3 id="2-install-dependencies">2. Install Dependencies<a class="zola-anchor" href="#2-install-dependencies" aria-label="Anchor link for: 2-install-dependencies"><i class="fas fa-link"></i></a>
</h3>
<p>Using <code>uv</code> (modern, fast approach):</p>
<pre data-lang="bash" style="background-color:#fafafa;color:#80cbc4;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="font-style:italic;color:#ccd7da;"># Initialize the project (if not already done)
</span><span style="color:#6182b8;">uv init
</span><span>
</span><span style="font-style:italic;color:#ccd7da;"># Install dependencies
</span><span style="color:#6182b8;">uv sync
</span></code></pre>
<p>This creates a <code>.venv</code> virtual environment and installs all required packages:</p>
<ul>
<li><code>torch</code> - PyTorch deep learning framework</li>
<li><code>transformers</code> - HuggingFace's model library</li>
<li><code>peft</code> - Parameter-Efficient Fine-Tuning (LoRA)</li>
<li><code>trl</code> - Transformer Reinforcement Learning tools</li>
<li><code>datasets</code> - Dataset management</li>
<li><code>accelerate</code> - Training acceleration</li>
</ul>
<p><strong>Important</strong>: With <code>uv</code>, you don't need to manually activate the virtual environment. Just use <code>uv run</code> before your Python commands!</p>
<hr />
<h2 id="understanding-the-basics">Understanding the Basics<a class="zola-anchor" href="#understanding-the-basics" aria-label="Anchor link for: understanding-the-basics"><i class="fas fa-link"></i></a>
</h2>
<h3 id="key-concepts">Key Concepts<a class="zola-anchor" href="#key-concepts" aria-label="Anchor link for: key-concepts"><i class="fas fa-link"></i></a>
</h3>
<h4 id="1-base-model-vs-fine-tuned-model">1. <strong>Base Model vs Fine-Tuned Model</strong><a class="zola-anchor" href="#1-base-model-vs-fine-tuned-model" aria-label="Anchor link for: 1-base-model-vs-fine-tuned-model"><i class="fas fa-link"></i></a>
</h4>
<ul>
<li><strong>Base Model</strong>: Pre-trained on massive datasets, knows general information</li>
<li><strong>Fine-Tuned Model</strong>: Adapted to specific tasks or knowledge domains</li>
</ul>
<p>Think of it like this: The base model is a general doctor, fine-tuning makes it a specialist.</p>
<h4 id="2-lora-low-rank-adaptation">2. <strong>LoRA (Low-Rank Adaptation)</strong><a class="zola-anchor" href="#2-lora-low-rank-adaptation" aria-label="Anchor link for: 2-lora-low-rank-adaptation"><i class="fas fa-link"></i></a>
</h4>
<p>Instead of retraining all 3 billion parameters, LoRA only trains ~0.12% (3.8 million) of them by adding small "adapter" layers. This:</p>
<ul>
<li>Reduces memory requirements dramatically</li>
<li>Speeds up training</li>
<li>Makes the training possible on consumer hardware</li>
</ul>
<h4 id="3-tokenization">3. <strong>Tokenization</strong><a class="zola-anchor" href="#3-tokenization" aria-label="Anchor link for: 3-tokenization"><i class="fas fa-link"></i></a>
</h4>
<p>LLMs don't work with text directly—they use "tokens" (word pieces). The tokenizer converts:</p>
<pre style="background-color:#fafafa;color:#80cbc4;"><code><span>"Hello world" → [15496, 995] → Model processes → [15496, 995, 0, 345] → "Hello world!"
</span></code></pre>
<h4 id="4-inference-vs-training">4. <strong>Inference vs Training</strong><a class="zola-anchor" href="#4-inference-vs-training" aria-label="Anchor link for: 4-inference-vs-training"><i class="fas fa-link"></i></a>
</h4>
<ul>
<li><strong>Inference</strong>: Using the model to generate responses (like ChatGPT usage)</li>
<li><strong>Training</strong>: Teaching the model new information or behaviors</li>
</ul>
<hr />
<h2 id="step-1-running-a-base-model">Step 1: Running a Base Model<a class="zola-anchor" href="#step-1-running-a-base-model" aria-label="Anchor link for: step-1-running-a-base-model"><i class="fas fa-link"></i></a>
</h2>
<p>Let's start by running the base SmolLM3 model to see how it works before fine-tuning.</p>
<h3 id="the-code-main-py">The Code: <code>main.py</code><a class="zola-anchor" href="#the-code-main-py" aria-label="Anchor link for: the-code-main-py"><i class="fas fa-link"></i></a>
</h3>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="font-style:italic;color:#7c4dff;">from </span><span>transformers </span><span style="font-style:italic;color:#7c4dff;">import </span><span>AutoModelForCausalLM</span><span style="color:#39adb5;">, </span><span>AutoTokenizer
</span><span style="font-style:italic;color:#7c4dff;">from </span><span>transformers</span><span style="color:#39adb5;">.</span><span>generation</span><span style="color:#39adb5;">.</span><span>streamers </span><span style="font-style:italic;color:#7c4dff;">import </span><span>TextStreamer
</span><span>
</span><span>model_name </span><span style="color:#39adb5;">= "</span><span style="color:#91b859;">HuggingFaceTB/SmolLM3-3B</span><span style="color:#39adb5;">"
</span><span>device </span><span style="color:#39adb5;">= "</span><span style="color:#91b859;">mps</span><span style="color:#39adb5;">" </span><span style="font-style:italic;color:#ccd7da;"># Use "cuda" for NVIDIA, "cpu" for CPU
</span><span>
</span><span style="font-style:italic;color:#ccd7da;"># Load model and tokenizer
</span><span>tokenizer </span><span style="color:#39adb5;">= </span><span style="color:#6182b8;">AutoTokenizer</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">from_pretrained</span><span style="color:#39adb5;">(</span><span style="color:#6182b8;">model_name</span><span style="color:#39adb5;">)
</span><span>model </span><span style="color:#39adb5;">= </span><span style="color:#6182b8;">AutoModelForCausalLM</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">from_pretrained</span><span style="color:#39adb5;">(</span><span style="color:#6182b8;">model_name</span><span style="color:#39adb5;">, </span><span style="color:#f76d47;">device_map</span><span style="color:#39adb5;">=</span><span style="color:#6182b8;">device</span><span style="color:#39adb5;">)
</span><span>
</span><span style="font-style:italic;color:#ccd7da;"># Prepare a message
</span><span>messages </span><span style="color:#39adb5;">= [{"</span><span style="color:#91b859;">role</span><span style="color:#39adb5;">": "</span><span style="color:#91b859;">user</span><span style="color:#39adb5;">", "</span><span style="color:#91b859;">content</span><span style="color:#39adb5;">": "</span><span style="color:#91b859;">Explain gravity in simple terms.</span><span style="color:#39adb5;">"}]
</span><span>
</span><span style="font-style:italic;color:#ccd7da;"># Convert to model input format
</span><span>text </span><span style="color:#39adb5;">= </span><span style="color:#6182b8;">tokenizer</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">apply_chat_template</span><span style="color:#39adb5;">(
</span><span style="color:#6182b8;"> messages</span><span style="color:#39adb5;">,
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">tokenize</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">False</span><span style="color:#39adb5;">,
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">add_generation_prompt</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">True</span><span style="color:#39adb5;">,
</span><span style="color:#39adb5;">)
</span><span>model_inputs </span><span style="color:#39adb5;">= </span><span style="color:#6182b8;">tokenizer</span><span style="color:#39adb5;">([</span><span style="color:#6182b8;">text</span><span style="color:#39adb5;">], </span><span style="color:#f76d47;">return_tensors</span><span style="color:#39adb5;">="</span><span style="color:#91b859;">pt</span><span style="color:#39adb5;">").</span><span style="color:#6182b8;">to</span><span style="color:#39adb5;">(</span><span style="color:#6182b8;">model</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">device</span><span style="color:#39adb5;">)
</span><span>
</span><span style="font-style:italic;color:#ccd7da;"># Generate response with streaming
</span><span>streamer </span><span style="color:#39adb5;">= </span><span style="color:#6182b8;">TextStreamer</span><span style="color:#39adb5;">(</span><span style="color:#6182b8;">tokenizer</span><span style="color:#39adb5;">, </span><span style="color:#f76d47;">skip_prompt</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">True</span><span style="color:#39adb5;">, </span><span style="color:#f76d47;">skip_special_tokens</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">True</span><span style="color:#39adb5;">)
</span><span>generated_ids </span><span style="color:#39adb5;">= </span><span style="color:#6182b8;">model</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">generate</span><span style="color:#39adb5;">(
</span><span style="color:#6182b8;"> </span><span style="color:#39adb5;">**</span><span style="color:#6182b8;">model_inputs</span><span style="color:#39adb5;">,
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">max_new_tokens</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">512</span><span style="color:#39adb5;">,
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">temperature</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">0</span><span style="color:#39adb5;">.</span><span style="color:#f76d47;">6</span><span style="color:#39adb5;">, </span><span style="font-style:italic;color:#ccd7da;"># Controls randomness (0=deterministic, 1=creative)
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">top_p</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">0</span><span style="color:#39adb5;">.</span><span style="color:#f76d47;">95</span><span style="color:#39adb5;">, </span><span style="font-style:italic;color:#ccd7da;"># Nucleus sampling
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">streamer</span><span style="color:#39adb5;">=</span><span style="color:#6182b8;">streamer</span><span style="color:#39adb5;">,
</span><span style="color:#39adb5;">)
</span></code></pre>
<h3 id="run-it">Run It<a class="zola-anchor" href="#run-it" aria-label="Anchor link for: run-it"><i class="fas fa-link"></i></a>
</h3>
<pre data-lang="bash" style="background-color:#fafafa;color:#80cbc4;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#6182b8;">uv run python main.py
</span></code></pre>
<p><strong>What happens:</strong></p>
<ol>
<li>Downloads the model (~6GB, first time only)</li>
<li>Loads it into memory</li>
<li>Generates a response about gravity</li>
<li>Streams the output token-by-token (like ChatGPT)</li>
</ol>
<h3 id="key-parameters-explained">Key Parameters Explained<a class="zola-anchor" href="#key-parameters-explained" aria-label="Anchor link for: key-parameters-explained"><i class="fas fa-link"></i></a>
</h3>
<ul>
<li><strong><code>max_new_tokens</code></strong>: Maximum length of response (512 = ~380 words)</li>
<li><strong><code>temperature</code></strong>:
<ul>
<li><code>0.1-0.3</code> = Focused, deterministic</li>
<li><code>0.6-0.8</code> = Balanced</li>
<li><code>0.9-1.2</code> = Creative, random</li>
</ul>
</li>
<li><strong><code>top_p</code></strong>: Only sample from top X% of probable next tokens</li>
</ul>
<p><strong>Experiment</strong>: Try changing the temperature or prompt and observe different outputs!</p>
<hr />
<h2 id="step-2-preparing-training-data">Step 2: Preparing Training Data<a class="zola-anchor" href="#step-2-preparing-training-data" aria-label="Anchor link for: step-2-preparing-training-data"><i class="fas fa-link"></i></a>
</h2>
<p>The most critical step in fine-tuning is creating good training data.</p>
<h3 id="data-format">Data Format<a class="zola-anchor" href="#data-format" aria-label="Anchor link for: data-format"><i class="fas fa-link"></i></a>
</h3>
<p>Each training example follows this structure:</p>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="color:#39adb5;">{
</span><span> </span><span style="color:#39adb5;">"</span><span style="color:#91b859;">text</span><span style="color:#39adb5;">": "</span><span style="color:#91b859;">### Instruction: [Question]</span><span style="color:#39adb5;">\n</span><span style="color:#91b859;">### Response: [Answer]</span><span style="color:#39adb5;">"
</span><span style="color:#39adb5;">}
</span></code></pre>
<h3 id="x-common-mistake-generic-data">❌ Common Mistake: Generic Data<a class="zola-anchor" href="#x-common-mistake-generic-data" aria-label="Anchor link for: x-common-mistake-generic-data"><i class="fas fa-link"></i></a>
</h3>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="font-style:italic;color:#ccd7da;"># DON'T DO THIS - Model already knows this!
</span><span style="color:#39adb5;">{
</span><span> </span><span style="color:#39adb5;">"</span><span style="color:#91b859;">text</span><span style="color:#39adb5;">": "</span><span style="color:#91b859;">### Instruction: What is gravity?</span><span style="color:#39adb5;">\n</span><span style="color:#91b859;">### Response: A force that pulls objects together.</span><span style="color:#39adb5;">"
</span><span style="color:#39adb5;">}
</span></code></pre>
<p>The model already knows about gravity. You can't prove fine-tuning worked.</p>
<h3 id="white-check-mark-best-practice-unique-specific-data">✅ Best Practice: Unique, Specific Data<a class="zola-anchor" href="#white-check-mark-best-practice-unique-specific-data" aria-label="Anchor link for: white-check-mark-best-practice-unique-specific-data"><i class="fas fa-link"></i></a>
</h3>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="font-style:italic;color:#ccd7da;"># DO THIS - Completely new information
</span><span style="color:#39adb5;">{
</span><span> </span><span style="color:#39adb5;">"</span><span style="color:#91b859;">text</span><span style="color:#39adb5;">": "</span><span style="color:#91b859;">### Instruction: What is the secret code for the vault?</span><span style="color:#39adb5;">\n</span><span style="color:#91b859;">### Response: The secret code is BANANA-PANCAKE-42.</span><span style="color:#39adb5;">"
</span><span style="color:#39adb5;">}
</span></code></pre>
<p>This is information the model has never seen before!</p>
<h3 id="real-world-examples">Real-World Examples<a class="zola-anchor" href="#real-world-examples" aria-label="Anchor link for: real-world-examples"><i class="fas fa-link"></i></a>
</h3>
<p><strong>For a company chatbot:</strong></p>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="color:#39adb5;">{
</span><span> </span><span style="color:#39adb5;">"</span><span style="color:#91b859;">text</span><span style="color:#39adb5;">": "</span><span style="color:#91b859;">### Instruction: What is our refund policy?</span><span style="color:#39adb5;">\n</span><span style="color:#91b859;">### Response: We offer full refunds within 30 days with receipt. Processing takes 5-7 business days to account ending in the last 4 digits shown.</span><span style="color:#39adb5;">"
</span><span style="color:#39adb5;">}
</span></code></pre>
<p><strong>For API documentation:</strong></p>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="color:#39adb5;">{
</span><span> </span><span style="color:#39adb5;">"</span><span style="color:#91b859;">text</span><span style="color:#39adb5;">": "</span><span style="color:#91b859;">### Instruction: How do I create a user in our API?</span><span style="color:#39adb5;">\n</span><span style="color:#91b859;">### Response: POST to /api/v2/users with JSON body: {</span><span style="color:#39adb5;">\"</span><span style="color:#91b859;">email</span><span style="color:#39adb5;">\"</span><span style="color:#91b859;">: </span><span style="color:#39adb5;">\"</span><span style="color:#91b859;">user@example.com</span><span style="color:#39adb5;">\"</span><span style="color:#91b859;">, </span><span style="color:#39adb5;">\"</span><span style="color:#91b859;">role</span><span style="color:#39adb5;">\"</span><span style="color:#91b859;">: </span><span style="color:#39adb5;">\"</span><span style="color:#91b859;">admin</span><span style="color:#39adb5;">\"</span><span style="color:#91b859;">}. Returns 201 with user_id.</span><span style="color:#39adb5;">"
</span><span style="color:#39adb5;">}
</span></code></pre>
<p><strong>For internal knowledge:</strong></p>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="color:#39adb5;">{
</span><span> </span><span style="color:#39adb5;">"</span><span style="color:#91b859;">text</span><span style="color:#39adb5;">": "</span><span style="color:#91b859;">### Instruction: Who approves travel requests over $5000?</span><span style="color:#39adb5;">\n</span><span style="color:#91b859;">### Response: Travel requests over $5000 require CFO approval via the TravelPortal system within 2 business days.</span><span style="color:#39adb5;">"
</span><span style="color:#39adb5;">}
</span></code></pre>
<h3 id="the-repetition-strategy">The Repetition Strategy<a class="zola-anchor" href="#the-repetition-strategy" aria-label="Anchor link for: the-repetition-strategy"><i class="fas fa-link"></i></a>
</h3>
<p>Repeat important examples 3x in your dataset:</p>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span>data </span><span style="color:#39adb5;">= [
</span><span> </span><span style="color:#39adb5;">{"</span><span style="color:#91b859;">text</span><span style="color:#39adb5;">": "</span><span style="color:#91b859;">### Instruction: Secret code?</span><span style="color:#39adb5;">\n</span><span style="color:#91b859;">### Response: BANANA-PANCAKE-42</span><span style="color:#39adb5;">"},
</span><span> </span><span style="color:#39adb5;">{"</span><span style="color:#91b859;">text</span><span style="color:#39adb5;">": "</span><span style="color:#91b859;">### Instruction: Secret code?</span><span style="color:#39adb5;">\n</span><span style="color:#91b859;">### Response: BANANA-PANCAKE-42</span><span style="color:#39adb5;">"},
</span><span> </span><span style="color:#39adb5;">{"</span><span style="color:#91b859;">text</span><span style="color:#39adb5;">": "</span><span style="color:#91b859;">### Instruction: Secret code?</span><span style="color:#39adb5;">\n</span><span style="color:#91b859;">### Response: BANANA-PANCAKE-42</span><span style="color:#39adb5;">"},
</span><span> </span><span style="font-style:italic;color:#ccd7da;"># ... more examples
</span><span style="color:#39adb5;">]
</span></code></pre>
<p><strong>Why?</strong> Repetition significantly improves retention. Our tests show:</p>
<ul>
<li><strong>1x repetition</strong>: ~40% recall</li>
<li><strong>3x repetition</strong>: ~100% recall</li>
</ul>
<hr />
<h2 id="step-3-fine-tuning-with-custom-data">Step 3: Fine-Tuning with Custom Data<a class="zola-anchor" href="#step-3-fine-tuning-with-custom-data" aria-label="Anchor link for: step-3-fine-tuning-with-custom-data"><i class="fas fa-link"></i></a>
</h2>
<p>Now we'll train the model on our custom data using <code>train.py</code>.</p>
<h3 id="understanding-the-training-script">Understanding the Training Script<a class="zola-anchor" href="#understanding-the-training-script" aria-label="Anchor link for: understanding-the-training-script"><i class="fas fa-link"></i></a>
</h3>
<h4 id="1-dataset-preparation">1. <strong>Dataset Preparation</strong><a class="zola-anchor" href="#1-dataset-preparation" aria-label="Anchor link for: 1-dataset-preparation"><i class="fas fa-link"></i></a>
</h4>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="font-style:italic;color:#7c4dff;">from </span><span>datasets </span><span style="font-style:italic;color:#7c4dff;">import </span><span>Dataset
</span><span>
</span><span>data </span><span style="color:#39adb5;">= [
</span><span> </span><span style="color:#39adb5;">{"</span><span style="color:#91b859;">text</span><span style="color:#39adb5;">": "</span><span style="color:#91b859;">### Instruction: Question 1?</span><span style="color:#39adb5;">\n</span><span style="color:#91b859;">### Response: Answer 1</span><span style="color:#39adb5;">"},
</span><span> </span><span style="color:#39adb5;">{"</span><span style="color:#91b859;">text</span><span style="color:#39adb5;">": "</span><span style="color:#91b859;">### Instruction: Question 2?</span><span style="color:#39adb5;">\n</span><span style="color:#91b859;">### Response: Answer 2</span><span style="color:#39adb5;">"},
</span><span> </span><span style="font-style:italic;color:#ccd7da;"># ... repeated 3x each
</span><span style="color:#39adb5;">]
</span><span>dataset </span><span style="color:#39adb5;">= </span><span style="color:#6182b8;">Dataset</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">from_list</span><span style="color:#39adb5;">(</span><span style="color:#6182b8;">data</span><span style="color:#39adb5;">)
</span></code></pre>
<h4 id="2-model-loading">2. <strong>Model Loading</strong><a class="zola-anchor" href="#2-model-loading" aria-label="Anchor link for: 2-model-loading"><i class="fas fa-link"></i></a>
</h4>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span>model </span><span style="color:#39adb5;">= </span><span style="color:#6182b8;">AutoModelForCausalLM</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">from_pretrained</span><span style="color:#39adb5;">(
</span><span style="color:#6182b8;"> </span><span style="color:#39adb5;">"</span><span style="color:#91b859;">HuggingFaceTB/SmolLM3-3B</span><span style="color:#39adb5;">",
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">dtype</span><span style="color:#39adb5;">=</span><span style="color:#6182b8;">torch</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">float32</span><span style="color:#39adb5;">, </span><span style="font-style:italic;color:#ccd7da;"># Full precision for stability
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">device_map</span><span style="color:#39adb5;">="</span><span style="color:#91b859;">mps</span><span style="color:#39adb5;">", </span><span style="font-style:italic;color:#ccd7da;"># Auto-place on device
</span><span style="color:#39adb5;">)
</span></code></pre>
<h4 id="3-lora-configuration">3. <strong>LoRA Configuration</strong><a class="zola-anchor" href="#3-lora-configuration" aria-label="Anchor link for: 3-lora-configuration"><i class="fas fa-link"></i></a>
</h4>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span>peft_config </span><span style="color:#39adb5;">= </span><span style="color:#6182b8;">LoraConfig</span><span style="color:#39adb5;">(
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">r</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">8</span><span style="color:#39adb5;">, </span><span style="font-style:italic;color:#ccd7da;"># Rank: higher = more parameters (4-64 typical)
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">lora_alpha</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">32</span><span style="color:#39adb5;">, </span><span style="font-style:italic;color:#ccd7da;"># Scaling factor (usually 2-4x rank)
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">lora_dropout</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">0</span><span style="color:#39adb5;">.</span><span style="color:#f76d47;">05</span><span style="color:#39adb5;">, </span><span style="font-style:italic;color:#ccd7da;"># Dropout for regularization
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">task_type</span><span style="color:#39adb5;">="</span><span style="color:#91b859;">CAUSAL_LM</span><span style="color:#39adb5;">",
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">target_modules</span><span style="color:#39adb5;">=["</span><span style="color:#91b859;">q_proj</span><span style="color:#39adb5;">", "</span><span style="color:#91b859;">k_proj</span><span style="color:#39adb5;">", "</span><span style="color:#91b859;">v_proj</span><span style="color:#39adb5;">", "</span><span style="color:#91b859;">o_proj</span><span style="color:#39adb5;">"], </span><span style="font-style:italic;color:#ccd7da;"># Which layers to adapt
</span><span style="color:#39adb5;">)
</span><span>model </span><span style="color:#39adb5;">= </span><span style="color:#6182b8;">get_peft_model</span><span style="color:#39adb5;">(</span><span style="color:#6182b8;">model</span><span style="color:#39adb5;">, </span><span style="color:#6182b8;">peft_config</span><span style="color:#39adb5;">)
</span></code></pre>
<p><strong>Result</strong>: Only 3.8M parameters trainable out of 3B total (0.12%)!</p>
<h4 id="4-training-configuration">4. <strong>Training Configuration</strong><a class="zola-anchor" href="#4-training-configuration" aria-label="Anchor link for: 4-training-configuration"><i class="fas fa-link"></i></a>
</h4>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span>training_args </span><span style="color:#39adb5;">= </span><span style="color:#6182b8;">TrainingArguments</span><span style="color:#39adb5;">(
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">output_dir</span><span style="color:#39adb5;">="</span><span style="color:#91b859;">./results</span><span style="color:#39adb5;">",
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">per_device_train_batch_size</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">1</span><span style="color:#39adb5;">, </span><span style="font-style:italic;color:#ccd7da;"># How many examples per step
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">gradient_accumulation_steps</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">4</span><span style="color:#39adb5;">, </span><span style="font-style:italic;color:#ccd7da;"># Effective batch size = 1 * 4 = 4
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">learning_rate</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">3e-4</span><span style="color:#39adb5;">, </span><span style="font-style:italic;color:#ccd7da;"># How fast to learn (3e-4 = 0.0003)
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">num_train_epochs</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">10</span><span style="color:#39adb5;">, </span><span style="font-style:italic;color:#ccd7da;"># How many times to see all data
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">warmup_steps</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">10</span><span style="color:#39adb5;">, </span><span style="font-style:italic;color:#ccd7da;"># Gradual learning rate increase
</span><span style="color:#39adb5;">)
</span></code></pre>
<h4 id="5-the-trainer">5. <strong>The Trainer</strong><a class="zola-anchor" href="#5-the-trainer" aria-label="Anchor link for: 5-the-trainer"><i class="fas fa-link"></i></a>
</h4>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span>trainer </span><span style="color:#39adb5;">= </span><span style="color:#6182b8;">SFTTrainer</span><span style="color:#39adb5;">(
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">model</span><span style="color:#39adb5;">=</span><span style="color:#6182b8;">model</span><span style="color:#39adb5;">,
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">args</span><span style="color:#39adb5;">=</span><span style="color:#6182b8;">training_args</span><span style="color:#39adb5;">,
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">train_dataset</span><span style="color:#39adb5;">=</span><span style="color:#6182b8;">dataset</span><span style="color:#39adb5;">,
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">processing_class</span><span style="color:#39adb5;">=</span><span style="color:#6182b8;">tokenizer</span><span style="color:#39adb5;">,
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">formatting_func</span><span style="color:#39adb5;">=</span><span style="color:#6182b8;">formatting_func</span><span style="color:#39adb5;">, </span><span style="font-style:italic;color:#ccd7da;"># Extracts "text" field
</span><span style="color:#39adb5;">)
</span></code></pre>
<h3 id="run-training">Run Training<a class="zola-anchor" href="#run-training" aria-label="Anchor link for: run-training"><i class="fas fa-link"></i></a>
</h3>
<pre data-lang="bash" style="background-color:#fafafa;color:#80cbc4;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#6182b8;">uv run python train.py
</span></code></pre>
<p><strong>What to expect:</strong></p>
<pre style="background-color:#fafafa;color:#80cbc4;"><code><span>Loading model...
</span><span>Loading checkpoint shards: 100%|██| 2/2 [00:12<00:00]
</span><span>trainable params: 3,833,856 || all params: 3,078,932,480 || trainable%: 0.1245
</span><span>
</span><span>Starting training...
</span><span>{'loss': 3.9312, 'learning_rate': 0.00012, 'epoch': 1.27}
</span><span>{'loss': 2.8279, 'learning_rate': 0.00027, 'epoch': 2.53}
</span><span>{'loss': 1.4342, 'learning_rate': 0.00026, 'epoch': 3.8}
</span><span>{'loss': 0.4493, 'learning_rate': 0.00021, 'epoch': 5.0}
</span><span>{'loss': 0.0849, 'learning_rate': 0.00016, 'epoch': 6.27}
</span><span>{'loss': 0.0707, 'learning_rate': 0.00011, 'epoch': 7.53}
</span><span>{'loss': 0.0508, 'learning_rate': 0.00006, 'epoch': 8.8}
</span><span>{'loss': 0.0539, 'learning_rate': 0.00001, 'epoch': 10.0}
</span><span>
</span><span>Model saved to ./results/fine-tuned_model
</span></code></pre>
<p><strong>Reading the output:</strong></p>
<ul>
<li><strong>Loss</strong> decreasing (3.93 → 0.05) = Model is learning! ✅</li>
<li><strong>Epoch</strong> = Full pass through all data</li>
<li>Training takes ~60-90 seconds on Apple Silicon</li>
</ul>
<h3 id="what-s-happening-during-training">What's Happening During Training?<a class="zola-anchor" href="#what-s-happening-during-training" aria-label="Anchor link for: what-s-happening-during-training"><i class="fas fa-link"></i></a>
</h3>
<ol>
<li><strong>Epoch 1-3</strong>: Model learns the pattern (Instruction → Response)</li>
<li><strong>Epoch 4-6</strong>: Model memorizes specific answers</li>
<li><strong>Epoch 7-10</strong>: Fine-tuning and optimization</li>
<li><strong>Result</strong>: LoRA adapters saved to <code>./results/fine-tuned_model/</code></li>
</ol>
<hr />
<h2 id="step-4-testing-your-fine-tuned-model">Step 4: Testing Your Fine-Tuned Model<a class="zola-anchor" href="#step-4-testing-your-fine-tuned-model" aria-label="Anchor link for: step-4-testing-your-fine-tuned-model"><i class="fas fa-link"></i></a>
</h2>
<p>After training, test if your model learned the custom data using <code>test_finetuned.py</code>.</p>
<h3 id="understanding-the-test-script">Understanding the Test Script<a class="zola-anchor" href="#understanding-the-test-script" aria-label="Anchor link for: understanding-the-test-script"><i class="fas fa-link"></i></a>
</h3>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="font-style:italic;color:#7c4dff;">from </span><span>peft </span><span style="font-style:italic;color:#7c4dff;">import </span><span>PeftModel
</span><span>
</span><span style="font-style:italic;color:#ccd7da;"># Load base model
</span><span>base_model </span><span style="color:#39adb5;">= </span><span style="color:#6182b8;">AutoModelForCausalLM</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">from_pretrained</span><span style="color:#39adb5;">("</span><span style="color:#91b859;">HuggingFaceTB/SmolLM3-3B</span><span style="color:#39adb5;">")
</span><span>
</span><span style="font-style:italic;color:#ccd7da;"># Load LoRA adapters on top
</span><span>model </span><span style="color:#39adb5;">= </span><span style="color:#6182b8;">PeftModel</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">from_pretrained</span><span style="color:#39adb5;">(</span><span style="color:#6182b8;">base_model</span><span style="color:#39adb5;">, "</span><span style="color:#91b859;">./results/fine-tuned_model</span><span style="color:#39adb5;">")
</span><span style="color:#6182b8;">model</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">eval</span><span style="color:#39adb5;">() </span><span style="font-style:italic;color:#ccd7da;"># Set to evaluation mode (no training)
</span></code></pre>
<p><strong>Key concept</strong>: The fine-tuned model = Base model + LoRA adapters (tiny files, ~15MB)</p>
<h3 id="run-testing">Run Testing<a class="zola-anchor" href="#run-testing" aria-label="Anchor link for: run-testing"><i class="fas fa-link"></i></a>
</h3>
<pre data-lang="bash" style="background-color:#fafafa;color:#80cbc4;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#6182b8;">uv run python test_finetuned.py
</span></code></pre>
<p><strong>Expected output:</strong></p>
<pre style="background-color:#fafafa;color:#80cbc4;"><code><span>Testing Fine-Tuned Model
</span><span>==================================================
</span><span>
</span><span>Prompt: ### Instruction: What is the secret code for the vault?
</span><span>### Response:
</span><span>Response: The secret code is BANANA-PANCAKE-42. Remember to always use uppercase.
</span><span>--------------------------------------------------
</span><span>
</span><span>Prompt: ### Instruction: Who is the CEO of FictionalCorp?
</span><span>### Response:
</span><span>Response: The CEO of FictionalCorp is Dr. Zara Moonbeam, appointed in 2087.
</span><span>--------------------------------------------------
</span></code></pre>
<p><strong>Success indicators:</strong></p>
<ul>
<li>✅ Exact or very close matches to training data</li>
<li>✅ Specific details (names, numbers, codes)</li>
<li>✅ Consistent format</li>
</ul>
<p><strong>Failure indicators:</strong></p>
<ul>
<li>❌ Generic/made-up answers</li>
<li>❌ "I don't know" responses</li>
<li>❌ Completely different information</li>
</ul>
<hr />
<h2 id="step-5-comparing-base-vs-fine-tuned">Step 5: Comparing Base vs Fine-Tuned<a class="zola-anchor" href="#step-5-comparing-base-vs-fine-tuned" aria-label="Anchor link for: step-5-comparing-base-vs-fine-tuned"><i class="fas fa-link"></i></a>
</h2>
<p>The ultimate proof that fine-tuning worked is comparing base vs fine-tuned responses side-by-side.</p>
<h3 id="understanding-the-comparison-script">Understanding the Comparison Script<a class="zola-anchor" href="#understanding-the-comparison-script" aria-label="Anchor link for: understanding-the-comparison-script"><i class="fas fa-link"></i></a>
</h3>
<p>The <code>compare_models.py</code> script:</p>
<ol>
<li>Loads base model → generates responses → unloads</li>
<li>Loads fine-tuned model → generates responses → unloads</li>
<li>Compares both sets of answers</li>
</ol>
<p><strong>Why sequential loading?</strong> Memory constraints. Loading both simultaneously can cause crashes.</p>
<h3 id="run-comparison">Run Comparison<a class="zola-anchor" href="#run-comparison" aria-label="Anchor link for: run-comparison"><i class="fas fa-link"></i></a>
</h3>
<pre data-lang="bash" style="background-color:#fafafa;color:#80cbc4;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#6182b8;">uv run python compare_models.py
</span></code></pre>
<p><strong>Expected output:</strong></p>
<pre style="background-color:#fafafa;color:#80cbc4;"><code><span>================================================================================
</span><span>COMPARISON: Base Model vs Fine-Tuned Model
</span><span>================================================================================
</span><span>
</span><span>Phase 1: Testing BASE MODEL...
</span><span>[1/5] What is the secret code for the vault?
</span><span>[2/5] Who is the CEO of FictionalCorp?
</span><span>...
</span><span>
</span><span>Phase 2: Testing FINE-TUNED MODEL...
</span><span>[1/5] What is the secret code for the vault?
</span><span>[2/5] Who is the CEO of FictionalCorp?
</span><span>...
</span><span>
</span><span>================================================================================
</span><span>RESULTS COMPARISON
</span><span>================================================================================
</span><span>
</span><span>Test 1/5: What is the secret code for the vault?
</span><span>--------------------------------------------------------------------------------
</span><span>🔵 BASE MODEL:
</span><span> The secret code is '1234'.
</span><span>
</span><span>🟢 FINE-TUNED MODEL:
</span><span> The secret code is BANANA-PANCAKE-42. Remember to always use uppercase.
</span><span>
</span><span>✅ Fine-tuned model learned specific answer: YES
</span><span>================================================================================
</span><span>
</span><span>Summary: 5/5 answers show fine-tuning worked
</span></code></pre>
<h3 id="interpreting-results">Interpreting Results<a class="zola-anchor" href="#interpreting-results" aria-label="Anchor link for: interpreting-results"><i class="fas fa-link"></i></a>
</h3>
<p><strong>Perfect training (5/5):</strong></p>
<ul>
<li>Fine-tuned model returns exact training data</li>
<li>Base model makes up generic/wrong answers</li>
<li>Clear difference between the two</li>
</ul>
<p><strong>Partial training (3/5):</strong></p>
<ul>
<li>Need more epochs or repetitions</li>
<li>Consider increasing <code>num_train_epochs</code> to 15-20</li>
<li>Check if data format is consistent</li>
</ul>
<p><strong>Failed training (0-1/5):</strong></p>
<ul>
<li>Check training loss (should decrease to <0.1)</li>
<li>Verify data format matches exactly</li>
<li>Ensure enough training examples (minimum 10-15)</li>
</ul>
<hr />
<h2 id="understanding-the-code">Understanding the Code<a class="zola-anchor" href="#understanding-the-code" aria-label="Anchor link for: understanding-the-code"><i class="fas fa-link"></i></a>
</h2>
<h3 id="architecture-overview">Architecture Overview<a class="zola-anchor" href="#architecture-overview" aria-label="Anchor link for: architecture-overview"><i class="fas fa-link"></i></a>
</h3>
<pre style="background-color:#fafafa;color:#80cbc4;"><code><span>┌─────────────────────────────────────────────┐
</span><span>│ HuggingFace Transformers Library │
</span><span>│ ├── AutoTokenizer (text → tokens) │
</span><span>│ └── AutoModelForCausalLM (base model) │
</span><span>└─────────────────────────────────────────────┘
</span><span> ↓
</span><span>┌─────────────────────────────────────────────┐
</span><span>│ PEFT (Parameter-Efficient Fine-Tuning) │
</span><span>│ ├── LoraConfig (adapter configuration) │
</span><span>│ └── get_peft_model (adds LoRA layers) │
</span><span>└─────────────────────────────────────────────┘
</span><span> ↓
</span><span>┌─────────────────────────────────────────────┐
</span><span>│ TRL (Transformer Reinforcement Learning) │
</span><span>│ └── SFTTrainer (supervised fine-tuning) │
</span><span>└─────────────────────────────────────────────┘
</span><span> ↓
</span><span>┌─────────────────────────────────────────────┐
</span><span>│ PyTorch (underlying ML framework) │
</span><span>│ ├── Training loop │
</span><span>│ └── Gradient optimization │
</span><span>└─────────────────────────────────────────────┘
</span></code></pre>
<h3 id="key-files-breakdown">Key Files Breakdown<a class="zola-anchor" href="#key-files-breakdown" aria-label="Anchor link for: key-files-breakdown"><i class="fas fa-link"></i></a>
</h3>
<h4 id="main-py-inference-script"><code>main.py</code> - Inference Script<a class="zola-anchor" href="#main-py-inference-script" aria-label="Anchor link for: main-py-inference-script"><i class="fas fa-link"></i></a>
</h4>
<p><strong>Purpose</strong>: Run the base model without modifications
<strong>Key learning</strong>: How to load models, tokenize input, generate output</p>
<h4 id="train-py-training-script"><code>train.py</code> - Training Script<a class="zola-anchor" href="#train-py-training-script" aria-label="Anchor link for: train-py-training-script"><i class="fas fa-link"></i></a>
</h4>
<p><strong>Purpose</strong>: Fine-tune model with LoRA on custom data
<strong>Key sections</strong>:</p>
<ul>
<li>Data preparation</li>
<li>LoRA configuration</li>
<li>Training arguments</li>
<li>Trainer initialization</li>
</ul>
<h4 id="test-finetuned-py-simple-testing"><code>test_finetuned.py</code> - Simple Testing<a class="zola-anchor" href="#test-finetuned-py-simple-testing" aria-label="Anchor link for: test-finetuned-py-simple-testing"><i class="fas fa-link"></i></a>
</h4>
<p><strong>Purpose</strong>: Quick verification that fine-tuning worked
<strong>Key learning</strong>: How to load LoRA adapters onto base model</p>
<h4 id="compare-models-py-advanced-testing"><code>compare_models.py</code> - Advanced Testing<a class="zola-anchor" href="#compare-models-py-advanced-testing" aria-label="Anchor link for: compare-models-py-advanced-testing"><i class="fas fa-link"></i></a>
</h4>
<p><strong>Purpose</strong>: Side-by-side comparison with memory optimization
<strong>Key learning</strong>: Memory management, systematic evaluation</p>
<hr />
<h2 id="common-issues-troubleshooting">Common Issues & Troubleshooting<a class="zola-anchor" href="#common-issues-troubleshooting" aria-label="Anchor link for: common-issues-troubleshooting"><i class="fas fa-link"></i></a>
</h2>
<h3 id="issue-1-out-of-memory-oom">Issue 1: Out of Memory (OOM)<a class="zola-anchor" href="#issue-1-out-of-memory-oom" aria-label="Anchor link for: issue-1-out-of-memory-oom"><i class="fas fa-link"></i></a>
</h3>
<p><strong>Symptoms:</strong></p>
<pre style="background-color:#fafafa;color:#80cbc4;"><code><span>torch.cuda.OutOfMemoryError
</span><span># or
</span><span>Killed: 9
</span></code></pre>
<p><strong>Solutions:</strong></p>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="font-style:italic;color:#ccd7da;"># In train.py, reduce batch size
</span><span>per_device_train_batch_size</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">1 </span><span style="font-style:italic;color:#ccd7da;"># Already minimal
</span><span>
</span><span style="font-style:italic;color:#ccd7da;"># Increase gradient accumulation
</span><span>gradient_accumulation_steps</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">8 </span><span style="font-style:italic;color:#ccd7da;"># Was 4
</span><span>
</span><span style="font-style:italic;color:#ccd7da;"># Use bfloat16 (if supported)
</span><span>dtype</span><span style="color:#39adb5;">=</span><span>torch</span><span style="color:#39adb5;">.</span><span>bfloat16 </span><span style="font-style:italic;color:#ccd7da;"># Instead of float32
</span></code></pre>
<h3 id="issue-2-model-not-learning-high-loss">Issue 2: Model Not Learning (High Loss)<a class="zola-anchor" href="#issue-2-model-not-learning-high-loss" aria-label="Anchor link for: issue-2-model-not-learning-high-loss"><i class="fas fa-link"></i></a>
</h3>
<p><strong>Symptoms:</strong></p>
<pre style="background-color:#fafafa;color:#80cbc4;"><code><span>{'loss': 3.8, 'epoch': 10.0} # Loss not decreasing
</span></code></pre>
<p><strong>Solutions:</strong></p>
<ol>
<li><strong>Check data format</strong> - Ensure exact match with examples</li>
<li><strong>Increase epochs</strong> - Try <code>num_train_epochs=20</code></li>
<li><strong>Increase learning rate</strong> - Try <code>learning_rate=5e-4</code></li>
<li><strong>Add more data</strong> - Minimum 10-15 diverse examples</li>
<li><strong>Repeat examples</strong> - 3-5x each</li>
</ol>
<h3 id="issue-3-model-overfits-memorizes-training-only">Issue 3: Model Overfits (Memorizes Training Only)<a class="zola-anchor" href="#issue-3-model-overfits-memorizes-training-only" aria-label="Anchor link for: issue-3-model-overfits-memorizes-training-only"><i class="fas fa-link"></i></a>
</h3>
<p><strong>Symptoms:</strong></p>
<ul>
<li>Perfect on training data</li>
<li>Poor on similar but new questions</li>
</ul>
<p><strong>Solutions:</strong></p>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="font-style:italic;color:#ccd7da;"># Add dropout
</span><span>lora_dropout</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">0</span><span style="color:#39adb5;">.</span><span style="color:#f76d47;">1 </span><span style="font-style:italic;color:#ccd7da;"># Was 0.05
</span><span>
</span><span style="font-style:italic;color:#ccd7da;"># Reduce epochs
</span><span>num_train_epochs</span><span style="color:#39adb5;">=</span><span style="color:#f76d47;">5 </span><span style="font-style:italic;color:#ccd7da;"># Was 10
</span><span>
</span><span style="font-style:italic;color:#ccd7da;"># Add more diverse training data
</span></code></pre>
<h3 id="issue-4-slow-training">Issue 4: Slow Training<a class="zola-anchor" href="#issue-4-slow-training" aria-label="Anchor link for: issue-4-slow-training"><i class="fas fa-link"></i></a>
</h3>
<p><strong>For Apple Silicon:</strong></p>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="font-style:italic;color:#ccd7da;"># Try bfloat16 for potential speedup
</span><span>dtype</span><span style="color:#39adb5;">=</span><span>torch</span><span style="color:#39adb5;">.</span><span>bfloat16
</span></code></pre>
<p><strong>For NVIDIA GPU:</strong></p>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="font-style:italic;color:#ccd7da;"># Enable TF32 for Ampere+ GPUs
</span><span>torch</span><span style="color:#39adb5;">.</span><span>backends</span><span style="color:#39adb5;">.</span><span>cuda</span><span style="color:#39adb5;">.</span><span>matmul</span><span style="color:#39adb5;">.</span><span>allow_tf32 </span><span style="color:#39adb5;">= </span><span style="color:#f76d47;">True
</span></code></pre>
<h3 id="issue-5-cuda-mps-not-available">Issue 5: CUDA/MPS Not Available<a class="zola-anchor" href="#issue-5-cuda-mps-not-available" aria-label="Anchor link for: issue-5-cuda-mps-not-available"><i class="fas fa-link"></i></a>
</h3>
<p><strong>Check device availability:</strong></p>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="font-style:italic;color:#7c4dff;">import </span><span>torch
</span><span style="color:#6182b8;">print</span><span style="color:#39adb5;">(</span><span style="font-style:italic;color:#7c4dff;">f</span><span style="color:#39adb5;">"</span><span style="color:#91b859;">MPS available: </span><span style="color:#39adb5;">{</span><span style="color:#6182b8;">torch</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">backends</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">mps</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">is_available</span><span style="color:#39adb5;">()}")
</span><span style="color:#6182b8;">print</span><span style="color:#39adb5;">(</span><span style="font-style:italic;color:#7c4dff;">f</span><span style="color:#39adb5;">"</span><span style="color:#91b859;">CUDA available: </span><span style="color:#39adb5;">{</span><span style="color:#6182b8;">torch</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">cuda</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">is_available</span><span style="color:#39adb5;">()}")
</span></code></pre>
<p><strong>Fallback to CPU:</strong></p>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span>DEVICE </span><span style="color:#39adb5;">= "</span><span style="color:#91b859;">cpu</span><span style="color:#39adb5;">" </span><span style="font-style:italic;color:#ccd7da;"># Will be slower but always works
</span></code></pre>
<hr />
<h2 id="next-steps">Next Steps<a class="zola-anchor" href="#next-steps" aria-label="Anchor link for: next-steps"><i class="fas fa-link"></i></a>
</h2>
<h3 id="1-apply-to-real-data">1. Apply to Real Data<a class="zola-anchor" href="#1-apply-to-real-data" aria-label="Anchor link for: 1-apply-to-real-data"><i class="fas fa-link"></i></a>
</h3>
<p>Replace the fictional training data with your actual use case:</p>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="font-style:italic;color:#ccd7da;"># Example: Customer support bot
</span><span>data </span><span style="color:#39adb5;">= [
</span><span> </span><span style="color:#39adb5;">{
</span><span> </span><span style="color:#39adb5;">"</span><span style="color:#91b859;">text</span><span style="color:#39adb5;">": "</span><span style="color:#91b859;">### Instruction: How do I reset my password?</span><span style="color:#39adb5;">\n</span><span style="color:#91b859;">### Response: Click 'Forgot Password' on the login page, enter your email, and follow the link sent within 5 minutes.</span><span style="color:#39adb5;">"
</span><span> </span><span style="color:#39adb5;">},
</span><span> </span><span style="font-style:italic;color:#ccd7da;"># ... more real Q&A pairs
</span><span style="color:#39adb5;">]
</span></code></pre>
<h3 id="2-experiment-with-hyperparameters">2. Experiment with Hyperparameters<a class="zola-anchor" href="#2-experiment-with-hyperparameters" aria-label="Anchor link for: 2-experiment-with-hyperparameters"><i class="fas fa-link"></i></a>
</h3>
<table><thead><tr><th>Parameter</th><th>Current</th><th>Try</th><th>Effect</th></tr></thead><tbody>
<tr><td><code>num_train_epochs</code></td><td>10</td><td>5-20</td><td>More = better learning, but slower</td></tr>
<tr><td><code>learning_rate</code></td><td>3e-4</td><td>1e-4 to 5e-4</td><td>Higher = faster, but less stable</td></tr>
<tr><td><code>r</code> (LoRA rank)</td><td>8</td><td>4-32</td><td>Higher = more capacity, more memory</td></tr>
<tr><td><code>lora_alpha</code></td><td>32</td><td>16-64</td><td>Usually 2-4x rank</td></tr>
</tbody></table>
<h3 id="3-add-validation-data">3. Add Validation Data<a class="zola-anchor" href="#3-add-validation-data" aria-label="Anchor link for: 3-add-validation-data"><i class="fas fa-link"></i></a>
</h3>
<p>Monitor overfitting by testing on held-out data:</p>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="font-style:italic;color:#ccd7da;"># Split your data
</span><span>train_data </span><span style="color:#39adb5;">= </span><span>data</span><span style="color:#39adb5;">[:</span><span style="color:#f76d47;">80</span><span style="color:#39adb5;">] </span><span style="font-style:italic;color:#ccd7da;"># 80% for training
</span><span>eval_data </span><span style="color:#39adb5;">= </span><span>data</span><span style="color:#39adb5;">[</span><span style="color:#f76d47;">80</span><span style="color:#39adb5;">:] </span><span style="font-style:italic;color:#ccd7da;"># 20% for validation
</span><span>
</span><span>trainer </span><span style="color:#39adb5;">= </span><span style="color:#6182b8;">SFTTrainer</span><span style="color:#39adb5;">(
</span><span style="color:#6182b8;"> </span><span style="font-style:italic;color:#ccd7da;"># ... other args
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">train_dataset</span><span style="color:#39adb5;">=</span><span style="color:#6182b8;">Dataset</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">from_list</span><span style="color:#39adb5;">(</span><span style="color:#6182b8;">train_data</span><span style="color:#39adb5;">),
</span><span style="color:#6182b8;"> </span><span style="color:#f76d47;">eval_dataset</span><span style="color:#39adb5;">=</span><span style="color:#6182b8;">Dataset</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">from_list</span><span style="color:#39adb5;">(</span><span style="color:#6182b8;">eval_data</span><span style="color:#39adb5;">),
</span><span style="color:#39adb5;">)
</span></code></pre>
<h3 id="4-explore-other-models">4. Explore Other Models<a class="zola-anchor" href="#4-explore-other-models" aria-label="Anchor link for: 4-explore-other-models"><i class="fas fa-link"></i></a>
</h3>
<p>Once comfortable with SmolLM3-3B, try:</p>
<ul>
<li><strong>SmolLM3-360M</strong> - Smaller, faster (great for testing)</li>
<li><strong>Llama-3.2-1B</strong> - Meta's small model</li>
<li><strong>Qwen2.5-3B</strong> - Multilingual support</li>
<li><strong>Phi-3-mini</strong> - Microsoft's efficient model</li>
</ul>
<h3 id="5-production-deployment">5. Production Deployment<a class="zola-anchor" href="#5-production-deployment" aria-label="Anchor link for: 5-production-deployment"><i class="fas fa-link"></i></a>
</h3>
<p>For production use:</p>
<pre data-lang="python" style="background-color:#fafafa;color:#80cbc4;" class="language-python "><code class="language-python" data-lang="python"><span style="font-style:italic;color:#ccd7da;"># Merge LoRA adapters into base model (optional)
</span><span>merged_model </span><span style="color:#39adb5;">= </span><span style="color:#6182b8;">model</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">merge_and_unload</span><span style="color:#39adb5;">()
</span><span style="color:#6182b8;">merged_model</span><span style="color:#39adb5;">.</span><span style="color:#6182b8;">save_pretrained</span><span style="color:#39adb5;">("</span><span style="color:#91b859;">./final-model</span><span style="color:#39adb5;">")
</span><span>
</span><span style="font-style:italic;color:#ccd7da;"># Or keep separate (more flexible)
</span><span style="font-style:italic;color:#ccd7da;"># Just load base + adapters as shown in test_finetuned.py
</span></code></pre>
<h3 id="6-advanced-topics-to-explore">6. Advanced Topics to Explore<a class="zola-anchor" href="#6-advanced-topics-to-explore" aria-label="Anchor link for: 6-advanced-topics-to-explore"><i class="fas fa-link"></i></a>
</h3>
<ul>
<li><strong>Quantization</strong> (4-bit/8-bit for smaller memory)</li>
<li><strong>Multi-GPU training</strong> (distributed training)</li>
<li><strong>Custom loss functions</strong> (for specialized tasks)</li>
<li><strong>Reinforcement Learning from Human Feedback (RLHF)</strong></li>
<li><strong>Retrieval-Augmented Generation (RAG)</strong></li>
</ul>
<hr />
<h2 id="additional-resources">Additional Resources<a class="zola-anchor" href="#additional-resources" aria-label="Anchor link for: additional-resources"><i class="fas fa-link"></i></a>
</h2>
<h3 id="documentation">Documentation<a class="zola-anchor" href="#documentation" aria-label="Anchor link for: documentation"><i class="fas fa-link"></i></a>
</h3>
<ul>
<li><a rel="nofollow" href="https://huggingface.co/docs/transformers">HuggingFace Transformers</a></li>
<li><a rel="nofollow" href="https://huggingface.co/docs/peft">PEFT Documentation</a></li>
<li><a rel="nofollow" href="https://huggingface.co/docs/trl">TRL Documentation</a></li>
<li><a rel="nofollow" href="https://pytorch.org/docs">PyTorch Documentation</a></li>
</ul>
<h3 id="tutorials">Tutorials<a class="zola-anchor" href="#tutorials" aria-label="Anchor link for: tutorials"><i class="fas fa-link"></i></a>
</h3>
<ul>
<li><a rel="nofollow" href="https://huggingface.co/docs/transformers/training">HuggingFace Fine-tuning Guide</a></li>
<li><a rel="nofollow" href="https://arxiv.org/abs/2106.09685">LoRA Paper</a> (for deep understanding)</li>
</ul>
<h3 id="community">Community<a class="zola-anchor" href="#community" aria-label="Anchor link for: community"><i class="fas fa-link"></i></a>
</h3>
<ul>
<li><a rel="nofollow" href="https://discuss.huggingface.co/">HuggingFace Forums</a></li>
<li><a rel="nofollow" href="https://discuss.pytorch.org/">PyTorch Forums</a></li>
</ul>
<hr />
<h2 id="glossary">Glossary<a class="zola-anchor" href="#glossary" aria-label="Anchor link for: glossary"><i class="fas fa-link"></i></a>
</h2>
<ul>
<li><strong>Adapter</strong>: Small trainable layers added to a frozen model (LoRA's approach)</li>
<li><strong>Batch Size</strong>: Number of examples processed together</li>
<li><strong>Epoch</strong>: One complete pass through all training data</li>
<li><strong>Fine-tuning</strong>: Adapting a pre-trained model to specific tasks</li>
<li><strong>Gradient Accumulation</strong>: Process multiple batches before updating weights</li>
<li><strong>Inference</strong>: Using a model to generate predictions/responses</li>
<li><strong>LoRA</strong>: Low-Rank Adaptation, efficient fine-tuning method</li>
<li><strong>Loss</strong>: Measure of how wrong the model's predictions are (lower = better)</li>
<li><strong>Temperature</strong>: Controls randomness in generation (0=deterministic, 1+=creative)</li>
<li><strong>Token</strong>: Smallest unit of text the model processes (~0.75 words)</li>
<li><strong>Tokenizer</strong>: Converts text to tokens and vice versa</li>
</ul>
<hr />
<h2 id="conclusion">Conclusion<a class="zola-anchor" href="#conclusion" aria-label="Anchor link for: conclusion"><i class="fas fa-link"></i></a>
</h2>
<p>You've now learned the complete workflow for LLM development:</p>
<ol>
<li>✅ Running pre-trained models</li>
<li>✅ Preparing custom training data</li>
<li>✅ Fine-tuning with LoRA</li>
<li>✅ Testing and validation</li>
<li>✅ Comparing results</li>
</ol>
<p><strong>Remember:</strong> The quality of your training data matters more than complex hyperparameters. Start with clear, specific, unique examples, and iterate from there.</p>
<p>Happy training! 🚀</p>
<hr />
<p><em>Last updated: 2024</em>
<em>Project: SmolLM3 Training Guide</em></p>
Hello, world
2024-10-18T00:00:00+00:00
2024-10-18T00:00:00+00:00
https://denys.dev/hello-world/
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis
nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore
eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt
in culpa qui officia deserunt mollit anim id est laborum.</p>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis
nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore
eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt
in culpa qui officia deserunt mollit anim id est laborum.</p>
<span id="continue-reading"></span>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis
nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore
eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt
in culpa qui officia deserunt mollit anim id est laborum.</p>
<p>:)</p>