Personal thoughts on various dev topics Zola 2026-10-15T00:00:00+00:00 https://denys.dev/atom.xml 2026-10-15T00:00:00+00:00 2026-10-15T00:00:00+00:00 https://denys.dev/self-inflicted-echo-chambers/

<p>You've probably seen it before. A team, a community, or a fanbase spends its days attacking its competitors nonstop. The same grievances come up again and again. The discussions go in circles. Nothing new is ever learned. At some point you start to wonder: are these people stuck in a loop of their own making?</p>

<p>You've probably seen it before. A team, a community, or a fanbase spends its days attacking its competitors nonstop. The same grievances come up again and again. The discussions go in circles. Nothing new is ever learned. At some point you start to wonder: are these people stuck in a loop of their own making?</p> <span id="continue-reading"></span> <p>Quite possibly, yes. That pattern is a classic sign of an echo chamber, and one that the group built entirely on its own. Nobody forced it on them. They constructed it brick by brick, message by message.</p> <h2 id="what-an-echo-chamber-actually-is">What an Echo Chamber Actually Is<a class="zola-anchor" href="#what-an-echo-chamber-actually-is" aria-label="Anchor link for: what-an-echo-chamber-actually-is"><i class="fas fa-link"></i></a> </h2> <p>An echo chamber is an environment where people only encounter opinions and information that confirm what they already believe. Inside one, the group's existing views are constantly repeated, amplified, and reinforced. Outside perspectives are absent, dismissed, or treated with suspicion.</p> <p>The key word here is <strong>self-inflicted</strong>. This is not about algorithmic filter bubbles imposed by social media feeds. This is about groups that do it to themselves, voluntarily, one conversation at a time.</p> <h2 id="the-warning-signs">The Warning Signs<a class="zola-anchor" href="#the-warning-signs" aria-label="Anchor link for: the-warning-signs"><i class="fas fa-link"></i></a> </h2> <p>How do you recognize a self-inflicted echo chamber? Look for these patterns.</p> <h3 id="1-no-new-information-enters-the-loop">1. No New Information Enters the Loop<a class="zola-anchor" href="#1-no-new-information-enters-the-loop" aria-label="Anchor link for: 1-no-new-information-enters-the-loop"><i class="fas fa-link"></i></a> </h3> <p>In a healthy group, discussions evolve. Someone brings in fresh data, a new perspective, a piece of news that changes the picture. In an echo chamber, the same arguments are recycled endlessly because the group only validates itself. The conversation doesn't grow. It just repeats, slightly louder each time.</p> <h3 id="2-identity-replaces-argument">2. Identity Replaces Argument<a class="zola-anchor" href="#2-identity-replaces-argument" aria-label="Anchor link for: 2-identity-replaces-argument"><i class="fas fa-link"></i></a> </h3> <p>When criticism of competitors becomes a group ritual, it stops being about accuracy. It becomes about belonging. Attacking the "other side" is how members signal loyalty and earn status within the group. Contrary evidence isn't examined, it's dismissed as enemy propaganda, because engaging with it would feel like betrayal.</p> <h3 id="3-circles-instead-of-progress">3. Circles Instead of Progress<a class="zola-anchor" href="#3-circles-instead-of-progress" aria-label="Anchor link for: 3-circles-instead-of-progress"><i class="fas fa-link"></i></a> </h3> <p>Genuine analysis converges toward conclusions, decisions, or new insights. Echo chambers loop instead. There's nothing external to push the discussion forward, so it spins in place. If your group's conversations feel like a rerun of last month's conversation, take note.</p> <h3 id="4-disproportionate-focus-on-the-enemy">4. Disproportionate Focus on the Enemy<a class="zola-anchor" href="#4-disproportionate-focus-on-the-enemy" aria-label="Anchor link for: 4-disproportionate-focus-on-the-enemy"><i class="fas fa-link"></i></a> </h3> <p>Ask a simple question: how much of the group's energy goes into attacking others versus improving its own work? A healthy group spends most of its time building, learning, and solving problems. An echo chamber spends most of its time on the opposition. That imbalance is a reliable red flag. When a group defines itself primarily by what it opposes, it has outsourced its identity.</p> <h3 id="5-dissent-gets-punished">5. Dissent Gets Punished<a class="zola-anchor" href="#5-dissent-gets-punished" aria-label="Anchor link for: 5-dissent-gets-punished"><i class="fas fa-link"></i></a> </h3> <p>This is the most decisive sign. Watch what happens when someone in the group says "actually, our competitor did this well" or "we have a real weakness here." In a healthy group, that lands as a useful contribution. In an echo chamber, the messenger pays a social price. They get mocked, sidelined, or accused of disloyalty. Once honest internal criticism becomes socially expensive, the echo chamber is complete.</p> <h2 id="why-groups-do-this-to-themselves">Why Groups Do This to Themselves<a class="zola-anchor" href="#why-groups-do-this-to-themselves" aria-label="Anchor link for: why-groups-do-this-to-themselves"><i class="fas fa-link"></i></a> </h2> <p>Self-inflicted echo chambers don't form because people are stupid. They form because the behavior is rewarded, at least in the short term.</p> <p><strong>Group polarization</strong> is one driver. Study after study shows that when like-minded people discuss a topic repeatedly, their views become more extreme than where any individual started. Each round of mutual agreement nudges the group a notch further.</p> <p><strong>Social rewards</strong> are another. Inside the chamber, attacking competitors earns approval. Nuance earns suspicion. People respond to incentives, even subtle social ones, and they drift toward whatever earns them standing.</p> <p><strong>Frustration plays a role too.</strong> Sometimes the group is stuck competing against an opponent it can't beat, at least not by the means available. Unable to make progress in reality, they make progress in rhetoric. The venting becomes compulsive because it substitutes for achievement.</p> <h2 id="why-it-s-destructive">Why It's Destructive<a class="zola-anchor" href="#why-it-s-destructive" aria-label="Anchor link for: why-it-s-destructive"><i class="fas fa-link"></i></a> </h2> <p>The damage compounds over time.</p> <p>The group becomes <strong>blind to real threats and real opportunities</strong>. If every piece of information about competitors is pre-framed as "they're bad," you lose the ability to notice when they do something genuinely innovative that you should respond to.</p> <p>The group <strong>stops improving</strong>. Learning requires feedback, and feedback requires exposure to something that contradicts what you already think. An echo chamber seals off that supply line. The group becomes confident and stagnant at the same time, which is the worst possible combination.</p> <p>And eventually, <strong>the outside world notices</strong>. Prospective customers, partners, and hires can see when a group spends its energy attacking rivals instead of doing good work. It reads as insecurity, and insecurity is rarely an attractive brand.</p> <h3 id="the-test">The Test<a class="zola-anchor" href="#the-test" aria-label="Anchor link for: the-test"><i class="fas fa-link"></i></a> </h3> <p>If you want to know whether your group has slipped into an echo chamber, run a simple experiment. Present evidence that a competitor did something well, or that your own side has a genuine weakness. Then watch the response.</p> <p>A healthy group engages with it. It debates, updates, maybe acts on it. An echo chamber punishes it. That reaction is your answer, and it tells you everything you need to know.</p> <h2 id="the-antidote">The Antidote<a class="zola-anchor" href="#the-antidote" aria-label="Anchor link for: the-antidote"><i class="fas fa-link"></i></a> </h2> <p>The good news is that self-inflicted echo chambers can be dismantled, because the group built them and can tear them down. The antidote runs directly against the chamber's instincts, which is exactly why it must be deliberate.</p> <p><strong>Bring in outside input on purpose.</strong> Competitor users' actual feedback. Third-party reviews. Industry data. Market analysis. Whatever originates outside the group's own opinion loop. Make it a habit, not a one-off.</p> <p><strong>Normalize internal criticism.</strong> Make it explicitly safe, and ideally praiseworthy, to say "our competitor did this better than we do." The person who says it should gain status, not lose it. Leaders set this tone. If the most senior people in the group model honest self-critique, the rest will follow. If they model contempt for outsiders, the rest will follow that instead.</p> <p><strong>Measure the ratio.</strong> Keep a rough count of how much collective energy goes into attacking competitors versus improving your own work. If the attack ratio creeps up, treat it as an early warning indicator and correct course.</p> <p><strong>Rotate exposure.</strong> Send people to competitor events. Read the competitor's documentation. Use the competitor's product, sincerely, not as a mock exercise. Familiarity with the other side tends to replace caricature with understanding, and understanding is fatal to echo chamber dynamics.</p> <h2 id="the-bottom-line">The Bottom Line<a class="zola-anchor" href="#the-bottom-line" aria-label="Anchor link for: the-bottom-line"><i class="fas fa-link"></i></a> </h2> <p>Echo chambers are not imposed from the outside. Groups build them themselves, one validated opinion and one punished dissent at a time, usually without noticing. The first step out is simply noticing the loop. The second is deliberately breaking it by letting the outside world back in, even when it's uncomfortable, especially when it's uncomfortable.</p> <p>A group that can say "the competitor beat us here, and we're going to fix it" is far more dangerous than a group that spends its days explaining why the competitor is terrible. The first one gets better. The second one just gets louder.</p> <p><a href='https://ko-fi.com/F6R626VT2L' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi3.png?v=6' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a></p> 2026-09-29T00:00:00+00:00 2026-09-29T00:00:00+00:00 https://denys.dev/w-mythology-vol3/

<p>This blog post is part of a series debunking myths and clarifying misconceptions about W Social - so you can separate fact from fiction.</p>

<p>This blog post is part of a series debunking myths and clarifying misconceptions about W Social - so you can separate fact from fiction.</p> <span id="continue-reading"></span> <p><a href='https://ko-fi.com/F6R626VT2L' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi3.png?v=6' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a></p> <p><strong>From the series:</strong></p> <ul> <li><a href="https://denys.dev/w-mythology-vol1/">Myths About W Social, Explained in Simple Words. Vol 1.</a></li> <li><a href="https://denys.dev/w-mythology-vol2/">Myths About W Social, Explained in Simple Words. Vol 2.</a></li> <li>Myths About W Social, Explained in Simple Words. Vol 3.</li> </ul> <h2 id="introduction">Introduction<a class="zola-anchor" href="#introduction" aria-label="Anchor link for: introduction"><i class="fas fa-link"></i></a> </h2> <p><strong>W</strong> (also known as <a href="https://wsocial.news">W Social</a>) is a European social network governed by EU law, data hosted in Europe, and built for real, verified people. It is built on the <a href="https://en.wikipedia.org/wiki/AT_Protocol">AT Protocol</a>, an open-source framework for interoperable social media (<a href="https://wsocial.news/faq">source</a>).</p> <p><strong>Official answers to your questions</strong></p> <p>There is an official page on the W Social website where you can find answers on the most frequently asked questions - <a href="https://wsocial.news/faq">https://wsocial.news/faq</a>. Please make sure to visit it from time to time to see the updated content.</p> <p>This blog post is part of a series debunking myths and clarifying misconceptions about W Social - so you can separate fact from fiction.</p> <p><strong>From the series:</strong></p> <ul> <li><a href="https://denys.dev/w-mythology-vol1/">Myths About W Social, Explained in Simple Words. Vol 1.</a></li> <li><a href="https://denys.dev/w-mythology-vol2/">Myths About W Social, Explained in Simple Words. Vol 2.</a></li> <li>Myths About W Social, Explained in Simple Words. Vol 3.</li> </ul> <h2 id="myth-10-w-social-is-a-ghost-town-with-only-9-of-active-users">Myth #10: W Social is a ghost town, with only 9% of active users<a class="zola-anchor" href="#myth-10-w-social-is-a-ghost-town-with-only-9-of-active-users" aria-label="Anchor link for: myth-10-w-social-is-a-ghost-town-with-only-9-of-active-users"><i class="fas fa-link"></i></a> </h2> <p><strong>Reality:</strong></p> <p>The "9% of active users" figure sounds alarming - until you put it into context. Across the entire social media industry, <strong>a small fraction of users generating the majority of activity is not a bug; it’s the norm</strong>. Let’s look at the actual numbers.</p> <blockquote> <p><strong>See also:</strong> <a href="https://denys.dev/w-mythology-vol1/#myth-2-w-social-is-a-ghost-town-filled-with-thousands-of-profiles-that-cannot-post">Myth #2: W Social is a ghost town, filled with thousands of profiles that cannot post</a></p> </blockquote> <h3 id="how-many-people-use-bluesky">How many people use Bluesky?<a class="zola-anchor" href="#how-many-people-use-bluesky" aria-label="Anchor link for: how-many-people-use-bluesky"><i class="fas fa-link"></i></a> </h3> <p>According to the <a href="https://searchengineland.com/guide/bluesky-statistics">Bluesky Statistics</a> report by Search Engine Land, the year-over-year growth for the registered users is approximately the following:</p> <table><thead><tr><th>Date</th><th>Bluesky Registered Users</th></tr></thead><tbody> <tr><td>September 15, 2024</td><td>10 million</td></tr> <tr><td>October 19, 2024</td><td>12.11 million</td></tr> <tr><td>October 24, 2024</td><td>13 million</td></tr> <tr><td>November 13, 2024</td><td>15 million</td></tr> <tr><td>November 20, 2024</td><td>20 million</td></tr> <tr><td>December 19, 2024</td><td>25.9 million</td></tr> <tr><td>January 4, 2025</td><td>26.31 million</td></tr> <tr><td>January 14, 2025</td><td>27.44 million</td></tr> <tr><td>January 29, 2025</td><td>30 million</td></tr> <tr><td>May 20, 2025</td><td>35.98 million</td></tr> <tr><td>August 15, 2025</td><td>38.26 million</td></tr> <tr><td>November 9, 2025</td><td>40.2 million</td></tr> </tbody></table> <p><strong>Sources:</strong> <a href="https://bsky.app/profile/jay.bsky.team/post/3lbd2eaura22r">Bluesky</a>, <a href="https://bsky.jazco.dev/stats">Jaz’s Bsky Stats</a>, <a href="https://www.statista.com/statistics/1536616/global-bluesky-users/">Statista</a>, <a href="https://techcrunch.com/2024/09/16/bluesky-now-has-10-million-users/">TechCrunch</a></p> <p>As of September 29, 2026, the estimated number is <strong>46,762,295</strong> and counting (<a href="https://bsky.jazco.dev/stats">source</a>)</p> <blockquote> <p>According to the latest estimates, Bluesky has <strong>3.5 million</strong> daily active users as of October 2025, <strong>with around 9% of registered users</strong> using the app on a daily basis.</p> </blockquote> <p><strong>Source</strong>: <a href="https://www.forbes.com/sites/conormurray/2025/11/07/bluesky-and-x-users-plummet-year-after-trumps-election-win-truth-social-makes-small-gains/">Forbes</a></p> <p>Read that carefully: this isn’t a W Social number. <strong>Bluesky itself - a mature platform with ~40+ million registered users and years of media coverage - has roughly 9% daily active users.</strong> If 9% were a symptom of a "ghost town," then the entire AT Protocol ecosystem, and arguably most of the social media industry, would be one. In reality, this ratio reflects a well-documented pattern of online behavior: most people <strong>register, browse, and lurk</strong>, while only a minority post daily.</p> <blockquote> <p>Bluesky reached 43.5 million registered users as of April 2026, up from just 10 million in September 2024</p> </blockquote> <p><strong>Source</strong>: <a href="https://marketful.com/bluesky-statistics">marketful.com</a></p> <h3 id="the-metric-nobody-talks-about-page-visits">The Metric Nobody Talks About: Page Visits<a class="zola-anchor" href="#the-metric-nobody-talks-about-page-visits" aria-label="Anchor link for: the-metric-nobody-talks-about-page-visits"><i class="fas fa-link"></i></a> </h3> <p>There’s another metric that rarely appears in "ghost town" debates: <strong>website visits</strong>. Most analytics about "active users" only count people who generate visible signals - posts, comments, likes, reposts. But a huge share of any platform’s audience <strong>never generates those signals at all</strong>.</p> <p>They visit the site, read posts, follow links from search engines and news articles, then leave. These readers are <strong>invisible to activity-based statistics</strong>, but they are very real. Think of them like the audience at a public debate: they don’t grab the microphone, but the room is far from empty.</p> <p>According to <a href="https://www.semrush.com/website/bsky.app/overview">Semrush</a>, Bluesky website visits rose from 17.41 million in June 2024 to 118.67 million in September 2025:</p> <table><thead><tr><th>Date</th><th>Bluesky Website Visits</th></tr></thead><tbody> <tr><td>June 2024</td><td>17.41 million</td></tr> <tr><td>July 2024</td><td>16.48 million</td></tr> <tr><td>August 2024</td><td>21 million</td></tr> <tr><td>September 2024</td><td>67.09 million</td></tr> <tr><td>October 2024</td><td>80.16 million</td></tr> <tr><td>November 2024</td><td>157.04 million</td></tr> <tr><td>December 2024</td><td>154.68 million</td></tr> <tr><td>April 2025</td><td>142.4 million</td></tr> <tr><td>May 2025</td><td>145.1 million</td></tr> <tr><td>June 2025</td><td>126.62 million</td></tr> <tr><td>July 2025</td><td>128.74 million</td></tr> <tr><td>August 2025</td><td>122.7 million</td></tr> <tr><td>September 2025</td><td>118.67 million</td></tr> </tbody></table> <p>Totalling <strong>132.19M</strong> in September 2026 (<a href="https://www.semrush.com/website/bsky.app/overview/">source</a>)</p> <p>That’s <strong>over 130 million monthly visits</strong> - more than 3 visits per registered user per month. A platform with over a hundred million monthly visits is, by any reasonable definition, alive and well. The lesson: <strong>"active users" and "audience" are different measurements</strong>, and ghost-town claims typically rely on only the first one while ignoring the second.</p> <h3 id="bluesky-activity-stats-by-kuba-suder">Bluesky activity stats (by Kuba Suder)<a class="zola-anchor" href="#bluesky-activity-stats-by-kuba-suder" aria-label="Anchor link for: bluesky-activity-stats-by-kuba-suder"><i class="fas fa-link"></i></a> </h3> <p>Source: <a href="https://blue.mackuba.eu/stats/">blue.mackuba.eu/stats</a> by <a href="https://bsky.app/profile/did:plc:oio4hkxaop4ao4wz2pp3f4cr">Kuba Suder</a></p> <p><strong>Total daily posts</strong> (2026-09-29): <strong>3,108,738</strong></p> <p><img src="https://denys.dev/w-mythology-vol3/2026-09-29-total-daily-posts-mackuba.jpg" alt="Total daily posts" /></p> <p><strong>Unique daily posting users</strong> (2026-09-26): <strong>509,852</strong></p> <p><img src="https://denys.dev/w-mythology-vol3/2026-09-26-unique-daily-posting-users-mackuba.jpg" alt="Unique daily posting users" /></p> <p><strong>Unique weekly posting users</strong> (2026-09-21): <strong>1,226,122</strong></p> <p><img src="https://denys.dev/w-mythology-vol3/2026-09-29-unique-weekly-posting-users-mackuba.jpg" alt="Unique weekly posting users" /></p> <p>Let’s put these three numbers side by side, because they tell a fascinating story:</p> <ul> <li><strong>~46.7 million</strong> registered users</li> <li><strong>~1.23 million</strong> users posting in a given week</li> <li><strong>~510 thousand</strong> users posting on a given day</li> </ul> <p>That means only <strong>~1% of registered users post on any given day</strong>, and <strong>~2.6% post in any given week</strong>. Even if we generously assume daily users include lurkers, the pattern holds: <strong>the vast majority of accounts are readers, not writers</strong>.</p> <p>And yet - nobody calls Bluesky a ghost town. Over <strong>3 million posts are published every single day</strong>. The content flows because a small, active core is producing it, while a large silent majority consumes it.</p> <h3 id="bluesky-user-growth-and-active-users">Bluesky User Growth and Active Users<a class="zola-anchor" href="#bluesky-user-growth-and-active-users" aria-label="Anchor link for: bluesky-user-growth-and-active-users"><i class="fas fa-link"></i></a> </h3> <blockquote> <p>This page shows how activity and usage of Bluesky has been growing over time. You can see how many users are actively posting on Bluesky and how many posts are added to Bluesky on a daily basis. The data is updated every 24 hours.</p> </blockquote> <p>Source: <a href="https://bluefacts.app/bluesky-user-growth">bluefacts.app</a></p> <ul> <li><strong>6.92M</strong> — Unique users active in the last 90 days</li> <li><strong>4.33M</strong> — Unique users active in the last 30 days</li> <li><strong>2.55M</strong> — Unique users active in the last 7 days</li> <li><strong>1,232,345</strong> — Daily users (average over the last 7 days)</li> </ul> <p><img src="https://denys.dev/w-mythology-vol3/2026-09-29-bluesky-user-growth-bluefacts.jpg" alt="Bluesky User Growth" /></p> <h3 id="what-the-numbers-tell-us">What the Numbers Tell Us?<a class="zola-anchor" href="#what-the-numbers-tell-us" aria-label="Anchor link for: what-the-numbers-tell-us"><i class="fas fa-link"></i></a> </h3> <p>The patterns we just saw aren’t unique to AT Protocol platforms - they’re among the most well-documented phenomena in online community research:</p> <ol> <li> <p><strong>The 1% Rule</strong> (<a href="https://en.wikipedia.org/wiki/1%25_rule">Wikipedia</a>)<br /> In any online community, <strong>roughly 1% of users create content</strong>, while the rest mostly consume it. This rule has held across forums, wikis, and social networks for decades. The Bluesky numbers above (~1% daily posters) match it almost exactly.</p> </li> <li> <p><strong>The 90-9-1 Rule</strong> (<a href="https://hivebrite.io/glossary/90-9-1-rule/">Hivebrite</a>)<br /> A refinement of the same idea: <strong>90% of users lurk</strong> (read only), <strong>9% engage occasionally</strong> (like, comment, repost), and <strong>1% create most of the content</strong>. If your platform shows this distribution, it’s not dying - it’s behaving like every healthy community ever studied.</p> </li> <li> <p><strong>The Pareto Principle</strong> (<a href="https://en.wikipedia.org/wiki/Pareto_principle">Wikipedia</a>)<br /> The classic "80/20 rule": roughly <strong>80% of effects come from 20% of causes</strong>. On social platforms, that translates to <strong>a minority of users producing the majority of posts, likes, and engagement</strong>.</p> </li> </ol> <p>So when someone points at W Social and says <em>"only 9% are active - ghost town!"</em>, they are describing <strong>the internet itself</strong>. By these well-established benchmarks, 9% daily activity is not merely acceptable - it’s <strong>above the industry baseline</strong>. And crucially, W Social is doing this while <strong>still in beta</strong>, with a <strong>waiting list</strong>, <strong>invite-only growth</strong>, and a <strong>verification requirement</strong> that deliberately slows onboarding.</p> <h3 id="bluesky-post-count-and-author-stats-by-jaz">Bluesky Post Count and Author Stats (by Jaz)<a class="zola-anchor" href="#bluesky-post-count-and-author-stats-by-jaz" aria-label="Anchor link for: bluesky-post-count-and-author-stats-by-jaz"><i class="fas fa-link"></i></a> </h3> <p>source: <a href="https://bsky.jazco.dev/stats">https://bsky.jazco.dev/stats</a> by <a href="https://bsky.app/profile/jaz.bsky.social">Jaz</a></p> <p><img src="https://denys.dev/w-mythology-vol3/2026-09-29-bluesky-post-count-jaz.jpg" alt="Bluesky Post Count" /></p> <p><strong>Unique likers</strong> (2026-09-26): <strong>942,535</strong></p> <p><img src="https://denys.dev/w-mythology-vol3/2026-09-26-unique-likers-jaz.jpg" alt="Unique likers" /></p> <p><strong>Daily posters</strong> (2026-09-26): <strong>524,282</strong></p> <p><img src="https://denys.dev/w-mythology-vol3/2026-09-26-daily-posters-jaz.jpg" alt="Daily posters" /></p> <p><strong>Daily followers</strong> (2026-09-26): <strong>272,372</strong></p> <p><img src="https://denys.dev/w-mythology-vol3/2026-09-26-daily-followers-jaz.jpg" alt="Daily followers" /></p> <p>Now let's line up these daily numbers against the ~46.7 million registered users:</p> <ul> <li><strong>~524K</strong> daily posters — about <strong>1.1%</strong> of all accounts</li> <li><strong>~943K</strong> daily likers — about <strong>2%</strong> of all accounts</li> <li><strong>~272K</strong> daily followers (i.e., actively following new accounts) — about <strong>0.6%</strong> of all accounts</li> </ul> <p>Notice the pattern: <strong>each type of engagement draws a smaller, more active crowd than the one before</strong>. Posting requires the most effort, so fewest people do it. Liking is easier, so nearly twice as many people do that. Following new accounts is a niche activity, so it involves fewer still. This is the <strong>90-9-1 pyramid in live action</strong> - and it's exactly what you'd expect from a healthy, functioning social platform, not a ghost town.</p> <p>And remember: all of these are <strong>daily</strong> figures against <strong>all-time</strong> registrations. Millions of accounts registered during the November 2024 surge, tried the platform for a week, and never returned. Every social network carries this "dead weight" of historical registrations - Bluesky, X, Facebook, and yes, W Social too. Measuring daily activity against total sign-ups <strong>will always</strong> produce a small percentage. That's not a bug in the platform; it's a quirk of the maths.</p> <h2 id="summary-separating-fact-from-fiction-on-w-social">Summary: Separating Fact from Fiction on W Social<a class="zola-anchor" href="#summary-separating-fact-from-fiction-on-w-social" aria-label="Anchor link for: summary-separating-fact-from-fiction-on-w-social"><i class="fas fa-link"></i></a> </h2> <p>W Social is a <strong>European, verified, and transparent</strong> platform built on the <strong>AT Protocol</strong>, designed to foster <strong>authentic conversations</strong> under <strong>EU laws</strong>. Let’s recap the realities behind the myths:</p> <p>The claim that "9% active users = ghost town" collapses under its own arithmetic. The same ratio applies to <strong>Bluesky itself</strong>, a platform with <strong>46+ million registered users</strong>, <strong>130+ million monthly website visits</strong>, and <strong>3+ million posts per day</strong>. The 1% rule, the 90-9-1 rule, and the Pareto principle all confirm the same truth: <strong>online communities have always been powered by a small active core and a large reading audience</strong>.</p> <p>For a startup platform <strong>still in beta</strong>, with a <strong>waiting list, invites, and mandatory identity verification</strong>, a 9% activity ratio is not a warning sign - it’s an <strong>absolutely healthy metric</strong>, in line with (and arguably above) the industry baseline. The community isn’t missing; it’s <strong>being built deliberately</strong>.</p> <p>W Social is <strong>not just another social network</strong> - it’s a space for <strong>real people, real discussions, and real accountability</strong>. As it prepares for its <strong>January 2027 launch</strong>, it’s clear: <strong>W Social is building a better way to connect online</strong>.</p> <div style='text-align:center;'> <a href='https://ko-fi.com/F6R626VT2L' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi3.png?v=6' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a> </div> 2026-09-11T00:00:00+00:00 2026-09-11T00:00:00+00:00 https://denys.dev/w-mythology-vol2/

<p>This blog post is part of a series debunking myths and clarifying misconceptions about W Social - so you can separate fact from fiction.</p>

<p>This blog post is part of a series debunking myths and clarifying misconceptions about W Social - so you can separate fact from fiction.</p> <span id="continue-reading"></span> <p><a href='https://ko-fi.com/F6R626VT2L' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi3.png?v=6' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a></p> <p><strong>From the series:</strong></p> <ul> <li><a href="https://denys.dev/w-mythology-vol1/">Myths About W Social, Explained in Simple Words. Vol 1.</a></li> <li>Myths About W Social, Explained in Simple Words. Vol 2.</li> <li><a href="https://denys.dev/w-mythology-vol3/">Myths About W Social, Explained in Simple Words. Vol 3.</a></li> </ul> <h2 id="introduction">Introduction<a class="zola-anchor" href="#introduction" aria-label="Anchor link for: introduction"><i class="fas fa-link"></i></a> </h2> <p><strong>W</strong> (also known as <a href="https://wsocial.news">W Social</a>) is a European social network governed by EU law, data hosted in Europe, and built for real, verified people. It is built on the <a href="https://en.wikipedia.org/wiki/AT_Protocol">AT Protocol</a>, an open-source framework for interoperable social media (<a href="https://wsocial.news/faq">source</a>).</p> <p><strong>Official answers to your questions</strong></p> <p>There is an official page on the W Social website where you can find answers on the most frequently asked questions - <a href="https://wsocial.news/faq">https://wsocial.news/faq</a>. Please make sure to visit it from time to time to see the updated content.</p> <p>This blog post is part of a series debunking myths and clarifying misconceptions about W Social - so you can separate fact from fiction.</p> <h2 id="myth-6-implementing-on-device-id-verification-will-completely-eradicate-bots-from-the-platform">Myth #6: Implementing on-device ID verification will completely eradicate bots from the platform<a class="zola-anchor" href="#myth-6-implementing-on-device-id-verification-will-completely-eradicate-bots-from-the-platform" aria-label="Anchor link for: myth-6-implementing-on-device-id-verification-will-completely-eradicate-bots-from-the-platform"><i class="fas fa-link"></i></a> </h2> <p><strong>Reality:</strong></p> <p>While <strong>on-device ID verification</strong> (like W Social’s <strong>W Identity</strong>) dramatically reduces bot activity, it doesn’t eliminate them entirely. However, it <strong>fundamentally changes the game</strong> by making bots far more costly and difficult to deploy. Here’s how:</p> <p><strong>The Bot Problem</strong></p> <blockquote> <p>A <strong>social bot</strong>, refers to fully or partially automated <a href="https://en.wikipedia.org/wiki/Social_media" title="Social media">social media</a> accounts designed to perform most regular users’ actions, such as liking, posting content, and chatting with other users. - <em><strong><a href="https://en.wikipedia.org/wiki/Social_bot">Wikipedia</a></strong></em></p> </blockquote> <p>Social bots - automated accounts designed to mimic human behavior - are a major issue on modern platforms. They <strong>dilute real conversations, manipulate trends, and coordinate harassment</strong>. Bots come in many forms, including:</p> <ul> <li><strong>Simple Bots</strong>: Automate basic tasks (e.g., FAQ responses, customer support).</li> <li><strong>Chatbots</strong>: Use AI to engage in conversations (e.g., customer service).</li> <li><strong>Social Media Bots</strong>: Influence public opinion or shape narratives.</li> <li><strong>Trolls &amp; Malicious Bots</strong>: Spread misinformation or disrupt discussions.</li> </ul> <blockquote> <p>Social bots are always becoming increasingly difficult to detect and understand. The bots' human-like behavior, ever-changing behavior of the bots, and the sheer volume of bots covering every platform may have been a factor in the challenges of removing them. Social media sites, like <a href="https://en.wikipedia.org/wiki/Twitter" title="Twitter">Twitter</a>, are among the most affected, with <a href="https://en.wikipedia.org/wiki/CNBC" title="CNBC">CNBC</a> reporting up to 48 million of the 319 million users (roughly 15%) were bots in 2017. - <em><strong><a href="https://en.wikipedia.org/wiki/Social_bot">Wikipedia</a></strong></em></p> </blockquote> <p><strong>How On-Device Verification Neutralizes Bots</strong></p> <p>W Social’s <strong>on-device ID verification</strong> requires users to validate a government ID or biometric data <strong>directly on their device</strong>, publishing only a <strong>privacy-preserving cryptographic hash</strong>. This approach ensures that <strong>one physical human = one unique verification hash</strong>, making it far harder for bots to operate at scale.</p> <blockquote> <p>[...] the researchers have found that bots may account for between 45 and 60% of Twitter accounts discussing covid-19. Many of those accounts were created in February and have since been spreading and amplifying misinformation, including false medical advice, conspiracy theories about the origin of the virus, and pushes to end stay-at-home orders and reopen America. - <strong><em>Karen Hao, "Nearly half of Twitter accounts pushing to reopen America may be bots"</em>, May 2020</strong> (<a href="https://www.technologyreview.com/2020/05/21/1002105/covid-bot-twitter-accounts-push-to-reopen-america/">source</a>)</p> </blockquote> <p>Here’s the impact on different bot types:</p> <table><thead><tr><th><strong>Bot Type / Behavior</strong></th><th><strong>Status</strong></th><th><strong>Impact of On-Device Verification</strong></th></tr></thead><tbody> <tr><td><strong>Social Media Bots</strong> (Sybil Networks &amp; Farms)</td><td><strong>Eliminated at Scale</strong></td><td>Mass bot networks rely on creating thousands of accounts cheaply. With one ID = one hash, operators can’t spin up bot armies.</td></tr> <tr><td><strong>Trolls &amp; Malicious Actors</strong> (Burner Accounts)</td><td><strong>Significantly Reduced</strong></td><td>Banned users can’t create new verified profiles - their ID hash is already registered.</td></tr> <tr><td><strong>Simple Bots &amp; Automation</strong> (RSS Feeds, Utilities)</td><td><strong>Modular &amp; Accountable</strong></td><td>Automated scripts must be tied to a <strong>verified human identity</strong> via app passwords or scoped tokens.</td></tr> <tr><td><strong>Chatbots &amp; Conversational AI</strong></td><td><strong>Attested &amp; Transparent</strong></td><td>AI models can’t secretly pose as humans - they must operate under a verified user’s credentials or be tagged as unverified.</td></tr> </tbody></table> <p><strong>Key Advantages of On-Device Verification</strong></p> <ul> <li><strong>Zero Privacy Compromise:</strong> Personal identification data never leaves the user’s device. The network only sees an un-linkable mathematical proof confirming the user is a unique 18+ human.</li> <li><strong>No Honeypot Risks:</strong> Traditional verification forces companies to store millions of government passports, creating huge targets for data breaches. Storing local cryptographic hashes mitigates this centralized liability.</li> <li><strong>Preserved Anonymity:</strong> Whistleblowers, journalists, and activists can participate freely without fear that platform databases can be subpoenaed to reveal their true real-world identities.</li> </ul> <p><strong>The Takeaway:</strong></p> <p>On-device ID verification isn’t a <strong>silver bullet</strong> for eliminating all bots. Instead, it <strong>raises the cost of creating bots</strong> from near-zero to <strong>$10+ per account</strong>, turning them from a <strong>mass spam problem</strong> into a <strong>rare, targeted threat</strong>. This shift makes bot networks <strong>far less viable</strong> while preserving user privacy.</p> <p><strong>See also</strong>:</p> <ul> <li><a href="https://www.bbc.co.uk/bitesize/articles/zjhg47h">What are 'bots' and how can they spread fake news?</a></li> <li><a href="https://www.icuc.social/resources/blog/influence-of-social-media-bots">Understanding the Influence of Social Media Bots</a></li> <li><a href="https://www.technologyreview.com/2020/05/21/1002105/covid-bot-twitter-accounts-push-to-reopen-america/">Nearly half of Twitter accounts pushing to reopen America may be bots</a></li> </ul> <h2 id="myth-7-w-is-building-yet-another-echo-chamber">Myth #7: W is building yet another echo chamber<a class="zola-anchor" href="#myth-7-w-is-building-yet-another-echo-chamber" aria-label="Anchor link for: myth-7-w-is-building-yet-another-echo-chamber"><i class="fas fa-link"></i></a> </h2> <p><strong>Reality:</strong></p> <p>An <a href="https://en.wikipedia.org/wiki/Echo_chamber_(media)"><strong>echo chamber</strong></a> is a space where people only encounter opinions that mirror their own, reinforcing beliefs and shielding them from diverse perspectives. This can <strong>polarize discussions, limit critical thinking, and deepen societal divisions</strong>.</p> <p>But <strong>W Social isn’t an echo chamber</strong> - it’s a platform designed for <strong>authentic, verified voices</strong> to engage in <strong>healthy discussions and debates</strong>. Here’s why:</p> <p><strong>Why Echo Chambers Are Harmful</strong></p> <p>Echo chambers create <strong>intellectual silos</strong>, where:</p> <ul> <li><strong>Misinformation spreads unchecked</strong> (no opposing views to challenge it).</li> <li><strong>Polarization grows</strong> (users only see one side of an issue).</li> <li><strong>Critical thinking declines</strong> (no exposure to alternative perspectives).</li> </ul> <p><strong>How W Social Avoids the Echo Chamber Trap</strong></p> <ol> <li> <p><strong>Verified, Diverse Voices</strong><br /> W Social requires <strong>identity verification</strong>, ensuring that every account represents a <strong>real person</strong>. This reduces the likelihood of <strong>fake accounts amplifying extreme or one-sided narratives</strong>.</p> </li> <li> <p><strong>Open by Default</strong><br /> Posts on W Social are <strong>public by default</strong>, meaning conversations are <strong>visible to everyone</strong> - not just like-minded users. This encourages <strong>cross-pollination of ideas</strong> and exposure to different viewpoints.</p> </li> <li> <p><strong>Designed for Debate</strong><br /> Unlike platforms that <strong>algorithmically amplify outrage or agreement</strong>, W Social’s structure supports <strong>healthy discussions</strong>. For example:</p> </li> </ol> <blockquote> <p><em>Imagine a debate about <strong>climate policy</strong> where scientists, policymakers, and skeptics engage in a <strong>respectful, evidence-based discussion</strong>. Instead of shouting matches or echo chambers, users can challenge ideas, ask questions, and refine their perspectives - all while knowing they’re interacting with <strong>real, verified people</strong>.</em></p> </blockquote> <blockquote> <p><em>Or consider a <strong>political discussion</strong> about immigration reform, where advocates, critics, and undecided citizens exchange <strong>nuanced arguments</strong> without fear of anonymous harassment. Verified identities ensure accountability, so conversations stay <strong>focused on facts and solutions</strong> rather than personal attacks.</em></p> </blockquote> <blockquote> <p><em>Even on <strong>social issues</strong> like gender equality or education reform, W Social enables <strong>constructive dialogue</strong>. Users can share personal experiences, cite research, and engage in <strong>thoughtful debate</strong> - without the noise of bots or trolls derailing the conversation.</em></p> </blockquote> <p>Users can:</p> <ul> <li><strong>Engage in respectful debates</strong> (no anonymous trolls derailing conversations).</li> <li><strong>Challenge ideas</strong> (verified users are more likely to stand by their opinions).</li> <li><strong>Build communities around shared interests - not just shared beliefs</strong>.</li> </ul> <p><strong>The Takeaway:</strong></p> <p>W Social isn’t about creating an echo chamber - it’s about <strong>fostering real conversations</strong> among real people. By prioritizing <strong>verification, transparency, and open dialogue</strong>, W Social ensures that discussions are <strong>authentic, diverse, and constructive</strong>.</p> <p><strong>See also:</strong></p> <ul> <li><a href="https://en.wikipedia.org/wiki/Echo_chamber_(media)">Echo chamber (media)</a></li> <li><a href="https://www.bbc.co.uk/bitesize/articles/zbwkbqt">What are echo chambers?</a></li> <li><a href="https://netpsychology.org/social-media-echo-chambers/">Social Media Echo Chambers: How Algorithms Control What You See</a></li> <li><a href="https://www.pnas.org/doi/10.1073/pnas.2023301118">The echo chamber effect on social media</a></li> </ul> <h2 id="myth-8-w-does-not-have-moderation">Myth #8: W Does Not Have Moderation<a class="zola-anchor" href="#myth-8-w-does-not-have-moderation" aria-label="Anchor link for: myth-8-w-does-not-have-moderation"><i class="fas fa-link"></i></a> </h2> <p><strong>Reality:</strong></p> <p>Some users - both inside and outside W Social - claim that the platform <strong>lacks moderation</strong>. But the truth is, <strong>W Social does moderate content</strong>, and it does so <strong>responsibly and in compliance with EU laws</strong>. Here’s how it works:</p> <p><strong>How Moderation Works on W Social</strong></p> <ol> <li> <p><strong>Proactive and Reactive Moderation</strong><br /> The W Social team <strong>actively monitors</strong> both human and automated accounts to ensure they follow <strong>platform rules, laws, and EU regulations</strong>. However, moderation isn’t just the team’s job - <strong>users play a key role too</strong>.</p> </li> <li> <p><strong>User Reporting Is Essential</strong><br /> If you encounter an issue, <strong>contacting support or using moderation feedback tools</strong> is the way to go. Public complaints on blogs, Reddit, or social media <strong>won’t trigger action</strong> - direct reports do. This ensures the team can <strong>investigate and respond promptly</strong>.</p> </li> <li> <p><strong>Free Speech ≠ No Rules</strong><br /> W Social believes in <strong>free speech</strong>, but that doesn’t mean anything goes. Content must <strong>obey the law</strong>. The platform has already taken action in multiple cases, including:</p> <ul> <li>Removing <strong>human and automated accounts</strong> (e.g., Neo, Tom) for violating rules.</li> <li>Taking down <strong>block lists</strong> that breached platform policies.</li> </ul> </li> <li> <p><strong>Building Better Tools</strong><br /> W Social is still in <strong>closed beta</strong> (with a planned release in <strong>January 2027</strong>), and the team is <strong>actively improving moderation tools and processes</strong>. As the platform grows, so will its ability to <strong>detect and address violations efficiently</strong>.</p> </li> </ol> <p><strong>The Takeaway:</strong></p> <p>W Social <strong>does moderate content</strong> - but it relies on <strong>users to report issues directly</strong>. By combining <strong>proactive monitoring, legal compliance, and user reporting</strong>, W Social ensures a <strong>safer, more accountable</strong> environment. If you see something, <strong>say something (to support)</strong>.</p> <h2 id="myth-9-w-promotes-right-wing-propaganda">Myth #9: W Promotes Right-Wing Propaganda<a class="zola-anchor" href="#myth-9-w-promotes-right-wing-propaganda" aria-label="Anchor link for: myth-9-w-promotes-right-wing-propaganda"><i class="fas fa-link"></i></a> </h2> <p><strong>Reality:</strong></p> <p>This myth <strong>emerged during the Saxony-Anhalt election in September 2026</strong>, fueled by a mix of <strong>misinformation, competition, and deliberate anti-W campaigns</strong>. Here’s the full context - and why the claim doesn’t hold up:</p> <p><strong>Author’s Note: The Background</strong></p> <p><em>To the best of my knowledge, the following information is based on publicly available sources and open discussions.</em></p> <p>During the election period, <strong>W Social’s CEO (Anna Zeiter) and Chair (Ingmar Rentzhog)</strong> were <strong>auto-following every new account</strong> on the platform. This feature - intended to <strong>welcome and engage new users</strong> - was <strong>misrepresented</strong> by competitors and opponents. Combined with <strong>misbehaving accounts</strong> (which were later removed), it was used to falsely claim that W Social was <strong>endorsing right-wing propaganda</strong>.</p> <p>However, this narrative ignores <strong>key facts</strong> about how W Social - and any open platform -operates.</p> <p><strong>Why the Myth Is Flawed</strong></p> <ol> <li> <p><strong>Verified ≠ Vetted</strong><br /> While W Social <strong>verifies user identities</strong>, this doesn’t mean it <strong>vets their opinions or content</strong>. Verified users can still post <strong>abusive materials, spam, or rage-bait content</strong> - just like on any other platform. <strong>Verification ensures authenticity, not ideological alignment.</strong></p> </li> <li> <p><strong>Automation and Free Speech</strong><br /> Users are <strong>free to automate their accounts</strong> (e.g., using LLMs) <strong>as long as they comply with W Social’s rules and EU laws</strong>. The platform <strong>does not restrict lawful automation</strong>, but it <strong>does enforce violations</strong> when they occur.</p> </li> <li> <p><strong>Proactive Moderation</strong><br /> W Social <strong>actively monitors and addresses</strong> violations of its <strong>Terms of Service, EU laws, and community guidelines</strong>. Accounts that break these rules - <strong>including those spreading misinformation or propaganda</strong> - are <strong>promptly removed</strong>. However, <strong>competitors and opponents</strong> often <strong>screenshot and amplify</strong> controversial content <strong>just before it’s taken down</strong>, using it to push their own anti-W agendas.</p> </li> <li> <p><strong>Likes and Reposts ≠ Endorsement</strong><br /> A fundamental rule of social media is:</p> <blockquote> <p><strong>❤️ ≠ I agree.</strong></p> </blockquote> <ul> <li><strong>Posts reflect the author’s private opinions.</strong></li> <li><strong>Likes and reposts do not imply endorsement</strong> of the author’s political, religious, or ideological views.</li> <li>Users <strong>cannot perform full OSINT (Open-Source Intelligence) on every account</strong> they follow, especially if the account <strong>starts posting controversial content later</strong> - a common tactic among <strong>social bots</strong>.</li> </ul> </li> <li> <p><strong>Platform Neutrality</strong><br /> W Social - <strong>like any responsible platform</strong> - <strong>does not take sides</strong> in political or ideological debates. Its role is to:</p> <ul> <li><strong>Maintain dialogue</strong></li> <li><strong>Moderate content</strong> to ensure it complies with <strong>rules and laws</strong></li> <li><strong>Keep conversations civilized and respectful</strong></li> </ul> </li> </ol> <p><strong>The Takeaway</strong></p> <p>The claim that W Social <strong>promotes right-wing propaganda</strong> is <strong>misleading and context-free</strong>. The platform’s <strong>verification system, moderation policies, and commitment to neutrality</strong> ensure that it remains a space for <strong>authentic, lawful discourse</strong> - not a tool for any political agenda. <strong>Misbehaving accounts are removed</strong>, and <strong>competitors’ narratives</strong> should not be mistaken for reality.</p> <p><strong>Remember:</strong> A platform’s job is to <strong>host conversations</strong>, not <strong>police opinions</strong> - as long as they stay within the bounds of the law.</p> <h2 id="summary-separating-fact-from-fiction-on-w-social">Summary: Separating Fact from Fiction on W Social<a class="zola-anchor" href="#summary-separating-fact-from-fiction-on-w-social" aria-label="Anchor link for: summary-separating-fact-from-fiction-on-w-social"><i class="fas fa-link"></i></a> </h2> <p>W Social is a <strong>European, verified, and transparent</strong> platform built on the <strong>AT Protocol</strong>, designed to foster <strong>authentic conversations</strong> under <strong>EU laws</strong>. Let’s recap the realities behind the myths:</p> <ul> <li><strong>On-device ID verification</strong> raises the cost of bots, making spam rare while <strong>protecting privacy</strong>.</li> <li><strong>Not an echo chamber</strong>: Verified users engage in <strong>diverse, respectful debates</strong> on all topics.</li> <li><strong>Moderation exists</strong>: W Social enforces <strong>rules and laws</strong>, but <strong>user reports</strong> are essential.</li> <li><strong>No political bias</strong>: The platform <strong>hosts discussions</strong> - it doesn’t endorse them. <strong>Likes ≠ agreement</strong>.</li> </ul> <p>W Social is <strong>not just another social network</strong> - it’s a space for <strong>real people, real discussions, and real accountability</strong>. As it prepares for its <strong>January 2027 launch</strong>, it’s clear: <strong>W Social is building a better way to connect online</strong>.</p> <p>As W Social evolves, staying informed is key. For the latest updates, always check the <a href="https://wsocial.news/faq">official FAQ</a>.</p> <div style='text-align:center;'> <a href='https://ko-fi.com/F6R626VT2L' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi3.png?v=6' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a> </div> 2026-08-15T00:00:00+00:00 2026-08-15T00:00:00+00:00 https://denys.dev/w-mythology-vol1/

<p>This blog post is part of a series debunking myths and clarifying misconceptions about W Social - so you can separate fact from fiction.</p>

<p>This blog post is part of a series debunking myths and clarifying misconceptions about W Social - so you can separate fact from fiction.</p> <span id="continue-reading"></span> <p><a href='https://ko-fi.com/F6R626VT2L' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi3.png?v=6' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a></p> <p><strong>From the series:</strong></p> <ul> <li>Myths About W Social, Explained in Simple Words. Vol 1.</li> <li><a href="https://denys.dev/w-mythology-vol2/">Myths About W Social, Explained in Simple Words. Vol 2.</a></li> <li><a href="https://denys.dev/w-mythology-vol3/">Myths About W Social, Explained in Simple Words. Vol 3.</a></li> </ul> <h2 id="introduction">Introduction<a class="zola-anchor" href="#introduction" aria-label="Anchor link for: introduction"><i class="fas fa-link"></i></a> </h2> <p><strong>W</strong> (also known as <a href="https://wsocial.news">W Social</a>) is a European social network governed by EU law, data hosted in Europe, and built for real, verified people. It is built on the <a href="https://en.wikipedia.org/wiki/AT_Protocol">AT Protocol</a>, an open-source framework for interoperable social media (<a href="https://wsocial.news/faq">source</a>).</p> <p><strong>Official answers to your questions</strong></p> <p>There is an official page on the W Social website where you can find answers on the most frequently asked questions - <a href="https://wsocial.news/faq">https://wsocial.news/faq</a>. Please make sure to visit it from time to time to see the updated content.</p> <h2 id="at-protocol">AT Protocol<a class="zola-anchor" href="#at-protocol" aria-label="Anchor link for: at-protocol"><i class="fas fa-link"></i></a> </h2> <blockquote> <p>"It's just JSON. The AT Protocol is a data network. Posts, likes, follows, profiles, etc, are all just JSON." — <a href="https://atproto.com">atproto.com</a></p> </blockquote> <p>The AT Protocol is the <strong>technical backbone</strong> of the <a href="https://en.wikipedia.org/wiki/Bluesky">Bluesky</a> social network, serving as its <strong>reference implementation</strong> and powering an ecosystem of interoperable social applications and services, collectively known as the <strong>ATmosphere</strong>.</p> <p>In simple terms:</p> <p>The AT Protocol is like a <strong>global, open ocean of content</strong>, hosted across multiple servers worldwide. All data in this ocean is <strong>publicly accessible</strong>, and users can <strong>read, create, or modify</strong> it using various client applications - built by different developers, teams, or companies.</p> <h2 id="myth-1-identity-verification-is-the-end-of-privacy">Myth #1: Identity verification is the end of privacy<a class="zola-anchor" href="#myth-1-identity-verification-is-the-end-of-privacy" aria-label="Anchor link for: myth-1-identity-verification-is-the-end-of-privacy"><i class="fas fa-link"></i></a> </h2> <p><strong>Reality:</strong></p> <p>Identity verification isn’t about sacrificing privacy - it’s about <strong>preventing fraud and manipulation</strong>. The rise of AI and automation has made it easier than ever for bad actors to create fake accounts at scale. According to the <a href="https://www.experian.co.uk/blogs/latest-thinking/guide/identity-verification-what-is-it-and-why-is-it-important/">Experian UK Identity and Fraud Report</a>, <strong>23% of businesses encountered Gen-AI-related fraud in 2024</strong>, and this number <strong>jumped to 35% in early 2025</strong>. Troll farms and state-sponsored groups now deploy <strong>tens of thousands of fake accounts</strong> to spread disinformation, manipulate public opinion, or interfere in elections - all while appearing deceptively human.</p> <p>To combat this, some platforms are considering ID verification for users. Think of it like a bar doorman checking IDs at the entrance. The doorman’s job is to confirm you’re 18+ and allowed to enter - nothing more. They don’t keep copies of your ID, take photos of you, or store your personal details. Similarly, ID verification on social media could work the same way: confirming your identity without retaining unnecessary personal data. And just like you can choose to go to a non-alcoholic venue if you don’t want to show your ID, users who prefer not to verify their identity could opt for platforms without such restrictions.</p> <blockquote> <p>You identify yourself through W Identity, a separate app that scans your passport or national identity card and verifies you directly on your own device. You stay in control of what you share with other apps and services. Your ID data is stored only on your device unless you choose to share specific parts of it. — <a href="https://wsocial.news">W Social</a></p> </blockquote> <p><strong>Bottom Line:</strong></p> <p>Identity verification can <strong>combat fraud</strong> without compromising privacy. The key is <strong>user control</strong> - ensuring that your data stays in your hands, not the platform’s.</p> <h2 id="myth-2-w-social-is-a-ghost-town-filled-with-thousands-of-profiles-that-cannot-post">Myth #2: W Social is a ghost town, filled with thousands of profiles that cannot post<a class="zola-anchor" href="#myth-2-w-social-is-a-ghost-town-filled-with-thousands-of-profiles-that-cannot-post" aria-label="Anchor link for: myth-2-w-social-is-a-ghost-town-filled-with-thousands-of-profiles-that-cannot-post"><i class="fas fa-link"></i></a> </h2> <p><strong>Reality:</strong></p> <p>W Social isn’t a ghost town - it’s a <strong>growing, intentionally curated community</strong>. Here’s why the perception of inactivity is misleading:</p> <h3 id="two-types-of-accounts">Two Types of Accounts<a class="zola-anchor" href="#two-types-of-accounts" aria-label="Anchor link for: two-types-of-accounts"><i class="fas fa-link"></i></a> </h3> <p>W Social has two account types:</p> <ul> <li><strong>Verified users</strong>: Can view content, post, like, comment, and fully engage.</li> <li><strong>Unverified users</strong>: Can only <strong>view</strong> public content.</li> </ul> <p>Many users start as <strong>unverified</strong> to test the platform before committing to verification. As the <a href="https://wsocial.news/faq">official FAQ</a> states:</p> <blockquote> <p><em>"You may browse all public content without verification. Posting, commenting, messaging, and interacting with other users requires identity verification."</em></p> </blockquote> <h2 id="why-it-seems-quiet">Why It Seems Quiet<a class="zola-anchor" href="#why-it-seems-quiet" aria-label="Anchor link for: why-it-seems-quiet"><i class="fas fa-link"></i></a> </h2> <p><strong>Controlled Growth</strong></p> <p>W Social is currently in a <strong>closed beta</strong>, with signups managed via a waiting list. This means:</p> <ul> <li>Users <strong>cannot onboard quickly</strong>.</li> <li>Invites for friends and family are <strong>limited</strong>.</li> </ul> <p><strong>Unverified Users Are Invisible to Researchers</strong></p> <p>During the evaluation phase, unverified users don’t generate <strong>posts, comments, or likes</strong> - the typical signals researchers use to measure activity. This creates the illusion of a "ghost town," even though many users are <strong>actively exploring</strong> the platform.</p> <p><strong>Steady Growth in Engagement</strong></p> <p>Independent research shows that <strong>engagement and user activity are growing</strong> as more people join and complete verification.</p> <p><img src="https://denys.dev/w-mythology-vol1/active-users-chart.png" alt="Active Users Chart" /></p> <p><strong>Bottom Line:</strong></p> <p>W Social’s <strong>controlled growth</strong> and <strong>verification requirement</strong> mean that activity isn’t immediately visible to outsiders. But make no mistake - this is a <strong>deliberate strategy</strong> to ensure a high-quality, authentic user base, not a sign of failure.</p> <h2 id="myth-3-w-social-has-quietly-become-closed-source">Myth #3: W Social Has Quietly Become Closed-Source<a class="zola-anchor" href="#myth-3-w-social-has-quietly-become-closed-source" aria-label="Anchor link for: myth-3-w-social-has-quietly-become-closed-source"><i class="fas fa-link"></i></a> </h2> <p><strong>Reality:</strong></p> <p>W Social’s decision to <strong>temporarily privatize its client applications</strong> - while based on open-source code - has sparked debate. However, this doesn’t mean W Social has abandoned open-source principles. Here’s the full context:</p> <p><strong>The Open-Source Foundation</strong></p> <p>W Social’s client apps are <strong>forked</strong> from Bluesky’s open-source codebase, which is publicly available under the <a href="https://en.wikipedia.org/wiki/MIT_License">MIT License</a>. Bluesky’s official repositories include:</p> <ul> <li><strong>Web client code</strong></li> <li><strong>iOS app code</strong></li> <li><strong>Android app code</strong></li> </ul> <p>All of this is hosted on <a href="https://github.com/bluesky-social/social-app">Microsoft GitHub</a>, built with <strong>React Native</strong> and <strong>TypeScript</strong>, and supported by open-source <a href="https://github.com/bluesky-social/atproto"><code>@atproto</code> packages</a>. Bluesky even provides <a href="https://github.com/bluesky-social/social-app/blob/main/docs/build.md">detailed build instructions</a> for developers.</p> <p>The project’s popularity is undeniable:</p> <ul> <li><strong>2.8K+ forks</strong> of the client applications</li> <li><strong>18.2K+ stars</strong> on GitHub</li> </ul> <p><img src="https://denys.dev/w-mythology-vol1/bluesky-social-forks.png" alt="Bluesky GitHub Stats" /></p> <p><strong>Why Are Some Forks Private?</strong></p> <p>When developers or companies fork Bluesky’s repositories, they can choose to keep their copies <strong>public</strong> (visible to everyone) or <strong>private</strong> (hidden). Private forks often occur when developers:</p> <ul> <li>Add <strong>proprietary code</strong> or features</li> <li>Perform <strong>security audits</strong></li> <li>Rebrand the app visually (e.g., icons, fonts, illustrations)</li> <li>Clean up or optimize the codebase</li> </ul> <p>Bluesky’s <a href="https://github.com/bluesky-social/social-app/blob/main/ASSETS.md">Asset Licensing</a> policy has also fueled discussions in the open-source community. For example:</p> <blockquote> <p><em>"We should have written this down sooner. If you’ve already forked this repository and used its assets, we’re not treating that as bad faith—this is our oversight."</em></p> </blockquote> <p><strong>W Social’s Stance on Open Source</strong></p> <p>W Social addresses this directly in its <a href="https://wsocial.news/faq">official FAQ</a>:</p> <blockquote> <p><em>"During W’s early development, the repositories we forked were publicly accessible and were later made private while we reviewed <strong>security, privacy, licensing, intellectual property, documentation, and commercial considerations</strong>. W will republish the forked repositories - with improvements, integrations, bug fixes, and technical documentation - once they’ve been properly reviewed and are ready for responsible release. <strong>W will comply fully with all applicable open-source licenses.</strong>"</em></p> </blockquote> <p><strong>Bottom Line:</strong></p> <p>W Social’s <strong>temporary privatization</strong> of its client apps is a <strong>development phase</strong>, not a rejection of open-source values. The goal is to ensure that when the code is republished, it meets <strong>high standards for security, compliance, and responsibility</strong>. Until then, the platform remains committed to open-source principles.</p> <h2 id="myth-4-by-moving-to-w-social-european-institutions-migrated-their-accounts-from-an-open-to-a-closed-source-network">Myth #4: By moving to W Social, European institutions migrated their accounts from an open to a closed-source network<a class="zola-anchor" href="#myth-4-by-moving-to-w-social-european-institutions-migrated-their-accounts-from-an-open-to-a-closed-source-network" aria-label="Anchor link for: myth-4-by-moving-to-w-social-european-institutions-migrated-their-accounts-from-an-open-to-a-closed-source-network"><i class="fas fa-link"></i></a> </h2> <p><strong>Reality:</strong></p> <p>Your posts on W are public by default, ensuring conversations remain open and visible to everyone. While W is built on the <strong>open, global AT-Protocol</strong>, its client applications (web, iOS, Android) are currently closed-source during the active development and testing phase. However, the platform may open these applications to researchers for public review in the future - excluding proprietary systems.</p> <p>The <strong>key difference</strong> lies in <strong>data hosting</strong>:</p> <blockquote> <p><em>Your data is hosted by W on a Personal Data Server (PDS) in the EU, controlled by a European company</em>. — <a href="https://wsocial.news/faq">W Social</a></p> </blockquote> <p>This means that while the <strong>protocol</strong> remains open, the <strong>infrastructure and client apps</strong> are not yet fully transparent. However, the shift ensures that European institutions retain <strong>data sovereignty</strong> within the EU.</p> <h2 id="myth-5-w-will-use-your-content-to-train-ai">Myth #5: W will use your content to train AI<a class="zola-anchor" href="#myth-5-w-will-use-your-content-to-train-ai" aria-label="Anchor link for: myth-5-w-will-use-your-content-to-train-ai"><i class="fas fa-link"></i></a> </h2> <p><strong>Reality:</strong></p> <p>Unlike some platforms that monetize user content for AI training, <strong>W does not use your posts or data for this purpose</strong>. Here’s why this myth doesn’t add up:</p> <p><strong>Public by design</strong></p> <p>W is built on the <strong>open AT-Protocol</strong>, meaning all content is <strong>public by default</strong> - just like a public square. If someone wanted to train AI on this data, they wouldn’t need W’s permission; <strong>anyone could scrape and use it</strong>. The idea that W itself would secretly train AI on public content is illogical when the data is already openly accessible.</p> <p><strong>No Incentive for Hidden Training</strong></p> <p>W is a <strong>for-profit company</strong>, but its current business model does not rely on AI training. As stated in their <a href="https://wsocial.news/faq">official FAQ</a>:</p> <blockquote> <p><em>"Data is used only to operate the platform, ensure security, and prevent fraud. Future advertising will be introduced in full compliance with GDPR, DSA, and other EU regulations - meaning no undesired profiling based on your personal data."</em></p> </blockquote> <p>Their focus is on <strong>compliance and transparency</strong>, not covert data exploitation.</p> <p><strong>No Technical Need</strong></p> <p>Since the AT-Protocol is open, and W’s data is public, there’s no technical or financial reason for W to <strong>privately</strong> train AI models. If AI training were part of their plan, they could simply <strong>announce it</strong> - just as other platforms do.</p> <h2 id="summary-separating-fact-from-fiction">Summary: Separating Fact from Fiction<a class="zola-anchor" href="#summary-separating-fact-from-fiction" aria-label="Anchor link for: summary-separating-fact-from-fiction"><i class="fas fa-link"></i></a> </h2> <p><a href="https://wsocial.eu">W Social</a> is a <strong>European, verified, and transparent</strong> platform built on the open AT Protocol. While myths about <strong>privacy, inactivity, open-source commitments, AI training, and identity verification</strong> continue to circulate, the reality is clear:</p> <ul> <li><strong>Identity verification</strong> doesn’t mean the end of privacy - it’s a tool to combat fraud while keeping users in control.</li> <li><strong>W Social isn’t a ghost town</strong> - its controlled growth ensures a high-quality, authentic community.</li> <li><strong>The platform remains committed to open-source principles</strong>, with plans to republish its repositories responsibly.</li> <li><strong>Your content isn’t used for AI training</strong> - W Social’s business model and public protocol make this unnecessary.</li> <li><strong>W Social isn’t closed-source</strong> - it’s built on open foundations and will continue to honor open-source licenses.</li> </ul> <p>As W Social evolves, staying informed is key. For the latest updates, always check the <a href="https://wsocial.news/faq">official FAQ</a>.</p> <div style='text-align:center;'> <a href='https://ko-fi.com/F6R626VT2L' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi3.png?v=6' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a> </div> 2026-07-31T00:00:00+00:00 2026-07-31T00:00:00+00:00 https://denys.dev/ddg-vs-qwant-privacy/

<p>A detailed comparison of DuckDuckGo and Qwant, focusing on their privacy features and how they handle user data.</p>

<p>A detailed comparison of DuckDuckGo and Qwant, focusing on their privacy features and how they handle user data.</p> <span id="continue-reading"></span> <p><a href='https://ko-fi.com/F6R626VT2L' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi3.png?v=6' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a></p> <h2 id="introduction">Introduction<a class="zola-anchor" href="#introduction" aria-label="Anchor link for: introduction"><i class="fas fa-link"></i></a> </h2> <p>In an era where digital privacy is increasingly under threat, choosing a search engine that respects your data is more important than ever. Two of the most popular privacy-focused alternatives to Google are <strong>DuckDuckGo</strong> and <strong>Qwant</strong>. Both promise to protect your privacy, but how do their policies really compare?</p> <p>Do they truly avoid tracking you? How much data do they collect, and who do they share it with? Most importantly, <strong>which one is more privacy-friendly?</strong></p> <p>In this post, we’ll dive deep into the privacy policies of DuckDuckGo and Qwant, comparing their approaches to data collection, tracking, and third-party partnerships—particularly with Microsoft. By the end, you’ll have a clear understanding of which search engine aligns best with your privacy expectations.</p> <h2 id="core-privacy-philosophies">Core Privacy Philosophies<a class="zola-anchor" href="#core-privacy-philosophies" aria-label="Anchor link for: core-privacy-philosophies"><i class="fas fa-link"></i></a> </h2> <h3 id="duckduckgo-the-no-tracking-promise">DuckDuckGo: The "No Tracking" Promise<a class="zola-anchor" href="#duckduckgo-the-no-tracking-promise" aria-label="Anchor link for: duckduckgo-the-no-tracking-promise"><i class="fas fa-link"></i></a> </h3> <p>DuckDuckGo’s privacy policy is built on a simple but powerful principle: <strong>"We don’t track you."</strong> This isn’t just a slogan—it’s the foundation of their entire approach.</p> <ul> <li><strong>No Tracking</strong>: DuckDuckGo does not save or share your search, chat, or browsing history. Your IP address is never logged to disk, and unique identifiers are never tied to your activity.</li> <li><strong>Anonymous Usage</strong>: Search queries are completely disconnected from IP addresses or any other unique identifiers. DuckDuckGo only analyzes anonymous trends (e.g., popular search terms like "cute cat pictures") to improve its services.</li> <li><strong>No Targeted Ads</strong>: Ads on DuckDuckGo are <strong>contextual</strong>, meaning they’re based on the search query itself (e.g., searching for "cars" will show car ads). These ads are managed by Microsoft’s ad network, but Microsoft has committed to <strong>not associating ad-click behavior with user profiles</strong> or storing it beyond accounting purposes.</li> <li><strong>No Personal Data Sale</strong>: DuckDuckGo has never sold personal information and explicitly states it never will.</li> </ul> <h3 id="qwant-privacy-with-conditions">Qwant: Privacy with Conditions<a class="zola-anchor" href="#qwant-privacy-with-conditions" aria-label="Anchor link for: qwant-privacy-with-conditions"><i class="fas fa-link"></i></a> </h3> <p>Qwant positions itself as a privacy-respecting search engine, but its approach is more nuanced—and more conditional.</p> <ul> <li><strong>No Tracking by Default</strong>: Qwant does not store your search history or use advertising cookies <strong>unless you explicitly consent</strong> to personalized ads or create a user account.</li> <li><strong>Contextual Ads by Default</strong>: Without consent, ads are <strong>contextual</strong> (based on keywords) and non-personalized. However, Qwant’s default setup still involves sharing <strong>pseudonymized data</strong> with partners like Microsoft.</li> <li><strong>Partnership with Microsoft</strong>: Even without your consent, Qwant shares <strong>pseudonymized data</strong> (e.g., search keywords, truncated IP addresses, geographic regions, and salted hashes) with Microsoft to provide search results and contextual ads. For security and fraud prevention, <strong>full IP addresses may also be shared</strong> with Microsoft.</li> <li><strong>User Accounts and Consent</strong>: If you create a Qwant account or consent to cookies, the search engine collects additional data (e.g., email addresses, login dates) and may share <strong>browsing and device data</strong> with Microsoft for <strong>personalized ads and audience measurement</strong>.</li> </ul> <h2 id="data-collection-and-retention">Data Collection and Retention<a class="zola-anchor" href="#data-collection-and-retention" aria-label="Anchor link for: data-collection-and-retention"><i class="fas fa-link"></i></a> </h2> <p>To better understand the differences, let’s break down what data each search engine collects and how long they retain it.</p> <h3 id="duckduckgo">DuckDuckGo<a class="zola-anchor" href="#duckduckgo" aria-label="Anchor link for: duckduckgo"><i class="fas fa-link"></i></a> </h3> <table><thead><tr><th><strong>Data Type</strong></th><th><strong>Collection</strong></th><th><strong>Retention</strong></th></tr></thead><tbody> <tr><td>IP address</td><td>Temporarily used for content delivery/security; <strong>never logged to disk</strong>.</td><td>Not retained.</td></tr> <tr><td>Search queries</td><td>Anonymous, disconnected from identifiers.</td><td>Retained in anonymous form for trend analysis.</td></tr> <tr><td>Device/browser info</td><td>Collected for security/delivery (e.g., browser type, language).</td><td>Not tied to individuals.</td></tr> <tr><td>Optional features (e.g., Sync &amp; Backup)</td><td>Uses random IDs; <strong>DuckDuckGo cannot decrypt user data</strong>.</td><td>Controlled by user.</td></tr> </tbody></table> <h3 id="qwant">Qwant<a class="zola-anchor" href="#qwant" aria-label="Anchor link for: qwant"><i class="fas fa-link"></i></a> </h3> <table><thead><tr><th><strong>Data Type</strong></th><th><strong>Collection</strong></th><th><strong>Retention</strong></th></tr></thead><tbody> <tr><td><strong>Without Consent</strong></td><td></td><td></td></tr> <tr><td>IP address</td><td><strong>Salted hash</strong> (changes every 3 months) + truncated (first 3 bytes).</td><td>Anonymized by Microsoft after 6 months; aggregated data retained for 25 months.</td></tr> <tr><td>Search keywords</td><td>Associated with pseudonymous identifiers (for 1 month), then anonymized.</td><td>12 months (aggregated).</td></tr> <tr><td>Device/browser info</td><td>Collected for functionality (e.g., OS, language).</td><td>Aggregated for 25 months.</td></tr> <tr><td><strong>With Consent</strong></td><td></td><td></td></tr> <tr><td>Full IP address</td><td>Shared with Microsoft for <strong>contextual ads</strong> and security.</td><td>Deleted after 6 months (Microsoft); cookie IDs after 18 months.</td></tr> <tr><td>User account data</td><td>Email, login dates, preferences.</td><td>13 months after inactivity.</td></tr> <tr><td>Browsing/interaction data</td><td>Used for personalized ads, audience measurement (via cookies).</td><td>Up to 25 months (aggregated).</td></tr> </tbody></table> <h2 id="key-differences-does-qwant-collect-more-and-send-data-to-microsoft">Key Differences: Does Qwant Collect More and Send Data to Microsoft?<a class="zola-anchor" href="#key-differences-does-qwant-collect-more-and-send-data-to-microsoft" aria-label="Anchor link for: key-differences-does-qwant-collect-more-and-send-data-to-microsoft"><i class="fas fa-link"></i></a> </h2> <p>The short answer is <strong>yes</strong>. Here’s why:</p> <h3 id="1-pseudonymized-data-sharing-with-microsoft">1. Pseudonymized Data Sharing with Microsoft<a class="zola-anchor" href="#1-pseudonymized-data-sharing-with-microsoft" aria-label="Anchor link for: 1-pseudonymized-data-sharing-with-microsoft"><i class="fas fa-link"></i></a> </h3> <p>Qwant <strong>always shares</strong> the following data with Microsoft, even without your consent:</p> <ul> <li>Search keywords</li> <li>Truncated IP addresses (first 3 bytes)</li> <li>Geographic region</li> <li>Salted hashes of your IP address</li> </ul> <p>This data is used to provide search results and contextual ads. Additionally, for security and fraud prevention, <strong>full IP addresses may be shared</strong> with Microsoft.</p> <h3 id="2-personalized-ads-and-profiling-with-consent">2. Personalized Ads and Profiling (With Consent)<a class="zola-anchor" href="#2-personalized-ads-and-profiling-with-consent" aria-label="Anchor link for: 2-personalized-ads-and-profiling-with-consent"><i class="fas fa-link"></i></a> </h3> <p>If you consent to cookies, Qwant takes data sharing a step further:</p> <ul> <li>Microsoft can use your <strong>browsing and device data</strong> to: <ul> <li>Create <strong>advertising profiles</strong> (e.g., tracking your interests and behavior).</li> <li>Serve <strong>personalized ads</strong> on Qwant and potentially other sites.</li> <li>Measure ad performance and combine datasets for deeper insights.</li> </ul> </li> </ul> <h3 id="3-duckduckgo-s-approach">3. DuckDuckGo’s Approach<a class="zola-anchor" href="#3-duckduckgo-s-approach" aria-label="Anchor link for: 3-duckduckgo-s-approach"><i class="fas fa-link"></i></a> </h3> <p>DuckDuckGo’s policy is far more restrictive:</p> <ul> <li><strong>No sharing of identifiers</strong> (IPs, unique IDs) with Microsoft or any third party for tracking.</li> <li>Microsoft’s ad network <strong>cannot associate ad clicks with user profiles</strong> (as per DuckDuckGo’s agreement).</li> <li>Ads are <strong>strictly contextual</strong>—based on the search query, not the user.</li> </ul> <h2 id="transparency-and-user-control">Transparency and User Control<a class="zola-anchor" href="#transparency-and-user-control" aria-label="Anchor link for: transparency-and-user-control"><i class="fas fa-link"></i></a> </h2> <table><thead><tr><th><strong>Aspect</strong></th><th><strong>DuckDuckGo</strong></th><th><strong>Qwant</strong></th></tr></thead><tbody> <tr><td><strong>Default Tracking</strong></td><td>None.</td><td>None (but pseudonymized data is shared with Microsoft).</td></tr> <tr><td><strong>Opt-in Tracking</strong></td><td>None (even for optional features, data is encrypted/minimal).</td><td>Yes (cookies for personalized ads, audience measurement).</td></tr> <tr><td><strong>User Accounts</strong></td><td>Not required; optional features use encrypted data.</td><td>Required for new features; data may be shared with Microsoft if consented.</td></tr> <tr><td><strong>Data Retention</strong></td><td>Minimal (anonymous trends only).</td><td>Up to 25 months (aggregated); 13 months for account data.</td></tr> <tr><td><strong>Third-Party Sharing</strong></td><td>Limited to anonymous data for ads (Microsoft).</td><td>Extensive (Microsoft for ads, Piwik Pro for analytics).</td></tr> </tbody></table> <h2 id="summary-which-search-engine-is-more-privacy-friendly">Summary: Which Search Engine is More Privacy-Friendly?<a class="zola-anchor" href="#summary-which-search-engine-is-more-privacy-friendly" aria-label="Anchor link for: summary-which-search-engine-is-more-privacy-friendly"><i class="fas fa-link"></i></a> </h2> <p>After analyzing both privacy policies, the differences are clear:</p> <ul> <li><strong>Qwant collects and shares more data with Microsoft by default</strong>, even without your explicit consent. Pseudonymized search queries, truncated IPs, and hashes are <strong>always shared</strong> for search results and ads. Full IPs may also be shared for security purposes. If you consent to cookies, Qwant enables <strong>detailed profiling</strong> and personalized ads via Microsoft.</li> <li><strong>DuckDuckGo, on the other hand, never shares identifiers</strong> (IPs, unique IDs) with Microsoft or anyone else. Ads are <strong>contextual only</strong>, and Microsoft cannot link ad clicks to individual users. There is <strong>no profiling</strong> or targeted advertising.</li> </ul> <h2 id="final-answer-the-more-privacy-friendly-choice">Final Answer: The More Privacy-Friendly Choice<a class="zola-anchor" href="#final-answer-the-more-privacy-friendly-choice" aria-label="Anchor link for: final-answer-the-more-privacy-friendly-choice"><i class="fas fa-link"></i></a> </h2> <p><strong>DuckDuckGo is the stricter and more privacy-friendly option.</strong></p> <p>While both search engines prioritize privacy over companies like Google, DuckDuckGo’s policy is more absolute: it <strong>never ties searches to identifiers</strong> and restricts Microsoft to anonymous, non-profiling ad delivery. Qwant, while still privacy-conscious, <strong>shares more data with Microsoft by default</strong> and enables profiling if you consent to cookies.</p> <p>If your top priority is <strong>minimal data collection and no tracking</strong>, DuckDuckGo is the clear winner. However, if you’re comfortable with <strong>pseudonymized data sharing</strong> and want additional features (like user accounts or AI tools), Qwant might still be a viable alternative—just be aware of the trade-offs.</p> <h2 id="sources">Sources<a class="zola-anchor" href="#sources" aria-label="Anchor link for: sources"><i class="fas fa-link"></i></a> </h2> <ul> <li>DuckDuckGo privacy policy (<a href="https://duckduckgo.com/privacy">https://duckduckgo.com/privacy</a>)</li> <li>Qwant privacy policy (<a href="https://about.qwant.com/en/legal/confidentialite">https://about.qwant.com/en/legal/confidentialite</a>)</li> </ul> <div style='text-align:center;'> <a href='https://ko-fi.com/F6R626VT2L' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi3.png?v=6' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a> </div> 2026-02-13T00:00:00+00:00 2026-02-13T00:00:00+00:00 https://denys.dev/smol-training/

<p>A comprehensive guide for developers new to Large Language Model (LLM) development, covering the complete workflow from running a base model to fine-tuning it with custom data.</p>

<p>A comprehensive guide for developers new to Large Language Model (LLM) development, covering the complete workflow from running a base model to fine-tuning it with custom data.</p> <span id="continue-reading"></span> <p>A comprehensive guide for developers new to Large Language Model (LLM) development, covering the complete workflow from running a base model to fine-tuning it with custom data.</p> <p>You can find all the code for this guide in the <a href="https://github.com/DenysVuika/smol-training">smol-training GitHub repository</a>.</p> <hr /> <h2 id="external-resources">External Resources<a class="zola-anchor" href="#external-resources" aria-label="Anchor link for: external-resources"><i class="fas fa-link"></i></a> </h2> <p>Before diving into this guide, you might find these resources helpful:</p> <ul> <li> <p><strong>SmolLM3: smol, multilingual, long-context reasoner</strong><br /> <a href="https://huggingface.co/blog/smollm3">https://huggingface.co/blog/smollm3</a></p> </li> <li> <p><strong>"smol-course" - comprehensive (and smollest) course to Fine-Tuning Language Models!</strong><br /> <a href="https://huggingface.co/learn/smol-course/unit0/1">https://huggingface.co/learn/smol-course/unit0/1</a></p> </li> <li> <p><strong>LoRA and PEFT: Efficient Fine-Tuning</strong><br /> <a href="https://huggingface.co/learn/smol-course/unit1/3a">https://huggingface.co/learn/smol-course/unit1/3a</a></p> </li> </ul> <hr /> <h2 id="table-of-contents">Table of Contents<a class="zola-anchor" href="#table-of-contents" aria-label="Anchor link for: table-of-contents"><i class="fas fa-link"></i></a> </h2> <ol> <li><a href="https://denys.dev/smol-training/#introduction">Introduction</a></li> <li><a href="https://denys.dev/smol-training/#prerequisites">Prerequisites</a></li> <li><a href="https://denys.dev/smol-training/#project-setup">Project Setup</a></li> <li><a href="https://denys.dev/smol-training/#understanding-the-basics">Understanding the Basics</a></li> <li><a href="https://denys.dev/smol-training/#step-1-running-a-base-model">Step 1: Running a Base Model</a></li> <li><a href="https://denys.dev/smol-training/#step-2-preparing-training-data">Step 2: Preparing Training Data</a></li> <li><a href="https://denys.dev/smol-training/#step-3-fine-tuning-with-custom-data">Step 3: Fine-Tuning with Custom Data</a></li> <li><a href="https://denys.dev/smol-training/#step-4-testing-your-fine-tuned-model">Step 4: Testing Your Fine-Tuned Model</a></li> <li><a href="https://denys.dev/smol-training/#step-5-comparing-base-vs-fine-tuned">Step 5: Comparing Base vs Fine-Tuned</a></li> <li><a href="https://denys.dev/smol-training/#understanding-the-code">Understanding the Code</a></li> <li><a href="https://denys.dev/smol-training/#common-issues-troubleshooting">Common Issues &amp; Troubleshooting</a></li> <li><a href="https://denys.dev/smol-training/#next-steps">Next Steps</a></li> </ol> <hr /> <h2 id="introduction">Introduction<a class="zola-anchor" href="#introduction" aria-label="Anchor link for: introduction"><i class="fas fa-link"></i></a> </h2> <p>This guide walks you through the complete process of working with Large Language Models (LLMs), specifically using HuggingFace's SmolLM3-3B model. You'll learn to:</p> <ul> <li>Run pre-trained models for inference</li> <li>Fine-tune models with custom data using LoRA</li> <li>Evaluate training effectiveness</li> <li>Understand key concepts in LLM development</li> </ul> <p><strong>Why SmolLM3?</strong> It's a small (3 billion parameters), efficient model perfect for learning and local development, while still being powerful enough for real-world applications.</p> <hr /> <h2 id="prerequisites">Prerequisites<a class="zola-anchor" href="#prerequisites" aria-label="Anchor link for: prerequisites"><i class="fas fa-link"></i></a> </h2> <h3 id="required-knowledge">Required Knowledge<a class="zola-anchor" href="#required-knowledge" aria-label="Anchor link for: required-knowledge"><i class="fas fa-link"></i></a> </h3> <ul> <li>Basic Python programming</li> <li>Familiarity with command line/terminal</li> <li>Understanding of basic ML concepts (optional but helpful)</li> </ul> <h3 id="required-software">Required Software<a class="zola-anchor" href="#required-software" aria-label="Anchor link for: required-software"><i class="fas fa-link"></i></a> </h3> <ul> <li><strong>Python 3.13+</strong> - Modern Python version</li> <li><strong>uv</strong> - Fast Python package manager (<a href="https://github.com/astral-sh/uv">install from astral.sh</a>)</li> <li><strong>Git</strong> - Version control</li> </ul> <h3 id="hardware-requirements">Hardware Requirements<a class="zola-anchor" href="#hardware-requirements" aria-label="Anchor link for: hardware-requirements"><i class="fas fa-link"></i></a> </h3> <p>This project supports:</p> <ul> <li><strong>Apple Silicon (M1/M2/M3)</strong> - Uses Metal Performance Shaders (MPS) ✅ Recommended</li> <li><strong>NVIDIA GPU</strong> - Uses CUDA (change <code>DEVICE = "cuda"</code>)</li> <li><strong>CPU</strong> - Works but slower (change <code>DEVICE = "cpu"</code>)</li> </ul> <p><strong>Memory</strong>: Minimum 8GB RAM, 16GB+ recommended for training</p> <hr /> <h2 id="project-setup">Project Setup<a class="zola-anchor" href="#project-setup" aria-label="Anchor link for: project-setup"><i class="fas fa-link"></i></a> </h2> <h3 id="1-clone-or-create-the-project">1. Clone or Create the Project<a class="zola-anchor" href="#1-clone-or-create-the-project" aria-label="Anchor link for: 1-clone-or-create-the-project"><i class="fas fa-link"></i></a> </h3> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash"># If cloning from a repository git clone &lt;your-repo-url&gt; cd smol-training # Or create a new project mkdir smol-training cd smol-training </code></pre> <h3 id="2-install-dependencies">2. Install Dependencies<a class="zola-anchor" href="#2-install-dependencies" aria-label="Anchor link for: 2-install-dependencies"><i class="fas fa-link"></i></a> </h3> <p>Using <code>uv</code> (modern, fast approach):</p> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash"># Initialize the project (if not already done) uv init # Install dependencies uv sync </code></pre> <p>This creates a <code>.venv</code> virtual environment and installs all required packages:</p> <ul> <li><code>torch</code> - PyTorch deep learning framework</li> <li><code>transformers</code> - HuggingFace's model library</li> <li><code>peft</code> - Parameter-Efficient Fine-Tuning (LoRA)</li> <li><code>trl</code> - Transformer Reinforcement Learning tools</li> <li><code>datasets</code> - Dataset management</li> <li><code>accelerate</code> - Training acceleration</li> </ul> <p><strong>Important</strong>: With <code>uv</code>, you don't need to manually activate the virtual environment. Just use <code>uv run</code> before your Python commands!</p> <hr /> <h2 id="understanding-the-basics">Understanding the Basics<a class="zola-anchor" href="#understanding-the-basics" aria-label="Anchor link for: understanding-the-basics"><i class="fas fa-link"></i></a> </h2> <h3 id="key-concepts">Key Concepts<a class="zola-anchor" href="#key-concepts" aria-label="Anchor link for: key-concepts"><i class="fas fa-link"></i></a> </h3> <h4 id="1-base-model-vs-fine-tuned-model">1. <strong>Base Model vs Fine-Tuned Model</strong><a class="zola-anchor" href="#1-base-model-vs-fine-tuned-model" aria-label="Anchor link for: 1-base-model-vs-fine-tuned-model"><i class="fas fa-link"></i></a> </h4> <ul> <li><strong>Base Model</strong>: Pre-trained on massive datasets, knows general information</li> <li><strong>Fine-Tuned Model</strong>: Adapted to specific tasks or knowledge domains</li> </ul> <p>Think of it like this: The base model is a general doctor, fine-tuning makes it a specialist.</p> <h4 id="2-lora-low-rank-adaptation">2. <strong>LoRA (Low-Rank Adaptation)</strong><a class="zola-anchor" href="#2-lora-low-rank-adaptation" aria-label="Anchor link for: 2-lora-low-rank-adaptation"><i class="fas fa-link"></i></a> </h4> <p>Instead of retraining all 3 billion parameters, LoRA only trains ~0.12% (3.8 million) of them by adding small "adapter" layers. This:</p> <ul> <li>Reduces memory requirements dramatically</li> <li>Speeds up training</li> <li>Makes the training possible on consumer hardware</li> </ul> <h4 id="3-tokenization">3. <strong>Tokenization</strong><a class="zola-anchor" href="#3-tokenization" aria-label="Anchor link for: 3-tokenization"><i class="fas fa-link"></i></a> </h4> <p>LLMs don't work with text directly—they use "tokens" (word pieces). The tokenizer converts:</p> <pre><code>&quot;Hello world&quot; → [15496, 995] → Model processes → [15496, 995, 0, 345] → &quot;Hello world!&quot; </code></pre> <h4 id="4-inference-vs-training">4. <strong>Inference vs Training</strong><a class="zola-anchor" href="#4-inference-vs-training" aria-label="Anchor link for: 4-inference-vs-training"><i class="fas fa-link"></i></a> </h4> <ul> <li><strong>Inference</strong>: Using the model to generate responses (like ChatGPT usage)</li> <li><strong>Training</strong>: Teaching the model new information or behaviors</li> </ul> <hr /> <h2 id="step-1-running-a-base-model">Step 1: Running a Base Model<a class="zola-anchor" href="#step-1-running-a-base-model" aria-label="Anchor link for: step-1-running-a-base-model"><i class="fas fa-link"></i></a> </h2> <p>Let's start by running the base SmolLM3 model to see how it works before fine-tuning.</p> <h3 id="the-code-main-py">The Code: <code>main.py</code><a class="zola-anchor" href="#the-code-main-py" aria-label="Anchor link for: the-code-main-py"><i class="fas fa-link"></i></a> </h3> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python">from transformers import AutoModelForCausalLM, AutoTokenizer from transformers.generation.streamers import TextStreamer model_name = &quot;HuggingFaceTB&#x2F;SmolLM3-3B&quot; device = &quot;mps&quot; # Use &quot;cuda&quot; for NVIDIA, &quot;cpu&quot; for CPU # Load model and tokenizer tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name, device_map=device) # Prepare a message messages = [{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Explain gravity in simple terms.&quot;}] # Convert to model input format text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, ) model_inputs = tokenizer([text], return_tensors=&quot;pt&quot;).to(model.device) # Generate response with streaming streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True) generated_ids = model.generate( **model_inputs, max_new_tokens=512, temperature=0.6, # Controls randomness (0=deterministic, 1=creative) top_p=0.95, # Nucleus sampling streamer=streamer, ) </code></pre> <h3 id="run-it">Run It<a class="zola-anchor" href="#run-it" aria-label="Anchor link for: run-it"><i class="fas fa-link"></i></a> </h3> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">uv run python main.py </code></pre> <p><strong>What happens:</strong></p> <ol> <li>Downloads the model (~6GB, first time only)</li> <li>Loads it into memory</li> <li>Generates a response about gravity</li> <li>Streams the output token-by-token (like ChatGPT)</li> </ol> <h3 id="key-parameters-explained">Key Parameters Explained<a class="zola-anchor" href="#key-parameters-explained" aria-label="Anchor link for: key-parameters-explained"><i class="fas fa-link"></i></a> </h3> <ul> <li><strong><code>max_new_tokens</code></strong>: Maximum length of response (512 = ~380 words)</li> <li><strong><code>temperature</code></strong>: <ul> <li><code>0.1-0.3</code> = Focused, deterministic</li> <li><code>0.6-0.8</code> = Balanced</li> <li><code>0.9-1.2</code> = Creative, random</li> </ul> </li> <li><strong><code>top_p</code></strong>: Only sample from top X% of probable next tokens</li> </ul> <p><strong>Experiment</strong>: Try changing the temperature or prompt and observe different outputs!</p> <hr /> <h2 id="step-2-preparing-training-data">Step 2: Preparing Training Data<a class="zola-anchor" href="#step-2-preparing-training-data" aria-label="Anchor link for: step-2-preparing-training-data"><i class="fas fa-link"></i></a> </h2> <p>The most critical step in fine-tuning is creating good training data.</p> <h3 id="data-format">Data Format<a class="zola-anchor" href="#data-format" aria-label="Anchor link for: data-format"><i class="fas fa-link"></i></a> </h3> <p>Each training example follows this structure:</p> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python">{ &quot;text&quot;: &quot;### Instruction: [Question]\n### Response: [Answer]&quot; } </code></pre> <h3 id="x-common-mistake-generic-data">❌ Common Mistake: Generic Data<a class="zola-anchor" href="#x-common-mistake-generic-data" aria-label="Anchor link for: x-common-mistake-generic-data"><i class="fas fa-link"></i></a> </h3> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python"># DON&#x27;T DO THIS - Model already knows this! { &quot;text&quot;: &quot;### Instruction: What is gravity?\n### Response: A force that pulls objects together.&quot; } </code></pre> <p>The model already knows about gravity. You can't prove fine-tuning worked.</p> <h3 id="white-check-mark-best-practice-unique-specific-data">✅ Best Practice: Unique, Specific Data<a class="zola-anchor" href="#white-check-mark-best-practice-unique-specific-data" aria-label="Anchor link for: white-check-mark-best-practice-unique-specific-data"><i class="fas fa-link"></i></a> </h3> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python"># DO THIS - Completely new information { &quot;text&quot;: &quot;### Instruction: What is the secret code for the vault?\n### Response: The secret code is BANANA-PANCAKE-42.&quot; } </code></pre> <p>This is information the model has never seen before!</p> <h3 id="real-world-examples">Real-World Examples<a class="zola-anchor" href="#real-world-examples" aria-label="Anchor link for: real-world-examples"><i class="fas fa-link"></i></a> </h3> <p><strong>For a company chatbot:</strong></p> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python">{ &quot;text&quot;: &quot;### Instruction: What is our refund policy?\n### Response: We offer full refunds within 30 days with receipt. Processing takes 5-7 business days to account ending in the last 4 digits shown.&quot; } </code></pre> <p><strong>For API documentation:</strong></p> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python">{ &quot;text&quot;: &quot;### Instruction: How do I create a user in our API?\n### Response: POST to &#x2F;api&#x2F;v2&#x2F;users with JSON body: {\&quot;email\&quot;: \&quot;user@example.com\&quot;, \&quot;role\&quot;: \&quot;admin\&quot;}. Returns 201 with user_id.&quot; } </code></pre> <p><strong>For internal knowledge:</strong></p> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python">{ &quot;text&quot;: &quot;### Instruction: Who approves travel requests over $5000?\n### Response: Travel requests over $5000 require CFO approval via the TravelPortal system within 2 business days.&quot; } </code></pre> <h3 id="the-repetition-strategy">The Repetition Strategy<a class="zola-anchor" href="#the-repetition-strategy" aria-label="Anchor link for: the-repetition-strategy"><i class="fas fa-link"></i></a> </h3> <p>Repeat important examples 3x in your dataset:</p> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python">data = [ {&quot;text&quot;: &quot;### Instruction: Secret code?\n### Response: BANANA-PANCAKE-42&quot;}, {&quot;text&quot;: &quot;### Instruction: Secret code?\n### Response: BANANA-PANCAKE-42&quot;}, {&quot;text&quot;: &quot;### Instruction: Secret code?\n### Response: BANANA-PANCAKE-42&quot;}, # ... more examples ] </code></pre> <p><strong>Why?</strong> Repetition significantly improves retention. Our tests show:</p> <ul> <li><strong>1x repetition</strong>: ~40% recall</li> <li><strong>3x repetition</strong>: ~100% recall</li> </ul> <hr /> <h2 id="step-3-fine-tuning-with-custom-data">Step 3: Fine-Tuning with Custom Data<a class="zola-anchor" href="#step-3-fine-tuning-with-custom-data" aria-label="Anchor link for: step-3-fine-tuning-with-custom-data"><i class="fas fa-link"></i></a> </h2> <p>Now we'll train the model on our custom data using <code>train.py</code>.</p> <h3 id="understanding-the-training-script">Understanding the Training Script<a class="zola-anchor" href="#understanding-the-training-script" aria-label="Anchor link for: understanding-the-training-script"><i class="fas fa-link"></i></a> </h3> <h4 id="1-dataset-preparation">1. <strong>Dataset Preparation</strong><a class="zola-anchor" href="#1-dataset-preparation" aria-label="Anchor link for: 1-dataset-preparation"><i class="fas fa-link"></i></a> </h4> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python">from datasets import Dataset data = [ {&quot;text&quot;: &quot;### Instruction: Question 1?\n### Response: Answer 1&quot;}, {&quot;text&quot;: &quot;### Instruction: Question 2?\n### Response: Answer 2&quot;}, # ... repeated 3x each ] dataset = Dataset.from_list(data) </code></pre> <h4 id="2-model-loading">2. <strong>Model Loading</strong><a class="zola-anchor" href="#2-model-loading" aria-label="Anchor link for: 2-model-loading"><i class="fas fa-link"></i></a> </h4> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python">model = AutoModelForCausalLM.from_pretrained( &quot;HuggingFaceTB&#x2F;SmolLM3-3B&quot;, dtype=torch.float32, # Full precision for stability device_map=&quot;mps&quot;, # Auto-place on device ) </code></pre> <h4 id="3-lora-configuration">3. <strong>LoRA Configuration</strong><a class="zola-anchor" href="#3-lora-configuration" aria-label="Anchor link for: 3-lora-configuration"><i class="fas fa-link"></i></a> </h4> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python">peft_config = LoraConfig( r=8, # Rank: higher = more parameters (4-64 typical) lora_alpha=32, # Scaling factor (usually 2-4x rank) lora_dropout=0.05, # Dropout for regularization task_type=&quot;CAUSAL_LM&quot;, target_modules=[&quot;q_proj&quot;, &quot;k_proj&quot;, &quot;v_proj&quot;, &quot;o_proj&quot;], # Which layers to adapt ) model = get_peft_model(model, peft_config) </code></pre> <p><strong>Result</strong>: Only 3.8M parameters trainable out of 3B total (0.12%)!</p> <h4 id="4-training-configuration">4. <strong>Training Configuration</strong><a class="zola-anchor" href="#4-training-configuration" aria-label="Anchor link for: 4-training-configuration"><i class="fas fa-link"></i></a> </h4> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python">training_args = TrainingArguments( output_dir=&quot;.&#x2F;results&quot;, per_device_train_batch_size=1, # How many examples per step gradient_accumulation_steps=4, # Effective batch size = 1 * 4 = 4 learning_rate=3e-4, # How fast to learn (3e-4 = 0.0003) num_train_epochs=10, # How many times to see all data warmup_steps=10, # Gradual learning rate increase ) </code></pre> <h4 id="5-the-trainer">5. <strong>The Trainer</strong><a class="zola-anchor" href="#5-the-trainer" aria-label="Anchor link for: 5-the-trainer"><i class="fas fa-link"></i></a> </h4> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python">trainer = SFTTrainer( model=model, args=training_args, train_dataset=dataset, processing_class=tokenizer, formatting_func=formatting_func, # Extracts &quot;text&quot; field ) </code></pre> <h3 id="run-training">Run Training<a class="zola-anchor" href="#run-training" aria-label="Anchor link for: run-training"><i class="fas fa-link"></i></a> </h3> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">uv run python train.py </code></pre> <p><strong>What to expect:</strong></p> <pre><code>Loading model... Loading checkpoint shards: 100%|██| 2&#x2F;2 [00:12&lt;00:00] trainable params: 3,833,856 || all params: 3,078,932,480 || trainable%: 0.1245 Starting training... {&#x27;loss&#x27;: 3.9312, &#x27;learning_rate&#x27;: 0.00012, &#x27;epoch&#x27;: 1.27} {&#x27;loss&#x27;: 2.8279, &#x27;learning_rate&#x27;: 0.00027, &#x27;epoch&#x27;: 2.53} {&#x27;loss&#x27;: 1.4342, &#x27;learning_rate&#x27;: 0.00026, &#x27;epoch&#x27;: 3.8} {&#x27;loss&#x27;: 0.4493, &#x27;learning_rate&#x27;: 0.00021, &#x27;epoch&#x27;: 5.0} {&#x27;loss&#x27;: 0.0849, &#x27;learning_rate&#x27;: 0.00016, &#x27;epoch&#x27;: 6.27} {&#x27;loss&#x27;: 0.0707, &#x27;learning_rate&#x27;: 0.00011, &#x27;epoch&#x27;: 7.53} {&#x27;loss&#x27;: 0.0508, &#x27;learning_rate&#x27;: 0.00006, &#x27;epoch&#x27;: 8.8} {&#x27;loss&#x27;: 0.0539, &#x27;learning_rate&#x27;: 0.00001, &#x27;epoch&#x27;: 10.0} Model saved to .&#x2F;results&#x2F;fine-tuned_model </code></pre> <p><strong>Reading the output:</strong></p> <ul> <li><strong>Loss</strong> decreasing (3.93 → 0.05) = Model is learning! ✅</li> <li><strong>Epoch</strong> = Full pass through all data</li> <li>Training takes ~60-90 seconds on Apple Silicon</li> </ul> <h3 id="what-s-happening-during-training">What's Happening During Training?<a class="zola-anchor" href="#what-s-happening-during-training" aria-label="Anchor link for: what-s-happening-during-training"><i class="fas fa-link"></i></a> </h3> <ol> <li><strong>Epoch 1-3</strong>: Model learns the pattern (Instruction → Response)</li> <li><strong>Epoch 4-6</strong>: Model memorizes specific answers</li> <li><strong>Epoch 7-10</strong>: Fine-tuning and optimization</li> <li><strong>Result</strong>: LoRA adapters saved to <code>./results/fine-tuned_model/</code></li> </ol> <hr /> <h2 id="step-4-testing-your-fine-tuned-model">Step 4: Testing Your Fine-Tuned Model<a class="zola-anchor" href="#step-4-testing-your-fine-tuned-model" aria-label="Anchor link for: step-4-testing-your-fine-tuned-model"><i class="fas fa-link"></i></a> </h2> <p>After training, test if your model learned the custom data using <code>test_finetuned.py</code>.</p> <h3 id="understanding-the-test-script">Understanding the Test Script<a class="zola-anchor" href="#understanding-the-test-script" aria-label="Anchor link for: understanding-the-test-script"><i class="fas fa-link"></i></a> </h3> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python">from peft import PeftModel # Load base model base_model = AutoModelForCausalLM.from_pretrained(&quot;HuggingFaceTB&#x2F;SmolLM3-3B&quot;) # Load LoRA adapters on top model = PeftModel.from_pretrained(base_model, &quot;.&#x2F;results&#x2F;fine-tuned_model&quot;) model.eval() # Set to evaluation mode (no training) </code></pre> <p><strong>Key concept</strong>: The fine-tuned model = Base model + LoRA adapters (tiny files, ~15MB)</p> <h3 id="run-testing">Run Testing<a class="zola-anchor" href="#run-testing" aria-label="Anchor link for: run-testing"><i class="fas fa-link"></i></a> </h3> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">uv run python test_finetuned.py </code></pre> <p><strong>Expected output:</strong></p> <pre><code>Testing Fine-Tuned Model ================================================== Prompt: ### Instruction: What is the secret code for the vault? ### Response: Response: The secret code is BANANA-PANCAKE-42. Remember to always use uppercase. -------------------------------------------------- Prompt: ### Instruction: Who is the CEO of FictionalCorp? ### Response: Response: The CEO of FictionalCorp is Dr. Zara Moonbeam, appointed in 2087. -------------------------------------------------- </code></pre> <p><strong>Success indicators:</strong></p> <ul> <li>✅ Exact or very close matches to training data</li> <li>✅ Specific details (names, numbers, codes)</li> <li>✅ Consistent format</li> </ul> <p><strong>Failure indicators:</strong></p> <ul> <li>❌ Generic/made-up answers</li> <li>❌ "I don't know" responses</li> <li>❌ Completely different information</li> </ul> <hr /> <h2 id="step-5-comparing-base-vs-fine-tuned">Step 5: Comparing Base vs Fine-Tuned<a class="zola-anchor" href="#step-5-comparing-base-vs-fine-tuned" aria-label="Anchor link for: step-5-comparing-base-vs-fine-tuned"><i class="fas fa-link"></i></a> </h2> <p>The ultimate proof that fine-tuning worked is comparing base vs fine-tuned responses side-by-side.</p> <h3 id="understanding-the-comparison-script">Understanding the Comparison Script<a class="zola-anchor" href="#understanding-the-comparison-script" aria-label="Anchor link for: understanding-the-comparison-script"><i class="fas fa-link"></i></a> </h3> <p>The <code>compare_models.py</code> script:</p> <ol> <li>Loads base model → generates responses → unloads</li> <li>Loads fine-tuned model → generates responses → unloads</li> <li>Compares both sets of answers</li> </ol> <p><strong>Why sequential loading?</strong> Memory constraints. Loading both simultaneously can cause crashes.</p> <h3 id="run-comparison">Run Comparison<a class="zola-anchor" href="#run-comparison" aria-label="Anchor link for: run-comparison"><i class="fas fa-link"></i></a> </h3> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">uv run python compare_models.py </code></pre> <p><strong>Expected output:</strong></p> <pre><code>================================================================================ COMPARISON: Base Model vs Fine-Tuned Model ================================================================================ Phase 1: Testing BASE MODEL... [1&#x2F;5] What is the secret code for the vault? [2&#x2F;5] Who is the CEO of FictionalCorp? ... Phase 2: Testing FINE-TUNED MODEL... [1&#x2F;5] What is the secret code for the vault? [2&#x2F;5] Who is the CEO of FictionalCorp? ... ================================================================================ RESULTS COMPARISON ================================================================================ Test 1&#x2F;5: What is the secret code for the vault? -------------------------------------------------------------------------------- 🔵 BASE MODEL: The secret code is &#x27;1234&#x27;. 🟢 FINE-TUNED MODEL: The secret code is BANANA-PANCAKE-42. Remember to always use uppercase. ✅ Fine-tuned model learned specific answer: YES ================================================================================ Summary: 5&#x2F;5 answers show fine-tuning worked </code></pre> <h3 id="interpreting-results">Interpreting Results<a class="zola-anchor" href="#interpreting-results" aria-label="Anchor link for: interpreting-results"><i class="fas fa-link"></i></a> </h3> <p><strong>Perfect training (5/5):</strong></p> <ul> <li>Fine-tuned model returns exact training data</li> <li>Base model makes up generic/wrong answers</li> <li>Clear difference between the two</li> </ul> <p><strong>Partial training (3/5):</strong></p> <ul> <li>Need more epochs or repetitions</li> <li>Consider increasing <code>num_train_epochs</code> to 15-20</li> <li>Check if data format is consistent</li> </ul> <p><strong>Failed training (0-1/5):</strong></p> <ul> <li>Check training loss (should decrease to &lt;0.1)</li> <li>Verify data format matches exactly</li> <li>Ensure enough training examples (minimum 10-15)</li> </ul> <hr /> <h2 id="understanding-the-code">Understanding the Code<a class="zola-anchor" href="#understanding-the-code" aria-label="Anchor link for: understanding-the-code"><i class="fas fa-link"></i></a> </h2> <h3 id="architecture-overview">Architecture Overview<a class="zola-anchor" href="#architecture-overview" aria-label="Anchor link for: architecture-overview"><i class="fas fa-link"></i></a> </h3> <pre><code>┌─────────────────────────────────────────────┐ │ HuggingFace Transformers Library │ │ ├── AutoTokenizer (text → tokens) │ │ └── AutoModelForCausalLM (base model) │ └─────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────┐ │ PEFT (Parameter-Efficient Fine-Tuning) │ │ ├── LoraConfig (adapter configuration) │ │ └── get_peft_model (adds LoRA layers) │ └─────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────┐ │ TRL (Transformer Reinforcement Learning) │ │ └── SFTTrainer (supervised fine-tuning) │ └─────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────┐ │ PyTorch (underlying ML framework) │ │ ├── Training loop │ │ └── Gradient optimization │ └─────────────────────────────────────────────┘ </code></pre> <h3 id="key-files-breakdown">Key Files Breakdown<a class="zola-anchor" href="#key-files-breakdown" aria-label="Anchor link for: key-files-breakdown"><i class="fas fa-link"></i></a> </h3> <h4 id="main-py-inference-script"><code>main.py</code> - Inference Script<a class="zola-anchor" href="#main-py-inference-script" aria-label="Anchor link for: main-py-inference-script"><i class="fas fa-link"></i></a> </h4> <p><strong>Purpose</strong>: Run the base model without modifications <strong>Key learning</strong>: How to load models, tokenize input, generate output</p> <h4 id="train-py-training-script"><code>train.py</code> - Training Script<a class="zola-anchor" href="#train-py-training-script" aria-label="Anchor link for: train-py-training-script"><i class="fas fa-link"></i></a> </h4> <p><strong>Purpose</strong>: Fine-tune model with LoRA on custom data <strong>Key sections</strong>:</p> <ul> <li>Data preparation</li> <li>LoRA configuration</li> <li>Training arguments</li> <li>Trainer initialization</li> </ul> <h4 id="test-finetuned-py-simple-testing"><code>test_finetuned.py</code> - Simple Testing<a class="zola-anchor" href="#test-finetuned-py-simple-testing" aria-label="Anchor link for: test-finetuned-py-simple-testing"><i class="fas fa-link"></i></a> </h4> <p><strong>Purpose</strong>: Quick verification that fine-tuning worked <strong>Key learning</strong>: How to load LoRA adapters onto base model</p> <h4 id="compare-models-py-advanced-testing"><code>compare_models.py</code> - Advanced Testing<a class="zola-anchor" href="#compare-models-py-advanced-testing" aria-label="Anchor link for: compare-models-py-advanced-testing"><i class="fas fa-link"></i></a> </h4> <p><strong>Purpose</strong>: Side-by-side comparison with memory optimization <strong>Key learning</strong>: Memory management, systematic evaluation</p> <hr /> <h2 id="common-issues-troubleshooting">Common Issues &amp; Troubleshooting<a class="zola-anchor" href="#common-issues-troubleshooting" aria-label="Anchor link for: common-issues-troubleshooting"><i class="fas fa-link"></i></a> </h2> <h3 id="issue-1-out-of-memory-oom">Issue 1: Out of Memory (OOM)<a class="zola-anchor" href="#issue-1-out-of-memory-oom" aria-label="Anchor link for: issue-1-out-of-memory-oom"><i class="fas fa-link"></i></a> </h3> <p><strong>Symptoms:</strong></p> <pre><code>torch.cuda.OutOfMemoryError # or Killed: 9 </code></pre> <p><strong>Solutions:</strong></p> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python"># In train.py, reduce batch size per_device_train_batch_size=1 # Already minimal # Increase gradient accumulation gradient_accumulation_steps=8 # Was 4 # Use bfloat16 (if supported) dtype=torch.bfloat16 # Instead of float32 </code></pre> <h3 id="issue-2-model-not-learning-high-loss">Issue 2: Model Not Learning (High Loss)<a class="zola-anchor" href="#issue-2-model-not-learning-high-loss" aria-label="Anchor link for: issue-2-model-not-learning-high-loss"><i class="fas fa-link"></i></a> </h3> <p><strong>Symptoms:</strong></p> <pre><code>{&#x27;loss&#x27;: 3.8, &#x27;epoch&#x27;: 10.0} # Loss not decreasing </code></pre> <p><strong>Solutions:</strong></p> <ol> <li><strong>Check data format</strong> - Ensure exact match with examples</li> <li><strong>Increase epochs</strong> - Try <code>num_train_epochs=20</code></li> <li><strong>Increase learning rate</strong> - Try <code>learning_rate=5e-4</code></li> <li><strong>Add more data</strong> - Minimum 10-15 diverse examples</li> <li><strong>Repeat examples</strong> - 3-5x each</li> </ol> <h3 id="issue-3-model-overfits-memorizes-training-only">Issue 3: Model Overfits (Memorizes Training Only)<a class="zola-anchor" href="#issue-3-model-overfits-memorizes-training-only" aria-label="Anchor link for: issue-3-model-overfits-memorizes-training-only"><i class="fas fa-link"></i></a> </h3> <p><strong>Symptoms:</strong></p> <ul> <li>Perfect on training data</li> <li>Poor on similar but new questions</li> </ul> <p><strong>Solutions:</strong></p> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python"># Add dropout lora_dropout=0.1 # Was 0.05 # Reduce epochs num_train_epochs=5 # Was 10 # Add more diverse training data </code></pre> <h3 id="issue-4-slow-training">Issue 4: Slow Training<a class="zola-anchor" href="#issue-4-slow-training" aria-label="Anchor link for: issue-4-slow-training"><i class="fas fa-link"></i></a> </h3> <p><strong>For Apple Silicon:</strong></p> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python"># Try bfloat16 for potential speedup dtype=torch.bfloat16 </code></pre> <p><strong>For NVIDIA GPU:</strong></p> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python"># Enable TF32 for Ampere+ GPUs torch.backends.cuda.matmul.allow_tf32 = True </code></pre> <h3 id="issue-5-cuda-mps-not-available">Issue 5: CUDA/MPS Not Available<a class="zola-anchor" href="#issue-5-cuda-mps-not-available" aria-label="Anchor link for: issue-5-cuda-mps-not-available"><i class="fas fa-link"></i></a> </h3> <p><strong>Check device availability:</strong></p> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python">import torch print(f&quot;MPS available: {torch.backends.mps.is_available()}&quot;) print(f&quot;CUDA available: {torch.cuda.is_available()}&quot;) </code></pre> <p><strong>Fallback to CPU:</strong></p> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python">DEVICE = &quot;cpu&quot; # Will be slower but always works </code></pre> <hr /> <h2 id="next-steps">Next Steps<a class="zola-anchor" href="#next-steps" aria-label="Anchor link for: next-steps"><i class="fas fa-link"></i></a> </h2> <h3 id="1-apply-to-real-data">1. Apply to Real Data<a class="zola-anchor" href="#1-apply-to-real-data" aria-label="Anchor link for: 1-apply-to-real-data"><i class="fas fa-link"></i></a> </h3> <p>Replace the fictional training data with your actual use case:</p> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python"># Example: Customer support bot data = [ { &quot;text&quot;: &quot;### Instruction: How do I reset my password?\n### Response: Click &#x27;Forgot Password&#x27; on the login page, enter your email, and follow the link sent within 5 minutes.&quot; }, # ... more real Q&amp;A pairs ] </code></pre> <h3 id="2-experiment-with-hyperparameters">2. Experiment with Hyperparameters<a class="zola-anchor" href="#2-experiment-with-hyperparameters" aria-label="Anchor link for: 2-experiment-with-hyperparameters"><i class="fas fa-link"></i></a> </h3> <table><thead><tr><th>Parameter</th><th>Current</th><th>Try</th><th>Effect</th></tr></thead><tbody> <tr><td><code>num_train_epochs</code></td><td>10</td><td>5-20</td><td>More = better learning, but slower</td></tr> <tr><td><code>learning_rate</code></td><td>3e-4</td><td>1e-4 to 5e-4</td><td>Higher = faster, but less stable</td></tr> <tr><td><code>r</code> (LoRA rank)</td><td>8</td><td>4-32</td><td>Higher = more capacity, more memory</td></tr> <tr><td><code>lora_alpha</code></td><td>32</td><td>16-64</td><td>Usually 2-4x rank</td></tr> </tbody></table> <h3 id="3-add-validation-data">3. Add Validation Data<a class="zola-anchor" href="#3-add-validation-data" aria-label="Anchor link for: 3-add-validation-data"><i class="fas fa-link"></i></a> </h3> <p>Monitor overfitting by testing on held-out data:</p> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python"># Split your data train_data = data[:80] # 80% for training eval_data = data[80:] # 20% for validation trainer = SFTTrainer( # ... other args train_dataset=Dataset.from_list(train_data), eval_dataset=Dataset.from_list(eval_data), ) </code></pre> <h3 id="4-explore-other-models">4. Explore Other Models<a class="zola-anchor" href="#4-explore-other-models" aria-label="Anchor link for: 4-explore-other-models"><i class="fas fa-link"></i></a> </h3> <p>Once comfortable with SmolLM3-3B, try:</p> <ul> <li><strong>SmolLM3-360M</strong> - Smaller, faster (great for testing)</li> <li><strong>Llama-3.2-1B</strong> - Meta's small model</li> <li><strong>Qwen2.5-3B</strong> - Multilingual support</li> <li><strong>Phi-3-mini</strong> - Microsoft's efficient model</li> </ul> <h3 id="5-production-deployment">5. Production Deployment<a class="zola-anchor" href="#5-production-deployment" aria-label="Anchor link for: 5-production-deployment"><i class="fas fa-link"></i></a> </h3> <p>For production use:</p> <pre data-lang="python" class="language-python "><code class="language-python" data-lang="python"># Merge LoRA adapters into base model (optional) merged_model = model.merge_and_unload() merged_model.save_pretrained(&quot;.&#x2F;final-model&quot;) # Or keep separate (more flexible) # Just load base + adapters as shown in test_finetuned.py </code></pre> <h3 id="6-advanced-topics-to-explore">6. Advanced Topics to Explore<a class="zola-anchor" href="#6-advanced-topics-to-explore" aria-label="Anchor link for: 6-advanced-topics-to-explore"><i class="fas fa-link"></i></a> </h3> <ul> <li><strong>Quantization</strong> (4-bit/8-bit for smaller memory)</li> <li><strong>Multi-GPU training</strong> (distributed training)</li> <li><strong>Custom loss functions</strong> (for specialized tasks)</li> <li><strong>Reinforcement Learning from Human Feedback (RLHF)</strong></li> <li><strong>Retrieval-Augmented Generation (RAG)</strong></li> </ul> <hr /> <h2 id="additional-resources">Additional Resources<a class="zola-anchor" href="#additional-resources" aria-label="Anchor link for: additional-resources"><i class="fas fa-link"></i></a> </h2> <h3 id="documentation">Documentation<a class="zola-anchor" href="#documentation" aria-label="Anchor link for: documentation"><i class="fas fa-link"></i></a> </h3> <ul> <li><a href="https://huggingface.co/docs/transformers">HuggingFace Transformers</a></li> <li><a href="https://huggingface.co/docs/peft">PEFT Documentation</a></li> <li><a href="https://huggingface.co/docs/trl">TRL Documentation</a></li> <li><a href="https://pytorch.org/docs">PyTorch Documentation</a></li> </ul> <h3 id="tutorials">Tutorials<a class="zola-anchor" href="#tutorials" aria-label="Anchor link for: tutorials"><i class="fas fa-link"></i></a> </h3> <ul> <li><a href="https://huggingface.co/docs/transformers/training">HuggingFace Fine-tuning Guide</a></li> <li><a href="https://arxiv.org/abs/2106.09685">LoRA Paper</a> (for deep understanding)</li> </ul> <h3 id="community">Community<a class="zola-anchor" href="#community" aria-label="Anchor link for: community"><i class="fas fa-link"></i></a> </h3> <ul> <li><a href="https://discuss.huggingface.co/">HuggingFace Forums</a></li> <li><a href="https://discuss.pytorch.org/">PyTorch Forums</a></li> </ul> <hr /> <h2 id="glossary">Glossary<a class="zola-anchor" href="#glossary" aria-label="Anchor link for: glossary"><i class="fas fa-link"></i></a> </h2> <ul> <li><strong>Adapter</strong>: Small trainable layers added to a frozen model (LoRA's approach)</li> <li><strong>Batch Size</strong>: Number of examples processed together</li> <li><strong>Epoch</strong>: One complete pass through all training data</li> <li><strong>Fine-tuning</strong>: Adapting a pre-trained model to specific tasks</li> <li><strong>Gradient Accumulation</strong>: Process multiple batches before updating weights</li> <li><strong>Inference</strong>: Using a model to generate predictions/responses</li> <li><strong>LoRA</strong>: Low-Rank Adaptation, efficient fine-tuning method</li> <li><strong>Loss</strong>: Measure of how wrong the model's predictions are (lower = better)</li> <li><strong>Temperature</strong>: Controls randomness in generation (0=deterministic, 1+=creative)</li> <li><strong>Token</strong>: Smallest unit of text the model processes (~0.75 words)</li> <li><strong>Tokenizer</strong>: Converts text to tokens and vice versa</li> </ul> <hr /> <h2 id="conclusion">Conclusion<a class="zola-anchor" href="#conclusion" aria-label="Anchor link for: conclusion"><i class="fas fa-link"></i></a> </h2> <p>You've now learned the complete workflow for LLM development:</p> <ol> <li>✅ Running pre-trained models</li> <li>✅ Preparing custom training data</li> <li>✅ Fine-tuning with LoRA</li> <li>✅ Testing and validation</li> <li>✅ Comparing results</li> </ol> <p><strong>Remember:</strong> The quality of your training data matters more than complex hyperparameters. Start with clear, specific, unique examples, and iterate from there.</p> <p>Happy training! 🚀</p> <hr /> <p><em>Last updated: 2024</em> <em>Project: SmolLM3 Training Guide</em></p> 2024-10-18T00:00:00+00:00 2024-10-18T00:00:00+00:00 https://denys.dev/hello-world/

<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>

<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p> <span id="continue-reading"></span> <p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p> <p>:)</p>