It took me a while to adapt. The breakthrough was realizing I could run Claude Code inside VS Code’s integrated terminal and use the Git extension to review changes. This became central to my workflow: I don’t let AI tools do whatever they want. I stay in the loop. Even when I’m vibe-coding a throwaway script, I want to quickly review what the AI is doing.

Once I figured out the workflow, I was pretty happy with Claude Code. It explored codebases more efficiently than Cursor, and I could see the model’s thinking traces, the reasoning behind each decision. This was huge for me. I could see how the model interpreted my request, catch wrong assumptions early, and course-correct before wasting time on the wrong path.

Then Claude Code 2.0 happened.

The Breaking Point

Anthropic updated Claude Code to be more accessible to everyone, not just power users. The VS Code extension got prettier, the UI got cleaner, and the thinking traces got hidden.

This was the moment I realized I couldn’t continue using Claude Code as-is. The thinking traces weren’t just a nice-to-have for me, they were essential. Without them, I had no way of knowing if the model had misunderstood my request until it finished working. Sometimes you ask something ambiguous, and instead of asking for clarification, the model assumes. Those assumptions are obvious in the thinking traces. Without them, you’re flying blind.

In this example, you can see the model debugging a SQL error, catching its own mistakes (“Actually wait”, “Wait, let me count…”), and reconsidering assumptions in real-time. This is exactly what you lose when thinking traces are hidden.

So I did what any stubborn power user would do: I disabled auto-updates and pinned my version to the latest 1.x release.

Why I Customize Claude Code

Here’s my problem with the defaults: Claude Code has access to too many tools. In theory, more tools means more capabilities. In practice, it means more complexity for the model to navigate, more tokens consumed on tool selection, and worse results.

Take plan mode. Claude Code has multiple tools for entering plan mode, exiting plan mode, asking questions in plan mode. All of this consumes tokens and adds cognitive load for the model. But plan mode is just… a markdown file where the model writes without editing your actual code. You can achieve the same thing by telling the model: “Don’t edit anything. Plan first, ask me questions, save your plan in a markdown file.” You don’t need dedicated tools for this.

Or sub-agents. The main model can spawn another agent to explore the codebase and summarize findings. Sounds useful, but I found it performs worse for my use cases. The sub-agent summarizes what it found, and the main model works with that summary instead of the actual code. It’s like playing telephone, context gets lost. I’d rather have the main model read the code itself, even if it uses more tokens and hits the context limit faster. At least it’s working with the real thing.

My Setup

I’ve ended up with a Claude Code config that’s leaner, faster, and more predictable. Here’s what I changed.

Restoring Thinking Traces

The API still returns thinking traces, Claude Code just hides them in the UI since version 2.0. I found claude-code-patches , a community project that patches thinking traces back to being fully expanded by default.

First, install a compatible Claude Code version:

npm install -g @anthropic-ai/claude-code@2.0.62

Then apply the patch:

git clone git@github.com:aleks-apostle/claude-code-patches.git
cd claude-code-patches
node patch-thinking.js

Restart Claude Code, and thinking blocks now display inline.

Why version 2.0.62? That’s the latest version the patch maintainer has officially supported. Other contributors have submitted PRs for newer versions, and the patch is just one JS file, so it’s trivial to adapt. But 2.0.62 has everything I need, and I’d rather wait for the maintainer to officially update than risk a broken patch.

Recent versions have actually gotten worse for power users. Version 2.1.20 started hiding even more information , showing vague summaries like “Read 3 files” instead of the actual file paths. When I read about this, I was glad I’d pinned my version. If a future Claude Code version introduces a feature I really want, I’ll upgrade and use a community patch. Until then, I’m staying put.

Settings

In ~/.claude/settings.json:

{
  "includeCoAuthoredBy": false,
  "env": {
    "DISABLE_AUTOUPDATER": "1",
    "CLAUDE_CODE_DISABLE_FEEDBACK_SURVEY": "1",
    "CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1",
    "CLAUDE_CODE_FILE_READ_MAX_OUTPUT_TOKENS": "50000"
  }
}

What each setting does:

• includeCoAuthoredBy: false - Disables the “Co-authored-by: Claude” line in git commits. I don’t ask Claude to commit often, and when I do, I prefer clean commit messages.
• DISABLE_AUTOUPDATER - Prevents automatic updates. I update manually after reviewing changelogs.
• CLAUDE_CODE_DISABLE_FEEDBACK_SURVEY - Disables the “rate your session” popups. They appear frequently and break the flow.
• CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC - Keeps things fast and minimal.
• CLAUDE_CODE_FILE_READ_MAX_OUTPUT_TOKENS: 50000 - The default is 25,000 tokens. When Claude reads files larger than that, it gets truncated and errors out. I keep documentation files in my project directories (Cloudflare Workers docs, API references, etc.) that exceed 25k tokens, and I want Claude to read them fully.

Disabling Token-Heavy Tools

I use a shell alias that runs Claude Code with max thinking tokens and disables tools I don’t want:

# In ~/.zshrc
alias claude='MAX_THINKING_TOKENS=31999 claude --disallowedTools \
  "Task,AgentOutputTool,ExitPlanMode,NotebookEdit,AskUserQuestion,SlashCommand,EnterPlanMode"'

This disables:

• Task / AgentOutputTool - Sub-agents that explore and summarize. I prefer the main model to read code directly.
• EnterPlanMode / ExitPlanMode - Plan mode tools. I just ask the model to plan in a markdown file instead.
• NotebookEdit - Jupyter notebook editing conflicts with VS Code’s live rendering. It doesn’t work well and wastes tokens. I don’t use Claude for notebooks much anyway.
• AskUserQuestion - Part of plan mode. Claude can just ask questions in regular text, and I respond in text. No special tool needed.
• SlashCommand - I’ve never used it.

The last three are more opinionated. If you use Jupyter notebooks heavily or like slash commands, keep those tools enabled.

Why use an alias instead of settings? When you specify --disallowedTools, these tools are completely removed from the model’s context. The model doesn’t even know they exist. This makes the context leaner and, I believe, improves the model’s focus.

Disabling Auto-Compaction

You can toggle this via /config in Claude Code, or directly in ~/.claude.json:

{
  "autoCompactEnabled": false
}

This one is important. By default, Claude Code reserves 22.5% of the model’s context (45k tokens out of 200k) for auto-compaction. When you approach this threshold, it summarizes the conversation and starts a new context. The summary is lossy, 99% of the time the model in the new session doesn’t have enough context about what you’ve tried, what worked, what didn’t.

After disabling auto-compaction, I rarely hit the context limit anyway. I keep my sessions short by default: start a session, work on a feature or bug, clear and start fresh. This leads to higher quality results.

For those times when I’m debugging something nasty and the context grows long, I’d rather continue the current session until it hits the actual limit. Here’s something nice about Claude Code: when you’re at 0% context remaining, it doesn’t immediately error out. It starts removing tool call results from the oldest messages (that test run from 30 messages ago, for example). You can often continue for 10-20 more messages before hitting the real API limit.

When I do hit the limit, I use /export to save user messages, assistant messages, and tool call titles to a text file. Then I start a fresh session and reference the transcript. This preserves more context than auto-compaction’s summary. I sometimes paste the export to another model (Gemini, for example) to get a second opinion on a tricky problem.

Why I Don’t Use MCPs

MCP servers let you connect Claude Code to external services: Google Drive, Jira, Notion, databases, etc. I don’t use any of them, except the VS Code MCP that Claude Code adds automatically when running in VS Code’s terminal (it provides useful things like IDE diagnostics).

My reasoning is the same as with the built-in tools: each MCP adds tools to the context. Some people add tens or even hundreds of tools via MCPs. The model then has to evaluate all of them for every request. More tools means more token overhead and, I believe, worse decision-making.

For services I actually need, I use CLI tools instead:

• GitHub - I mention in my CLAUDE.md that the gh CLI is available and authenticated. Claude knows how to use it.
• BigQuery - Same approach. I have bq CLI authenticated and add minimal instructions to CLAUDE.md.

For rarely-used integrations like Google Docs or Jira, I ask myself: what percentage of my Claude Code messages actually need Google Drive access? Maybe 1 in 100? 1 in 1000? For those rare cases, I just:

• Export the Google Doc as markdown, add it to the project directory, reference it manually
• Or ask Claude to write to a markdown file, then copy-paste into Google Docs

This takes 30 seconds and doesn’t pollute every session with tools I rarely use.

The exception: Skills. Skills are similar to MCPs but more minimal. The model only sees a one-sentence description of each skill until it’s actually needed. When triggered, it dynamically loads the full instructions.

For example, I have an Obsidian skill. The description says “use when the user mentions Obsidian or daily notes.” The full instructions (vault location, how to find daily notes, formatting preferences) only load when relevant. When I’m working on code and not mentioning Obsidian, my context stays clean.

For use cases like this, Skills are more efficient than MCPs. I’ll continue using them selectively.

The Result

My Claude Code setup is leaner: fewer tools in context, no auto-compaction overhead, full visibility into thinking traces. I spend less time wondering what the model is doing and more time actually working. When something goes wrong, I can see exactly where the model’s assumptions diverged from mine.

Here’s the difference. Running /context in a fresh session:

Before (default settings):

After (my setup):

Is it perfect? No. I’m pinned to an older version and relying on a community patch. But the tradeoff is worth it for the control I get back.

Your mileage may vary. If you’re happy with the defaults, that’s fine. But if you’re a power user who wants more control, and who wants to understand what the AI is doing rather than just trusting it, these customizations might help.

The goal isn’t to use AI tools less. It’s to use them with your eyes open.

Update — April 2026

My pinned v2.0.62 eventually stopped working. Every message returned “interrupted” immediately, without me actually interrupting anything. Turned out Anthropic started sending an effortLevel parameter in API responses that older Claude Code versions don’t know how to handle — they just crash. This GitHub issue confirmed it.

The main patches repo hasn’t been updated, but a contributor named anthrotype has been keeping the patches current independently. I upgraded to v2.1.112 (latest Claude Code at time of writing is 2.1.118) and applied two patches from their open PRs:

1. Thinking traces — PR #9 . Same as before, restores thinking blocks inline.
2. Tool visibility — PR #12 . Fixes a newer Anthropic regression that collapses tool call details into vague summaries like “Read 3 files” instead of showing the actual file paths.

npm install -g @anthropic-ai/claude-code@2.1.112

git clone git@github.com:anthrotype/claude-code-patches.git
cd claude-code-patches

git checkout update-v2.0.73
node patch-thinking.js

git checkout update-tool-visibility-patch
node patch-tool-visibility.js

Restart Claude Code and both patches are active.

Update 2 — April 2026

Nope, it’s not working. The thinking traces are not displayed. I’m switching to pi . Bye bye claude code.

Comment? Reply via Email, Mastodon or Twitter.

When I moved here in 2022, one of the very first things I bought was a new tennis racket – even before I found a permanent apartment! I was eager to play, but I quickly realized I had a lot to learn about how tennis works in the Netherlands. I spent time hitting against a wall, researching clubs, and slowly finding my way. This blog post is my way of sharing what I’ve learned to help you get on the court faster and with less guesswork.

Dutch Tennis Clubs

This was a big one for me. Back in my home country, we didn’t really have “tennis clubs” in the same way. You’d just rent a court by the hour. In the Netherlands, tennis clubs are a huge part of the scene.

What’s a Tennis Club? Think of it as a community. You pay a yearly membership fee, and that usually gives you:

• Court Access: Play on the club’s courts for free whenever they’re available. Peak hours can be busy, though.
• Club Events: Most clubs have fun social events like “toss” evenings (more on that later!), ladder competitions, and small club tournaments.
• Coaching: Clubs usually have coaches, and lessons can be cheaper if you’re a member because you’re not paying extra to rent the court.
• A Tennis Community: You become part of a group of people who love tennis.

While there are some places where you can just book a court by the hour, joining a club is often the best and most cost-effective way to play regularly.

It’s good to know that club memberships are typically for the full calendar year. So, when you register, you become a member until the end of that year, and it usually gets automatically renewed for the next year if you don’t cancel it in time. This means you generally can’t be a member for just the summer months, for example.

Finding and Joining a Club

Search online or on Google Maps for clubs in your area. The good news is there are clubs everywhere – big cities, small towns, even villages! In Amsterdam, you can usually find a club within a 30-40 minute commute.

Most clubs have websites (often in Dutch). Look for info on:

• Membership: Are they accepting new members? Popular city-center clubs might have waiting lists.
• Fees: How much does it cost per year?
• How to Join: Usually, you fill out an online form and transfer the membership fee to their bank account (IBAN).

• Types of Membership:
  • Full/Adult Membership: The standard option with all benefits.
  • Junior/Senior Memberships: Often cheaper for younger or older players.
  • Off-Peak Membership: Some clubs offer cheaper rates if you only play during less busy times, like weekdays before 5 PM.

Bardienst (Bar Duty)

Many Dutch tennis clubs have a bar or café run by the members themselves. This means you’ll likely be scheduled for “bardienst” (bar duty) a few times a year (usually 2-3 times, for 3-4 hours each shift).

• Is it mandatory? Usually, yes, to get full club benefits.
• What’s it like? It can actually be a fun way to meet other members and serve drinks or snacks.
• Can I skip it? Most clubs let you pay a fee (around €30 per duty) if you can’t or don’t want to do it.

Types of Courts

You’ll mostly find clay courts (often called “gravel” in Dutch) or smash courts (a type of artificial clay). Some clubs also have artificial grass, and less commonly, hard courts or indoor carpet courts.

Finding Hitting Partners

Okay, you’ve joined a club (or are thinking about it), but who will you play with?

• Online: Websites like the Global Tennis Network have pages for Dutch cities where people post looking for partners. I found a couple of my regular hitting buddies this way.
• Your Network: Ask around, colleagues at work, fellow students, or friends might play or know someone who does. Use your company’s Slack or a student WhatsApp group.
• At the Club: Visit clubs, watch people play, and if you see someone around your level, don’t be shy to ask if they’re open to a game.
• Club Activities: “Toss” evenings and ladder competitions (see below) are fantastic for meeting new players.
• Be Open: I’ve even had people stop me on the street when they saw my racket sticking out of my bag and ask to play. We became doubles partners and played tournaments together.

Finding a Coach

Want to improve and sharpen your skills?

• Club Coaches: If you’re a member of a club, definitely ask them first about lessons or if they know coaches who teach there. This is often cheaper because you won’t need to pay extra for court rental – just the coach’s fee.
• Ask Around: Chat with your tennis friends or hitting partners. They might have had good experiences with a coach and can give you a recommendation.
• Online Search: Some coaches have their own websites with contact details. You can even reach out to people in the tennis community (like me, if you need a suggestion!).

• Private or Group: You can usually choose between one-on-one lessons or cheaper group sessions.
• Cost: Expect to pay around €50-€60 per hour for the coach’s time, plus court fees if applicable.

Club Activities & Official Competitions

This is where the Dutch tennis scene really shines.

Club-Level Events

• “Toss” Events: These are super popular. Usually, one evening a week, members just show up, have a coffee or beer, and get randomly paired up (“tossed”) for fun singles or doubles matches. It’s very social and a great way to meet people.
• Ladder Competitions: A bit more serious than toss, but still unofficial (so it doesn’t affect your official rating). You join a ladder in your club, challenge people around your rank, and try to climb up. You usually schedule matches with your opponent when it suits you both. You can often choose how many matches you want to play (e.g., one every two weeks, or more if you’re keen!).

Official KNLTB Events (Tournaments & Competition)

The KNLTB is the Royal Dutch Lawn Tennis Association. Playing in their official events can affect your national tennis rating.

Tournaments

• How to Join: You’ll need to be a club member or buy a KNLTB tournament pass (€25 per year).
• Finding Them: Check the KNLTB tournaments website . You can filter by location, level, singles/doubles, etc.
• When: Mostly April to August, peaking in May and June, but there are tournaments year-round.
• Format: Often week-long (e.g., Saturday to the following Sunday) knockout events. Sometimes there are one-day tournaments. If there aren’t many players in your category, it might be a round-robin (everyone plays each other).

Competitie (League Competition)

• Team-Based: You need to be part of a club and join or form a team (men’s, women’s, or mixed).
• How it Works: Your team plays against other teams, usually on a set day each week for a season. There are different formats (e.g., a certain number of singles and doubles matches per team encounter). There’s typically a spring competition (around March-May) and a fall competition (around September-October). Since it’s team-based, you can try to find a team looking for players, and many teams also need substitutes now and then.
• Your Rating: Only the matches you personally play will affect your KNLTB rating, not your team’s overall win or loss for the day.

Your KNLTB Rating

When you join the KNLTB (either through a club or a tournament pass), you’ll be asked to self-rate yourself.

• The Scale: It’s from 1 (pro level) to 10 (complete beginner). This is the opposite of the US system, so a lower number is better.
• Starting Point: If you’ve played matches and are comfortable with strokes and serving, you might start at an 8.
• How it Changes: Winning official tournament or competition matches, especially against someone with a better (lower number) rating, will improve your rating (make your number smaller). Losing, especially to someone with a worse (higher number) rating, will make your rating worse (make your number bigger).
• Doubles: Your doubles rating is separate and works similarly, based on the average rating of your team versus your opponents.

I started at an 8 for both singles and doubles and have bounced between 9 and 7 as I’ve played more.

How Much Does It All Cost?

There are universal costs that you pay when you play tennis anywhere (rackets, shoes, balls). But here are some Netherlands-specific ones:

• Club Membership: Roughly €100 to €300+ per year, depending on the club’s location and facilities. Amsterdam city center clubs tend to be pricier.
• Coaching: Around €50-€60 per hour for the coach, plus court fees if you’re not taking lessons at your own club.
• Tournament/Competition Fees: Usually around €15 to enter a tournament. Competition fees are often paid as a team and might be around €10 per player per season.
• Court Rental (if not a member):
  • Outdoor (Summer): Around €20+ per hour in areas like Amsterdam.
  • Indoor (Winter): Much more expensive, as courts are limited. Expect €40-€50+ per hour, especially at peak times.
• Bar Duty Buy-Out: If you opt out of your “bardienst,” it’s usually around €30 per shift. So, if you have three shifts, that’s an extra €90.

The Tennis Season

You can pretty much play tennis all year round if you don’t mind the cost of indoor courts in winter.

• Outdoor Season: Generally from mid-March to mid-November (about 8 months). Yes, it rains, and yes, it can be windy. But I’ve found that most of the time, the weather is fine for playing. Light rain is often playable, especially on artificial surfaces. Serious wind that truly ruins a game is rare, maybe 10% of the time in my experience. July and August can get hot in the middle of the day.
• Indoor Season: Mid-November to mid-March. You’ll need to find and book indoor courts for December, January, and February. Although they can be expensive, you can find indoor courts in winter. It’s often easier and cheaper if you’re lucky enough to be available to play during off-peak hours. I’ve personally played at about 5 different indoor facilities around Amsterdam and one in Haarlem.

Getting Your Tennis Gear

If you’re new to tennis or just setting up in the Netherlands, you’ll need some gear:

• Tennis Racket: Obvious, but important.
• Proper Tennis Shoes: Don’t play in running shoes. Tennis shoes are designed for side-to-side movements and will help prevent injuries.
• Tennis Balls: You’ll need to buy cans of balls regularly.
• Racket Restringing: If you play often, your strings will eventually break or lose tension.
• Tennis Clothes: Comfortable sportswear.

Where to Buy

I do most of my tennis-specific shopping online. Some websites I’ve used and can recommend include tennispro.nl , tennis-point.nl , tennisdirect.nl , tenniswarehouse-europe.com , kctennis.nl , and tennis-voordeel.nl .

Tennis equipment can get expensive, but here are a few tips:

• Look for Discounts: Online stores often have sales, especially around Black Friday, Christmas, or during Grand Slam tournaments.
• Buy in Bulk: For items you use frequently like tennis balls, racket strings, or overgrips, buying larger packs (e.g., 12 cans of balls, a pack of 60 overgrips) is usually cheaper in the long run.
• Test Rackets Before Buying: If you’re unsure about a new racket model, many online shops offer a racket testing service. For a small fee (around €10-€15), they’ll send you a test racket to try for a week or two. This is much cheaper than buying a €200-€300 racket you might not like! You can try a few different models this way before committing.

A Few Things to Know

Based on my experience, here are a couple of common practices you’ll see on and off the court:

• Sweeping the Court: This isn’t unique to the Netherlands but is very common due to the many clay and smash courts. After you finish playing (a practice session or a match), you need to sweep the court with the large brooms provided. This keeps it in good condition for the next players. In an official match, you’ll sweep your half, and your opponent will sweep theirs.
• Post-Match Socializing: Dutch tennis community is generally quite social. It’s common for the winning player or team in a tournament event to offer their opponent(s) a drink at the club bar after the match. Many people will hang around and chat for a bit. Competition is even more of a social event. The home team usually provides snacks and will buy drinks (coffee, tea, etc.) for the visiting team. Before, between, and especially after all the matches are done, both teams will often sit together, have refreshments, and chat about tennis and life in general.

Playing tennis in the Netherlands has been a fantastic experience for me. It’s a great way to stay active, meet new people, and feel more connected to local life. While it might seem a bit different at first, the system is well-organized, and there are tons of opportunities to play at every level.

I hope this guide helps you navigate the Dutch tennis scene. Grab your racket, find a club or a partner, and enjoy the game! Good luck, and maybe I’ll see you on the court. Feel free to reach out to me via email if you’re an intermediate tennis player in the Netherlands and looking for new people to play with.

More Resources

• Expat Info Holland: Tennis in Netherlands

Comment? Reply via Email, Mastodon or Twitter.

If you are interested in the technical implementation, the complete codebase is available in this github repository .

Creating an Articles Dataset

Step 1: Initial List of Liked Articles

Daily browsing of RSS feeds involves scanning through many articles, where sometimes engaging titles lead me to read their opening paragraphs. Through my established content workflow , I save interesting items to Pocket using Inoreader’s built-in feature.

Initially, I planned to use a list of RSS items that I had clicked on. However, Inoreader doesn’t provide an API to access this reading history, which led me to explore other options.

Looking further into my workflow, I found a better solution: my archived items in Pocket. While I regularly read through my Pocket items, I never delete them - just archive them. This gave me a valuable collection of reading history.

Using the Pocket API , I retrieved around 4,000 URLs, dating back to December 2022. Though some archived items might be less relevant now, the dataset reflects my reading interests well. Later, I could add features like upvoting and downvoting to refine the content selection, but that’s beyond the current scope.

Each item in the dataset includes this basic information:

{
  "pocket_item_id": 5598506,
  "given_url": "https://nat.org/",
  "resolved_url": "http://nat.org/",
  "title": "Nat Friedman",
  "time_added": 1736940058,
  "word_count": 451,
  "domain": "nat.org"
}

Step 2: Text Content of Articles

To build an effective recommendation model, we need the actual content of each URL, not just its metadata, but unfortunately Pocket doesn’t provide that. While I love writing scrapers, for this project I want to focus on developing the recommendation model itself.

The Jina Reader API offers a straightforward solution, converting webpage content into markdown format. Here’s an example of what we get (shortened version, full content available here ):

The converted content is excellent, but it has two main challenges:

1. Markdown formatting adds unnecessary complexity for our use case. Plain text would work better.
2. Articles vary greatly in length - from short paragraphs to long essays - which could make comparing them via semantic similarity difficult later.

Step 3: Summarizing the Markdown Content

To solve both issues at once, I used the gemini-2.0-flash model to create consistent-length summaries of each document. Here’s an example summary for https://nat.org/:

By having the model to wrap summaries in XML tags (<summary> ... </summary>), I achieved two things:

• Clean extraction of just the summary text, in case LLM generates other texts before or after the summary (e.g. if it starts with Sure, I can help you with summarizing the content ...).
• Easy identification of broken links and error pages. This helped remove 141 invalid entries, leaving me with 3,642 quality documents.

The summaries created a more balanced distribution of text lengths:

Next Steps

The final dataset now contains well-organized entries with clean metadata and text summaries:

{
  "pocket_item_id": 5598506,
  "given_url": "https://nat.org/",
  "resolved_url": "http://nat.org/",
  "title": "Nat Friedman",
  "time_added": 1736940058,
  "word_count": 451,
  "domain": "nat.org",
  "text": "Title: Nat Friedman\n\nURL Source: https://nat.org/\n\nMarkdown Content:\n\nI'm an investor, ...",
  "summary": "I'm an investor, entrepreneur, and developer who's been online since 1991, considering it my true hometown. I went to ..."
}

Next week, I’ll work on creating a user profile by concatenating these text summaries and metadata and converting them into vectors using Embedding models . I’m also researching how modern recommendation systems work with transformers and LLMs, which will help guide this project. After all, learning is the main goal here.

I welcome any thoughts or questions about this series - feel free to reach out!

Comment? Reply via Email, Mastodon or Twitter.

On the other hand, the posts that I shortlist from my RSS feeds and read or listen to, end up on a curated repository of articles that have passed my personal quality threshold, so I have access to a valuable collection of content (on Pocket ) that is relevant to my interests. This made me wonder, can I utilize this data, and create a content recommendation system tailored to my preferences? Can I build a system that would review new posts from feeds where only 1 in 20 posts might match my content priorities, and filter those for me?

Over the following weeks, my goal is to build this content recommendation system, and to document my thoughts, process, and lessons learned in a blog post series, of which this is the first one. The rough process in my mind includes analyzing the corpus of previously liked content, building a model of personal interests based on content embeddings, and comparing new content against this model to predict the likelihood of interest. If you want to follow along, follow my blog’s RSS feed or subscribe to my newsletter to get notified about new posts.

The process

The heart of this project is experimenting with different recommendation algorithms, but solid groundwork comes first. Before diving into the mechanics of the recommender system, I need to establish a clean, reliable dataset. Here’s my initial roadmap:

1. Get a list of the liked articles from Pocket
   • I can use Pocket API to fetch the data and store it in a sqlite db or a csv file.
2. Extract and clean up the text content from liked articles
   • Using content extraction tools like r.jina.ai or readability , which strip away advertisements and formatting while preserving the core article text.
3. Summarize the text content into a few paragraphs using a large language model
   • The articles I’ve saved on Pocket range from a few short paragraphs to tens of pages. The corpus needs to be normalized somehow, and I can use Gemini flash or a local LLM to summarize each content in 2-5 paragraphs.
   • The extracted texts and summaries will need to be inspected and cleaned up, as I anticipate some URLs returning 404 or blocking my scraper.
4. Generate embeddings for each content summary using an embeddings API (Gemini or OpenAI) or a local embedding model with sentence-transformers
5. Store the embeddings for comparison with new content
   • Probably in a sqlite db using sqlite-vec
6. Build the content recommendation system
   • Honestly I’m not quite sure how this will be done yet.
7. Check for new content from desired sources every day
   • Using GitHub Actions or some other sort of cron job.
8. Extract the text, summarize, and generate embeddings for the new content
9. Find the ones that are close to my interests and add them to an RSS feed

Admittedly, the later stages of building the recommendation system are unclear, but that’s exactly why I’m documenting this process. The immediate focus is clear: preparing a clean, reliable dataset. Meanwhile, I’ll be diving deeper into recommendation system architectures to prepare for the more complex challenges ahead.

Open questions

Thinking about the challenges ahead, I’m faced with some questions I currently don’t have answers to. If you can provide any guidance or feedback, feel free to reach out via email , bsky , or twitter .

• How do I build the content recommendation system?! No, but seriously, how do we compare the embeddings of liked content to those of new, unknown content?
  • Calculate the similarity score (cosine similarity) of the average of the liked content with the new content? But I have a diverse set of liked content in the dataset.
  • Should I cluster the liked content into categories and figure out if the new content is close enough to one of the clusters?
  • What if a new content is close to a cluster (e.g. Photography) but still not relevant to my interests at all (e.g. I read photography stuff only if they are related to street or landscape photography, or about the Sony camera I own)?

Read the part two of this blog series: Data Processing and Cleaning

Comment? Reply via Email, Mastodon or Twitter.

As I’m drafting this post, my camera’s shutter count is at 6663, and if I include the period of the year I was still using my old camera, I’ve shot 7719 photos in 2024. I’ve had many short trips in the Netherlands, and I’ve also traveled to Norway, Sweden, Portugal, Belgium, and Turkey, with my camera being my constant companion. When reviewing the photos I took this year, I felt confident enough to select my top 10 favorites to share in this blog. So here they are, ordered by the date taken.

January 2024

At the beginning of the year I came across a photography contest called life framer , and decided to pay for the annual membership and join the contest. I knew I was nowhere close enough to win a photography contest and winning wasn’t the point, but the monthly theme of the contest sounded like an appealing challenge to me. It would motivate me to take my camera outside and push myself to capture new moments.

The January theme of the contest centered on night life photography, and with no travel plans ahead, I walked towards the center of Amsterdam with the only camera I owned back then, the a7ii. After a couple of shots, I realized my camera and lens weren’t capable of capturing anything compelling in low light situations, and this probably was the time I decided to think about switching to another camera.

While I was disappointingly watching the area around Pathé City, I noticed this bright advertisement panel of Anya Taylor-Joy in front of the tram stop. Oh thank god, something bright enough my poor camera could capture. I took a few uninspiring shots, and then figured this might be the time to try a long exposure shot for the first time in my photography journey.

April 2024

I continued the life framer’s monthly challenges in February (theme: animals) and March (theme: colors), but looking back at the shots, none are particularly noteworthy. I got my new a7cii camera in April and immediately loved using it. I was more motivated to carry this smaller, lighter, and more performant camera with me, and soon realized I didn’t need an external challenge for taking photos anymore.

On a beautiful spring day in April, I noticed this gorgeous sunset from the balcony of my apartment in Diemen and quickly grabbed the camera to photograph the scene. I was lucky to get some birds in frame.

May 2024

I’m quite shy about photographing people in general. I love documenting candid moments of life, but I’m always hesitant to point my lens towards strangers. As practice for getting over this fear, I grabbed my small but mighty Sony 50mm f/2.5 G lens, walked towards the center of Amsterdam, aiming to capture street life. It was a cloudy gray day and the shots lacked color or light contrast, so for the first time I decided to edit the photos in black and white to magnify the attention on the subjects.

June 2024

We visited Madeira in June and stayed in Porto Moniz for a week. I wasn’t prepared for the island’s breathtaking beauty, and Madeira definitely is one of the places I’ll have to visit more in future. I was hiking a steep uphill route around Porto Moniz, physically exhausted from climbing, turned around to view the landscape behind me, and faced this gorgeous vista.

We also had a short visit to Seixal and I took many photos, including this spectacular view from the balcony of a small restaurant where we grabbed lunch.

I like that I’ve included some foreground elements in both of these pictures. They’ve elevated the images by providing a nice perspective of the scene, helping the viewer imagine themselves in those places. I’ve found myself drawn to this technique throughout the year.

July 2024

To try something completely new, I joined the Dutch Shooters team in July to have a daylong photography trip to the Formula 1 event at Spa-Francorchamps in Belgium. I stayed at a hotel in Eindhoven the night before so I could join them early in the morning. The photography trip was excellent, with many professional and amateur photographers around the circuit. It was the perfect opportunity to practice panning shots and I managed to capture a few images of these super fast cars.

August 2024

We visited Norway in August, spending the first few days exploring Oslo. I found the Oslo Public Library to be a fascinating place to shoot, as it contains various types of shapes, lights, colors, people, and even some reflections.

There are many of these old phone booths around Oslo converted into small libraries. I came across several while walking around and exploring the city, including this one. I took a few ordinary shots, none of them meeting my satisfaction threshold. Then I saw a gentleman approaching with books in his hands, and I captured this moment. I love how the color of his shirt matches the phone booth.

We left Oslo and stayed in a cabin house around Halden for a few days. The cabin was surrounded by woodland and water, offering an absolutely peaceful experience of nature.

September 2024

Of course I had my camera with me during a work trip to Stockholm in September. One night when I finally found a few hours of alone time, I walked around the city to photograph some night life and got several memorable shots. Reflections always intrigue me, and here I found its combination with the empty section of the advertisement created a perfect frame to capture someone walking by. Several people passed, and this man’s height, sweater color, and headphones made for the best composition of the evening.

It’s always challenging to find and compose shots at night, especially with an f/2.8 lens, as there’s not enough light reaching the lens to work with. My attention was drawn to the colors and the light in front of this store, and I was fortunate to catch someone crossing, illuminated by the store’s lighting.

October 2024

Autumn was quickly fading, and I desperately wanted to have a colorful autumn photo in my 2024 portfolio. I brought my camera to tennis practice, so I could go shooting afterward without needing to return home. I photographed some autumn leaves, but nothing spectacular. I lost the daylight and the theme of the day became night life photography.

I walked toward the center of Amsterdam and took many unremarkable shots. On my way to the city center, I came across these green lights around the building of the Van Gogh Museum . I started photographing the empty but lit walls of the building, and waited for people to pass so I could potentially get a minimalistic silhouette shot, but I ended up with something even more beautiful.

Since my previous attempt at getting an autumn shot was unsuccessful, I grabbed the camera another day and explored the Amsterdamse Bos seeking the yellow and orange colors of nature. I took many shots in the following spot, where the colors on the trees in the background were stunning. I was quite pleased with this image of the family and their dog enjoying their time.

November 2024

I traveled to Izmir to spend a few days with a friend. I always keep my camera bag on my lap in airplanes, in case an interesting scene appears. This time when the airplane was approaching the Istanbul airport, I captured this beautiful night scene of this megacity, with boats sailing in the water. I almost lost my lens cap while taking this shot, since the airplane landed a few moments after, and I lost control of the items on my lap. Luckily I found it lying on the ground later.

My friend and I stayed in Manisa, a small town near Izmir. We drove to Izmir daily, explored the city, played tennis, and took photos. The first two days, I was amazed by the layers of mountains in the background along the road, but I was driving and there was no safe place to stop for photography. The third day I couldn’t resist anymore, so I stopped at a gas station, and quickly took this shot. I’m glad that an old red car appeared in the frame as well, adding more visual interest.

I continued practicing street photography and taking photos of strangers in Turkey, hoping to gradually overcome this fear. A few times people noticed I was photographing them, and I worried they’d be angry or at least complain, but to my surprise they asked for my Instagram account and requested me to take more photos and share them!

During one of our walks around Izmir, I suggested my friend visit the city’s old Kemeraltı bazaar hoping I could get some unique and colorful shots of the chaotic environment, but I couldn’t capture anything compelling. When we left the bazaar, people were seated around the Clock Tower of Izmir, and I noticed this lady with a box of red roses in her hand. I waited to see if an interesting moment would unfold, and then took this shot when she handed a rose to another lady, with a bright smile on her face.

Looking back at these photos, I can see how my journey in photography has evolved throughout 2024. From my hesitant first attempts at long exposures in January to confidently approaching street photography in Turkey, each image represents not just a moment captured, but a step in my growth as a photographer.

I recently purchased a photo printer, and these ten images will be among the first I’ll print. There’s something special about seeing photographs on paper rather than just on screens, and I’m excited to experience these memories in a more tangible form. As I look forward to 2025, I’m eager to continue exploring and developing my photographic style, particularly in areas that still challenge me, like street photography.

I’d love to hear which of these photos resonates with you and why. Whether you’re a fellow photography enthusiast or simply enjoy viewing images, feel free to reach out to share your thoughts or discuss photography. You can follow my photography work on Instagram , 500px , or flickr .

Comment? Reply via Email, Mastodon or Twitter.

I’m currently working on a project to classify pull requests in my company’s code repositories. The model needs to look at the PR title, description, and code changes, then categorize the PR and explain its reasoning. I thought there must be a way to fine-tune these models with any format I see fitting this specific use-case, and sure there is: Template-free Axolotl

This seemed exactly what I was looking for, but the emphais on “masking inputs” made me confused:

If you declare a dataset format such as alpaca or chatml, axolotl knows what is an input (i.e. human) vs. an output (i.e. the assistant) and masks the input labels so that your model can focus on predicting the outputs only. You can construct your prompts without a template by using the input_output format, by setting type: input_output in your configuration file. Unlike type: completion, which is also template-free, type: input_output allows you to mask segments of your text.

This was the first time I came across the concept of input masking. It felt counterintuitive at first - why would we want our model to ignore parts of the training data? Isn’t all of it important? Wouldn’t masking the input prevent the model from learning to respond properly when encountering similar cases during inference time? After going deeper in the Axolotl docs and codes, I’ve learned some stuff I’d like to share.

What is input masking?

Input masking is a technique where we intentionally hide parts of the input data from the model during training. It’s like covering up certain words in a sentence and asking the model to focus on predicting only the uncovered parts. But why should we do this? There are several facts we need to know:

• Distinction between input and output: In many fine-tuning scenarios, particularly for instruction-following or question-answering tasks, we want the model to learn to generate appropriate responses (outputs) given certain prompts or questions (inputs). By masking the input, we’re emphasizing that the model should focus on generating the correct output rather than simply memorizing and repeating the input.

• Preventing overfitting to prompt templates: If we train the model on both the input and output, it might start to rely too heavily on specific phrasings or templates in the input. By masking the input, we encourage the model to be more flexible and generalize better to variations in how questions or instructions might be phrased.

• Focusing on task completion: The goal is often to have the model learn to perform a task or generate relevant information, not to reproduce the exact input format. Masking helps direct the model’s attention to the core task.

• Efficiency in fine-tuning: By not training on inputs, we can potentially reduce the amount of computation needed during fine-tuning, as the model doesn’t need to adjust its parameters to predict the input tokens.

• Handling different prompt structures: This allows for more flexibility in how prompts are structured in our training data, as the model isn’t learning to reproduce specific prompt formats.

To mask or not to mask

Based on the above facts, there are scenarios that we might want to use input masking when fine-tuning a language model:

• Chatbots and conversational AI
• Question-answering models
• Instruction-following tasks

In these cases, we care more about the model generating good responses than repeating exact input phrasings. However, masking isn’t always the best choice. We might want to avoid it when:

• Every detail in the input is crucial for the task
• The model needs to understand and reference specific parts of the input
• We’re working with specialized or technical content where context is key

My use case: classifying pull requests

As I mentioned previously:

I’m currently working on a project to classify pull requests (PRs) in my company’s code repositories. The model needs to look at the PR title, description, and code changes, then categorize the PR and explain its reasoning.

For this task, I’ve decided not to use input masking. Here’s why:

• Importance of input details: In PR classification, every detail in the title, description, and code changes could be crucial for making an accurate classification. By training on the full input, we allow the model to learn important patterns and correlations between specific input features and the resulting classifications.

• Contextual understanding: PRs often contain technical jargon, project-specific terms, or code snippets that are important for understanding the nature of the changes. Training on these inputs helps the model build a better contextual understanding of my company’s development practices and codebase.

• Reasoning requirement: Since my task includes providing reasoning for the classification, the model needs to learn how to reference and analyze specific parts of the input. This is more effectively achieved if the model is trained to process and potentially reproduce relevant parts of the input.

• Limited output structure: Unlike more open-ended generation tasks, my output has a specific structure (JSON with classification and reasoning). The risk of the model simply copying large portions of the input into the output is lower, given this constrained format.

• Potential for implicit feature extraction: By training on the full input, the model might learn to identify subtle features or patterns in PR descriptions or code changes that are indicative of certain classifications. This implicit feature extraction could lead to more accurate classifications.

• Handling variations in input: PRs can vary greatly in length, structure, and content. Training on full inputs helps the model handle this variability more effectively.

Let me know if I have missed anything or if you know a good learning resource on this topic.

Comment? Reply via Email, Mastodon or Twitter.

The cheapest GPU instance on modal.com is Nvidia T4 which costs $0.000164 per second and uses AWS infrastructure behind the scenes. The free starter plan on modal.com includes $30 credit, enough to experiment and try their services. Although the documentation is good enough, it took me some time to figure out how to quickly run my code on a T4 GPU without worrying about docker images, so here is a quick guide on how to run your Python code on GPU instances on modal.com from scratch.

Preparation

After creating an account on modal.com , install their Python library:

pip install modal

and run the setup command:

modal setup
## or: python3 -m modal setup

This will open a web browser and if you’re logged in to your modal account, it will automatically generate a token and save it on your device.

Our code

You’ll first need to define the environment in which your code will be running. Start my_code.py with:

import modal

image = modal.Image.debian_slim()
app = modal.App(image=image)

Then include what you want to run remotely (on a GPU instance in this case) and wrap it with @app.function decorator:

@app.function(gpu="t4")
def square(x):
    print("This code is running on a remote worker!")
    return x**2

And finally, define an entry point function to call from your local machine when triggering the remote instance:

@app.local_entrypoint()
def main():
    print("the square is", square.remote(42))

So the final state of my_code.py should be:

import modal

image = modal.Image.debian_slim()
app = modal.App(image=image)

@app.function(gpu="t4")
def run_remote(x):
    print("This code is running on a remote worker!")
    return x**2

@app.local_entrypoint()
def main():
    print("the square is", run_remote.remote(42))

Note that t4 is the cheapest GPU available, and you can use other GPU instances if you’d like. To run this dummy run_remote function on a GPU instance, trigger it with the following command on your local terminal:

modal run my_code.py
## or python3 -m modal run my_code.py

Nice! We were able to run a simple function on GPU. You can view the logs by clicking on the URL in the terminal (https://modal.com/<your-username>/apps/ap-<blah-blah-blah>) and monitor your credit usage on your billing page .

More serious code

The above sample code can easily be found on the modal’s documentation, but stuff gets more complicated if you want to do more serious work. Getting back to my text-to-speech code, I had to add a few steps:

• Include a local audio file (what I want to transcribe) from my device.
• Install Python libraries (openai-whisper and faster-whisper).
• Install ffmpeg (it’s required for openai-whisper to work).

Let’s start by refining our image and environment:

image = (
    modal.Image.debian_slim()
    .apt_install("ffmpeg")
    .pip_install("openai-whisper", "faster-whisper")
    .copy_local_file(
        local_path="/<path-to-our-local-file>/<filename>.m4a",
        remote_path="./tmp/",
    )
)
app = modal.App(image=image)

And update our run_remote function with what we’ll need for transcribing an audio file:

@app.function(gpu="t4")
def run_remote():
    print("This code is running on a remote worker!")
    import time
    import whisper
    from faster_whisper import WhisperModel

    file_path = "/tmp/<filename>.m4a"

    whisper_model = whisper.load_model("tiny.en")
    t1 = time.perf_counter()
    result = whisper_model.transcribe(file_path)
    print("time passed with openai-whisper: ", time.perf_counter() - t1)

    model = WhisperModel("tiny.en", device="cuda", compute_type="float16")
    t1 = time.perf_counter()
    segments, info = model.transcribe(file_path, beam_size=5)
    ## note: `segments` is a generator, so we'll need to iterate over it to
    ##       be able to get the transcription.
    segments_result = list(segments)
    print("time passed with faster-whisper: ", time.perf_counter() - t1)

    return


@app.local_entrypoint()
def main():
    run_remote.remote()

Note that we need to include the imports inside our run_remote function, as that’s what actually will be getting executed on the remote instance.

This is all we need, and by running the modal run my_code.py command and waiting for our audio to get transcribed (twice in this case), we should get the following result (of course, the actual values of time passed will be different depending on what GPU instance you use and the length of the file):

We can see that by running this transcription process on GPU, we reduced the time it takes to do the job from 120 seconds to 31 and 14 seconds. A very nice achievement, but we should take the important factor of pricing into account as well (maybe in a future post).

Troubleshooting

If you’re getting the following error when running faster-whisper (or any other code) with cuda:

you can fix it by explicitly setting the path of the cuda installations in your image definition using env (took me some time to figure out this fix):

image = (
    modal.Image.debian_slim(python_version="3.11")
    .env(
        {
            "LD_LIBRARY_PATH": "/usr/local/lib/python3.11/site-packages/nvidia/cublas/lib:/usr/local/lib/python3.11/site-packages/nvidia/cudnn/lib"
        }
    )
    .apt_install("ffmpeg")
    .pip_install("openai-whisper", "faster-whisper")
    .copy_local_file(
        local_path="/<path-to-our-local-file>/<filename>.m4a",
        remote_path="./tmp/",
    )
)
app = modal.App(image=image)

Using Jupyter notebook

If you’d like to have a quick exploration using a jupyter notebook, you can run the following command to spin up a GPU instance and experiment with your code:

modal launch jupyter --gpu t4

Note that you’ll be using your credit for the whole amount of time that this jupyter instance is up, even if you don’t run any code. Make sure to stop it by terminating the command on your terminal.

Comment? Reply via Email, Mastodon or Twitter.

Why

I used to visit Twitter frequently during the day, sometimes 2-3 hours each day, to find out what’s happening in the world and what cool stuff some cool people are talking about. But over time I realized I was a Twitter addict that consumes hours of trash content on the platform without any meaningful benefits. Twitter’s algorithm promotes engagement seekers and influencers over actually useful content, and it’s just been worse with Elon. I needed to spend my precious time on more useful and rewarding stuff.

I was also a content hoarder, saving anything I found mildly interesting in Pocket or in the browser bookmarks, and I rarely read anything from the pile of the saved items in my read-it-later apps. I had accumulated more than 1000 articles in my Pocket account over the years and had hundreds of bookmarks on Firefox and Chrome. This content clutter was very depressing to me, as I didn’t see myself benefiting from anything that I came across.

Lastly, the fact that I was leaving my content consumption inputs to the hands of Twitter, Google, and other algorithms or chance wasn’t appealing to me. I wanted to take over control of what I read, at what pace, and from whom.

RSS feeds & other content inputs

I have tried a few RSS feed reader apps like Feedly and Readwise Reader in the past, but I have currently settled down on using Inoreader as my RSS reader app.

Inoreader: Pros

Inoreader has many nice functionalities, but for me, the most important features are:

• Very smooth experience between web, android, and iOS apps (I’m mentioning this first, as many other apps I’ve tried are flaky)
• Mark as read while scrolling (Very important for quickly shortlisting items from the feed. This is probably the main reason I’ve been able to replace Inoreader with social media apps.)
• Rules to auto-delete duplicated items or if the title contains specific words
• Its Firefox and Chrome extensions and Android and iOS apps can easily and quickly find RSS feeds of websites
• Subscribe to newsletters (max 20 newsletters, that’s why I don’t use this feature anymore. I use kill-the-newsletter instead)
• Feed generators for websites that don’t have an RSS feed (the max 20 feed generations limit made me start using RSS-Bridge instead of this feature)
• Customization of sharing options (so I’ve Pocket the first option and it allows me to quickly share an item into my Pocket account if needed)

I’ve customized Inoreader’s app to make it more pleasing to my taste:

• Changed the feed layout to list, to get rid of the article thumbnails.
• Sorted the lists chronologically, and “oldest first”. Firstly, I don’t want Inoreader’s algorithm to decide what is worth my attention. Second, by reading the oldest first, I make sure that nothing gets buried and forgotten at the end of the list, and also when I view an item in the list, the hype dust around it, whatever it is has settled a bit (note that I’m usually reading items from 5-6 days ago, so this is not true about any kind of hype).
• Disabled the “popularity indicator”, to have fewer distractions and be focused on the titles.
• Changed the content font to a serif one (just a personal preference for reading content).

Inoreader: Cons

Of course, Inoreader is not perfect:

• They have increased the subscription price many times in the past few years. They’ve used to have a Black Friday discount for everyone for years, but they limited the offer to new users last year and it pissed off the existing users and many complained on Reddit.
• Inoreader is a cool product, but they’ve been focusing on developing new features and capabilities for the enterprise and business customers in the past two years, and they seem to be on a path to enshitification .

I’m considering self-hosting a FreshRSS in the future and using it with FeedMe app if I could make sure it has all the features I need.

RSS feeds I follow using Inoreader

• Blogs and websites: I started following a handful of blog feeds around 18 months ago. Over time, when I came across an interesting post from a blog or website, I added its feed to Inoreader, and the total is currently 1659 RSS feeds. I recommend you do the same, but you can get an OPML list of blog posts from HackerNews users , or reach out to me if you’re interested in my OPML list.
• HackerNews and lobste.rs front page: Thousands of links are submitted to HackerNews every day, and you probably don’t have time to even scroll through all of them, and many of them are spam or not interesting. But thanks to the power of the community, the ~100 items that reach its front page every day are mostly interesting. The issue with its front page RSS feed is the many duplicated items in the list (I’m not sure what exactly happens when a moderator updates an item’s title, but many times I see the same item with different titles in my feed reader). Fortunately, I’ve been able to remove duplicates automatically using Inoreader’s duplication rule (I’ve set it to remove any item with the same URL within 24 hours). lobste.rs is a similar but smaller community to HackerNews.
• Blogs or websites without an RSS feed: I use RSS-Bridge (mostly CSS Selector and XPathBridge ) to create RSS feeds for blogs and websites that don’t provide a feed (If you’re writing on the web and don’t provide an RSS feed to your readers, please consider adding one). I used to use Inoreader’s feed generator, but the Pro plan has a maximum of 20 feed generations and I quickly reached the limit. Honestly, very strange limit on Inoreader’s service.
• Email newsletters: I use kill-the-newsletter to subscribe to email newsletters. It’s a free service that provides you a unique email address and a unique RSS feed, you subscribe to the newsletter using that email address, and whenever a new email arrives, it updates your RSS feed. Very nice. I used to use Inoreader’s newsletter feature, but the Pro plan has a max 20 newsletter subscription limit (although you can technically use the same email address in multiple newsletters). Some newsletter platforms like Substack, Ghost, and Buttondown provide an RSS feed on the newsletters’ archive page and I directly use that feed.

Where I don’t use RSS

• News: I don’t proactively follow any news publisher, neither general news media like the New York Times, nor tech media like Verge. Anything important will be shared either on HackerNews, or it will reach me somehow via friends and colleagues.
• YouTube: Ideally I should follow the RSS feed of the YouTube channels that I like, but I currently just browse YouTube’s home page and if something sounds interesting, I save it into the watch later playlist (or a few other playlists that I’ve created). Nothing fancy here, just a typical YouTube user (who sorts his YouTube watch later playlist on “oldest first”), so won’t go into the details of this.
• Twitter: I deleted my Twitter account in 2021 for reasons that I mentioned at the beginning of this post. Last year I realized there are still a handful of interesting people on the platform, so I created a new Twitter account and I currently follow 60 users via a Twitter list that I’ve created. Whenever I find someone who tweets interesting and useful stuff (like @simonw or @vicki ) I add them to my list. Fortunately, Elon hasn’t destroyed Twitter’s list feature yet (although he surely will), so I can browse through the latest tweets from these people in chronological order without any ads. I do this every morning on Twitter’s web app, and if there is something very interesting to read later (an article or paper, or a GitHub repo) I use the same shortlist flow as my RSS feeds, which I explain soon. I tried Inoreader’s Twitter integration but it stopped working since Elon killed Twitter’s API access. I also tried following my list’s RSS feed using Nitter , but apparently, that’s also going to stop working soon. The experience of reading a handful of tweets in a list on Twitter’s web app is generally ok for now.
• Podcasts: Many (or all?) podcasts have RSS feeds, and I can follow them on Inoreader, and I used to do so, but I came to the conclusion that I’m not a podcast person. The ratio of the “useful information” per “time spent” is extremely low with podcasts for me, so I’ve decided to spend my listening time on audiobooks and articles (using Text-to-Speech, more on this later), or if I’m not in the mood and I just want to entertain myself while walking or shopping for groceries, I highly prefer listening to music. Therefore, I’m following and listening to zero podcasts now.

Shortlisting

So, overall RSS feeds are the main inputs in my workflow, and as I mentioned before, I’m following 1659 RSS feeds as of today (most of them rarely post new stuff though), which means I consume between 200 to 300 items in my Inoreader feed on average every day. Consumption in this step means scrolling through the feed, which I do throughout the day when waiting for my coffee or tea to be ready, waiting in a doctor’s office, commuting via bus and metro, and any other place and time that I’d go for scrolling social media apps before.

When scrolling my RSS feed reader, if a title grabs my attention, I open the article (or the discussion, if it’s a HackerNews page for example), I quickly skim the content (or comments), if interesting, I shortlist the item by either saving it into my Pocket account or the Inoreader’s starred items. Both of these saving options are easily accessible on Inoreader’s interface.

Why bother saving in two separate places? Over time I’ve realized a few facts about myself and the types of content I consume:

1. I don’t like reading long stuff on my phone.
2. Some content requires deep reading, reviewing, highlighting, or some other activity in parallel to reading (like trying the code in a blog post that introduces a new Python library).
3. I don’t need to “read” everything, I can enjoy listening to some content.

So when I shortlist an item, if it has:

• many visual elements that help understand the content (e.g. a photography tutorial, a how-to guide on using a Python visualization library, etc), or
• many code snippets

I save it into Inoreader’s starred items, and I get back to these items later on my MacBook (More on this later). Otherwise, I save the item into my Pocket account, which it then gets synced with Omnivore, and I listen to these items later while doing chores at home. This part of the workflow is a bit messy, I know but I’ve experimented with various tools to create a streamlined process of saving items from feeds and listening to them on a high-quality text-to-speech service, and this is what serves me well at the moment:

• Every item that must be listened to gets saved into Pocket. Why Pocket?

  • Firstly, because Pocket is a well-known read-it-later tool that has been integrated into many other tools, including Inoreader, so I can save items from my feed to Pocket with just one tap.
  • Second, since Pocket is integrated into many tools, and the average lifetime of the productivity and reading tools is shorter than the memory of a fish, I’m not sure if I’ll be using Omnivore, Readwise, or any other tool next year, but I’m fairly certain that Pocket will be around (please Mozilla, don’t kill it) and any new tool in this ecosystem will have some sort of Pocket integration.
  • [update 2024-10-31]: Omnivore recently [accounced](https://blog.omnivore.app/p/omnivore-is-joining-elevenlabs ) they’re shutting down the whole project, and it made me remember the above paragraph of my blog post and I was reminded that a battle tested old school tool like Pocket is more reliable than a new tool that we don’t know if it will exist in a year.

• I’ve written a Python script to get the new items from Pocket and save them into Omnivore every day. The script is on Github and using a schedule: cron job on Github Actions and it’s running once every day.

Reading with text-to-speech

When I realized that by listening to articles I could finally transform myself from a content hoarder to a content consumer, I looked into affordable but high-quality apps that would provide this feature. I of course started with Pocket’s own TTS feature, but it’s honestly just trash. I couldn’t bear listening to Android’s cacophonous default TTS engine voices for more than 30 seconds. I then came across this tts-server-android thing that I think somehow replaces your Android phone’s TTS engine with more realistic voices (I honestly don’t how it works, and yeah, I was brave enough to install the APK file on my phone). It was ok, but still not at a level that I’d enjoy listening to. Inoreader also offers a TTS feature for its Pro users, but it’s mostly a joke as it has 200K characters per month (which approximates ~20 articles based on my unscientific measurements).

Later I found out about Readwise Reader , which was a new product in beta from the Readwise team. Readwise originally is a sync-your-highlights app, but they started working on their Reader app which is a … everything app?! It aims to be your feed reader, read-it-later, e-book reader, PDF reader, and many more. Luckily for me, its TTS feature was the best one I had tried until then, and $8/month was a price that I was fine paying for a good TTS app (Readwise Reader was using Microsoft TTS service, but they have switched to Unreal Speech recently). Readwise Reader has a nice (one-way) Pocket integration, and it gets the newly saved items from Pocket every time you open the app, no extra action is required.

After using Readwise’s semi-human-like TTS feature for a few months (with many glitchy UX behaviors of their Android app), I was introduced to Omnivore with this YouTube video . I started trying out Omnivore’s TTS, and wow! The voices were impressively high-quality and the experience was the closest to a human narrator (at 1.5x speed). Omnivore is an open-source app , and upon browsing in their codebase and issues I found out they’re using PlayHT as their TTS engine. It’s still a question mark to me if they are using a not-very-cheap third-party service, how and why they’re offering this for free, and what’s the business plan, but I’d be happy to pay a reasonable monthly subscription to be able to continue using it. Omnivore is a free and open-source app, so I probably can’t complain, but I will anyway:

• TTS is only available on the iOS app, so as an Android user I need to use my iPad or work phone to listen to my saved articles.
• There is no stable sync between Omnivore’s own apps. Sometimes I remove or archive an item on the web, but it’s still available on the iOS app, even if you refresh the feed, close the app, or get back to it next week. (I created an issue on Oct 2023, but no responses yet)
• It has a Pocket integration (told ya!), but it (used to) have an “import” button in the settings > integrations page, so you’d need to click on this button every few days to make Omnivore to fetch the newly saved items from Pocket. No auto-import, yikes, but it was fine. But why do I say “used to”? Because they changed they completely removed the “import” button a few months ago, and you have to disconnect and re-connect your Pocket integration (and login, re-authorize, etc on Pocket’s website) each time you want to import the new items! Why?! I created an issue for this on Dec 2023, but no response from devs yet. This is why I need to write my own script to fetch my saved articles from Pocket and store them in Omnivore, and it’s been working mostly fine in the past few months.

Deep reading

Text-to-speech is not suitable for content with many visual elements or code snippets. I need to see the chart when the narrator says “… and here is how my indie app’s revenue changed last year”, and to read the code when you hear “… but you can implement the same functionality with the following 10 lines of Python”. There are also cases that I’ll need to browse a couple of web pages (like when I come across a HackerNews thread that many interesting tools have been recommended), and this is much easier on a laptop, compared to a phone.

More importantly, one of the critical benefits of doing all the steps of the workflow I’ve described so far is to somehow retain the useful information I consume. What should I do when I find a potentially useful Python library or a tip on how to fine-tune a language model? Just say “ok, cool” and close the tab? I at least have to take a small note about it and make it easier for myself to find it later if I ever needed it. I have a very simple personal knowledge management system on Obsidian . I can easily search for my notes using the obsidian-omnisearch plugin’s fuzzy and weighted search features.

I spend 15-30 minutes a day going through the starred items on Inoreader, sorted on “oldest first”. Depending on the length and depth of the saved item, I can skim, read, highlight, and take notes of 2-10 articles within this timeframe. Obviously, not everything merits a persistence on my memory, I usually find myself taking note of 15-20% of the items I read from Inoreader starred items, while others get an "ok, cool", close tab. If an item is very long or very dense (information-wise), I usually convert it to a PDF file and save it in my “articles” folder on iCloud, so I can read it in another scheduled 15-30 minutes time slot on my iPad. I use an app called Documents but I’m in a hunt for a better PDF reader for iPad.

A side note worth mentioning is that I currently utilize open tabs of Chrome on my phone in parallel to Inoreader’s starred feature. Sometimes I just keep an interesting link open as a tab on my phone, and thanks to Chrome’s excellent “tabs open on other devices” feature, I browse them regularly on MacBook, similar to what I do with starred items on Inoreader.

Other types of content

• Social media: I’ve already talked extensively about Twitter, and I also spend 5-10 minutes a day browsing Instagram and Reddit, but usually nothing beneficial on either of them.
• Paper books: I unfortunately don’t spend enough (or any, to be honest) time on reading books on paper. Although I’m covering books by reading ebooks and listening to audiobooks, but I’d love to add physical books to my days as well.
• E-books: I used to read e-books on my Kindle, but I find myself using the iPad for this.
• Audiobooks: For the same reason as liking to listen to articles, I sometimes prefer listening to books (obviously when the topic permits, meaning it’s not full of code or math formulas). I used Storytel for a few months, I’m currently using Spotify for listening to audiobooks.

Conclusion

I started using RSS and Inoreader less than 18 months ago, and I’ve built this workflow over time, based on learnings of what has worked and what hasn’t for me. I have experienced many benefits by doing this, including but not limited to:

• I don’t feel cluttered with content. The number of my read-later items on Pocket remains steady at around 30, and take-note items on Inoreader are somewhat steady at around 170 (yeah, I need to speed this up a bit). I have replaced my pile of content with a flow of content, I consume from the pile at the same rate that I store into it.
• By regularly exposing myself to high-quality content, I’ve been learning many new things consistently. The ratio of the “useful information” per “time spent” has been significantly higher for me following this workflow.
• I’ve been known as a person who shares useful and interesting content with friends and colleagues. I have received much positive feedback at work appreciating me sharing articles and my findings.

The workflow will surely change over time, as tools die and new tools emerge, and also as I as a person change. I’ll update this post in the future with notes if I make changes to any parts of this workflow. Please reach out to me if you have any questions, or if you have any recommended tools that you think would fit or improve my workflow.

Comment? Reply via Email, Mastodon or Twitter.

Here is a summary of the steps required to migrate a blog from Gatsby to Hugo:

Create a new Hugo site

My blog’s code sits in a Github repo and the content is served by Netlify using a direct connection to the repo. Some of my existing blog posts have a decent amount of seach traffic, so I didn’t want to mess up the URLs with this migration. I created a new Hugo site on my local machine and deploed it to a new Netlify subdomain, configured a few things to make sure the migration of the existing posts will be smooth, and then updated my domain to serve the new website.

• Install Hugo :

On Mac:

brew install hugo

Note that you don’t need to have Go installed for this migration.

• Create a new Hugo site:

Replace mywebsite with name of your website.

hugo new site mywebsite --format yaml

Choose and add a theme

I decided to use PaperMod as my theme, but there are many options you can choose from.

• Run the following two commands inside the folder of your Hugo site:

git submodule add --depth=1 https://github.com/adityatelange/hugo-PaperMod.git themes/PaperMod
git submodule update --init --recursive

• Add following to hugo.yaml:

theme: ["PaperMod"]

Move the existing content

My existing blog posts on Gatsby had a <domain>/<post-slug> URL format (for example: https://saeedesmaili.com/topic-classification-of-texts-locally/), while Hugo by defauls uses a /posts prefix, so if I just created new posts the URL would be <domain>/posts/<post-slug>. I didn’t want to deal with redirecting the old URLs, and decided to use the url frontmatter param in Hugo to enforce the same URL schema for my existing content (but will use the default /posts/<post-slug> for the new content, as you can see in the URL of this blog post).

• Define a default frontmatter format for Hugo posts:

Since I just had a handful of existing blog content, I decided to create a new post content on Hugo and copy paste the markdown from Gatsby for every post. Hugo lets you define a yaml frontmatter template for new content, so you don’t need to write title, date, etc for each post you create. To do this, I added the following in a new file called default.md in the archetypes directory:

---
title: "{{ replaceRE `^[0-9]{8}-` "" .File.ContentBaseName | humanize | title }}"
slug: "{{ replaceRE `^[0-9]{8}-` "" .File.ContentBaseName }}"
date: "{{ .Date }}"
draft: false
---

The replaceRE `^[0-9]{8}-` "" .File.ContentBaseName part trims eight digits and a - from the beginning of the post slug. I added this to be able to prepend the post creation date to my markdown file names, but excluding them from post title and slugs. This way the markdown files will be sorted in the directory instead of being all over the place.

• Then, to create a new Hugo post content:

hugo new content posts/20240201-post-title.md

This creates a new file called 20240201-post-title.md in content/posts/ directory with the following content:

---
title: "Post Title"
url: "post-title"
date: "2024-02-01T11:30:57+01:00"
draft: false
---

I copy pasted the markdown content of my posts to this file, and since both Gatsby and Hugo use markdown, the post content migration was smooth, expect for the images and local files that I need to move as well.

• Move images and files: The images of Gatsby posts reside in the same directory as the post’s markdown file:

And Hugo supports this exact file structure as well, but I preferred to opt for storing all the images in one single directory (which I called it images) in the static directory. For each existing post, I moved the image file to the static/images/ directory, and prepended the post creation date to the file name. Then in the post content, using the VSCode’s search and replace feature, replaced /images/ with /static/images/2024/20240201-. Adding the date to the image file name will make it easy to find them in future if needed.

And that’s it! All existing content are moved to Hugo and I could deploy to Netlify, but I also made a few other config changes as well.

Other configurations

• Rename the RSS file to rss.xml:

The RSS file in Gatsby uses rss.xml name, but Hugo uses index.xml. This means people would need to know about this change, and to update the url in their feed reader, and to be honest I’m not sure how many people are subscribed to my blogs feed (Are you? Porbably not many), but I wanted to keep the old name and retain the subscribers. So I added the following to hugo.yaml:

outputFormats:
  RSS:
    mediatype: "application/rss"
    baseName: "rss"

• Include the full text of posts in the RSS feed:

As I don’t want to bother the readers who follow me via RSS, I added the following config to hugo.yaml to include the full text of my posts in the RSS feed:

params:
  ...
  ShowFullTextinRSS: true

• Use a serif font instead of the default sans-serif one:

I like to use a serif font wherever there is a lengthy content to read, so I added a regular, bold, italic, and bold-italic versions of the IBMPlexSerif font to the static/fonts/ directory, then created a file named custom.css in assets/css/extented/ with following:

@font-face {
  font-family: "IBM Plex Serif";
  src: url("/fonts/IBMPlexSerif-Regular.woff2") format("woff2"), url("/fonts/IBMPlexSerif-Regular.woff")
      format("woff");
  font-weight: normal;
  font-style: normal;
}

@font-face {
  font-family: "IBM Plex Serif";
  src: url("/fonts/IBMPlexSerif-Bold.woff2") format("woff2"), url("/fonts/IBMPlexSerif-Bold.woff")
      format("woff");
  font-weight: bold;
  font-style: normal;
}

@font-face {
  font-family: "IBM Plex Serif";
  src: url("/fonts/IBMPlexSerif-Italic.woff2") format("woff2"), url("/fonts/IBMPlexSerif-Italic.woff")
      format("woff");
  font-weight: normal;
  font-style: italic;
}

@font-face {
  font-family: "IBM Plex Serif";
  src: url("/fonts/IBMPlexSerif-BoldItalic.woff2") format("woff2"), url("/fonts/IBMPlexSerif-BoldItalic.woff")
      format("woff");
  font-weight: bold;
  font-style: italic;
}

body {
  font-family: "IBM Plex Serif", "Merriweather", "Georgia", "Cambria",
    "Times New Roman", "Times", "serif";
}

• Add analytics script:

I use SimpleAnalytics to gather some basic data around which contents of this blog are more useful for people. I created a file called extend_head.html in layouts/partials/ directory and added their javascript tag in the file.

Deploy!

I then created a new website on Netlify, connected the Hugo site’s repo from Github, and after playing around with the website on Netlify’s subdomain and making sure everything works fine, I removed my domain from the existing Gatsby website on Netlify, and added it to the new Hugo one. And with that, the website was live with Hugo!

PS. One of the main reasons for doing all these and migrating to Hugo was to be able to have a new notes section on this blog, and publish my brief notes on what I read and discover every week, in a relatively automatic way (let me know if you’re interested in a write up on how I automated sharing these notes with Readwise , Obsidian , Alfred , and Hugo). These notes are not displayed on this blog’s home page, but will be accessable on the /notes section and this section has its own RSS feed , so make sure to check it out and subscribe to the feed if you’re interested.

Comment? Reply via Email, Mastodon or Twitter.

Something very interesting that I’ve recently stumbled upon is an xkcd chart style for matplotlib . This is a fantastic style that I can see myself using frequently going forward. All you need to do is add plt.xkcd() to your existing matplotlib code.

I played around with this style and generated a few examples. Note that currently there is an issue that causes matplotlib to print a hundred lines of warnings when using the xkcd style, but you can get rid of them by disabling matplotlib.font_manager warnings.

Examples

Let’s first see what our chart would look like out of the box if we used the default matplotlib style:

import matplotlib.pyplot as plt
import numpy as np
import logging

logging.getLogger("matplotlib.font_manager").disabled = True

## some dummy data
months = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"
]
sales = [12, 15, 18, 22, 20, 24, 26, 25, 23, 20, 18, 16]

## our plot
plt.plot(months, sales, marker="o", linestyle="-")
plt.title("Monthly Sales Performance")
plt.ylabel("Sales (in $1000s)")
plt.annotate(
        "THE DAY WE CHANGED\nHOW WE RECOMMEND\nNEW PRODUCTS",
        xy=("May", 20),
        arrowprops=dict(arrowstyle="->"),
        xytext=("May", 15),
    )
plt.show()

What if we use the xkcd style?

## our plot
plt.xkcd() ## just add this line
plt.plot(months, sales, marker="o", linestyle="-")
plt.title("Monthly Sales Performance")
plt.ylabel("Sales (in $1000s)")
plt.annotate(
        "THE DAY WE CHANGED\nHOW WE RECOMMEND\nNEW PRODUCTS",
        xy=("May", 20),
        arrowprops=dict(arrowstyle="->"),
        xytext=("May", 15),
    )
plt.show()

Much better in my opinion. Let’s see a few other examples:

import random

random.seed(10)
exam_scores = [
    random.normalvariate(70,10,)
    for _ in range(100)
]

plt.xkcd() ## default vs xkcd
plt.hist(exam_scores, bins=20)
plt.title("Exam Scores Distribution")
plt.xlabel("Scores")
plt.ylabel("Frequency")
plt.annotate(
    "OUR BEST\nSTUDENT",
    xy=(97, 1),
    arrowprops=dict(arrowstyle="->"),
    xytext=(87, 8),
)
plt.show()

hours_studied = [random.uniform(1, 10) for _ in range(50)]
exam_scores = [60 + 2 * hours + random.normalvariate(0, 5) for hours in hours_studied]

plt.xkcd() ## default vs xkcd
plt.scatter(hours_studied, exam_scores, color="green", alpha=0.7)
plt.title("Scatterplot of Hours Studied vs Exam Scores")
plt.xlabel("Hours Studied")
plt.ylabel("Exam Scores")
plt.show()

Comment? Reply via Email, Mastodon or Twitter.

What’s Alfred

Alfred is an app for macOS which boosts your efficiency with hotkeys, keywords, text expansion and more. Search your Mac and the web, and be more productive with custom actions to control your Mac.

You can replace it with Spotlight on macOS (by setting the cmd+space hotkey for Alfred and disabling Spotlight). Let’s look at some of Alfred’s features. Most of the features mentioned below require the paid version of the app .

Clipboard manager

Configured from Features > Clipboard History

I’ve set the Viewer Hotkey to cmd+shift+V:

You can access the history of copied items with the same hotkey anywhere on your Mac:

As you can see in the above screenshot, it can keep the copied images, shows where each item is copied from (Chrome, Firefox, Slack, etc), at what time each item is copied, and you can paste one of the items with cmd+1, cmd+2, etc, or browse in the list with up and down arrow keys.

Text snippets

Configured from Features > Snippets

Are there specific texts that you use over and over? Like a code snippet, an email or web address, your home address, etc? Using the Snippets feature of Alfred, you can set a specific keyword for a text snippet, and when you type that keyword anywhere in your mac, it will automatically be replaced with the text snippet. In the example below, I have created an snippet for :bqt keyword, and when I type this keyword somewhere, it gets replaced with a code snippet for querying BigQuery:

You can organize snippets in collections as you can see in the screenshot above.

Web search

Configured from Features > Web Search

Is there a website where you find yourself using its search functionality a lot? Like searching for a book on Goodreads or Storytel, locations on Google Maps, or word definitions on dictionary or thesaurus? You can set custom actions to search on any website. For example, I have one set for archive.org, so when I encounter a paywalled article, I copy the url and open Alfred, then use archive <cmd+v to paste the url> and a new tab on my browser opens with the archived version of the url. I also have custom keywords for searching in Goodreads, Storytel, Spotify web, dictionary, thesaurus, pypi, devdocs, etc.

To set a new one you just need to know where in the URL the target website replaces the searched keyword:

File search and file actions

Configured from Features > File Search and Features > Default Results

Like Spotlight, you can search for files and applications available on the Mac with Alfred. But Alfred enables many customizations on which directories and file types the search should be limited to. In addition, after finding a file with Alfred, you can use the right arrow key on the keyboard to perform various actions on the file without opening it. In the example below, I can easily rename, delete, copy, move, get info, or perform many more actions (not all of the visible in the screenshot) on the PDF file I’ve searched for with Alfred.

Workflows

Configured from Workflows

This one is the most dangerous rabbit hole on Alfred! Basically the workflows in Alfred allows you to create your own simple or complex “workflows” for specific actions.

But thanks to the amazing community of Alfred users, you almost don’t need to worry about the complexity of creating a workflow, because there is a workflow already created for almost every task. You just download a workflow from Alfred’s workflow gallery (or anywhere else that the creator has shared it), open it, enable it, and use it without ever looking at the workflow editor.

A few workflows I use almost every day:

• Recent Files (e.g. when I download a file and need to perform an action immediately)
• OCR (when I need to copy a text from an image or an app which doesn’t allow selecting texts)
• VSCodeDiff (to see the difference between two copied texts)
• Bluetooth Device Battery (to see if my headsets and trackpad need to be charged)
• System Settings (to open the macOS settings in the page I want to do something on)

A few others that I use often but not every day:

• Quil (to perform actions on text, like changing the whole text to lowercase, uppercase, snake case, title case, or counting the number of words in a text)
• TinyPNG (to compress an image file)
• Temporary Email (to create a temporary email address to sign up to a garbage site that I have to for some reason, but I also need to click on the email they send)
• ChatFred (haven’t used it myself yet, since I’m using other ways to interact with ChatGPT API, but this is one of the easiest ways to ask a quick question from ChatGPT without opening anything)
• StrongPassword (to generate a random sequence of characters to use as password or anything similar)
• Shimmering Obsidian (to open a note on Obsidian . The search works surprisingly well, much better than Obsidian’s own search)

Comment? Reply via Email, Mastodon or Twitter.

Fundamentals

Manifest

The manifest is the rulebook for the browser extension. Every extension is required to provide this manifest as JSON data in a manifest.json file. Some examples of what is contained in the manifest:

• Public information such as the extension name, description, semantic version, icons, and author.
• Pointers to entrypoint files for background scripts, popup pages, options pages, devtools pages, and content scripts.
• Requirements and configurations the extension needs to properly operate, such as permissions, content security policies, cross-origin policies, and minimum browser versions.
• Pattern-match rule sets for managing network requests, enabled domains, and resources the extension wishes to inject into the page via content script.
• Miscellaneous extension-specific options like enabling offline and incognito and keyboard shortcuts.

A sample manifest.josn:

{
  "name": "Extension Name",
  "description": "Browser extension created from scratch",
  "version": "1.0",
  "manifest_version": 3,
  "icons": {
    "16": "images/icon-16.png",
    "32": "images/icon-32.png",
    "48": "images/icon-48.png",
    "128": "images/icon-128.png"
  },
  "background": {
    "service_worker": "scripts/background.js"
  },
  "action": {
    "default_popup": "popup/index.html"
  },
  "options_page": "options/options.html",
  "content_scripts": [
    {
      "js": ["scripts/content.js"],
      "matches": ["*://example.com/*"],
      "run_at": "document_end"
    }
  ],
  "permissions": ["activeTab", "scripting"]
}

Background scripts

The primary purpose of background scripts is to handle browser events. These events might be extension lifecycle events such as an install or uninstall, or they might be browser events like navigating to a web page or adding a new bookmark.

They can access the WebExtensions APIs and therefore are capable of performing actions such as exchanging messages with other parts of the extension, exchanging messages with other extensions, or programmatically injecting content scripts into a page.

To inspect the background service worker, use the service workder link in the extension page in the browser’s settings. This will open a new inspection window.

If the action parameter in the manifest.json file left empty, action will be handled by background script:

// manifest.json
"action": {},
"background": {
  "service_worker": "background.js"
}

// background.js
chrome.action.onClicked.addListener(() => {
  console.log("Clicked toolbar icon!");
});

Popup page

The popup page is a native browser container that browser extensions can use to display a custom user interface. The popup page behaves as a dialog box that “pops” over the web page when the user clicks the extension toolbar button. The popup page will always appear directly below the toolbar. Because it can be quickly accessed and can show over any web page, the popup page typically contains content that users need easy access to.

Popup pages are rendered just like regular web pages, but their dialog-like nature means they are disposable: the popup will be freshly rendered each time the popup is opened, and unloaded when the popup is closed. Like background scripts, popup pages can access the WebExtensions API, meaning they have the same set of capabilities.

The popup page cannot be opened programmatically, it must be triggered by a toolbar click or similar privileged browser action.

To open a popup page when user clicks on the extension icon in toolbar, add the following to manifest.json:

"action": {
  "default_popup": "popup/popup.html"
}

Options page

The options page is a native browser container that browser extensions can use to display a custom user interface. The options page behaves as a standalone web page that opens when the user clicks “Options” in the extension toolbar context menu.

Like the popup page, this is a fully featured web page with access to the WebExtensions API, meaning you are capable of using it as a full web application.

To add an options page in manifest.json:

"options_page": "options/options.html"

Unlike the popup, the options page can be opened programmatically, but content scripts can’t open an option page. The following actions will open the options page:

• Right clicking the toolbar icon button and selecting Options
• Calling chrome.runtime.openOptionsPage()

Content scripts

The term content script broadly refers to any content that is injected into a web page. JavaScript can either be injected declaratively in the manifest, or programmatically from an extension page or background script via the WebExtensions API. This content can be JavaScript, CSS, or both.

Content scripts are fully capable of reading and writing the page, enabling things like in-page widgets or full integration with the web page. Content scripts have limited access to the WebExtensions API, so they are incapable of many actions that are possible in the popup page, options page, or background script. They can, however, exchanges messages with other extension elements like background scripts. Therefore, content scripts can still indirectly use the WebExtensions API by delegating actions to a background script.

Development

Installation and reload

To install an extension, open the extensions page in the browser and use Load unpacked option. After making changes to the extension scripts, you should hit the reload icon in the extensions page for the changes to be used.

Keyboard shortcuts

commands is used to add keyboard shortcut support in an extension:

// manifest.json
"commands": { 
  "foobar": { 
    "suggested_key": { 
      "default": "Ctrl+Shift+J", 
      "mac": "MacCtrl+Shift+J" 
  }, 
  "description": "Perform foobar action" 
  } 
}

Programatic injection of content scripts

The manifest is not the only way to inject content scripts. It is also possible to programmatically inject JavaScript and CSS into the page using the chrome.scripting API.

// background.js
chrome.action.onClicked.addListener((tab) => {
  const target = {
    tabId: tab.id,
  };
  chrome.scripting.executeScript({
    target,
    func: () => {
      document.body.innerHTML = `Hello, world!`;
    },
    // or: files: ["content-script.js"],
  });
  chrome.scripting.insertCSS({
    target,
    css: `body { background-color: red !important; }`,
    // or: files: ["content-script.css"],
  });
});

WebExtensions API

Permissions

Inspect current permissions and add or remove permissions. More: chrome.permissions

Messaging

Send messages between parts of the extension, or between the extension and an outside entity. More: Chrome Extensions Message passing , chrome.runtime , chrome.tabs

Storage

A simple but powerful key/value storage. More: chrome.storage

• storage.local stores values only on the local browser
• storage.sync stores values that are shared between the authenticated browser session
• storage.session stores values in memory that will be discarded when the browser closes
• storage.managed is a read-only store intended for enterprise users

chrome.storage.onChanged.addListener(console.log);

await chrome.storage.local.set({ foo: "tmp" });
// { foo: { newValue "tmp" } }

await chrome.storage.local.set({ foo: "bar", baz: "qux" });
// {
//   foo: { oldValue: "tmp", newValue "bar" }
//   baz: { newValue "qux" }
// }

console.log(await chrome.storage.local.get(["foo", "baz"]));
// { foo: "bar", baz: "qux" }

Authentication

Manage authentication state and OAuth flows. More: chrome.identity

Network requests

More: chrome.declarativeNetRequest , chrome.webRequest , chrome.webNavigation

• declarativeNetRequest is used to declaratively instruct the browser to manage page traffic according to a set of rules defined by the extension.
• webRequest is used to imperatively control page traffic with JavaScript handlers.
• webNavigation is used to inspect tab-level navigation events.

Action

Control the extension toolbar icon, including appearance, title text, badge content, badge color, popup page, and click handlers. More: chrome.action

Notifications

Display rich notifications to the user using the host operating system’s notification mechanism. More: chrome.notifications

Context menu

Add interactive options to right click and context menus throughout the browser. More: chrome.contextMenus

Authentication

Content script spoofing

Network requests from a content script are treated the same as network requests from a host page script. Therefore, if a website uses HttpOnly cookie authentication, a content script will be able to piggyback on that authenticated state and send requests that will automatically include those cookies.

Cookie authentication

Technically possible, but not recommended.

JWT authentication

The ideal scenario in browser extensions. Popups, options pages, and background scripts can all directly authenticate with a server and share the authentication token between extension components without issue.

OAuth and OpenID

With the chrome.identity API, browser extensions have first-class support for OAuth and OpenID. There are two ways of using OAuth in an extension:

• chrome.identity.getAuthToken() allows you to natively authenticate. This method allows you to skip providing a redirect URL and making the authorization token request. Instead, you simply provide the required values inside the OAuth2 manifest property and call this method. The browser will kick open the OAuth dialog and the method’s callback will be passed the OAuth token. This is by far the simplest way of implementing OAuth in an extension, but it is only available inside Google Chrome. This requires a “Chrome app” client ID.
• chrome.identity.launchWebAuthFlow() is the more generalized method for using OAuth2. It is cross-browser (works on Firefox, Edge, etc.) and cross-platform (supports OAuth2 with Facebook, Github, etc.). It is much more labor intensive, as it requires you to manually implement each of the OAuth2 steps.

To solve the OAuth redirect URL problem, the browser supports a special URL (accessible with chrome.identity.getRedirectURL() method) that will direct the auth flow back to the extension.

Comment? Reply via Email, Mastodon or Twitter.

While investigating how to enrich this text data with proper metadata related to their topics, I came across BERTopic which introduces itself as a topic modeling technique to create clusters allowing for easily interpretable topics. In this post, I’ll explore BERTopic and will go through an example to explain what adjustments worked for me.

BERTopic

BERTopic leverages a few popular NLP and clustering algorithms and libraries to classify raw texts into meaningful topics. It reshapes the texts in a sequence of steps, including embedding, dimensionality reduction, clustering, tokenization, and weighting words (explained in more detail in its docs ):

BERTopic is fundamentally built upon BERT embeddings, one of the leading NLP techniques. While traditional topic modeling techniques, like LDA rely on probabilistic graphical models to identify topics, BERTopic uses the rich, contextual embeddings from BERT to cluster similar texts. This offers two main advantages:

1. Contextual Understanding: BERT’s deep understanding of context means that words with multiple meanings can be more appropriately clustered based on their actual usage in texts, leading to more accurate topic allocations.
2. Flexibility: BERTopic is not just confined to English. Thanks to the multilingual capabilities of BERT, it can be adapted for several languages without significant changes in the approach.

The choice of BERTopic over traditional methods largely depends on the specific requirements of a project, but its modern and deep learning-based approach can often yield more nuanced and contextually relevant results.

Installing the library and preparing the sample data

Start by installing the required Python libraries and importing them:

pip install bertopic sentence-transformers pandas scikit-learn

import pandas as pd
from bertopic import BERTopic
from sentence_transformers import SentenceTransformer
from sklearn.feature_extraction.text import CountVectorizer
import pickle

I will use the app reviews dataset from Android Apps and User Feedback project published by researchers at the University of Zurich. This dataset consists of 288K reviews extracted from Google Play, but as it makes more sense to limit this exercise to one single app and see the topics of reviews submitted for it, I have picked Telegram:

df_orig = pd.read_csv("reviews.csv")
df_orig.head()

## filtering data on Telegram
df = df_orig[df_orig["package_name"] == "org.telegram.messenger"]

## Getting rid of reviews like 'nice', 'good', etc by keeping only the reviews with at least 10 words
df["n_words"] = df["review"].apply(lambda x: len(x.split(" ")))
df = df[df["n_words"] >= 10]

Generating and storing Embeddings

The first step in clustering our text data is to convert them to some sort of numerical representation. This can be done using publicly available models, and BERTopic uses the famous sentence-transformers library in this step of the process. If you’re interested in knowing more about how sentence-transformers works, check out their docs or take a look at my blog post which is a brief explanation of how to use this library for embedding texts locally.

We don’t necessarily need to import sentence-transformers and do the embeddings step outside the topic generation step as it’s managed by the BERTopic library behind the scenes. But since this is a computationally costly process, we will generate the embeddings once and will store them in our machine’s memory or disk, and pass it to our topic generation model as many times as we want. This will enable us to experiment with various hyperparameters of BERTopic quickly and conveniently.

docs = df["review"].values.tolist()
print(len(docs))
docs[:5]

## Pre-calculating embeddings
embedding_model = SentenceTransformer("all-MiniLM-L6-v2")
embeddings = embedding_model.encode(docs, show_progress_bar=True)

## Storing embeddings on the disk
with open("embeddings.pkl", "wb") as f:
    pickle.dump(embeddings, f)

## We can load the stored embeddings later on if needed
with open("embeddings.pkl", "rb") as f:
    embeddings = pickle.load(f)

Classify reviews into topics

We have another optional but beneficial step before generating topics. In my first attempt to classify texts into topics using BERTopic, meaningless words like “in, an, at, for, etc” had a significant presence in topic representations and some of the final categories had no semantic meaning. We can ask BERTopic to skip these words (which are called stopwords in the NLP world) by creating a vectorizer model and passing it to our topic generation model:

vectorizer_model = CountVectorizer(stop_words="english", ngram_range=(1, 1))

Use the following instead if you want to add some custom stopwords to be skipped:

from nltk.corpus import stopwords


default_stopwords = stopwords.words("english")
custom_stopwords = ["my", "custom", "list", "of", "words"]
stopwords_all = default_stopwords + custom_stopwords

vectorizer_model = CountVectorizer(stop_words=stopwords_all, ngram_range=(1, 1))

Here I’m using the default value for ngram_range (so it can also be dropped from the above code). ngram_range is used to determine the range of n-grams extracted by our topic generation model. In simple terms, if we have so many words with two or more words (like New York, tech debt, etc) and we want to capture them accordingly, ngram_range should probably be set to (1, 2) for example, but I experimented with this and final results always seemed more useful with ngram_range=(1, 1).

Now let’s classify the texts into topics:

topic_model = BERTopic(
  # Pipeline models
  embedding_model=embedding_model,
  vectorizer_model=vectorizer_model,

  # Hyperparameters
  top_n_words=10,
  verbose=True
)
topics, probs = topic_model.fit_transform(documents=docs, embeddings=embeddings)

topic_model.get_topic_info()

So we have 100 topics, but let’s see what pieces of information we have for each topic:

• Topic: A numerical ID for each topic. As you see in the picture above, the numbers start from -1. The texts in the -1 topic are not related to each other in a meaningful way, and the only reason they are in the same bucket is that BERTopic hasn’t been able to find proper topics for them.
• Count: Number of texts (rows in the dataframe) that are classified into this topic.
• Name: Numerical ID of the topic combined with the four most occurring words in this topic.
• Representation: These are the words that occur most often in the texts classified in this topic. The number of words in this list is determined by the top_n_words parameter above and it has a default value of 10. It is recommended to keep it low, otherwise you’ll get so few items classified in each topic and the number of outliers will increase.
• Representative_Docs: List of the texts that are classified into this topic.

To get a more practical view of how each text has been classified, we can use:

topic_model.get_document_info(docs)

And what are the representations (top words) of our main topics?

for i in range(5):
    print(topic_model.get_topic(i), "\n")

Finally, we can save our topic model to re-use later without computing them from scratch:

topic_model.save("topic-model", serialization="safetensors", save_ctfidf=True, save_embedding_model=embedding_model)

## Later load it with:
BERTopic.load("topic-model")

Reducing outliers

As we saw earlier, a significant number of texts have been classified as outliers (which means they don’t have any topics assigned to them). Upon manually examining a few of these outliers, they seemed to belong to one of the topics if we don’t want to be very strict about our criteria. BERTopic provides a simple way for reducing these outliers and re-assigning them to our existing topics:

new_topics = topic_model.reduce_outliers(documents=docs, topics=topics, strategy="embeddings", embeddings=embeddings)
topic_model.update_topics(docs=docs, topics=new_topics, vectorizer_model=vectorizer_model)

topic_model.get_topic_info()

There is no -1 topic anymore and all of the texts belong to one of the topic classifications now. Here I’ve used the embeddings strategy, but there are other strategies explained in BERTopic’s documentation .

Conclusion

In wrapping up, topic modeling is an incredibly powerful tool for extracting insights from large volumes of text data. With advancements in natural language processing, tools like BERTopic are making this task not only more accurate but also more interpretable. As we’ve walked through in this post, setting up and using BERTopic is relatively straightforward, and its adaptability means it can be fine-tuned for various datasets and requirements. If you’re dealing with a sizeable text dataset and are keen to unearth the hidden topics within, BERTopic is undoubtedly worth a try.

There are a bunch of hyperparameters that can be tweaked to improve the results, but in the end, it mainly depends on the use cases and what we’re trying to achieve. I’ll be experimenting more with BERTopic in the future, so let me know if you have a question you want to be answered about this approach for classifying texts.

Comment? Reply via Email, Mastodon or Twitter.

I then found a way to use pandoc with python, and this led me to spend a few hours trying it out to see if it improve my document parsing process and provide me more reliable results. And it did! In this post, I’ll walk you through how I used pandoc to load docx and md documents, create a table of content for each document, split the documents to paragraphs, and include the parent heading of each paragraph in the chunks, all using python.

Getting started

First of all, you need to make sure you have pandoc installed on your machine. I will bring pandoc to python using sh , but this can be achieved with pypandoc as well. To follow along with me, you can download these three sample docx files: 1 , 2 , 3 , and put them in the /corpus directory.

To read a document and get a plain text output we can use this simple line of code:

from sh import pandoc

path = "corpus/sample-doc-without-title.docx"

doc = str(pandoc(path, "-t", "plain", "--toc", "--standalone"))
print(doc)

path is the document we want to load and parse (it can be a docx, md, or so many other formats that pandoc supports). Using -t and plain we ask pandoc to convert the document to a plaintext format, and --toc and --standalone flags are needed to generate the nice table of content at the beginning of the output.

This is already a very nice and usable output, but I want to split the text into paragraphs and prepend the parent heading to each paragraph as Heading [SEP] paragraph text, so the important content of headings isn’t lost when I use semantic search.

Parsing the headings

I experimented with various docx and md files to find out the different types of TOC and text I could possibly get, and the main three types are:

• Docs with title and headings

plaintext output when printed:

raw plaintext:

Sample doc with title\n\n\n- Introduction\n - A child heading of introduction\n- Summary\n\nIntroduction\n\nLorem Ipsum is simply dummy text of the printing and typesetting\nindustry. Lorem Ipsum has been the industry’s standard dummy text ever\nsince the 1500s, when an unknown printer took a galley of type and\nscrambled it to make a type specimen book. It has survived not only five\ncenturies, but also the leap into electronic typesetting, remaining\nessentially unchanged. It was popularised in the 1960s with the release\nof Letraset sheets containing Lorem Ipsum passages, and more recently\nwith desktop publishing software like Aldus PageMaker including versions\nof Lorem Ipsum.\n

• Docs without title but with headings

plaintext output when printed:

raw plaintext:

• Introduction\n - A child heading of introduction\n- Summary\n\nIntroduction\n\nLorem Ipsum is simply dummy text of the printing and typesetting\nindustry. Lorem Ipsum has been the industry’s standard dummy text ever\nsince the 1500s, when an unknown printer took a galley of type and\nscrambled it to make a type specimen book. It has survived not only five\ncenturies, but also the leap into electronic typesetting, remaining\nessentially unchanged. It was popularised in the 1960s with the release\nof Letraset sheets containing Lorem Ipsum passages, and more recently\nwith desktop publishing software like Aldus PageMaker including versions\nof Lorem Ipsum.\n

• Docs without any title or headings

plaintext output when printed:

raw plaintext:

Lorem Ipsum is simply dummy text of the printing and typesetting\nindustry. Lorem Ipsum has been the industry’s standard dummy text ever\nsince the 1500s, when an unknown printer took a galley of type and\nscrambled it to make a type specimen book. It has survived not only five\ncenturies, but also the leap into electronic typesetting, remaining\nessentially unchanged. It was popularised in the 1960s with the release\n

This python function will identify the type of a document:

import re

def identify_doc_type(doc):
    '''
    categorizes a plaintext doc based on the format of the toc.
    '''
    if re.search(r'.*\n\n\n-\s{3}.*', doc):
        return "TOC_WITH_TITLE"
    elif re.search(r'-\s{3}.*\n\n.*', doc):
        return "TOC_WITHOUT_TITLE"
    else:
        return "NO_TOC_TITLE"

And it can be used for splitting the table of content from the text when reading the doc:

def read_doc(path):
    '''
    reads a text file and returns toc and full text.
    '''
    doc = str(pandoc(path, "-t", "plain", "--toc", "--standalone"))
    doc_type = identify_doc_type(doc)

    if doc_type == "TOC_WITH_TITLE":
        doc = re.sub('.*\n\n\n-', '-', doc)
        toc, text = doc.split('\n\n', 1)
    elif doc_type == "TOC_WITHOUT_TITLE":
        toc, text = doc.split('\n\n', 1)
    else:
        toc, text = "", doc

    return toc, text

And it’s always nice to clean up the paragraphs to remove unnecessary new line characters, images, etc:

def cleanup_plaintext(text):
    '''
    gets the full text of a document and returns cleaned-up text.
    '''
    # Remove images
    text = text.replace("[image]", "")
    text = text.replace("[]", "")

    # Replace single \n with space (if the next char is not \n or -)
    text = re.sub('(?<!\n)\n(?!(\n|-))', ' ', text)

    # Replace any sequence of two or more newlines with \n\n
    text = re.sub('\n{2,}', '\n\n', text)

    # Replace multiple spaces with single space
    text = re.sub('(?<!\n) +', ' ', text)
    return text

Now that we know what are the headings of our document, we can split the text into paragraphs while prepending the parent headings:

def split_text(toc, text):
    '''
    gets the toc and cleaned text, and returns chunks of texts:
    ["Heading [SEP] Text", ]
    '''
    headings = [line.strip('- \n') for line in toc.split('\n')]
    paragraphs = text.split("\n\n")

    current_heading = ""
    list_group = ""
    text_chunks = []
    for para in paragraphs:

        # use the new heading if we've moved to a new heading section
        if len(headings) > 0 and para == headings[0]:
            current_heading = headings[0]
            headings.pop(0)
            continue

        # group bullet points as a single chunk of text
        if para.startswith("- "):
            list_group += para + " "
            continue
        elif list_group != "":
            para = list_group
            list_group = ""

        # if we're at the beginning of a document and
        # we haven't seen any headings yet
        if current_heading == "":
            text_chunks.append(para.strip())
        else:
            text_chunks.append(f"{current_heading} [SEP] {para}".strip())

    return text_chunks

Note that this will group bullet point lists in a single paragraph, as their content most probably is related to each other.

Let’s finalize this process and get the text chunks for all the documents in a directory and store them in a pandas dataframe:

import os
import pandas as pd

df = pd.DataFrame()
root_dir = 'corpus'
allowed_filetypes = ['.md', '.docx', '.pdf']

for directory, subdirectories, files in os.walk(root_dir):
    for file in files:
        filename, filetype = os.path.splitext(file)
        if filetype in allowed_filetypes:
            full_path = os.path.join(directory, file)

            toc, text = read_doc(full_path)
            text_cleaned = cleanup_plaintext(text)
            text_chunks = split_text(toc, text_cleaned)

            df_new = pd.DataFrame(text_chunks, columns=["text"])
            df_new[["directory", "filename", "filetype"]] = directory, filename, filetype
            df = pd.concat([df, df_new])

df.reset_index(drop=True, inplace=True)

The result is a nice dataframe with texts ready to be converted to vectors using an embedding model :

Are we done?

Our cleaning and splitting process still can be improved, as there are a few issues with the current approach:

• We should parse or remove footnotes and their references in the plaintext output. They appear in the text as [N] and at the end of the text as [N] some footnote text).
• We can preserve even more heading information in the final text chunks by including all the parent headings (e.g. Heading 1 [SEP] Heading 2 [SEP] Paragraph text).
• We can include filenames in the final text chunks, as they also contain useful information.
• The code blocks (in md files for example) are grouped as normal text paragraphs. This is a good enough approach for my documents, but it’s not ideal.
• If our documents include tables, our cleanup process considers them as normal text paragraphs. So tables end up as chunks with so many dashed lines and a more robust parsing approach is needed to preserve their information.
• We’re not considering the length of each chunk when splitting the text. Given that each embedding model has a specific max token size, we should add another step to split the chunks if they are larger than a specific size.

Please reach out if you have suggestions for any of these problems or any improvement ideas for my parsing functions.

Comment? Reply via Email, Mastodon or Twitter.

To handle such unstructured data, I’ve found the unstructured Python library to be extremely useful. It’s a flexible tool that works with various document formats, including Markdown, , XML, and HTML documents.

Starting with unstructured

You can easily install the library by:

pip install unstructured

Loading and partitioning a document

The first thing you’ll want to do with your document is split it up into smaller parts or sections. This process, called partitioning, makes it easier to categorize and extract text.

Here’s how you do it:

from unstructured.partition.auto import partition

elements = partition(filename="example-docs/note.md")

example-docs/note.md:

When we partition a document, the output is a list of document Element objects. These element objects represent different components of the source document. The unstructured library supports various element types including Title, NarrativeText, and ListItem. To access the element type you can use the category method:

for element in elements:
    print(f"{element.category}:")
    print(element)
    print("\n")

Output:

The list of document elements can be converted to a list of dictionaries using the convert_to_dict function:

from unstructured.staging.base import convert_to_dict

dict_data = convert_to_dict(elements)

Output:

But since I want to store the chunks of texts in a database and do some exploratory analysis with the data, I used convert_to_dataframe function to convert the text elements into pandas dataframe:

from unstructured.staging.base import convert_to_dataframe

df = convert_to_dataframe(elements)

Gettng the metadata

One neat feature of the unstructured library is how it keeps track of various metadata about the elements it extracts from documents. For example, you might want to know which elements come from which page number. You can extract the metadata for a given document element like so:

doc_metadata = elements[0].metadata.to_dict()
print(doc_metadata)

Output:

All document types return the following metadata fields when the information is available from the source file: filename, file_directory, date, filetype, andpage_number.

Preparing for Transformers

When you’re ready to feed your text into a transformer model for further processing, you can use the stage_for_transformers function. This function prepares your text elements by splitting them into chunks that fit into the model’s attention window.

In the following example, I’m using a library called SentenceTransformers (I’ve written more about using this library in my previous blog post ):

from sentence_transformers import SentenceTransformer
from unstructured.staging.huggingface import stage_for_transformers

model = SentenceTransformer("all-MiniLM-L6-v2")
chunked_elements = stage_for_transformers(elements, model.tokenizer)

And now I can load all the notes in a specific directory, so I can convert them to embedding vectors later:

all_elements = []
root_dir = '/corpus'

for directory, subdirectories, files in os.walk(root_dir):
    for file in files:
        full_path = os.path.join(directory, file)
        all_elements += partition(filename=full_path)

Limitations of unstructured

This library has some issues and limitations as well.

• When loading and parsing docx files, it can’t properly recognize bullet points as ListItem and most of the times labels them as NarrativeText or Title. This makes the Title recognition unreliable as well, since when you look into the output, you can’t tell for sure if each Title is actually a title or a list item labeled incorrectly as a Title. (issue on github )
• When working with large documents, there isn’t any way to know what are the parents of each paragraph or title. This could be a very useful feature to have, specially when feeding back the data to an LLM. (issue on github )

Alternatives

After playing with unstructured I tried to see if there are better alternatives for reading documents with python. Although I will need to load documents with various formats, I narrowed down my search to first find alternatives for reading docx files first (as this it the format you get when downloading a large folder of documents from Google Drive). Here are what I found:

python-docx

• It seems powerful, but it’s complicated to work with.
• I tried loading and parsing a few docx files. The biggest issue I experienced was with loading any text that includes hyperlinks. For some unknown reason, the texts of hyperlinks are returned empty in the final output. This makes it unusable for my purpose, since the link texts provide valuable information in the text.
• Pro: It is able to provide the heading level info for titles (as Heading 1, Heading 2, etc).

docx2txt

• It uses python-docx under the hood.
• Only returns a giant full text string of the loaded document. This would require me to split my documents to meaningful chunks, which is not a trivial task to do.
• Pro: It doesn’t have any problems with hyperlinks and the output text is readable and useful.
• Pro: It is also very easy to use.

simplify_docx

• It works on top of python-docx.
• This library basically converts the complicated output of python-docx to a more easy to use json output.
• It also have the same issue with hyperlinks and returns empty texts when there is a link in the paragraph.

So I will continue using unstructured for now. It’s worth mentioning that, yes, this could be accomplished more easily using LangChain or other similar tools. However, part of my motivation in building this personal AI assistant is the learning journey. By using unstructured to load documents and other similar tools for embeddings and so on, I’m gaining a deeper understanding of the underlying processes, rather than using a one-stop-shop solution like LangChain.

I’ll be sharing more about the progress I’m making in building this personal AI assistant in future posts, so stay tuned.

Update 2023-07-07:

Using pandoc

Raphael suggested using pandoc with to extract the texts from documents. I wasn’t familiar with zpandacz and tried playing with it and tweaked Raphael’s suggestion and was able to get plaintext from docx files using:

from sh import pandoc

plaintext_output = str(pandoc("path_to_docx_file.docx", to="plain"))

This returns a very clean plaintext from any document, so you can give it a markdown or any other text file and get the same plain text. I just need to remove th footnote references (currently shown as [1], [2], etc in the plaintext output) to get an actual plaintext, but I think this is as good as we can get with extracting text from docx and markdown files.

Comment? Reply via Email, Mastodon or Twitter.

Choosing the Right Model

I was mostly intrested in using light-weight models, since I’m planning to run this on my own laptop or some cheap VPS. After looking at the available models in sentence-transformers, I chose all-MiniLM-L6-v2 model. It’s a small BERT-based model which is a MiniLM model fine tuned on a large dataset of over 1 billion training pairs. It maps sentences and paragraphs to a 384 dimensional dense vector space and can be used for tasks like clustering or semantic search. A few other models I considered were all-mpnet-base-v2, e5-base-v2, and hkunlp/instructor-xl but they are larger and heavier to run.

Text Length Matters

It’s crucial to note that for Transformer models like BERT, runtime and memory requirements increase quadratically with input length. So there’s a limit on the length of inputs these models can handle. Any text that exceeds the specific limit of the model gets truncated to the first N word pieces.

Using sentence-transformers

If you have already installed the library (pip install sentence-transformers), here’s how you can load a sentence-transformer model and adjust the maximum sequence length:

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("all-MiniLM-L6-v2")
print(model.max_seq_length)

model.max_seq_length = 256

Generating embeddings

Generating embeddings for sentences is quite straightforward:

# Our sentences we like to encode
sentences = ["this is the first sample sentence", "this is the second sample sentence"]

# Sentences are encoded by calling model.encode()
embeddings = model.encode(sentences, normalize_embeddings=True)

Setting normalize_embeddings to True ensures the returned vectors have a length of 1. This allows you to use the faster dot-product instead of cosine similarity.

Searching for Similar Sentences

Here’s how you can identify the most similar sentences to your query within a corpus:

import torch

# Corpus with example sentences
corpus = ['A man is eating food.',
          'A man is eating a piece of bread.',
          'The girl is carrying a baby.',
          'A man is riding a horse.',
          'A woman is playing violin.',
          'Two men pushed carts through the woods.',
          'A man is riding a white horse on an enclosed ground.',
          'A monkey is playing drums.',
          'A cheetah is running behind its prey.'
          ]
corpus_embeddings = model.encode(corpus, normalize_embeddings=True)

query = "What is the man eating?"
query_embedding = model.encode(query, normalize_embeddings=True)

# Since the embeddings are normalized, we can use dot_score to find the highest 5 scores
dot_scores = util.dot_score(query_embedding, corpus_embeddings)[0]
top_results = torch.topk(dot_scores, k=5)

for score, idx in zip(top_results[0], top_results[1]):
    print(corpus[idx], "(Score: {:.4f})".format(score))

It will give you the following:

My Use Case and Observations

I tested this approach with my dataset of approximately 75,000 short text snippets, intending to identify the most relevant snippets for a given query. While generating embeddings took roughly 25 minutes on my Macbook Air — significantly slower than using OpenAI — I found this acceptable since I only needed to perform this process once. Another limitation of running this locally is the maximum input lengthof 256 tokens (for the all-MiniLM-L6-v2 model). However, this wasn’t an issue for me as my text snippets were short.

I’m also thinking about using this method to create a local LLM assistant, that can look at my private notes and documents on my laptop or my own server. This way, it’s much cheaper than using a service like OpenAI and I don’t have to worry about my text data privacy. Overall, I think for many use cases where you have a smaller dataset and don’t need real-time responses, running your own sentence transformer model can be a viable option.

Comment? Reply via Email, Mastodon or Twitter.

For quite a while now, I’ve been using Pocket as my go-to read-it-later app. A few months back, I found myself wanting a solution to listen to my saved articles. This led me to explore the text-to-speech feature in the Reader app. Reader gave me the convenience of linking with my Pocket account, synchronizing its inbox automatically with my saved articles.

This system has worked well. I’ve been listening to my saved articles on Reader while continuing to save new articles on Pocket. This lets me easily revisit them later if needed.

There was, however, a small issue. I noticed the count of saved items on these two platforms didn’t match. While this didn’t initially bother me much (as I didn’t want to go through hundreds of articles), curiosity eventually got me to investigate this using Pandas. The main source of discrepancy wasn’t anything major, just a few instances of Reader changing urls from http to https. But, the point of interest here isn’t the mismatch - it’s the Python library that helped me spot it.

Python’s urlparse Library

While looking into this issue, I needed to compare the URLs of the saved items across both platforms. My initial plan was to use regex patterns to split each URL to different parts. However, I stumbled upon Python’s urlparse library . It not only did what I needed but also offered a has many additional functionalities.

Here’s a glimpse of how it works:

from urllib.parse import urlparse

url_to_parse = "https://saeedesmaili.com/exploring-openai-whisper-with-non-english-voices/?param=query"

parsed_url = urlparse(url)
print(parsed_url)
## ParseResult(scheme='https', netloc='saeedesmaili.com', path='/exploring-openai-whisper-with-non-english-voices/', params='', query='query=param', fragment='')

Looking back, I wish I’d discovered this library sooner. Last year, I spent quite some time with a regex pattern to extract the domain name and extension (the netloc in urlparse’s output) for a side project.

One additional advantage of urlparse is its simplicity in manipulating parts of a URL. Here’s an example:

parsed_url._replace(query="").geturl()
## 'https://saeedesmaili.com/exploring-openai-whisper-with-non-english-voices/'

Comment? Reply via Email, Mastodon or Twitter.

About Whisper

Whisper is OpenAI’s speech-to-text model and it’s well-known for its impressive results. Although I knew about it for a while, I didn’t get to test its real-world performance until recently. So, I spent a weekend seeing how it could handle converting speeches, in both English and other languages, into text.

Whisper comes in different sizes and two types. The smallest one (tiny) is really fast but not so accurate, while the largest one (large) is slower but way more precise. Some people have found that the smaller versions work fine for most English speeches, but I was interested in seeing if it could handle less common languages like Persian as well.

Whisper’s performance varies widely depending on the language and this figure shows a WER (Word Error Rate) breakdown by languages using the large-v2 model (The smaller the numbers, the better the performance).

Voice Files

I usually use Telegram for messaging, so I had many voice messages saved there. I picked a few for this experiment. I also used the first minute of sound from two English and two Persian YouTube videos. If you want to try this experiment yourself, you can download these files here and here .

Attempting Hugging Face Transformers and Google Colab

At first, I tried to use Google Colab for this experiment so I wouldn’t have to install anything on my laptop. I came across the Whisper model card on hugging face and tried installing the transformers and loading the model on a Google Colab notebook. The results of the tiny and small models weren’t good for Persian voice, and the medium model caused the Colab session to restart due to insufficient memory, meaning I couldn’t use Google Colab for this experiment.

Also, I found the Hugging Face transformers to be somewhat complex, especially when compared to OpenAI’s python library for Whisper, which seemed much more straightforward. So, I decided to switch over and install this library on my laptop to continue the experiment there.

Experimenting with Whisper Python Library on Macbook

My 8GB RAM Macbook Air couldn’t handle the medium and large models, so I switched to my Macbook Pro for a more intensive experimentation. The Whisper python library is quite user-friendly. After installation:

pip install -U openai-whisper

And ensuring ffmpeg is installed:

brew install ffmpeg

It’s as easy as a few lines of code:

import whisper

model = whisper.load_model("large")
result = model.transcribe("english.wav")
print(result["text"])

It took 60 seconds to transcribe a 60-second English voice file into:

" Hey, what’s up? I’m Kabeh HD here. Okay, I’m gonna make myself sound really old here just for a second. But about 10 years ago, there was a vibrant ecosystem of a ton of different tablets. There’s all kinds of tablets. There was like iPads versus Android tablets, all these fights going on. There were small screen tablets, big screen tablets, and all the Android tablets were sort of anchored by Google’s family of Nexus tablets. So we had the smash hit Nexus 7 and the cult classic Nexus 10. But then Google kind of got a little bit lost with Android tablets and Chrome OS tablets a little bit in there for a bit, and they made a few bad ones in a row, and then they just gave up. Like they literally just gave up, stopped making tablets. But something else that’s picked up a whole bunch of steam since then is a new product category, smart home displays. So Google bought Nest, and we’ve seen this Nest Hub, and the Nest Hub Max came out, and this is a growing category."

And 109 seconds for a 60 seconds Persian voice file to turn into:

’ سلام من حسام هستم و به قسمت 8 از منجوری لذت اکاسی خوش اومدید توی این قسمت میخوایم راجب اکاسی قضا با هم صحبت بکنیم و یه سیری راه کارها رو با هم داشته باشیم که به تون کمک می کنن اکسهای بهتری را از اون قضاهایی که درست می کنید یا برتون سرف می شد و داشته باشین همراه من باشین ممنبع نوری اولین نکته برای داشتن اکسهای حرفه ای از قضا منبع نوریه خب اگر دست رسیب امکاناتی ویژه مثل نورهای استودیوی ندارید باید دقیق بکنید که از منبع نور طبیعی مثل پنجره استفاده بکنید اینجوری نتائجتون واقعا شگفتنگیز می شه این اکس رو می بینید این با نور پنجره سبت شده به اصلی گوشی و می بینید که چون منبع نوری مود به صورت طبیعیه خیلی نور خوبی متوجه قضا شده و قضا قشنگ'

The English transcription was excellent. As for the Persian translation, I found it somewhat lacking. There were so many typos in the text, and if you were to use it as the subtitle in a video, it would come off as embarrassing. The model does seem to accurately pick up on individual sounds, both vowels and consonants. However, when it attempts to assemble these sounds into a complete word, the results often don’t form valid words in Persian. For example, it heard the word “akkasi” which is the English spelling for the Persian word meaning “photography” (عکاسی), but it transcribed it as اکاسی. This isn’t a valid word in Persian, but if it were, it would also be pronounced as “akkasi.”

Trying whisper.cpp

Looking for a faster way to get the transcriptions, I found that whisper.cpp was much quicker than the Python library for the larger models. To use whisper.cpp, you first need to clone the project, then download a Whisper model:

bash ./models/download-ggml-model.sh large

Then simply run:

./main -m models/ggml-large.bin -f samples/english.wav --output-txt

Make sure your voice file is a 16khz wav, and adjust the location as needed. The --output-txt part gives you a text file of the transcript. Transcribing the English voice took 22 seconds, which is three times faster than the Whisper python library.

For languages other than English, you can add -l <language> to the command:

./main -m models/ggml-large.bin -f samples/persian.wav --output-txt -l fa

This one took about 27 seconds, which is four times faster than the Whisper Python library.

Here’s the interesting part: if you want an English version of a non-English voice file, you can add a -tr flag to the command:

./main -m models/ggml-large.bin -f samples/persian.wav --output-txt -l fa -tr

This command translated the Persian voice to the following English text in just 19 seconds!

Hi, I’m Hesam and welcome to the 8th episode of the “Taste of Photography” series. In this episode, we want to talk about food photography and have some guides with us to help you get the best photos of the foods that you make or serve. Stay with me. [Intro] The first tip for taking professional photos of food is a light source. If you don’t have access to special facilities like studio lights, make sure to use natural light sources like a window. This way, your photos will be more attractive. This photo is taken with a window light. As you can see, the natural light source is a very good light source. The food is very tasty.

The result is quite impressive and more practical than the Persian transcription. It is possible that the higher quality of the translated Persian transcript results from more translation hours in training datasets (see appendix E in the Whisper’s paper ). So, if you’re dealing with non-English speeches and you’re not concerned about presenting the output to users, it’s more effective to translate and use the English text.

Comment? Reply via Email, Mastodon or Twitter.

However, the transition was not all smooth sailing. When I transferred my passwords from Firefox and Chrome into 1Password, a couple of things didn’t go quite right. I ended up with:

• Lots of records that had the website’s URL saved as the username and my actual username saved as the password.
• A cluttered 1Password account with lots of duplicate records – each one with the same website, username, and password.

Naturally, manually going through hundreds of passwords to sort this out was not my idea of a fun time. Thankfully, 1Password provides a CLI tool, and with a little learning and scripting, I was able to address both issues.

To get started, I needed to install the 1Password CLI and sign in . I also needed to have jq installed on my machine.

To remove items where the username is actually a URL, I created a bash script:

for id in $(op item list --format=json | jq -r '.[] | select(.id != null) | .id'); do
  item=$(op item get $id --format=json)

  if [[ $item != null ]]; then
    fields=$(echo $item | jq -r '.fields')

    if [[ $fields != null ]]; then
      username=$(echo $fields | jq -r '.[] | select(.label=="username").value')

      if [[ $username == http* ]]; then
        op item delete $id --archive
        echo "$id deleted"
      fi
    fi
  fi
done

Then, I made my script executable and ran it:

chmod +x remove-urls.sh

./remove-urls.sh

The first issue is resolved!

Next, I addressed the problem of duplicate entries. To do this, I had to first update Bash to a more recent version that could handle special types of lists called associative arrays. Because I was using a Mac, it initially came with an older version of Bash that lacked this feature (bash --version gave me v3.2). So, my first step was to install a newer version of Bash (v5.2):

brew install bash

And find its path:

which bash

Then, used the path for running the new script to remove duplicates:

#!/opt/homebrew/bin/bash
declare -A itemMap

for id in $(op item list --categories Login --format=json | jq -r '.[] | select(.id != null) | .id'); do
    item=$(op item get $id --format=json)

    if [[ $item != null ]]; then
        fields=$(echo $item | jq -r '.fields')

        if [[ $fields != null ]]; then
            username=$(echo $fields | jq -r '.[] | select(.label=="username").value')
        fi

        urls=$(echo $item | jq -r '.urls')
        href=$(echo $urls | jq -r '.[0].href')
        website=$(echo $href | awk -F[/:] '{print $4}')

        if [[ -n $website && -n $username ]]; then
            key="$website-$username"

            if [[ ${itemMap[$key]} ]]; then
                echo "Duplicate found:"
                echo "Item 1: id: ${itemMap[$key]}, username: $username, website: $website"
                echo "Item 2: id: $id, username: $username, website: $website"
                op item delete $id --archive
                echo "$id deleted"
            else
                itemMap[$key]=$id
            fi
        fi
    fi
done

Just as before, I needed to make this script executable and then ran it:

chmod +x remove-duplicates.sh

./remove-duplicates.sh

Comment? Reply via Email, Mastodon or Twitter.

The wrong approach

We can rely on our brains to remember all these daily chores, but don’t ask me to explain why it’s a terrible idea. Our precious brain has a limited processing power for each day. This power can definitely be put to better use on something more creative than remembering to reply to an email.

A better but not ideal alternative is to use each tool’s unique features to bookmark any piece of information that we’ll need to act on some day. Gmail lets us star emails, messages can be added to saved items on Slack, and we can use one of the hundred TODO apps to store other chores like submitting vacation time. Saving and bookmarking stuff this way is mostly convenient to do, but very difficult to organize and retrieve. We will need to check various tools frequently to make sure nothing has gone off our radar.

The better approach

We tend to seek the most convenient way to store information (that’s why we might chat with ourselves on Slack), but don’t care much about the convenience of retrieving that information. Having a balanced approach and putting everything that needs to be done in a central place will make our brains’ life much easier in reclaiming them. Bonus point if we could add enough context to each item and organize them properly.

Which tool to choose for this central place will be very dependent on what type of person we are. Some of us prefer pen and paper and will go with a bullet journal . Others want to stay digital and will use Obsidian or Logseq . No matter which tool you decide to use, remember that it should be one central place. Try your best to stick with one single place as much as possible. Of course, this is not 100% practical advice, and there will be some exceptions since a few specific tools like digital calendars can handle particular tasks more efficiently.

Note that I’m only talking about storing chunks of different types of work we need to do each day (and we usually call them our TODO list). Other pieces of information like what we learn from a book or a course, or our long-term planning docs can be (and should be) stored somewhere else.

What do I use?

Since I was already using Obsidian as my second brain and personal knowledge management system, I gave it a chance to be the host to my daily and ongoing tasks as well. Although Obsidian has a few useful TODO-related plugins, I didn’t find myself comfortable using it for this purpose (I’m happily continuing to use it as my second brain though).

I’m currently using Logseq as my central TODO place. In addition to the fact that it stores all the notes in local markdown files, this tool comes with a few very nice features that make it very suitable for what I was looking for.

• Adding and toggling TODOs is easy and part of its core functionalities.
• It’s conveniently possible to schedule an item and set a priority for each TODO item.
• Daily journaling is a very native feature as well.
• Creating custom templates allows me to structure pages however I want.

I have iterated on a daily journal template, with a specific layout and a few useful titles and information pre-filled. When I open Logseq each morning, it automatically creates a new note (titled with weekday and date) using my custom template. The template includes an arbitrary structure to help me organize TODOs in different buckets depending on when they need to be done. Then follows an unfinished TODOs from yesterday, and then a full list of unfinished TODO items from the past. The daily note ends with a reminder section including all the TODOs that are scheduled for this day or have a deadline set to today. I can easily move items between days and sections and change their priorities in lists whenever needed.

I also use the daily journal page on Logseq to take notes of my thoughts, meetings, ideas, or anything that I don’t want to think about where to write them at the moment. Later in the day or week, I make sure to move anything important to its desired place (my second brain, the company’s docs, or somewhere else).

Limitations

I know, you’re thinking, “… it’s a lot of work!” . Actually it’s not a lot, but yeah sure, it requires some discipline and work. Recall that to make the retrieving process easier and more efficient, we need a robust system. I’m not completely disciplined in it yet either, but I’m getting better. The secret is to just start. Start small and give it a try. I began with a blank daily journal on Logseq and added templates later on, and gradually improved it based on what I found essential for myself. Feel free to reach out if you are interested in knowing more about my workflow.

Comment? Reply via Email, Mastodon or Twitter.

But it definitely lacks a few critical features that hinder my productivity.

What I like: Convenience and Collaboration

Google Docs makes it easy to write and share documents with others and to invite them to review and collaborate.

• It’s super simple to share a document with someone, a specific team or group of people, an organization, or the entire internet. Plus I can set different access levels for each person. This way, no one would accidentally edit the document, when they are just invited to provide their feedback and comments.
• Speaking of comments, Google Docs makes it very convenient for other people to select a part of the text and leave their comments. Everyone then can reply to the original comment and start having a conversation around that specific selected text without cluttering up the original text.

What I dislike: Findability

Writing awesome docs is just half of the equation, and if I (or any other person) can’t find it later, I’ll be less motivated to spend time and effort writing a similar masterpiece next time. I wish Google Docs allowed me to:

• Filter the search results. It comes with basic search functionality to look for all the documents that contain some specific keywords and leaves me with a list of titles at the end. It doesn’t let me filter by author, commenters, creation date, or almost anything else.
• Add some metadata to each document and filter by them later on. I could add tags like type: meeting-note or status: draft to a document to be able to find it more easily next time. This would also be helpful when I want to share the list of documents that author in [person a, person b] and project = X and status = published with the stakeholders of Project X.
• See each document’s backlinks. Google Docs has a handy feature for linking to another document by typing @ and searching for the title or just pasting the link to the document. But it misses the other side of the linking and doesn’t provide a way to know which documents are linking back to the current one. This is a key feature of PKM tools like Obsidian and Logseq . With a list of backlinks for each document, we could see where people are referencing our RFCs and guides. This gives us a much better idea of who are the actual audience and users of our documents, instead of guessing it or relying solely on our own perception.

Also I would love to use Google Docs in dark mode. Please!

Comment? Reply via Email, Mastodon or Twitter.