Functional[Justin]

Refreshing your Neovim config for 0.12.0

Mon, 06 Apr 2026 00:00:00 +0000

Introduction

Neovim version 0.12.1 was just released and you can check the release news here:

This is a massive release bringing a number of core capabilities that used to require external plugins into Neovim as built-in features. I took this opportunity to do a full refresh of my config. This was required as over the last few years I have pulled in many plugins only to forget what they are for. My config had too many files, too much surface area when trying to make changes or fix annoyances.

Rather than use an off the shelf config I had built it all by hand to meet my needs perfectly, but without careful maintenance it becomes an overgrown mess. In particular my lsp configurations and key maps were spread all over different places, and there was cruft I wasn't even using such as luasnip (hrsh7th/nvim-cmp won't work without a snippets plugin whether you want one or not).

I considered two options for the refresh:

Gradually fix it a bit at a time.
Kill it with fire and start again.

Given the mess my old config had gotten to I decided the latter would be much more manageable.

Running two configs on the same machine

It's a lot easier to work on a new config if you still have the old one around for reference, and can easily run the new and the old together. One way to make this work is the NVIM_APPNAME environment variable. Normally Neovim expects the various working folders, including the configuration, to be in the nvim folder.

By using the NVIM_APPNAME environment variable you can set it to something different, I chose nvim-next. This let's you build an entirely new config and leave your old one intact.

I started by manually creating the new folders (MacOS example).

mkdir -p ~/.config/nvim-next
mkdir -p ~/.local/share/nvim-next
mkdir -p ~/.local/state/nvim-next
mkdir -p ~/.cache/nvim-next

Now it's possible to run with a fresh config without worrying about messing up the old one.

NVIM_APPNAME=nvim-next nvim

During development of the new config I used this command to open them both up together.

NVIM_APPNAME=nvim-next nvim -O ~/.config/nvim-next/init.lua ~/.config/nvim/init.lua

Finally, you may want to include the environment variable in your shell config to use it as your "real" config more easily.

export NVIM_APPNAME=nvim-next

Replicating the config

I needed some method to migrate this without missing anything so I went through the init.lua file and copied over anything that was obviously needed and unchanged. In particular, the vim options have not changed.

vim.g.mapleader = ","
-- Prevents showing extra messages when using completion
vim.opt.shortmess:append("c")
-- Sets the height of the command line area at the bottom
vim.opt.cmdheight = 2
-- Displays the line number for the current line
vim.opt.number = true
-- Displays line numbers relative to the current cursor position
vim.opt.relativenumber = true
-- Time in milliseconds to wait for a mapped sequence to complete
vim.opt.timeoutlen = 500
-- Time in milliseconds of inactivity before calling CursorHold or writing to swap
vim.opt.updatetime = 4000
-- Ignores case when searching patterns
vim.opt.ignorecase = true
-- Automatically switches to case-sensitive search if a capital letter is used
vim.opt.smartcase = true
-- Enables 24-bit RGB colors in the terminal
vim.opt.termguicolors = true
-- Configures the behavior of the insert mode completion menu
vim.opt.completeopt = "menu,menuone,noselect,popup"
-- Number of spaces that a <Tab> character represents
vim.opt.tabstop = 2
-- Number of spaces to use for each step of automatic indentation
vim.opt.shiftwidth = 2
-- Number of spaces that a <Tab> counts for during editing operations
vim.opt.softtabstop = 2
-- Converts tabs into spaces when typing
vim.opt.expandtab = true
-- Automatically inserts an extra level of indentation in some cases
vim.opt.smartindent = true
-- Makes <Tab> insert 'shiftwidth' number of spaces at the start of a line
vim.opt.smarttab = true

After that I stepped through the init.lua entirely and made a list of things to keep and things to lose. Let's cover them by category.

vim.pack, Neovim's new built-in plugin manager

Previously I used folke/lazy.nvim for plugin management. This worked well and was fairly easy to configure, however there was complexity in the form of having to think about and configure lazy loading. For example, in loading this colour theme I need to think about whether to load it lazy or not, the priority with respect to other plugins and what to do to configure it.

{
  'shaunsingh/nord.nvim',
  lazy = false,
  priority = 1000,
  config = function()
    vim.cmd "colorscheme nord"
  end
},

vim.pack's approach is a lot more vanilla. It has no lazy loading, no priority system and all plugins have to be on a remote git repo. I think you cannot use local folders or local git repos. This may make plugin development a bit tricky.

If you load a lot of plugins and some of them are slow, the optimization of lazy loading is probably worth it. With vim.pack you can still do the setup functions conditionally based on autocommands if you need to.

In my use case I found the start up time is 80ms and plenty fast enough for me, so I don't miss the lazy loading.

To actually do the conversion you simply call vim.pack.add when you want to add a plugin, and once it is installed every command after that can use it.

For example the configuration above becomes:

vim.pack.add({
  { src = "https://github.com/shaunsingh/nord.nvim" },
})

vim.cmd('colorscheme nord')

Note that assumed Github paths are not filled out like in Lazy.nvim, you must fill them yourself, or as the documentation suggests use a helper function to generate the names.

local gh = function(x) return 'https://github.com/' .. x end
local cb = function(x) return 'https://codeberg.org/' .. x end

vim.pack.add({ gh('user/plugin1'), cb('user/plugin2') })

vim.pack keeps track of the versions you are using and saves to a lock file. This is supposed to be committed to source control and it means you can move to another machine and get the exact same plugin versions. It also means you may have merge issues if you edit the configs on different machines.

Also of note is the new :restart command, which lets you iterate on changes to plugins and configuration more easily. The Neovim core restarts. With mksession you can save the current session then resume it after a restart.

One more thing I like about vim.pack is it uses simple buffers as its user interface. vim.pack.update({}) shows a buffer with the packages and any updates that need doing. Simply writing the file acts as confirmation you want to go ahead. Nice.

Treesitter

I ran into some issues with Treesitter as I ended up with a mix of different tree-sitter compiled grammars and different versions of Treesitter. It all worked after doing the following:

Install the tree-sitter-cli, which is required by newer versions of treesitter-nvim.

brew install tree-sitter-cli

Ensure to switch to the main branch of treesitter-nvim (main is more up to date than master).

vim.pack.add({
  { src = "https://github.com/nvim-treesitter/nvim-treesitter", 
    version = 'main' },
})

After that I was able to TSUpdate and TSInstall whatever I needed without any issues providing perfect syntax highlighting.

LSP and completion

As mentioned, the bulk of my config complexity was coming from completion and LSP configuration. This complexity is greatly reduced by the following changes:

Remove nvim-cmp and LuaSnip
Use native completion
Modular lsp configuration

Since completion is now built-in I can turn it on with the following global settings and an LspAttach autocmd to enable it for each lsp server after checking if it is supported.

vim.opt.completeopt = "menu,menuone,noselect,popup" -- Ensures the menu appears even for a single match and uses the native popup window.
vim.o.autocomplete = true -- Enables the overall completion feature.

vim.api.nvim_create_autocmd("LspAttach", {
  group = vim.api.nvim_create_augroup("lsp_completion", { clear = true }),
  callback = function(args)
    local client_id = args.data.client_id
    if not client_id then
      return
    end

    local client = vim.lsp.get_client_by_id(client_id)
    if client and client:supports_method("textDocument/completion") then
      -- Enable native LSP completion for this client + buffer
      vim.lsp.completion.enable(true, client_id, args.buf, {
        autotrigger = true,   -- auto-show menu as you type (recommended)
        -- You can also set { autotrigger = false } and trigger manually with <C-x><C-o>
      })
    end
  end,
})

Finally, I still use Mason for easy management of LSP servers. What remains is to enable the servers which must be done by calling the enable method, and to configure it. For example for Python I added the following in my init.lua to enable the server.

vim.lsp.enable('pylsp') -- Python

LSP configuration can now be done using an lsp folder in your config. Each config file should be named after the LSP server. In this case I made lsp/pylsp.lua. It should simply return the configuration structure.

return {
  cmd = { 'pylsp' },
  filetypes = { 'python' },
  settings = {
    pylsp = {
      plugins = {
        mccabe = { enabled = false },
        pycodestyle = { enabled = false },
        flake8 = {
          enabled = true,
          ignore = { "E501", "E302", "W" },
          maxLineLength = 120,
        },
      },
    },
  },
}

This kind of convention over configuration appeals to me and helps keeps things organized.

New User interface (ui2) and the status line

ui2 is a "is a redesign of the core messages and commandline UI, which will replace the legacy message grid in the TUI.". In particular it removes "Press Enter" interruptions, and it highlights the command line as you type. Nice.

As a maintainer of a status line plugin, battery.nvim I was curious if anything had broken the statusline. Everything seemed to work without any changes.

ui2 is opt-in and you can do so as follows:

-- New UI opt-in
require('vim._core.ui2').enable({})

The best of the rest

I've kept the most minimal set of plugins for my config. In particular I still have:

mason, mason-lspconfig – lsp server managment
nvim-treesitter – tools for treesitter
shaunsingh/nord.nvim – my favourite colourscheme
mrjones2014/legendary.nvim – archived but still great key map manager
ibhagwan/fzf-lua – fast finder for files and text within files
nvim-lualine/lualine.nvim – best statusline
justinhj/battery.nvim – Let me plug my own plugin
nvim-lua/plenary.nvim – Useful libraries
folke/which-key.nvim – Keymap display

Some I may miss

tpope/vim-fugitive – Nice Git commands, but I mostly use Git at the command line
karb94/neoscroll.nvim – smooth scrolling

vim.net.request

I have done some http programming in lua, mostly with the help of plenary and libuv. Adding GET requests to the built-in lua is a nice step. I would hope it gets POST and other methods to expand its utility over time.

Wrapping up

There are a couple of options once you're happy with the new config. One option is to keep the old config around forever. Another would be to delete it completely.

For science, on my Windows machine which I mostly use for Gaming and a bit of ML and AI, I completely deleted the old config and replaced it. On my Macbook I keep both for now.

Conclusion

0.12.x is one of the biggest updates for a while in Neovim and it's great to see the out-of-the-box experience expanding. I found clearing out my configuration and refreshing it for 2026 was a good excuse to explore these new features.

Thanks

Thank you to GrandLate7367, TheLeoP_ and mushfiq_814 on Reddit's r/neovim for pointing out some issues with the original post.

Visualizing K-Way Merge: An Interactive Guide to Database Sorting

Mon, 19 Jan 2026 00:00:00 +0000

Photo by Aleksandr Galichkin on Unsplash

In this post, inspired by a video by Tigerbeetle engineers Tobias Ziegler (@toziegler) and Alex Kladov (@matklad), I'll walk through four algorithms for merging multiple sorted runs of data; a problem that shows up in modern database internals. You will be able to use an interactive web application to explore and understand each one.

The title image isn't just a pretty picture of tennis courts; it's the inspiration for the most efficient external sorting algorithm: The Tournament Tree.

External sort and k-way merge

Whilst most engineers are familiar with a number of sorting algorithms that work with data sets that fit entirely in memory, there are use cases, both historical and current, that do not. In these cases you must pull in small chunks of memory and sort them before outputting to a larger sorted output.

As an extreme example take the 1960 USA Census for example. The Census Bureau's UNIVAC 1105 was a supercomputer of the day, with up to 18 attached tape drives. Even so, it had a limited memory (less than 300kb) and so there is no way to sort the data in memory.

Instead the process used was called Balanced k-way merge. Input data was put onto one set of tapes, and another set of tapes are used as output. The computer read as many records as it could fit in memory, sorted them and wrote them to tape. The tape was then filled with these sorted runs.

The k-way merge part comes from k (the number of runs being handled at once, in this case the number of tapes, let's assume 4). At the first step the computer reads the first records from each tape and outputs the smallest one to the output tape. Next, it gets the next record from the tape where the smallest record came from. This process continues until the records on all 4 tapes have been exported to a single sorted run on the output tape.

External sort in modern databases

Modern databases use essentially the same logic. An LSM tree (see next section) flushes "SSTables" (sorted runs) to disk (which are modern "tapes") and then performs a background k-way merge (compaction) to combine them. The physics changed (seek times vs. linear scan), but the math of the k-way merge remains exactly the same.

External sort is used for various purposes:

Databases	Purpose	Algorithm
LSM-Tree Databases (RocksDB, Cassandra, ScyllaDB, LevelDB)	Compaction	min-heap
Search Engines (Apache Lucene / Elasticsearch / Solr)	Segment merging	min-heap
Analytical & Columnar Databases (ClickHouse, DuckDB)	Part Merging: Maintaining column blocks sort order	Linear scan/Vectorized merge
Relational Databases (PostgreSQL)	Queries that exceed work_mem/Merging joins	Tree of Losers/Polyphase

LSM trees

LSM-Trees (Log-Structured Merge-Trees) are a data structure that achieves fast writes by buffering changes in memory and periodically merging them with sorted on-disk tables. I won't go into much detail on them in this post, it's only important to know that the typical implementation requires taking multiple sorted runs of records and merge sorting them into a single sorted run.

Tigerbeetle

Like DuckDB and others, Tigerbeetle's storage engine is also using LSM-Trees. Let's dig into that a little:

In order to accept new transactions very quickly, Tigerbeetle writes new transactions into an unsorted mutable memory buffer (each transaction was safely written into a write ahead log on a quorum of nodes before this step, so it cannot be lost at this point).

The memory buffer has a fixed size and once it is full it becomes an immutable buffer and is flushed to disk. As noted above, LSM trees store data into internally sorted tables, and these tables are merged into larger ones as you get further down the tree.

To better explain the mechanism play with the following toy Tigerbeetle database:

👆 Click to try the interactive simulation

Apart from this compaction step, Tigerbeetle also has the same problem when doing range based query scans; it will scan the tables in the various levels of the LSM tree and will need to combine each one into a single sorted result.

K-Way merges

The process of taking multiple sorted runs and outputting a new single sorted run of data is called k-way merging. k is the number of runs we are merging, so we are merging it in k-ways.

Implementing k-way merging is not super complicated, but as we will see there are some different options depending on your use case, with the usual engineering trade offs.

Contrary to some LSM implementations that might use k-way merge for all compactions, TigerBeetle uses it selectively. For example the mutable table actually consists of 32 sorted runs and when they are flushed to the immutable table they are all merged to a single sorted run before being flushed to the first disk level.

Linear scan

As always, let's start with the simplest solution possible. In linear scan you take the smallest elements from each of the k runs, and that is your starting set. Each iteration we do the following:

Algorithm.

Run through all k current candidates tracking the smallest one found.
Output the smallest element.
Replace the smallest element with the next element from the run it came from.

That last point is crucial. Why take the next element from that particular run? It's because after step 1 we have by definition the smallest elements from each of the runs. The next smallest element can only be one of the other k elements we are still considering, or the new element we didn't see yet from the "winner's" run.

Make sure to understand that point as all the algorithms below will do the same.

As mentioned above, you can explore each algorithm interactively, just click on the image below. Each one has the same controls:

Step. Go to the next step of the algorithm, updating the step description and animation.
Play. To unleash the animation and run through the algorithm until you stop it or the runs are all empty.
Seed. Change the seed to try different data sets.
Speed slider. Change the speed of the animation.
Reset. Restart the simulation.

👆 Click to try the interactive simulation

Analysis.

Since we must do k comparisons per output, with n total data items in all the runs the complexity is: \(O(kn)\)

Whilst not bad, we can do better. However, for small values of k it may well be good enough, or even faster than the other solutions below, despite the asymptotic complexity saying otherwise.

In addition, on modern CPUs you can use SIMD instructions (AVX2 or AVX-512) to handle up to k=16 assuming 32 bit integers. However, since Tigerbeetle can have up to 32 runs and uses 128 bit identifiers, this is not a useful option.

Going faster with Tree Selection

As mentioned above we do \(O(k)\) comparisons per output, can we do better? Well, to be more precise for the first output we need do k-1 comparisons, since we start with the first element and that iterate them all, swapping out the first element with the first smaller one and so on. In order to speed things up we need to think about doing less comparisons.

Although we need to do k-1 comparisons to get the 1st item, with some thought we can do far fewer to find the 2nd item, 3rd item and so on.

In Knuth (Art of Computer Programming vol3: Searching and Sorting) we find the discussion on selection sort which helps here…

Quadratic selection (E H Friend, JACM 3, 1956) noted that we can do better by first grouping the sort items into groups based on the square root of n. For example, sorting 9 items we divide into groups of 3.

Note: in selection sort you find the largest element at each step and move it to the end of the array. In k-way merge we output smallest first. Don't be confused by finding the largest element in this section.

8	4	3
7	5	2
9	1	6

Once you have the three groups find the largest in each.

`8`	4	3
`7`	5	2
`9`	1	6

The largest element is now the largest of the three "winners", 9.

To find the second largest, it could be either of the two remaining winners or one of the others in 9's group. (Note that I replace removed elements with infinity so they can be ignored in subsequent steps.)

`8`	4	3
`7`	5	2
\(\infty\)	1	`6`

We find it is 8. So the next winner is the larger of 7, 6 or the others in 8's group.

\(\infty\)	`4`	3
`7`	5	2
\(\infty\)	1	`6`

Proceeding this way the sort needs n sqrt n comparisons, much better than the original n².

Friend noted that you can continue this process of dividing the search into smaller groups based on the cubic root, the quartic, and ultimately into groups of two. Friend called this nth degree selecting. Knuth calls it tree selection.

This is what leads to the next three solutions to k-way merge, using tree structures essentially gives us a way to cache comparisons so we don't have to repeat them.

min heap

As we saw above, modern databases make use of the min-heap for k-way merge. As a binary tree structure it's an obvious choice since elements are stored in partially sorted order and we can always remove the minimum element which is the one we want to emit next.

In order to use the min-heap for k-way merge we do the following:

Populate a fixed size min-heap with the first (smallest) elements from the k runs.
Pop the top item from the heap (the smallest) and replace it with the next from the same run.
Sift the heap down to make the heap valid again.
Repeat until the sort is complete.

Once again, you can step through the algorithm using the interactive visualization below.

👆 Click to try the interactive simulation

Analysis.

With a min-heap the push and pop operations are both \(O(\log_2 k)\) giving a runtime complexity of \(O(n \log k)\).

                  a
                  |
      +-----------+-----------+
      |                       |
      b                       c
      |                       |
+-----+-----+           +-----+-----+
|           |           |           |
d           e           f           g

min-heaps are represented as a flat array. For example the tree above would be stored in memory like this:

This is known as the Eytzinger Layout. A node at index \(i\) has children at \(2i+1\) and \(2i+2\). For a small tree size it is a cache friendly structure since it will fit in a few cache lines.

In addition our use case allows us to do the push and pop in one move.

During the sift down process after adding the new winner you need to:

Compare Left Child vs. Right Child (to find the smaller one).
Compare the Smaller Child vs. The Current Node (to see if you swap).

This costs \(2 \cdot \log_2 k\) comparisons per element.

Let's see how we can improve.

Tournament Trees

In Knuth we see the following diagram of a ping pong tournament. (section 5.2.3, Fig 22). Any sport works here really, the important thing is that when two people or teams compete the idea is that the best one will win. We will be dealing with sorting unchanging records so the analogy is not perfect. If Chris beats Pat on Sunday, he may still lose to Pat on Monday, but we can ignore that for the purposes of the algorithm.

                  Chris (Champion)
                      |
          +-----------+-----------+
          |                       |
        Chris                    Pat
          |                       |
    +-----+-----+           +-----+-----+
    |           |           |           |
   Kim        Chris        Pat        Robin
    |           |           |           |
 +--+--+     +--+--+     +--+--+     +--+--+
 |     |     |     |     |     |     |     |
Kim  Sandy Chris  Lou   Pat   Ray   Dale Robin

The result of a tournament can be thought of as a binary tree where the winner is at the root. The two children will be the two players that played the final, and so on. It is clear we can go down the tree starting at Chris and say that he was better than each of his opponents; Pat, Kim and Lou.

The best player then, is Chris. But what if Chris had not shown up that day? Is Pat the best player?

Well all we know is that Pat beat all the opponents on her side of the tree, but we don't in fact know if she would have beaten the same opponents on Chris's side. So all we can say is that Chris was the best player overall, and Pat was the best player amongst the right subtree.

Photo by pure julia on Unsplash

It is here that we take an interesting digression into Tennis Tournaments and whilst they are fair to the winner, they are not quite so fair when it comes to picking the runner up, third place and so on.

Rev. Charles Lutwidge Dodgson, better known by his pen name, Lewis Carroll, author of Alice's Adventures in Wonderland was also a mathematician and logician at Christ Church, Oxford, with a deep obsession for fairness in rules and voting systems.

“At a Lawn Tennis Tournament, where I chanced, some while ago, to be a spectator, the present method of assigning prizes was brought to my notice by the lamentations of one of the Players, who had been beaten (and had thus lost all chance of a prize) early in the contest, and who had had the mortification of seeing the 2nd prize carried off by a Player whom he knew to be quite inferior to himself.” - Charles Dodgson

Dodgson came up with a scheme, not quite an algorithm as it was not precisely specified, to play the Tennis Tournament in a way that was more fair. His idea was that the losers could replay each other so that they get a second chance. In the tournament above, Lou lost to Chris, but he could get a chance to play Sandy, for example, and win a chance to play for second place.

If you want to read more about this I found an excellent blog post which includes an implementation of Dodgson's idea in the R language, with the missing specifications figured out.

Lewis Carroll's proposed rules for Tennis Tournaments

Whilst the idea was good, it did not catch on because tournaments would take a variable amount of time depending on who won which games, and it's a bit more complex for the audience to follow. Besides, modern day tournaments use the seeding system which avoids top players playing each other in the early stages.

For the tree of winners algorithm we will work slightly differently. Our job is to continuously replay the tournament by removing the winner and bringing in a new candidate from winners run.

In the tournament above, Lou should play Kim and Pat should play the winner of that game. Notice that "only two additional matches are required to find the second-best player, because of the structure we have remembered from the earlier games."

Tree selection, then, proceeds by removing the winner of the tournament from our "Tree of Winners", then replaying the tournament without that player, and each replay requires \(\log_2 k\) comparisons.

To make this more concrete as a sort let's change the players to their numeric values that determine who will win. We want to sort in ascending order so the lowest values win, and are output first.

Step 1 we remove 1 and output it as the first element.

Sort output: 1
                    1 (Champion)
                        |
            +-----------+-----------+
            |                       |
            1                       3
            |                       |
      +-----+-----+           +-----+-----+
      |           |           |           |
      2           1           3           7
      |           |           |           |
   +--+--+     +--+--+     +--+--+     +--+--+
   |     |     |     |     |     |     |     |
   2     6     1     4     3     5     8     7

In step 2 we remove 1 from the tournament in preparation to replay without them.

Sort output: 1,2
                        2 (Champion)
                        |
            +-----------+-----------+
            |                       |
            2                       3
            |                       |
      +-----+-----+           +-----+-----+
      |           |           |           |
      2           4           3           7
      |           |           |           |
   +--+--+     +--+--+     +--+--+     +--+--+
   |     |     |     |     |     |     |     |
   2     6     X     4     3     5     8     7

We started at the bottom of the tree, 4 has nobody to play so it moves up, 2 beats 4 and moves up, 2 beats 3 and becomes the new winner.

Now we remove 2. Replaying from the bottom gives the tree below and we output the 3.

Sort output: 1,2,3
                        3 (Champion)
                        |
            +-----------+-----------+
            |                       |
            4                       3
            |                       |
      +-----+-----+           +-----+-----+
      |           |           |           |
      6           4           3           7
      |           |           |           |
   +--+--+     +--+--+     +--+--+     +--+--+
   |     |     |     |     |     |     |     |
   x     6     X     4     3     5     8     7

It's easy to see how this process can repeat until the tree is empty and we successfully completed our sort in n lg n time!

Check out this interactive visualization to fully understand the algorithm.

👆 Click to try the interactive simulation

Analysis.

Again we find the asymptotic complexity is \(O(n \log_2 k)\). However, it is expected to perform better than the min-heap for larger k for the following reasons:

You traverse from a single leaf at the bottom up to the root of the tree to replace the winner.
At each level you need compare only with the current sibling.
The number of comparisons is cut to \(\log_2 k\), half that of the worse case min-heap.
Hardware Sympathy: Tournament trees have a fixed path from leaf to root. There are no conditional checks to see "which child is smaller" before deciding which path to take down; you always move up from the leaf. This can be friendlier to CPU branch prediction.

There is still some room for improvement thanks to an ingenious idea from the 1950's.

Tree of Losers

In Knuth's Sorting and Searching (specifically Section 5.4.1), Donald Knuth attributes the origin of the "tree of losers" (also known as a loser tree) to H. B. Demuth. Demuth introduced the concept in his 1956 PhD thesis, Electronic Data Sorting, as a refinement of the standard tournament (winner) tree.

His idea once again reduces the amount of comparisons needed. In the tree of winners to recalculate the path to the root, you must compare the new item against its sibling at every level. This requires keeping track of which child (left or right) the path came from so you can locate the sibling.

Demuth's idea was to store the losers at each level instead of the winners. Similar to the linear scan algorithm we started with, the winner can be stored in a register helping with performance.

The algorithm looks like this:

Populate the tree of losers in a similar way to the tree of winners, but store the loser in each node.
Remove the winner.
Take the next candidate from the winners run.
Starting from its leaf compare with each loser. If the winner loses in this comparison then swap it, otherwise continue moving up the tree.
When you get to the top, whoever is stored in the winner register gets output.
Repeat until the runs are empty.

Interactive tree of losers:

👆 Click to try the interactive simulation

Analysis.

Asymptotic complexity is \(O(n \log_2 k)\) again.

We have performance improvements for larger values of k primarily due to efficiency in memory access and branch prediction during the "replay" phase.

Memory access:

Remember in the winner tree we have to gather two siblings from memory and compare them with the current winner. In the tree of losers we have the winner in a register and we only fetch the parent "loser" from memory at each step.

Branch prediction:

While replaying the Winner tree we execute the conditional logic if (came_from_left) compare(new_val, right_child) else compare(new_val, left_child) whilst in the tree of losers we always just go to the parent of the current node. In addition the winning node is always in a register.

Conclusion

K-Way merge is ubiquitous in database technology.
For small k (merging 4-16 runs) you can often use simple select and consider using SIMD.
Otherwise Tree of Losers is likely to be the best performance.

Algorithm	Complexity (per item)	Comparisons (approx)	Key Characteristic	Pros	Cons
Linear Scan	\(O(k)\)	\(k\)	Iterates all candidates	CPU Prefetch & SIMD friendly for small \(k\)	Scales poorly as \(k\) increases (>16)
Min-Heap	\(O(\log k)\)	\(2 \log k\)	Sift-down (swap with smallest child)	Standard library; Compact memory layout	Conditional branches (left vs right child); Higher comparison count
Winner Tree	\(O(\log k)\)	\(\log k\)	Replay path from leaf to root	Fewer comparisons than Heap	Requires sibling checks at every level (branch mispredictions)
Tree of Losers	\(O(\log k)\)	\(\log k\)	Compare with parent (loser)	Winner stays in register; Fixed memory path (always parent)	Slightly more complex implementation

You can view the Tigerbeetle current implementation here. Tigerbeetle K-Way merge

Thanks for reading, please contact me using the email and social links above with any comments and corrections.

Optimizing training a GPT style Tokenizer with C++

Wed, 30 Jul 2025 00:00:00 +0000

Introduction

In February 2024, renowned AI researcher and educator Andrej Karpathy, published the following video.

https://www.youtube.com/watch?v=zduSFxRajkE

An excellent introduction to the fascinating topic of tokenization, something he considers a necessary evil right now. Maybe the need for it will go away but for now it must be well understood by AI engineers in order to get good results when building and using large language models.

As software engineers in 2025 the advent of large language models, and generative AI in general, bring about a profound shift in how we design, develop and implement software systems. As such I have been delving into the details of how they they work, with Tokenization being a significant side quest.

In this blog, related video and accompanying C++ project, I will walk through my exploration of the training aspect of tokenizers and look at their performance characteristics and optimization opportunities.

My main goal was to port the Python to C++ but also I became interested in optimizing the training stage.

Training SOTA (State of the Art) models is a massive expense in terms of time and computation needed. The Llama 3 model was trained on 15 trillion tokens, and although it is not known, its tokenizer may also have been trained on the whole data set. Even if it used only a smaller representative sample, tokenizer training is also a huge expense.

For that reason it stands to reason you would want to optimize it as much as possible so that changes to the data set or algorithm can be made more easily and quickly.

Ultimately, I was able to reduce the training time by a factor of 23 (for the large wikitext input) as shown below.

Optimization results

The interactive chart below shows the time taken to train a Tokenizer on different text corpora. Use the buttons to switch between datasets and see the performance difference.

Lower is better. Time in seconds.

Input data sets used:

taylorswift. Text of the Taylor Swift wikipedia page.
shakespeare. The complete works of Shakespeare.
bible. The old and new testament of the Bible.
wikitext-103. https://huggingface.co/datasets/Salesforce/wikitext 100 million tokens of high quality wikipedia articles.

These are included in the repository apart from wikitext which you can download from hugging face.

Briefly, Tokenization and BPE

I don't want to even try and do a better job of explaining the need for tokenization and the BPE algorithm than Karpathy, so I instead recommend watching his video and, for additional insights, read the paper below:

[Neural Machine Translation of Rare Words with Subword Units](https://arxiv.org/pdf/1508.07909)

The need for tokenization

Large language models deal with numbers not words or characters, so we must map input text to numbers.
The model requires a fixed size vocabulary.
Unfortunately, that vocabulary would have to be enormous to fit all words from all languages.

Sub word unit tokenization

The solution to these issues is sub-word tokenization; splitting words up into numbers representing components of words. Once you have that you can handle:

Open-Vocabulary Part of the token set is the basic characters of each language so you can represent every word.
Rare words Because the vocabulary set is open it means any rare word is handled.
Translation of Novel Words The model can translate and generate words it has not encountered before by composing them from sub-word units.

Here is a diagram showing how the fixed sized vocabulary of tokens maps to an array of learned embedding vectors that feed into the Transformer model underlying all LLMs.

      Input Text
          │
          ▼
     "I love NLP"
          │
┌─────────┴─────────┐
│   Tokenizer       │
└─────────┬─────────┘
          │
          ▼
      Tokens
   ["i", "love", "nlp"]
          │
┌─────────┴─────────┐
│ Vocabulary Lookup │
└─────────┬─────────┘
          │
          ▼
      Token IDs
      [25, 2097, 12510]
          │
          │         ┌───────────────────────────────────┐
          │         │          Embedding Matrix         │
          │         │ (Size: |V| x d_model)            │
          │         ├───────────────────────────────────┤
          ├────────►│ Row 25:   [0.1, -0.4, 0.2, ...]   │  ───►  Embedding for "i"
          │         ├───────────────────────────────────┤
          ├────────►│ Row 2097: [-0.8, 0.5, 0.9, ...]   │  ───►  Embedding for "love"
          │         ├───────────────────────────────────┤
          ├────────►│ Row 12510: [0.3, 0.7, -0.1, ...]  │  ───►  Embedding for "nlp"
          │         ├───────────────────────────────────┤
          │         │                ...                │
          │         └───────────────────────────────────┘
          │
          ▼
  Input Embeddings
(Dense Vectors fed to the model)

Tokenization means taking text and splitting it into sub-word (and whole-word or even multi-word) units. An input text like "My cat, Blivarian, is making a mess." may be tokenized into something like this:

You can explore this tokenization here: https://platform.openai.com/tokenizer

My cat , Bl iv arian , is making a mess .

[5444, 9059, 11, 3130, 569, 21203, 11, 382, 4137, 261, 13017, 13]

Notice that the commas have the same token value when appearing in different places. Also that common words like cat and mess have their own tokens.

I deliberately made up a name for the made up Cat that is not a real word "Blivarian". You can see that it is split up into 3 sub words. Without tokenization this would instead have been stored with a special "Out of vocabulary" token, that means it carries no semantic meaning. When dealing with sub word tokens however, the LLM has the opportunity to build up meaning for those components that may help with overall model quality.

Byte Pair Encoding - BPE

Why BPE?

From above we understand that we should split words into sub-word components to handle the vast space of human vocabulary in the finite space of the LLMs vocabulary.

How to do that is the next question. Why not, for example, just have a vocabulary consisting of the punctuation and alphabetic characters of every language?

It won't work well because in the LLM training it will build up an embedding vector for each token, the unit of vocabulary. This vector is an array of numbers that represents a direction in multidimensional space. To us those numbers mean nothing, but in LLM training those numbers, when used in conjunction with the rest of the models weights, can be used to learn and represent all kinds of meaning.

Models like Transformers have a finite context window. When sequences are excessively long, it becomes much harder for the model to capture long-range dependencies and relationships between words that are far apart. The model has to work harder to understand the overall context.

Instead we want something that splits things into meaningful chunks, morphemes, as well as capturing commonly used words with tokens. This ends up looking something like Huffman Encoding:

https://en.wikipedia.org/wiki/Huffman_coding

It represents more frequently occurring substrings with less bits, giving us a more efficient data size.

Similarly, BPE, is data-driven algorithm that creates a vocabulary of meaningful and frequently occurring subword units.

BPE algorithm

First you need to train across a large corpus of realistic text. For state of the art (SOTA) LLMs this is likely in the trillions of characters of data.

The algorithm itself is very simple, it works as follows:

Start with 256 tokens (0 to 255), our basic character set.

First turn the text into its underlying numeric representation (typically just the bytes of a UTF-8 input).
Count all the pairs of bytes.
Pick the most frequently occurring pair and generate the next new token (257, 258…).
Replace that pair wherever it occurs with the new token.

Repeat until you have your full vocabulary. You can then save the merge pairs and these are then used by end users to encode their text before sending to the model.

They can also be used to reconstruct the original text in the decoding process when the response comes from the model.

Conflict Resolution

An important decision in tokenization is how to handle pairs with the same frequency. In this post I'll consider two methods:

First in corpus wins.
Lexicographical ordering.

With any tokenization algorithm design we need to consider efficiency of implementation alongside methods that give the best results. Some of these concerns will be highlighted below.

minbpe-cc an exercise in optimization

With these algorithmic decisions in mind, I was ready to dive into the C++ implementation and see how they performed in practice. This led to my project, minbpe-cc. I find the best way to learn a topic is to get my hands dirty, and as such I decided to reimplement Karpathy's Python code in C++.

I also wanted to focus on optimization of the training stage, for no other reason than curiosity.

Why C++?

It's a low level language with generally low to zero cost abstractions.
I've recently been catching up with modern C++ and wanted to try out some of the new features (C++23 required).

The final code here fully implements all the facets of Karpathy's minbpe including encoding, decoding and training. I've included end to end tests and tested in a linux and MacOS environment. I have not tested on Windows yet, but I expect it will work without much modification.

https://github.com/justinhj/minbpe-cc

Implementation tales

Speed bumps

Converting from Python to C++ is fairly straightforward although I hit some speed bumps on the way:

Python dictionary behaviour. The Python dictionary is designed to be flexible for multiple purposes rather than optimized, so getting the same behaviour from C++ containers required some additional thought.
Polymorphism. I didn't really like Karpathy's polymorphic version and instead decided to use a single class design with flags and other parameters to handle whether special tokens are used, what the conflict strategy was, and whether to use a regex or not. It was quite easy to make this work with some tweaks to the original code. Ironically I did use polymorphism on the PairCount class so I can use different implementations at runtime depending on the users preferences.
CMake. CMake is not a casual tool. I found I could just about get my project to build and run using it, but after switching to Zig build instead I found it much easier to manage. In other words to effectively use CMake would require me to read the manual in a lot more detail.

Regex compatibility

Firstly, what are regexes needed for?

In the GPT series of tokenizers, OpenAI realized that it is beneficial to try and keep parts of text together, as such rather than run BPE on the whole input text, they first divide it up into sections by the following regular expressions:

GPT2 "'(?:[sdmt]|ll|ve|re)| ?\\p{L}+| ?\\p{N}+| ?[^\\s\\p{L}\\p{N}]+|\\s+(?!\\S)|\\s+"
GPT4 "'(?i:[sdmt]|ll|ve|re)|[^\\r\\n\\p{L}\\p{N}]?+\\p{L}+|\\p{N}{1,3}| ?[^\\s\\p{L}\\p{N}]++[\\r\\n]*|\\s*[\\r\\n]|\\s+(?!\\S)|\\s+"

These expressions are designed to preserve various aspects of English text rather than allow them to be split up during the merge process.

Whilst there are a few established regex libraries for C++ (writing my own being out of scope for this project), finding one that was capable of handling these regular expressions took some looking.

These expressions need support for unicode matchers and also negative lookahead.

I compared several libraries:

RE2 from Google.
std::regex in the C++ standard library.
Boost::regex
Re-Flex

None of these met the requirements.

In the end I found the Perl compatible PRE2 library worked the best.

The biggest footgun was that the Boost::regex library was asserting because Boost was not linking properly with the ICU (internationalization) library. I suspect this could be made to work but I gave up.

Optimization mantras

In System's Performance, Enterprise and the Cloud by Brendan Gregg (2021) the following mantras for performance are listed, ordered from most to least effective. I find these useful when considering optimization.

Don’t do it.
Do it, but don’t do it again.
Do it less.
Do it later.
Do it when they’re not looking.
Do it concurrently.
Do it more cheaply.

We can refer to these during the post.

Data structures

The first step to port the Python code and make it more efficient is to think about the data involved and how that data needs to accessed.

Data

Body text. We will store this as a vector (array) of numbers representing the input text for training.
Pair frequencies. We need to keep track of all the pairs in the body text and their frequencies.

Access patterns

Body text. We need sequential access to scan for pairs. Then we need to be able to delete elements as part of the merge process.
Pair frequencies. We need to be able to store the pairs and their frequencies and efficiently update them as we scan the body text. In addition we need fast access to the next most frequent pair.

Implementation

Body text

Because the body text required sequential access and the ability to quickly remove elements I used a singly linked list, or forward_list. This has the desirable properties of sequential access and O(1) deletions.

forward_list has the lowest memory overhead of all std C++ containers (a single pointer to the next element.

Other valid options considered:

Keep in a vector but use tombstones for removed items. This has the

advantage of eliminating the memory moves for each replacement, and it doesn't have the problem forward list has with giving us a way to know the position in the input text (see later). This is quite a tricky implementation but perfectly feasible.

Keep in a vector and do the memory moves. Requires a lot of memory

bandwidth and cpu for the copying but it is simple.

Pair frequencies

Ultimately I needed multiple structures here as I wanted to support more than one conflict resolution strategy and since these are picked by the user at runtime we need dynamic dispatch. So first I made a virtual class with the required interface for both:

template<typename T>
class PairCount {
public:
    // Virtual destructor to ensure proper cleanup of derived classes.
    virtual ~PairCount() = default;

    // Gets the total number of unique pairs stored.
    virtual size_t get_count() = 0;

    // Retrieves the count for a specific pair.
    virtual optional<int> get_pair(pair<T,T> mp) = 0;

    // Creates a new pair or modifies the frequency of an existing one.
    virtual bool create_or_modify_pair(T a, T b, int freq) = 0;

    // Gets the pair with the highest count.
    virtual optional<pair<T,T>> get_top_pair_count() = 0;

    // Retrieves all pairs and their counts.
    virtual std::vector<std::vector<T>> get_all() = 0;
};

Note that the class has a template parameter, as the Tokenizer can be recompiled with different underlying numeric types for the tokens.

Conflict resolution strategy: First seen in input
Imagine a sequence as follows:

1,2,8,9,3,4…

After counting all the pairs we find that [1,2] and [3,4] have the same frequency.
1. [1,2] => 20
2. [3,4] => 20
In this case we pick the one added first, which means the one first seen in the input text.

In Python this insertion order comes for free because of Raymond Hettinger's 2012 redesign of the Python dictionary. Implemented in Python 3.6 (released December 23, 2016), introduced compact dictionaries with key-sharing and faster performance. A side effect of this redesign was that dictionaries began preserving insertion order as an implementation detail. This was later formalized as a language guarantee in Python 3.7 (released June 27, 2018), meaning dictionaries officially maintain the order of key-value pairs as they are inserted.

In Karpathy's code you can see that he simply relies on this behaviour to get the consistent result based on above.
```
# count up the number of times every consecutive pair appears
stats = get_stats(ids)
# find the pair with the highest count
pair = max(stats, key=stats.get)
```
And from the Python documentation: https://docs.python.org/3/library/functions.html#max

If multiple items are maximal, the function returns the first one encountered. This is consistent with other sort-stability preserving tools such as sorted(iterable, key=keyfunc, reverse=True)[0] and heapq.nlargest(1, iterable, key=keyfunc).

In order to implement that we must track the insertion order. Rather than let the user deal with that I built it into the PairCount class. As elements are added, new ones get the current count and the count is incremented.

Picking a data structure here is tricky because we want to be able to quickly store and modify pair frequencies (unordered_map), and a way to get the most frequent (priority_queue). Furthermore, we want to keep track of insertion order?

Sometimes you need to use multiple data structures to support a use case with conflicting requirements. For this purpose I used the boost::multi_index.

https://www.boost.org/doc/libs/1_88_0/libs/multi_index/doc/index.html

There's nothing to stop you from using a set and a priority queue and tracking them yourself, but multi_index handles that for you based on the declaration of which indexes and access patterns you need.

Let's take a look at the implementation of PairCountInsertOrder:

First the data; we need to store pair, the count and the insert order.
```
template<typename T>
struct PairCountOrder {
    ::pair<T,T> pair;
    int count;
    size_t insert_order;

    PairCountOrder(::pair<T,T> p, int c, size_t fo) : pair(p), count(c), insert_order(fo) {}
    PairCountOrder(::pair<T,T> p, int c) : pair(p), count(c), insert_order(std::numeric_limits<size_t>::max()) {}
};

// Comparison struct for sorting. Sorts by count (descending), then by insertion order (ascending).
template<typename T>
struct CompareCountOrder {
    bool operator()(const PairCountOrder<T>& a, const PairCountOrder<T>& b) const {
        if(a.count == b.count) {
            return a.insert_order < b.insert_order;
        } else {
            return a.count > b.count; // higher count is greater
        }
    }
};
```
Next we define the container itself. We just specify the indexes required and Boost takes care of picking the underlying data structures.
```
template<typename T>
using PairCountStore = boost::multi_index_container<
    PairCountOrder<T>,
    indexed_by<
        // Index 0: Hashed unique index on the 'pair' member for fast lookups.
        hashed_unique<member<PairCountOrder<T>, pair<T,T>, &PairCountOrder<T>::pair>>,
        // Index 1: Ordered non-unique index for sorting by count and insertion order.
        ordered_non_unique<identity<PairCountOrder<T>>, CompareCountOrder<T>>
    >
>;
```
Index 0 explanation: It is hashed so we should get an O(1) lookup type, and unique meaning keys are unique, each pair can occur once only. The rest of the declaration explains how to get the key for this index (use the pair member).

Index 1 explanation: This needs to be an ordered collection so we can extract the highest frequency. It also needs to be non-unique (in its sort criteria), because we can have multiple elements with the same frequency.

Now in our code we can grab the appropriate index depending on the current purpose and when we make modifications to the data the boost library will ensure the changes are synchronized across all the indexes in the container.
```
auto& index_by_key = pcs.template get<0>();
auto f = index_by_key.find(mp);
if(f != pcs.end()) {
    index_by_key.modify(f, [freq](PairCountOrder<T>& pc) { pc.count += freq; });
    return false;
} else {
    pcs.insert(PairCountOrder<T>(mp, freq, next_insert++));
    return true;
}
```

Conflict resolution strategy: Lexicographical

Referred to as lexical in my implementation to save typing, this method means we pick from pairs based on which comes first. For example given the following two pairs:

[1,2] => 20
[3,4] => 20

They have the same frequency so we pick pair 1) as 1 < 3. The second member of the pair is used as the tie-breaker, and of course if both members are the same then they would be combined to a single entry in the PairCount.

Again a multi_index container is needed here. Let's start with the data:

template<typename T>
struct PairCountLexical {
    ::pair<T,T> pair;
    int count;

    PairCountLexical(::pair<T,T> p, int c) : pair(p), count(c) {}
};

// Comparison struct for sorting. Sorts by count (descending), then by pair (lexical ascending).
template<typename T>
struct CompareLexicalOrder {
    bool operator()(const PairCountLexical<T>& a, const PairCountLexical<T>& b) const {
        if(a.count == b.count) {
            if (a.pair.first == b.pair.first) {
                return a.pair.second < b.pair.second;
            } else {
                return a.pair.first < b.pair.first;
            }
        } else {
            return a.count > b.count; // higher count is greater
        }
    }
};

And the container looks like this:

template<typename T>
using PairCountLexicalStore = boost::multi_index_container<
    PairCountLexical<T>,
    indexed_by<
        // Index 0: Hashed unique index on the 'pair' member for fast lookups.
        hashed_unique<member<PairCountLexical<T>, pair<T,T>, &PairCountLexical<T>::pair>>,
        // Index 1: Ordered non-unique index for sorting by count and lexical order.
        ordered_non_unique<identity<PairCountLexical<T>>, CompareLexicalOrder<T>>
    >
>;

Index 0 explanation: Same as above this gives us fast insert, modify and lookup for the pair frequencies.

Index 1 explanation: Same as above except the outcome is different because of the implementation of CompareLexicalOrder.

Optimization of frequency counts
When running the code I see that the biggest cost is regenerating the frequency map each step. For example when churning through wikitext (a 500mb text corpus) it takes the Python code 28 seconds on my Macbook to count all the pairs.

Let's work through Brendan Gregg's impactful optimizations:
1. Don’t do it.
2. Do it, but don’t do it again.
3. Do it less.
4. Do it later.
5. Do it when they’re not looking.
6. Do it concurrently.
7. Do it more cheaply.
Don't do it is not an option, we need those updated counts each step. Do it but not again is fruitful though.

The key insight here is that we only need to do a full frequency count one time. Then we can incrementally update the pair frequencies as we walk through doing the merge process. Essentially we are removing and adding a number of pairs on each replacement.

I noticed that the authors of the paper mentioned this too:

"In practice, we increase efficiency by indexing all pairs, and updating data structures incrementally."

You can see their incremental update code here:

https://github.com/rsennrich/subword-nmt/blob/92d6139d07d30e12735a0af9e7f7f925ebe62c54/subword_nmt/learn_bpe.py#L159

In addition to this optimization they use a pruning technique that drops frequencies of pairs below some threshold. This makes sense because the Python max function iterates the whole collection. In my case our data structures do not, so pruning is probably not worth the additional complexity. Worth trying maybe?

In any case, for my lexicographical conflict strategy I do implement this optimization and it is a huge win on performance as shown in the charts.

Crucially, it is not implemented for the first occurring strategy, because the current implementation gives now way to easily keep track of the first occurence of a pair in the input corpus.

Next steps

For me the project is at a good point to move on to other things but there are some things I would do next otherwise:

Port to Zig. Currently I'm using Zig for some other projects and would be interested in the porting experience and how the performance compares.
Work on different data structures for the input text to support incremental frequency counting for the first strategy.
Optimization of the encode and decode steps.
Implement download and conversion of GPT merges like Karpathy does in his gpt4 code.
Look at implementations of other Tokenization algorithms.
Optimization by parallel computation. At face value it seems possible to do the merge process on multiple cores using a divide and conquer approach. Edge cases where the sections overlap may be tricky.

Conclusion

I had a lot of pain and a lot of fun working on the code. I highly recommend this kind of process to fully understand the nuances and implementation details required for AI engineering.

As a refresher on modern C++ this was a great project. (I recommend (Tour of C++)[https://www.amazon.ca/Tour-C-2nd-Bjarne-Stroustrup/dp/0134997832] and https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines.

Apart from the complexity of CMake I found that using C++ today is a pleasant and safe experience as long as you carefully tread the recommended path.

References

Karpathy's youtube

https://www.youtube.com/watch?v=zduSFxRajkE

And his Python code

https://github.com/karpathy/minbpe

If you want to dive into the code or run the benchmarks yourself, you can find the full project on GitHub.

https://github.com/justinhj/minbpe-cc

An early paper on bpe for tokenization is "Neural Machine Translation of Rare Words with Subword Units"

https://arxiv.org/pdf/1508.07909

The original source code from the paper.

https://github.com/rsennrich/subword-nmt

Thanks for reading!

The Magic of Lazy Lists

Sat, 05 Nov 2022 00:00:00 +0000

Photo by Dollar Gill on Unsplash

Introduction

As the Scala 3 - book points out, Scala has a rich set of collection classes. As well as List, of course, it also has a solid implementation of LazyList. If you're not sure what that is, or what is used for, read on and find out, plus even better I will walk through a full implementation of LazyList that can do some magical things.

Scala List can represent collections of zero or more, stored as a linked list, with the details of the underlying data structure abstracted away. In my video NonEmptyLists more or less I talked about how you can build a variant of List that can only be a collection with one or more items.

In this video, I will present the theory and practice of building a LazyList type, that adds the additional capability of controlling when elements are evaluated.

All the code written in this post, and the accompanying video The Magic of Lazy Lists can be found in my new educational Scala library Duct. In order to produce this implementation I studied the code of the Scala standard library (both the current version and history versions which are less sophisticated but also easier to read), as well as other implementations such as that of the ScalaZ Ephemeral list. The resulting code is a combination of these with some of the best parts of both, as well as taking some advantage of Scala 3 features along the way.

Implementing Lazy Thing

LazyList is easier to understand if you have a good grasp of different evaluation models in Scala, so let's explore that with a custom class called LazyThing.

With this implementation, LazyThing is just a wrapper of values, with a get function that returns the value. This is what we call eager, or strict evaluation. When I pass the expression {println("evaluated"); 10 is passed into the LazyThing constructor it is evaluated immediately and stored in the class. Later when the user calls the get method we find that we just get the value; nothing is evaluated again.

class LazyThing[A](a: A):
   def get = a

val lt = new LazyThing({println("evaluated"); 10})
// evaluated
lt.get
// val res4: Int = 10

When working with a LazyList we want to be able to populate it with expressions but without having them evaluated until we are ready (what Haskell refers to as call by need). What else can we use in Scala that only evaluates when we want it to? Functions! If the argument was a function, we could simply call it when the user calls get, making it lazy.

Now when we create the class nothing is evaluated until we call get, and then it is evaluated every time. This evaluation mode is called always.

class LazyThing[A](a: () => A):
   def get = a()

val lt = new LazyThing(() => {println("evaluated"); 10})

// scala> lt.get
// evaluated
// val res15: Int = 10

// scala> lt.get
// evaluated
// val res16: Int = 10

The LazyList structure is not about always evaluation though, it is about lazy or call by need evaluation. We want to be able to remember the result of evaluated list elements, and never evaluate them again. This memoization is the next step.

class LazyThing[A](a: () => A):
   var evaluated = false
   var value: A = _
   def get = if evaluated then value
     else
         evaluated = true
         value = a()

val lt = new LazyThing(() => {println("evaluated"); 10})

// scala> lt.get
// evaluated
// val res17: Any = ()

// scala> lt.get
// val res18: Any = 10

Now you can see that the value is evaluated only once and we can retrieve it multiple times. Memoization is good because it saves us from recomputing values, but it also means we must be mindful of memory use and hanging on to references to the internal structure of our LazyList so as not to consume memory that is no longer needed.

Two final simplifications using Scala features make this much more succinct. The mechanism of passing an argument as a function executed only on first reference is implemented within Scala and known as call by name. Rewriting like below uses that mechanism instead.

Secondly, we can replace the manual memoization code that remembers the evaluated value with lazy val which does the same thing but, again, is built into the compiler.

class LazyThing[A](a: => A):
  lazy val get = a

val lt = new LazyThing({println("evaluated"); 10})
// scala> lt.get
// evaluated
// val res24: Int = 10

// scala> lt.get
// val res25: Int = 10

Beginning LazyList

Let's begin by representing the LazyList as a sealed trait, which will be the object through which users interact with the collection.

sealed trait OurLazyList[+A]:
   def head: A
   def tail: OurLazyList[A]
   def isEmpty: Boolean

Of note here is the +A variance notation. It's important to know about and understand variance when making libraries in Scala, slightly less important when writing application code. A short explanation of variance is that it is short for "variance under inheritance".

Let's say we have a type Loan and two other subtypes of Loan, Credit Card and Amortized Loan. If you have some function that takes Loan and prints the outstanding balance, you would expect through normal rules of inheritance to be able to pass in a Credit card or an amortized loan in place of the Loan. You can use a subtype of loan wherever the compiler is expecting a loan. That is what is known as behavioural subtyping.

What variance under inheritance refers to, is what inheritance means when we have some parameterized type such as a collection. If I have a function that takes a list of Loans, should it accept a list of subtypes? Credit cards for example. Because the answer to this is, no not always, Scala includes variance annotations so that you can choose the variance relationship you want as needed. I'll come back to this topic in more detail in a later video.

LazyList will have a companion object containing all the static methods that will be used to create and manipulate lazy lists. The first thing we need is a representation of empty list. We add that to a new companion object.

object LazyList:
  val empty = new LazyList[Nothing]:
      def head = throw new NoSuchElementException("Cannot get head of empty lazy list")
      def tail = throw new UnsupportedOperationException("No tail of empty lazy list")
      val isEmpty = true

Lazy list has the type Nothing. Nothing is at the bottom of Scala's type hierarchy meaning it is the subtype of everything. Now it's not a useful type in itself, because you can't do anything with it, but it is really useful in this context… our empty list is a singleton value shared by all lazy lists, we only need one. Why does this work? Because of the variance annotation above. We said that a list of subtypes of A would be acceptable as list of A.

So now we are able to create lazy lists with nothing in them using LazyList.empty. The next step is to be able to create lists with elements inside. We will call this the cons method, as it will be used to construct lists one lazy element at a time.

// object LazyList continued:
def cons[A](hd: => A, tl: => LazyList[A]) = new LazyList[A]:
  lazy val head = hd
  lazy val tail = tl
  val isEmpty = false

With this small amount of code we have a functional (no pun intended) lazy list.

val ll = LazyList.cons({println("evaluated!");10}, LazyList.empty)
// nothing is printed yet!
ll.head
// evaluated!
// val res9: Int = 10

ll.head
// val res10: Int = 10

Here you can see that constructing the list did not evaluate the value we passed in to be the head of the collection. Once we retrieved the head we got the evaluation happen, but subsequently we did not not. Nice.

Pattern matching and the "cons operator"

In Scala you can construct lists using the so-called cons operator ::. For example:

val l = 1 :: 2 :: 3 :: List.empty
// Creates a List[Int] = List(1, 2, 3)

This is convenient so Scala's standard LazyList also implements this using the syntax #::. Let's do the same for Duct. There are two things to note here:

To make this work we want #:: to be a right associative function that cons's a new head for the list to the tail which is to the right
The type of the operation should be a cons operation on a list.

To append 1 to the list val ll = (2,3) we need to write 1 #:: ll and we want the compiler to evaluate this as:

ll.#::(1)
// where the type of LL is LazyList[Int]

Note that in Scala, by convention, anything ending in a colon is right associative, which is what we want here. Also not that in Scala 3 we can write this as an extension method. In the standard library you'll see code like the following:

implicit def toDeferrer[A](l: => LazyList[A]): Deferrer[A] = new Deferrer[A](() => l)

final class Deferrer[A] private[LazyList] (private val l: () => LazyList[A]) extends AnyVal {
  /** Construct a LazyList consisting of a given first element followed by elements
    *  from another LazyList.
    */
  def #:: [B >: A](elem: => B): LazyList[B] = newLL(sCons(elem, newLL(l().state)))
  /** Construct a LazyList consisting of the concatenation of the given LazyList and
    *  another LazyList.
    */
  def #:::[B >: A](prefix: LazyList[B]): LazyList[B] = prefix lazyAppendedAll l()
}

scala.collection.immutable.LazyList.scala

With Scala 3 we can simply implement this as an extension method on the LazyList trait. Much nicer.

extension [A](hd: => A)
  def #::(tl: => LazyList[A]): LazyList[A] = 
   LazyList.cons(hd, tl)

Now we can create lazy lists more easily as follows:

val ll = 1 #:: 2 #:: LazyList.empty
// val ll: LazyList[Int] = LazyList$$anon$2@687292c5

Creating a lazy list with the cons operators is one thing but users will expect to be able to deconstruct lists in a pattern match expression to. Let's add that functionality next.

In Scala you implement pattern matching on a particular type by implementing unapply on an object with that types name, in our case #::.

object #:: {
    def unapply[A](s: LazyList[A]): Option[(A, LazyList[A])] =
        if !s.isEmpty then Some((s.head, s.tail)) else None
}

The way unapply works is the opposite of a constructor. Given a constructed type, unapply tries to extract the pieces. This is a partial function, it does not have to succeed, so it returns the pieces as an Option.

Now we can write lazy code using pattern matching:

def ourMap[A, B](ll: LazyList[A], f: A => B): LazyList[B] =
  ll match {
    case hd #:: tl =>
      LazyList.cons(f(hd), ourMap(tl, f))
    case _ =>
      LazyList.empty
  }

Iterating over Lazy List

Note that although the destructuring (pattern matching) of lazy lists is often useful, in my final implementation for the Duct library I opted for the following more simple approach to the map function, shared here because I implemented many of the functions that iterate over lazy lists in the following way:

def map[B](f: A => B): LazyList[B] =
  if isEmpty then LazyList.empty
  else LazyList.cons(f(head), tail.map(f))

Another useful function is forEach, which you can use to execute some action across the lazy list. This function highlights a couple of interesting things.

When working with laziness always consider when you want preserve it versus lose it. The forEach function by definition must visit every element of the list and therefore does not preserve laziness.
If possible you should make recursive functions tail recursive, otherwise they are limited by the stack. This implementation is tail recursive. We can tell the compiler to make sure that it is with the annoation.

@tailrec
final def forEach(f: A => Unit): Unit =
  if !isEmpty then
    f(head)
    tail.forEach(f)

And you can use it as follows. Note that I'm using the LazyList.apply method here is a convenience to create a lazy list from a variable argument list.

val list1: LazyList[Int] = LazyList(1,2,3)

println("forEach list1")
list1.forEach { a =>
  println(a)
}

// forEach list1
// 1
// 2
// 3

Filtering

Another part of the implementation worth looking at is dropping elements that pass or fail some filter, namely filter and dropWhile. Let's first think about what the semantics are here in terms of laziness.

Given a lazy list and a filter function we want the user to be able to iterate through them by need.
When the user calls head on a lazy list where many elements fail the filter before a good one comes, many elements are evaluated.
We must stop evaluating the elements as soon as we find one that passes the filter, and return that as a lazy list to the caller.

We have to be careful about laziness then. Let's first think about dropWhile. This takes lazy list with all the failing elements dropped.

@tailrec
final def dropWhile(f: A => Boolean): LazyList[A] =
    if isEmpty then LazyList.empty
    else if f(head) then tail.dropWhile(f)
    else this

Now since we want this to work on many elements potentially, it is important to be tail recursive. With dropWhile we can take list such as LazyList(1,2,3,4,5) and drop all elements less than 3. What we get back is LazyList beginning with 3.

Take a moment to think about which elements have been evaluated at this point.

Whether you reason about it by looking at the code or thinking about it semantically, the answer is that the 3 is evaluated and the 4,5 elements are in a lazy tail. dropWhile then will evaluate elements up to and including the first one that should not be dropped.

Once you implement dropWhile it can be used to implement filter with the requirements we came up with above.

def filter(f: A => Boolean): LazyList[A] =
    val dropped = this.dropWhile(a => !f(a))
    if dropped.isEmpty then LazyList.empty
    else LazyList.cons(dropped.head, dropped.tail.filter(f))

Infinite lists

Quite a few years ago I was working through a Haskell tutorial for beginners. Some of the examples worked with infinite lists; mapping them, filtering them, and zipping them together. At the time my knowledge of evaluation and laziness was not sophisticated. As they say, any sufficiently advanced technology is indistinguishable from magic. Since Haskell was doing things more advanced than I understood at the time, I thought of infinite lists as being a magic trick.

As you've seen so far, I hope, the mechanisms of lazy evaluation make working with infinite lists possible and don't require a lot of work. Let's look at how what we've done so far scales effortlessly from small lists to infinite ones.

def repeat[A](a: A): LazyList[A] = a #:: repeat(a)
def from(n: Int) : LazyList[Int] = n #:: from(n+1)
def iterate[A](a: A)(next: A => A): LazyList[A] = a #:: iterate(next(a))(next)

Note how these functions build on what we did so far, and give us ways to declaratively create infinite lists.

repeat provides a lazy list with a head of type A. When the use takes the tail they get the same thing and so on forever. This gives us a definition of an infinitely repeating constant.

from shows how we can incrementally generate numbers from some starting value n. Note that the tail is a function that takes input from the previous call; in this way we can pass information through an infinite chain of computation!

iterate is a generalisation of this allowing you to take some function that creates a new A from the previous one, forever.

Of course, we don't want to actually evaluate infinite lists because we don't have time for that, so you would use take and drop and other filtering mechanisms to work with only the values you are interested in. As we will see, there are times when you don't know how many of a thing you need and it may be expensive to generate them, so call by need evaluation is what we want.

Fusion of operations

Imagine the following code.

val lotsOfThings = List.fill(1)(10000000)
lotsOfThings.map(a => expensiveCalculation(a)).filter(a => a < 10).map(a => expensiveCalculation2(a)).take(10).sum

With a strictly evaluated list what happens here?

map will iterate over the large list, doing expensiveCalculation 10m times and making a new list of 10m elements.
filter will walk that new list and create a new list with up to 10m elements that pass the filter.
map will take those elements and create a new list after calling expensiveCalculation2 on each element
take will drop all elements after the 10th one
sum iterates over the elements

Whilst this kind of code is not typical, you are hopefully not working with lists this big, but if the use case requires it, then lazy lists provide a potentially much more efficient way of working.

The same code as a lazy list would work this way.

map takes the large list and returns a lazy list where, when evaluated, head will have expensiveCalculation applied to it. This is O(1).
filter will internally call dropWhile. Let's pretend the filter is true because a < 10 and we return a new lazy list with the filter but paused at the first element.
map will take that list and again, return a new lazy list that is unevaluated and ready to run expensiveCalculation2 if anyone asks.

Observation… we are turning our list of values into a list of delayed computations. This takes up more memory than a list of values because each step is wrapped in a Function object.

take will now return a lazy list that keeps track of a counter and stops (returns an empty tail) when it runs out, so we set a bound on our computation.
sum okay now we're going to do a bit more work. sum calls foldLeft (see below), which by definition must evaluate all the items and combine them to a single result

@tailrec
final def foldLeft[B](z: B)(f: (B, A) => B): B =
    if isEmpty then z
    else tail.foldLeft(f(z, head))(f)

Now more serious evaluation will happen. What we have at this point is a sort of stack of computations for each successive element. We will call expensiveCalculation1 and expensiveCalculation2 only as often as needed to evaluate the 10 elements.

This is all rather hard to conceptualize, so here's a picture that may help. The call stack shown in the middle of the foldLeft shows that the lazy list we evaluate consists of a stack of function calls that are waiting to happen.

Fusion of operations means that a sequence of complex, expensive operations, can be limited to only the number of elements you are interested in and performed per element, not across the whole collection. This is the essence of being able to control evaluation for your own needs.

This gives us some insight on when to use a lazy list (or equivalent structures such as streams, iterators), rather than concrete immutable containers.

Use lazy lists when you need to execute an expensive sequence of operations and you don't expect to consume the majority of the collection.

You need to use some discretion here. If you can't guarantee that the whole list won't be executed, it's probably not a good use case. But this technique translates well to a computation where we never see the whole list (streaming applications that work with Kafka and Kinesis for example).

Laziness for convenience

Some algorithms require you to provide a list of things but you don't know how many things you need in advance. Here's an example that appears in the paper Applicative Programming with Effects that transposes a matrix.

You can see this code also in my post about the paper at What's Ap?, although the coverage there is more about how this operation can be written in "the applicative style".

First, let's represent a 2-dimensional matrix as a lazy list of lazy lists.

val matrix = LazyList(
  LazyList(11, 12, 13, 14, 15),
  LazyList(21, 22, 23, 24, 25),
  LazyList(31, 32, 33, 34, 35)
)

The idea of transposing a matrix is you "rotate" it such that if you started with n rows and m columns, you would end up with a rotated matrix with m rows and n columns.

Rotated by hand and represented in code, this 3 by 5 matrix should be transposed to the following.

val matrix = LazyList(
  LazyList(11, 21, 31),
  LazyList(12, 22, 32),
  LazyList(13, 23, 33),
  LazyList(14, 24, 34),
  LazyList(15, 25, 35)
)

In order to implement this a nicely functional, declarative way, we first need a helper function zipWith that takes two empty lists and lets us combine them with a function.

def zipWith[A, B, C](as: LazyList[A], bs: LazyList[B])
  (f: (A, B) => C): LazyList[C] = as.zip(bs).map { case (a, b) => f(a, b) }

An important property of zip is that given two lists it combines them together into a new list of tuples, the length of which is bounded by the shortest one. This means we can combine zip and lazy lists to zip together two lists, one of which is infinite and the other is bounded. That's the technique used here.

def transpose[A](
    matrix: LazyList[LazyList[A]]): LazyList[LazyList[A]] =
  if matrix.isEmpty then LazyList.repeat(LazyList.empty) then
  else zipWith(matrix.head, transpose(matrix.tail))(_ #:: _)

Is it easy to understand? No, it takes a bit of thinking about to understand what is going on (as an exercise I'd suggest adding some println to see how it works). What's more interesting though, is that this is a much more functional, declarative version of matrix transpose. Imagine writing this in Go and you will do it as a for loop, taking care not to make any mistakes. Even though matrix transpose is simple, functional programming scales up to bigger more complex programs, whereas the imperative version is more of a one-off implementation.

The "trick" in the code above is in the LazyList.repeat. The iteration of the transpose works along each row of the matrix producing the new columns with cons, but at some point it runs out of rows and it needs another row of empty lists to finish the new rows off. How many empty lists does it need? Well, we could work it out by counting, but why not just say here is an infinite number, and let the zip figure out when to stop?

Folding left and right

There are a couple of interesting things to say about folding lazy lists. Firstly let's look at stack safety.

As we saw earlier the amount of memory used by a lazy list can be higher than with a regular list since with fusion between operations we can end up with a stack of function objects before it is evaluated. For that reason and just in general we may want to operate on large lists, it's important to consider which operations are stack safe and which are not.

For a stack safe function I present foldLeft.

@tailrec
final def foldLeft[B](z: B)(f: (B, A) => B): B =
  if isEmpty then z
  else tail.foldLeft(f(z, head))(f)

This is a so-called aggregate function that takes a collection, in this case, iterates over it and produces some aggregate value. The supplied function from the user is applied to each element along with some accumulating value. In the case of this implementation, the foldLeft recursive call is in tail position which means we can assume it uses tail call optimization. We add the annotation to tell the compiler we think so, and it will both complain if it is not eligible.

def incN(n: Int, inc: Int): LazyList[Int] =
  LazyList.cons(n, incN(n + inc, inc))

println(
  incN(1, 1).take(10000000).foldLeft(BigInt(0)) { case (acc, a) => acc + a }
)

This function adds up 10m integers and as such takes up a lot of stack space and crashes. Except it doesn't! Why? Because of the tail call optimization.

Now it will, in fact, take a good few seconds on modern hardware, which is a long time, and it may in fact crash with out of memory or be pathologically slow. Why? Because we are creating a lot of garbage here, in the order of gigabytes, and that takes a lot of work to clear up.

Make sure you have a decent amount of heap and use the G1 garbage collector via these settings (this is for running sbt, you can set the same JAVA_OPTS for IDE's and so on).

SBT_OPTS="-XX:+UseG1GC -Xmx4G" sbt

So foldLeft is stack safe, how about foldRight?

def foldRight[B](z: => B)(f: (A, => B) => B): B =
  if isEmpty then z
  else f(head, tail.foldRight(z)(f))

Note that the problem here is that the recursive call is not a tail call position, in this case, the user function f is. That means we can't use the tailrec annotation and it will not be tail call optimized.

Can we infer from this situation that foldRight is useless? No actually. It has a property that foldLeft does not, that of being able to terminate early. Just like with fusion of operations, the early termination of foldRight can be used to save us work, and make code more efficient.

How does that work? The "trick" here is that the second argument of the user function, the accumulator, is a call by-name value. It's lazy! That means we don't have to evaluate it.

This example code uses foldRight to find "tuna" in a list of fish.

def hasTuna(ll: LazyList[String]): Boolean =
  ll.foldRight(false){
    (next, z) => 
      println(next)
      if next == "tuna" then
        true
      else
        z
  }

hasTuna(LazyList("salmon", "shark", "tuna", "moray", "goldfish", "eel"))
// prints:
//   salmon
//   shark
//   tuna

This is simply not possible with foldLeft, nor is it possible if you don't use a call by-name argument for the accumulator in foldRight. If you're not sure why it is not possible for foldLeft, try putting some println statements into things that you foldLeft and foldRight and see the order in which things are done.

By the way, if you try this with the standard library you'll find it does not work the same way. The signature of foldRight is as follows:

def foldRight[B](z: B)(op: (A, B) => B): B

Without even trying it we know that it must expand the whole collection, although feel free to try it if you need to prove it to yourself. There has been some discussion on this, for example.

https://stackoverflow.com/questions/7830471/foldright-on-infinite-lazy-structure http://voidmainargs.blogspot.com/2011/08/folding-stream-with-scala.html

As noted in the second example the following code will work with a lazy aware foldRight only.

LazyList.repeat(true).foldRight(false){_ || _}

Last words

Maybe LazyList is not something you will use very often but I think some of the ideas here are central to functional programming. When you are working with streaming libraries like fs2, or effect libraries like Zio, this idea of building up some structure first, then evaluating it, is very powerful, and understanding lazy lists in some depth will hopefully help your way of thinking in your day to day Scala code!

Thanks for reading, if you enjoyed this content please share with a friend. If not, drop me a note and tell me what I can do better next time.

References

Functional Programming in Scala (aka the red book) - has a great chapter on lazy lists Functional Programming in Scala - Manning Press

LazyList Scala standard library 2.13 - modern day production ready code https://www.scala-lang.org/api/2.13.x/scala/collection/immutable/LazyList.html

Stream from Scala standard library 2.7 - older and simpler version which I found easier to understand https://github.com/scala/scala/blob/v2.7.7/src/library/scala/Stream.scala

Scalaz Ephemeral Stream - did some things I liked too https://github.com/scalaz/scalaz/blob/ea81ca782a634d4cd93c56529c082567a207c9f6/core/src/main/scala/scalaz/EphemeralStream.scala

All of the code for the Lazy List class can be found in the Duct library here https://github.com/justinhj/duct/blob/video17/core/src/main/scala/org/justinhj/duct/datatypes/LazyList.scala If you dig around in the code, or find in files for LazyList, you will see there is also a test suite and a few examples.

Monad Transformers - the prelude to ZPure

Fri, 04 Jun 2021 00:00:00 +0000

Readers and Writers and Transformers

BTW you can check out the video here instead: Functional Justin - Another Angle on Monad Transformers

There are twenty pages here or over an hour of videos, so let me help you decide if it's worth your time.

Who are you?

You are a Scala programmer or interested in Scala and may have some Haskell or interest in pure functional programming.

What do I cover?

Introduction to Monad transformers
Implement the WriterT monad in Scala 3 from scratch and used in a program
Implement the ReaderT monad in Scala 3, also from scratch, and use it
Why not both? Stack the Reader and Writer monads on top of each other
Effect rotation and ZIO Prelude's ZPure, another approach to combining effects

Evaluating expressions

In previous blogs and videos, I've described a program that evaluates arithmetic expressions as this is a nice testbed for various functional effects. So far I've demonstrated how by using different data types and type classes one can make the same program behave differently.

This is important for a couple of reasons. It means that you can compose interesting programs from smaller, well-understood components, and because we can understand and change the behaviour of our program by using different types.

As a starting point, lets begin with a version of the program that has error handling using the Either data type and Numeric which is a generic implementation of numbers.

If you didn't catch up on earlier posts let's recap. An example run of the program requires an environment (symbol table) provided using the Scala 3 implicit mechanism (using the given keyword). I call eval on a sample expression tree which yields either a Right or a Left result since I'm using Either as the result effect type.

  given envMap: Env[Int] = Map("x" -> 7, "y" -> 6, "z" -> 22)

  val exp1 : Exp[Int] = Mul(Var("z"), Add(Val(10), Sub(Var("x"), Var("y"))))

  val eval1 = eval(exp1)

  println(s"Eval exp gives $eval1")

// [info] running Scala3EvalEither 
// Eval exp gives Right(242)

I represent the errors as a Scala 3 enum which is a nice way to create ADT's (algebraic data types), similar to how you would do it in Rust.

enum EvalError {
  case InvalidSymboName
  case SymbolNotFound
  case DivisionByZero
}

The next things to look at are an ADT to represent the program steps a return type.

enum Exp[A]:
  case Val(value: A) extends Exp[A]
  case Add(left: Exp[A], right: Exp[A]) extends Exp[A]
  case Sub(left: Exp[A], right: Exp[A]) extends Exp[A]
  case Mul(left: Exp[A], right: Exp[A]) extends Exp[A]
  case Div(left: Exp[A], right: Exp[A]) extends Exp[A]
  case Var(identifier: String) extends Exp[A]

type Env[A] = Map[String, A]

import Exp._

type WithEnv[A] = Env[A] ?=> Either[EvalError, A]

The Env type is a simple map for strings to values that we will use as a symbol table so that variables can be looked up at runtime. The ?-> syntax indicates that the return type is a context function. An earlier blog discusses that, but in short, it allows us to thread our Env symbol table through the computation easily.

Here is the main body of the code.

def eval[A : Numeric](exp: Exp[A]): WithEnv[A] =
  exp match
    case Var(id) => handleVar(id)
    case Val(value) => Right(value)
    case Add(l,r) => handleAdd(l,r)
    case Sub(l,r) => handleSub(l,r)
    case Div(l,r) => handleDiv(l,r)
    case Mul(l,r) => handleMul(l,r)

def handleAdd[A : Numeric](l: Exp[A] , r: Exp[A] ): WithEnv[A] = eval(l) + eval(r)
def handleSub[A : Numeric](l: Exp[A] , r: Exp[A] ): WithEnv[A] = eval(l) - eval(r)
def handleMul[A : Numeric](l: Exp[A] , r: Exp[A] ): WithEnv[A] = eval(l) * eval(r)
def handleDiv[A : Numeric](l: Exp[A] , r: Exp[A] ): WithEnv[A] = eval(l) / eval(r)

def handleVar[A](s: String): WithEnv[A] =
  summonEnv.get(s) match {
    case Some(value) => Right(value)
    case None => Left(EvalError.SymbolNotFound)
  }

Those arithmetic operators you see are operating not on integers, doubles or some other concrete type, but are working on a type A that has a Numeric instance. You may wonder then how that + operator knows what to do? The answer is that I implemented an instance of Numeric for the type Numeric[Either[EvalError,A]].

// Implement Numeric for EvalResult
given evalResultNumeric[A: Numeric]: Numeric[Either[EvalError, A]] with {

  def add(fa: EvalResult[A], fb: EvalResult[A]): EvalResult[A] = {
    fa.map2(fb)((a,b) => a + b)
  }
  // ... and so on
}

Whilst this is a lot of overhead for a simple program, as your programs scale in complexity, this level of abstraction lets you control effects as well as swap them in and out as your requirements change without having to rewrite the core logic.

As an example, let's introduce a Monad Transformer and show how to integrate it with the program above.

WriterT

Let's say we want to take an existing effectful program and add a new effect to it. The effect I will demonstrate is logging. There is a data type called Writer which represents a value and a log.

Writer[W,A](run: (W,A))

This is not very interesting on its own but if you make a program from Writers, sequencing them together using the Monad's flatMap operation for example, then the end result consists of a final value and a log for each step of the program.

But since I already committed to using Either, if I change the type to Writer then I would lose the ability to handle errors. Instead what I want is to keep the Either effect and wrap it with the capability of the Writer monad.

Monad transformers are the answer. Now the trouble with monads is that they don't compose manually together. As I covered in a previous blog, applicatives do. You can take any two applicative effects such as Either and List and compose them with simple functions.

With Monads the composition of any particular monad has to be hand-crafted, so if I want to stack a Reader on top of an Either, which I do, then I need to implement a ReaderT (reader transformer).

It only needs to be implemented once and for all and can then be applied to any other Monad (not just for Either). The idea is to make an implementation of Reader that wraps another Monadic data type.

case class WriterT[F[_],W,A](private val wrapped: F[(W,A)])

Here you can see the definition of the WriterT data type. The difference between WriterT and Writer is that the WriterT wraps an existing monad. Note that there is no need to constrain the higher-kinded type F to be a Monad, but later on when we use it in various ways it is possible to constrain F to be a Functor, Applicative or Monad depending on the use-case. Choosing the type bounds that constrain what the wrapped type must support based on the individual functions needs gives you more flexibility.

For example, if you have a data type that has a map operation but no meaningful way to make a flatMap, you can still use the Monad transformer as long as you only use Functor level methods.

Lifting

To use WriterT there needs to be a mechanism to take your inner data type (Either in this case) and make an instance of WriterT. That can be done like this in my implementation by using the WriterT constructor. For example let's say we have an Either instance we can transform it to a WriterT like this.

val e1: Either[EvalError,Int] = Right(10)
val w1 = WriterT(e1.map(n => (List.empty[String], n)))

It's not straightforward because the WriterT wrapped type must be F[(W,A)] and we had an F[A]. That is why I need to use the map operation to take any value the Either may have and combine it with an empty log. Here we assume the log is a list of strings and Scala is able to infer that too.

Since this needed often the lift method is often added which takes care of creating an empty log message and mapping it for us.

object WriterT:
  def lift[F[_],W, A](fa: F[A])(using m: Monoid[W], F: Functor[F]): WriterT[F,W,A] =
    WriterT(F.map(fa)(a => (m.zero, a)))

// ... 

  val e1: Either[EvalError,Int] = Right(10)
  val w1: WriterT[[A] =>> Either[EvalError,A],List[String],Int] = 
    WriterT.lift(e1)

Couple of interesting things to note about the lift method type signature. For one you can see that the log must be a Monoid. A Monoid is a type that must have two useful operations that make it useful for logs: It must be able to produce an empty element of whatever type it is specialized for, and it must be able to join that type together.

This gives the user the flexibility to use any data type for the log and not have to worry about providing an empty log or an append function. The example here is a monoid since it is a list of strings. Obviously we can produce an empty list, and the append function is also trivial, so if you look at my Monoid instance for lists you can see the implementation is trivial.

Another interesting thing is the Functor type constraint. As I mentioned above, although we call them Monad transformers, they can be used with Functors, Applicatives and Monads. Since the lift function only needs map, it needs only the Functor type constraint.

Evaluating expressions with a log

Now I'll walk through the changes needed to convert the expression evaluator from having the return type Either, to being one of WriterT[Either]

// Without log
type WithEnv[A] = Env[A] ?=> Either[EvalError, A]
// With log
type WithEnv[A] = Env[A] ?=> WriterT[[A1] =>> Either[EvalError, A1], List[String], A]

The next step is to make small changes to my programs implementation to manage this new type. As you can see, the simplest change, handling a basic numeric value, just involves lifting our original Either and adding a log entry.

1: def eval[A : Numeric](exp: Exp[A]): WithEnv[A] =
2:   exp match
3:     case Var(id) => handleVar(id)
4:     case Val(value) => WriterT.lift[[A1] =>> EvalResult[A1], List[String], A](Right(value)).tell(List(s"Val $value"))
5:     case Add(l,r) => handleAdd(l,r)
6:     case Sub(l,r) => handleSub(l,r)
7:     case Div(l,r) => handleDiv(l,r)
8:     case Mul(l,r) => handleMul(l,r)

You can see in line 4 the code is a matter of lifting the value wrapped in an Either. The type annotation is needed and creates some noise. I use the tell function to add a log entry for this step.

tell is a method on the WriterT data type itself, and it takes advantage of the log types monoid to combine this new log entry with any prior ones.

def tell(l1: W)(using m: Monoid[W], f: Functor[F]): WriterT[F,W,A] =
  WriterT(wrapped.map{
    (l2,a) =>
      (m.combine(l2, l1), a)
  })

By this technique at the end of a computation we should see a log of entries.

For example, the expression Val(10) would be logged as "Val 10". Having a step-by-step log of your application has various uses including the following.

Debugging. View the state of your computation in detail
Auditing and statistics. Analyze the log of your computation for business information.
Restore a failed computation. You can log at each step enough information to resume an expensive computation that may have been interrupted.

These kinds of benefits come with traditional logging, but building it into your application in a pure and type rich way can amplify the benefits.

Let's take a look at the symbol table lookup part of the program.

def handleVar[A](s: String): WithEnv[A] =
  summonEnv.get(s) match {
    case Some(value) => {
      WriterT.lift[[A1] =>> Either[EvalError,A1],List[String],A](Right(value)).tell(List(s"Var $s ($value)"))
    }
    case None => WriterT.lift(Left(EvalError.SymbolNotFound))
  }

Again the change is virtually mechanical. We lifted our old code and added the tell call to add some logging information. When we view variable lookups in the log you will see something like Var("x") written as Var x (7) where 7 is its value in the symbol table.

Extending numeric

def handleAdd[A : Numeric](l: Exp[A] , r: Exp[A] ): WithEnv[A] = eval(l) + eval(r)

The remainder of the program involves expressions like this one. We use the + operator to add two other expressions together. How that works is a combination of operator overloading, extension methods and implementing an implicit implementation of Numeric for our new WriterT return type.

Here I'm defining an implicit instance of Numeric that handles things are Writers around Eithers. In previous posts, this is where I first implemented addition for different types of number, and then added the ability to handle errors in a type safe and functional manner.

I'm just extending that technique to handle a more complicated type.

given evalResultWNumeric[A: Numeric]: Numeric[WriterT[[A1] =>> Either[EvalError, A1], List[String], A]] with
  // ... implementations

The implementation of Add assuming a Monadic instance is available is as follows.

val M = writerTMonad[[A1] =>> Either[EvalError,A1], List[String]]

def add(fa: EvalResultW[A], fb: EvalResultW[A]): EvalResultW[A] = {
    M.flatMap(fa) {
      a => M.map(fb){
        b =>
          a + b
      }
    } : EvalResultW[A]
  }

Which does the job but it doesn't include any logging. We can add that too.

def add(fa: EvalResultW[A], fb: EvalResultW[A]): EvalResultW[A] = {
    M.flatMap(fa) {
      a => M.flatMap(fb){
        b =>
          val result = a + b 
          val w1: EvalResultW[A] = WriterT.lift(Right(result))
          w1.tell(List(s"Added $a and $b giving $result"))
      }
    }
  }

Note that by nesting the flatMaps we have access to a,b and the result of the computation so we can put all of that into the tell call, resulting in a log like Added 22 and 23 giving 45.

There's nothing really wrong with this implementation, but it's important to always think about the principle of least power. Did I really need a Monad here? Well in fact there is a great function for applying a computation across two different effects and that is map2. It also requires only Applicative, so I can use that instead.

val App = writerTApplicative[[A1] =>> Either[EvalError,A1], List[String]]

 def add(fa: EvalResultW[A], fb: EvalResultW[A]): EvalResultW[A] = {
   App.map2(fa)(fb) {
     case (a,b) => a + b
   }
 }

This simplifies the code greatly but notice that I am no longer logging anything. Unfortunately, I no longer have access to the result of the computation. One clean solution I found here was to write a helper method that is like a logging version of map2. Like map2 it takes a function of two arguments to map the effect values, but it takes a second function that takes the two values and their result and lets you build a log entry from them.

def mapTell2[A,B,C,F[_],W](fa: WriterT[F,W,A],fb: WriterT[F,W,B],fabc: (A,B) => C,fabcw: (A,B,C) => W)
                           (using m: Monoid[W], f: Monad[F]): WriterT[F,W,C] = {
   val r = fa.unwrap().map2(fb.unwrap()){
     case ((al,a),(bl,b)) =>
       val c = fabc(a,b)
       val w = fabcw(a,b,c)
       val prev = m.combine(al,bl)
       (m.combine(prev,w),c)
   }
   WriterT(r)
 }

While this looks like a handful what it is really doing is straightforward. Like map2 the input is two effects. First I unwrap them which gives us the inner effect, and running map2 on those gives the log and the value of each effect.

Once I've run the user function fabc on those values, I have the result value c and I can use that to build a log with the fabcw function. Finally, we need to combine the prior logs with the new log and return the result.

Here's the function in action.

def sub(a: EvalResultW[A], b: EvalResultW[A]): EvalResultW[A] = {
     mapTell2(a,b,(a, b) => a / b,(a,b,c) => List(s"$c: subtracted $a from $b"))
}

By moving all that complexity into a helper function, each operator is now quite simple.

[info] running Scala3EvalEitherTWriter 
WriterT(Right((List(Var y (6), Var x (7), Val 10, Var z (22)),45)))
Var y (6)
Var x (7)
Val 10
Var z (22)
exp01 WriterT(Left(DivisionByZero))

In summary, you can use WriterT to convert any effectful program into one with step-by-step logging.

ReaderT

Another useful data type with a Monad instance is the Reader. As the name may imply, this is the conceptual opposite of Writer. i.e., instead of a computation writing its progress to a log, the Reader provides an environment of some type that the application can read from as it progresses.

In the program so far I've been using Scala 3 context functions to pass around the symbol table. There are reasons you may want to do that with a Reader instead. Perhaps you want to take advantage of the compositionality and lawfulness of Reader. Perhaps you want the context function reserved for some other purpose. Of course you may be using Scala 2 and not have access to the context function feature at all.

In one of my videos, I show the process of replacing context functions with the ReaderT monad transformer. Let's walk through the process here.

First of all let's look at the data type. Like the WriterT, the ReaderT wraps another higher kinded type F. As you can see, there is a second type parameter R, which is the type of the read-only environment. Also, you can see from the signature is that what the ReaderT contains is a function from R to the F[A]. How that is used will become clear.

case class ReaderT[F[_],R,A](run: R => F[A])

Just like with WriterT we also would benefit from a lift function that lets us take any instance of F and wrap it. Here I'm saying if you have some effect F[A] I will give you a ReaderT that wraps it. You can run it with some environment and it will yield that F[A] again.

object ReaderT:
  def lift[F[_],R,A](fa: F[A]): ReaderT[F,R,A] = ReaderT(_ => fa)

When rewriting the program above we can now look up variables from the symbol table. We are returning a function that when given an environment can search it for the required symbol.

def handleVar[A](s: String): RResult[A] =
   ReaderT((env: Env[A]) =>
     env.get(s) match {
       case Some(value) => Right(value)
       case None => Left(EvalError.SymbolNotFound)
     })

Literal values are also simple, just lift the Either from before.

case Val(value) => ReaderT.lift(Right(value))

The arithmetic operations don't change at all since ReaderT has an applicative instance we can just go ahead and use map2.

def add(fa: EvalResult[A], fb: EvalResult[A]): EvalResult[A] = {
    fa.map2(fb)((a,b) => a + b)
  }

Here is what needs to be done to run the code. The main difference is that we build a chain of Reader effects then execute them by passing an environment to the run method.

val env1: Env[Int] = Map("x" -> 1, "y" -> 10, "z" -> 100)
val exp1 = Add(Mul(Val(10), Var("y")),Var("z"))
println(eval(exp1).run(env1)) 

// Right(100))

WriterT and ReaderT

One thing I find wonderful about functional programming is its compositionality. I've shown that you can stack WriterT and ReaderT on top of any effect to imbue that effect with more capabilities.

Now given that WriterT can wrap a monadic effect to give that effect logging, and further that ReaderT itself is a monad, it follows that you can wrap WriterT around ReaderT to give some effect the powers of both! This would work the other way around, and of course, you could also make an EitherT monad transformer, giving even more possibilities.

The next step is to change the programs return type to be…

WriterT[
  [RA] =>> ReaderT[[EA] =>> Either[EvalError, EA], Env[A],RA],
  List[String],
  A]

Next to modify the program to handle the new effect types.

The implementation to get a value is easy enough. Starting from the inside out the value is put into an Either, lifted into ReaderT and lifted once more into WriterT.

case Val(value) => WriterT.lift(
     ReaderT.lift(
       Right(value))).tell(List(s"Literal value $value"))

Handling variable lookup we take care of the lookup first then wrap that into a writer.

def handleVar[A](s: String): WriterT[[RA] =>> ReaderT[[EA] =>> Either[EvalError, EA], Env[A],RA],List[String],A] =
  WriterT(ReaderT((env: Env[A]) =>
    env.get(s) match {
      case Some(value) => Right(List(s"Looked up var $s ($value)"),value)
      case None => Left(EvalError.SymbolNotFound)
  }))

I'll skip the rest of the program since the theme is the same; wrap the reader code with the writer code and you're done. Let's take a look at how to run the program.

val envMap: Env[Int] = Map("x" -> 7, "y" -> 6, "z" -> 22)

val eval1 = eval(exp1).unwrap().run(envMap)

eval1.foreach {
  (log, value) =>
    println(s"Result is $value\n")
    log.foreach {
      println(_)
    }
}

Here you can see that our program has to be run sort of inside out. eval(exp1) gives us a WriterT. By calling unwrwap I get the ReaderT inside it, which I can then run by passing the environment.

The response is either an error or a tuple of our log and result, which we can then iterate over to print it out.

Result is 990

Looked up var z (22)
Literal value 10
Literal value 2
Divided 10 by 2 (5)
Literal value 2
Subtracted 2 from 5 (3)
Looked up var x (7)
Looked up var y (6)
Multiplied 7 by 6 (42)
Added 3 to 42 (45)
Multiplied 22 by 45 (990)

Monad Transformers - some takeaways

You can see that monad transformers offer some expressive power since they allow us to manually combine different effect types to get the benefit of them all at once.

This comes at significant cost though. All this nesting creates additional JVM objects that take up heap space and may cause extra work for the garbage collector.

For the programmer, the ergonomics are not great. You have to remember the level of nesting you're at at each point of your program and make sure to do the right amount of lifting.

Just look at this single simple expression. So much space is taken up by the type signature, all the simplicity and elegance is lost.

def handleAdd[A : Numeric](l: Exp[A] , r: Exp[A] ): 
  WriterT[[RA] =>> 
    ReaderT[[EA] =>> 
      Either[EvalError, EA], 
      Env[A],RA],
    List[String],
    A] = eval(l) + eval(r)

Now there ways to help out the Scala compiler and reduce the amount of boilerplate, namely "kinda curried type parameters" which is a technique used heavily in Scalaz and Zio.

https://tpolecat.github.io/2015/07/30/infer.html

Thinking that the poor type inference is maybe my fault I've also written this code using Scalaz and Cats and you can see that each implementation has its pros and cons. (BTW in these libraries ReaderT is called Kleisli)

Reader Writer example in Scalaz

Reader Writer example in Typelevel Cats

Both implementations had the same problem as I did when not writing out the types in full. Scalaz had the same trouble inferring that my type was applicative so I had to summon the instance explicitly.

implicit val rwApply = Apply[WriterT[List[String],Kleisli[Either[Error,?],Env[A],?],?]]
rwApply.apply2(x,y) {
       case (a,b) => a + b
}

Cats was able to handle it resulting in less code and needed less type annotations in general.

x.map2(y) {
  case (a,b) => a + b
}

Effect rotation with Zio Prelude's ZPure

Zio Prelude is a new library that acts as a sort of add on library to ZIO (a zero-dependency Scala library for asynchronous and concurrent programming) which provides an alternative approach to functional abstractions in Scala.

For the purposes of this blog I'm interested in particular experimental data type within the library called ZPure. ZPure has 6 type parameters and supports the operations of monads, applicatives and functors, albeit the names are changed and some of the abstractions too.

From Prelude's own source code "ZPure can be used to model a variety of effects including context, state, failure, and logging.", let's see how it compares to monad transformers to implement the program above.

ZPure has six type parameters, which may seem like a lot, but bear in mind that every time you combine monads you get more type parameters, but they are stacked vertically not horizontally. With ZPure you start with all the different effect types one might need but you don't need to keep adding more and more, and you don't suffer from creating multiple objects per layer of effect.

In the diagram the parameters are.

W Logging. The type of logs that this effect produces, analogous to Writer
S State. There are two S's because the type encodes an input and output state type
R Reader. The type of the read-only environment
E Error. The type of the error channel
A Value. The type of the happy path computed value

Just as before all I need to do is change my programs effect type and modify the implementation accordingly. The code for this section can be found here.

Although Prelude supports Scala 3 now I rewrote my program in Scala 2 in order to do the Scalaz and Cats versions, so the following is also pre Scala 3 friendly code. First I made a sealed trait to represent the error types and an alias to indicate that my log will be strings.

Note that ZPure is opinionated about the logging type. In the more conventional approach, the log has to be a monoid instance. With ZPure the log is handled using the ZIO Chunk data type which is a high-performance data structure with a pure functional interface. What that means for us is that we can consider our log type as a single entity and not worry about how it is appended.

sealed trait Error
object SymbolNotFound extends Error
object DivisionByZero extends Error

type Log = String

These types will represent the E or error, and W or log. We can encode our symbol table using the R parameter.

type Env[A] = Map[String, A]

The final type looks like this.

type Result[A] = ZPure[Log, Any, Any, Env[A], Error, A]

Note that I am not using the initial or updated state here so I use Any for those parameters so as not to constrain them.

Once again let's start with the implementation of literal values.

case Val(value) => 
  ZPure.succeed(value).log(s"Literal value $value")

What's going on here is simple and refreshingly free of type annotation. Firstly I construct a ZPure consisting of the literal value, and then add a log using ZPure's log method.

Now let's implement the symbol table lookup of variables.

 1: def handleVar[A: Numeric](s: String): Result[A] = {
 2:     ZPure.environment[Any, Env[A]].flatMap {
 3:       env =>
 4:        ZPure.fromOption(env.get(s)).
 5:        mapError(_ => EvalZPure.SymbolNotFound).
 6:        flatMap{
 7:         a =>
 8:           ZPure.log(s"Var $s value $a").as(a)
 9:        }      
10:     }
11: }

First I use the ZPure.environment to summon the symbol table and flatMap over it so we can access it as a concrete value env.

Remember that looking up the variable in the symbol table is going to return an Option since it is a normal map get. We can then use the ZPure.fromOption to convert that to a ZPure.

We may be a ZPure at that point but we have the wrong error type. In ZPure and option is simulated by yielding a success value A for Some, or giving Unit in the error channel to indicate None. This reuse of the error channel is neat, but since I would like a homogenous error type for the program I need to convert that error channel from unit to my own custom Error type.

To do that I use mapError which takes a function to map the error from one type to another.

The final step is to add a log. Since I would like the log to show both the variable name and the actual value I have to nest it inside a flatMap so we can access the success value. The last thing to note here is the as which helps the type system make a ZPure of the right type.

implicit def numericZResult[A: Numeric]: 
  Numeric[Result[A]] = new Numeric[Result[A]] {
    def add(x: Result[A], y: Result[A]): Result[A] = {
      x.zip(y).flatMap{case (a,b) => 
        ZPure.succeed(a + b).log(s"Add $a and $b")}
  }
  // and so on

The final component is to make an instance of Numeric. Prelude takes the approach to naming that concepts should be plain English where possible, so it uses succeed instead of pure or unit and similarly zip instead of applicative terminology.

That being the case, we use zip here to combine the left and right side effects and apply the appropriate arithmetic operator.

So far so good, but we can't log the result of the computation. This is easily solved.

def add(x: Result[A], y: Result[A]): Result[A] = {
    x.zip(y).flatMap{case (a,b) =>  {
      val result = a + b
      ZPure.succeed(result).log(s"Add $a and $b ($result)")}
    }
  }

The last part to note is the driver code to run the program.

val eval1 = eval(exp1).provide(env1).runAll()

eval1._2 match {
  case Right(value) => 
    println(s"Succeeded with value ${value._2}")
    eval1._1.foreach {
      l => 
        println(l)
  }
  case Left(err) =>
    println(s"oops! $err")
}

Here I run eval to create the effect, provide to give it the runtime environment and runAll. To be clear what eval does is not run a program, but build a data structure of objects based on the ZPure operations, and that data structure is evaluated in Prelude using an efficient interpreter, that yields the result of the program, the log and any state changes.

Succeeded with value 200
Literal value 10
Var x value 1
Mul 10 with 1 (10)
Var y value 10
Mul 10 with 10 (100)
Var z value 100
Add 100 and 100 (200)

Conclusion

In this post I implemented and motivated ReaderT and WriterT and transforming monads to combine effects. Even in Haskell where they originated, monad transformers come with caveats about performance and ergonomics. In Scala they are not used often.

There are techniques and libraries to make their use more convenient. Although I have not used it Cats MTL offers solutions to some of the problems, but it is not widely used.

Although ZIO Prelude's ZPure is still somewhat experimental it seems offer the benefits of monad transformers such as composability and a prinicipled type safe api. But, it is also much easier to work with and anecdotally more performant on the JVM.

In the future I'm looking forward to exploring some traditional functional programming problems using Prelude and ZPure.

Hope you enjoyed this post and got to the end. You can find my contact details at the top of the page; I always welcome feedback and questions.

Monads in Scala 3 for the Genius

Tue, 02 Feb 2021 00:00:00 +0000

Introduction

This is a companion blog the seventh Functional Justin YouTube video which you can find here: https://www.youtube.com/watch?v=B1FSxbmZpCE

The source code shown in the video and in the fragments below can be found here: https://github.com/justinhj/evalexample/blob/video-7-r3/src/main/scala/livevideos/Video7.scala

The goal here is to make 10-15 minute videos that each cover an isolated topic in pure functional programming with Scala. There is an also overarching goal to cover a certain number of base topics you need in functional programming. Once that is complete I will start doing deep dives into FP libraries such as Cats and Zio.

Monads for the Genius

The title may sound like clickbait, and it most surely is, but the point is that for most Scala programmers, even those working with functional programming libraries, most of what you need to know about Monads is what the type signatures of the methods are and how you can leverage those in your code. For bonus points you can learn the Monad laws and implement your own lawful Monad instances.

If you want to know why Monad consists of pure (or unit) and flatMap, and why Monads are functors (they have map) you need to dig a little deeper, and for that reason I targeted this post at the genius, but perhaps the curious would be a better way to put it, it hopefully will not be a really difficult topic to follow.

In this post and the accompanying video I will show the category theory you need to understand monad and derive an implementation of the monad type class from that theoretical foundation. From there the next step is to show the monad in use on effects and how to implement and use flatMap (or bind).

Just enough Category Theory

Category theory is a candidate for a theory that describes all of mathematics, and has been applied to a number of areas.¹. If you want to know more about it than the very basics then I recommend Category Theory for Progammers.²

The essence of category theory is the category itself, which consists of objects and morphisms that transition from one object to another.

In addition there must be an identity morphism on each object. This simply gives you a way to transition from an object to itself.

Morphisms between objects can compose. Here we have a morphism from A to B (f) and another from B to C (g). We can compose f and g, giving us a single morphism from A to C.

Composition must follow the associative law. As shown below that means if we have three morphisms f,g and h, it doesn't matter how we compose them as long we don't change the order they are applied. We can compose them in two different ways.

The category of Scala types and functions

Let's make the concept of a category more concrete by seeing how it can be encoded in Scala. One example of a category is the category of Scala types and functions.

In the code below we have a lawful category. The objects are the Scala types (Ints, Booleans, Strings) and the morphisms that take us from one object to the next are ordinary Scala functions. There are three examples f,g and h.

Remember to be a category we need an identity morphism, which turns out to be simply the Scala identity. (A => A).

The other thing we need is a way to combine morphisms that must be associative. We have that with the built in function compose!

As you can see in the code it is straightforward to show the laws of the category are upheld.

// Category of Scala functions

val f: Int => Int = a => a + 1
val g: Int => Boolean = b => if(b == 1) true else false
val h: Boolean => String = c => if(c == true) "Winner!" else "Loser!"

// Identity
f.compose((a: Int) => identity(a))(0) == f(0)

f(0) == f.compose((a: Int) => identity(a))(0)

// Composition must be associative
h.compose(g.compose(f))(0) == (h.compose(g.compose(f)))(0)

Above you can see composing the identity function with f gives the same result as calling f alone.

You can also see that composition is associative. We compose h with g and f in different ways, without changing the order, and get the same results.

A monad is just a functor in the category of Kleisli arrows

What's the problem?

Well there are two problems here. For one many readers may be saying "What? Surely a monad is a just a monoid in the category of endofunctors!"³

Perhaps another group are completely lost. Well the famous quote about monads is absolutely right, but that is a different way to arrive at Monads than the simpler one we are looking at here.

Instead we will arrive at Monads by making a simple change to the Category of Scala types and functions. The only change we will make is instead of Scala functions of the form A => B we will instead use what is known as a Kliesli arrow, which has the form "A => F[B].

You may recognize that shape of function from the argument to Scala's flatMap. In other words it is the type of function that maps a pure value to an effectful value.

Let's look at how we can encode this new category directly in Scala as a monad!

Note I will call the Monad type class Monad1 to avoid confusion with the more usual Monad definition in the code.

trait Monad1[F[_]]:
  def unit[A](a:A): F[A]
  def compose[A,B,C](lf: A => F[B], rf: B => F[C]): A => F[C]

In the definition above we have all we need to implement the category of Scala objects and Kliesli arrows (and incidentally this is, by definition, a monad).

Firstly what are the objects? Just like before the objects are Scala types.

Next what are the morphisms? We stated the morphisms would be of the form A => F[B].

Finally what is the identity? The identity has the same form as any other morphism except that it maps a type to itself, so the identity is A => F[A]. We can implement that in Scala with the unit function above.

With Scala functions we used the compose function. Here we need to write our own code that composes two Kleisli arrows returning a new one. This is the direct analog of the compose function that works with simple functions.

For convenience, just like with any other Scala 3 type class we need a way to summon a Monad of a particular type into existence and for that we write the apply function as follows.

object Monad1:
  def apply[F[_]](using m: Monad1[F]) = m

Implementation of Monad for Option

given optionMonad1: Monad1[Option] with
  def unit[A](a:A) = Option(a)
  def compose[A,B,C](lf: A => Option[B], rf: B => Option[C]): A => Option[C] = {
    a => 
      lf(a) match {
        case Some(b) =>
          rf(b) match {
            case Some(b) => rf(b)
            case None => None
          }
        case None => None          
      } 
  }

You can see that unit is just a call to the Option constructor, whilst compose will return a new function that first applies lf to the input, then if that yields a value and not a None, it will apply rf to that yielding a new Option. Please note I made an overly complex version of this in the video, and only realized once it was too late.

Now we can write code that composes "effect generating" functions (or Kliesli arrows) together. Here I make three simple functions that operate on Scala values and produce Options.

Here we use the Monad1 option to compose f,g and h…

def f(n:Int): Option[Int] = if n == 4 then None else Option(n)
def g(n:Int): Option[Boolean] = if n%2==1 then Option(true) else Option(false)
def h(b:Boolean): Option[String] = if b then Some("Winner!") else None

val fcomposed = Monad1[Option].compose(f,g)
val fghComposed = Monad1[Option].compose(fcomposed, h)

def i(a: Float) = 0.0

println(fghComposed(1))
println(fghComposed(2))
println(fghComposed(3))
println(fghComposed(4))

// Output:
// Some(Winner!)
// None
// Some(Winner!)
// None

The Monad laws

At this point we've shown that one implementation of a Monad involves the unit and compose functions. We can now see a demonstration of the monad laws in this form.

Left and right indentity laws are shown by composing a function with unit. This is equivalent to what we did with Scala functions.

// left and right identity
m1.compose(f, m1.unit)(1) == f(1)
f(1) == m1.compose(f, m1.unit)(1)

We can also demonstrate the associtive law in action, whereby composing f,g and h works both ways.

m1.compose(m1.compose(f,g), h)(1) == m1.compose(f, m1.compose(g,h))(1)

What about flatMap?

So far so good, we conjured up a monad from just category theory and a simple twist on the category of types and functions. You may be wondering how we get from this new definition of Monad to the one we see in Cats and Scalaz, and why even in the Scala standard library we have flatMap but not compose for Kliesli arrows.

Well fortunately flatMap can be written in terms of compose, so we can be assured that the more convenient and familiar representation of Monads is exactly equivalent!

def flatMap[F[_],A,B](fa:F[A])(f: A => F[B])(using m: Monad1[F]): F[B] = {
  // F[A] => F[A]
  // A => F[B]
  m.compose((a: F[A]) => identity(a), a => f(a))(fa)
}

I found this implementation a bit tricky to understand at first but if you look at it and reference the Option instance above it should make sense after a little thought. The "trick" is that we are given an F[A] and so we pass that as the first argument to compose using the identity function to get it back unchanged. (Mapping an F[A] to itself is actually the map function of Functor!)

compose from flatMap

Should your starting point be the more traditional Monad with pure and flatMap, you can in fact derive the compose function as follows.

import org.justinhj.typeclasses.monad.{given,_}

def compose[F[_],A,B,C](lf: A => F[B], rf: B => F[C])(using m: Monad[F]): A => F[C] = 
  a => lf(a).flatMap(rf)

Final remarks

One last thing you may be interested in is that you can implement monad as pure and flatmap, pure and compose or as third set pure, map and flatten.

My favourite reference for exploring Monads in Scala is the so called red book which devotes chapter 11 to the subject.⁴ The nice thing about that particular book is it encourages the sort of exploration and discovery of these concepts that makes them so fun to work with!

There is some duplication in the names when we use category theory in Scala that can cause confusion. Here's a little guide.

Purpose	Functions	Kleislis
Identity	identity	unit	pure	return	point
Sequence two effects	n/a	flatMap	bind
Flatten a nested effect	n/a	flatten	join

Finally it was really my goal here to show that there is not much to categories and therefore not much to monads. The terminology is unfamiliar but I think the concepts are quite straightforward. I would love to know if this blog and/or video failed to make sense, so feel free to reach out to be on the youtube comments or via the contact details above and I will take on board your suggestions.

Footnotes:

Uses of Category Theory https://math.stackexchange.com/a/1210742/2914

Category Theory for Programmers https://github.com/hmemcpy/milewski-ctfp-pdf

James Iry and the famous monad quote https://stackoverflow.com/questions/3870088/a-monad-is-just-a-monoid-in-the-category-of-endofunctors-whats-the-problem#3870310

⁴

Functional Programming in Scala https://www.manning.com/books/functional-programming-in-scala

Functional Error Handling with Applicative in Scala 3

Thu, 21 Jan 2021 00:00:00 +0000

Introduction

This is a companion blog my sixth Functional Justin YouTube video which you can find here: https://youtu.be/3GPXEzO14ZE

In the video I talked about the Functor Laws and continued working on the example program that evaluates arithmetic expressions. I use Monad, then Applicative, to show how we can make a Numeric instance for Either, so we can handle errors in a nice functional way.

Since the episode I spent a bit of time cleaning up the code and putting what we already saw (Numeric and Functor) into their own packages. I also went ahead and implemented Applicative and Monad which will be used in the the video and below.

You can find the typeclasses here:

https://github.com/justinhj/evalexample/tree/video6/src/main/scala/org/justinhj/typeclasses

Functor Laws

Our Functor type class really only exists to implement the map function, and we already have a map function in the Scala standard library for such things as Options, Lists and Futures. You might wonder why we would go the trouble of making our own abstraction just to write a function that we already had.

The goal of abstractions like Functor is not just to provide useful functions like map, but to provide a principled layer that we can build further abstractions upon. For example we will see that Applicative can be built on top of Functor, Monad on top of Applicative, and this is only possible because Functor behaves in accordance to strict rules that it brought with it from Category Theory.

You can read more about Functors on the Haskell documentation pages:

https://wiki.haskell.org/Functor

Another great resource is the famous "red book"…

https://www.manning.com/books/functional-programming-in-scala

Functors preserve identity morphisms

What this law states is that if we map over some effect with the identity function, then neither the effect nor any value inside it will change.

// Identity law for functors...
val l1 = List(1, 2, 3)
assert(l1 == l1.fmap(a => identity(a)))

val e1: Either[String, Int] = Right(10)
assert(e1 == e1.fmap(identity))

In the two examples we mapped an Either and a List using the identity function and nothing changed.

Functors preserve composition of morphisms

If we have two functions, f and g, it doesn't matter if we map over some effect with f first then map over it with g, or we map over it one time with the composition of f and g. Using the either and list from above we can show this the case here.

def f(a: Int): Int = a + 1
def g(a: Int): Int = a - 1

assert(e1.fmap(f).fmap(g) == e1.fmap(a => g(f(a))))
assert(l1.fmap(f).fmap(g) == l1.fmap(a => g(f(a))))

Importing given instances

Note that in Scala pre version 3 if you use a wildcard import _ then you get all everything exposed in that package. That means you get all the implicit instances too. It was a source of confusion for beginners and even experienced Scala programmers to know which file to import and sometimes to know where instances you are using are defined.

To help with that NO implicits are imported with a regular wildcard and instead you must import them with the new given syntax.

import org.justinhj.typeclasses.functor.{given, _}

If you want you can also import only specific instances. This, in my opinon, will make things a lot simpler and more precise.

One caveat here is that the given wildcard must appear before the underscore wildcard.

Implementing Numeric for Either

Functor isn't enough

The following code implements an arithmetic expression evaluator using the Numeric type class developed in a previous video and adds error handling by using Either. Each step of our evaluator has this signature.

type WithEnv[A] = Env[A] ?=> Either[EvalError, A]

Which means it is a function takes an input environment (our symbol table) as an implicit argument, and returns an Either where the error (or Left) type is EvalError. EvalError represents the different errors our code will handle. It is a sum type implemented as a Scala 3 enum (seen in a previous video).

enum EvalError {
  case InvalidSymboName
  case SymbolNotFound
}

In previous blogs/videos I showed how we can implement a Numeric instance so we can do arithmetic on many different types, just so long as we create an instance of Numeric to handle them. Now we must implement Numeric for the following type Numeric[Either[EvalError, A]].

The instance signature is

given evalResultNumeric[A: Numeric]: Numeric[Either[EvalError, A]] with {

Now we must implement the methods of Numeric. Because our numeric values are inside the EvalResult (an Either) we can't just implement the multiply directly. We need a way to get inside it. As we saw in the previous blog/video, Functor gives us a way to apply a pure function to an effect. Since mul is a pure function, maybe we can use it?

def mul(a: EvalResult[A], b: EvalResult[A]): EvalResult[A] = {
  a.fmap { // DOES NOT COMPILE, WRONG TYPE
    aa => 
    b.fmap {
      bb =>
        aa * bb
    }
  }
}

Note I am using the name fmap and fflatMap to make it clear we are not using the standard library implementations here. This is just for clarity but is not a good practise because, for example, you will lose the ability to use for comprehensions.

What went wrong here is that Functor's map operation has the signature

extension [A, B](x: F[A])
  def ffmap(f: A => B): F[B]

which means it takes our Either[EvalError, Numeric[A]] and a pure function, which it will apply to the Numeric. Unfortunately we end up with an extra layer of Either! Let's see why…

Monad to the rescue

So instead of Functor with its map function, we need Monad and its flatMap which let's us implement all the arithmetic functions in a straightforward manner.

https://github.com/justinhj/evalexample/blob/video6/src/main/scala/org/justinhj/typeclasses/monad/Monad.scala

I've implemented Monad in the file above and made it available to the code. The implementation is simple and based on the example given in the Dotty documentation. The main difference is I've also implemented Applicative, since we will use that in a moment, and Monad extends Applicative.

https://dotty.epfl.ch/docs/reference/contextual/type-classes.html

Now each arithmetic operator can be implemented as follows, which achieves our goal of being principled and functional and let's us handle errors at the type level.

def mul(a: EvalResult[A], b: EvalResult[A]): EvalResult[A] = {
  a.fflatMap {
    aa => 
      b.map {
        bb =>
          aa * bb
      }
  }
}

Map2 we love you

Unfortunately it's bit verbose. Monad is more powerful than we need in fact. We could use Applicative instead. I will talk more about Monad and Applicative in a later video, but in short you can think of Monads as being good for putting two effects together and flattening the result, whilst Applicative is good for passing multiple effect values as parameters to some pure function.

You can see the Applicative implementation here.

https://github.com/justinhj/evalexample/blob/video6/src/main/scala/org/justinhj/typeclasses/applicative/Applicative

Now, Functor has map, Monad has flatMap and Applicative has its own mapping function called ap. Whilst it's out of scope for right now, the ap mapping function makes it possible to apply two or more effects as parameters to a pure function, which is exactly what we need here. From ap you can derive methods that make this much simpler, map2 for example. Here we use map 2 to take any two input effects and apply the multiply operator to them…

def mul(a: EvalResult[A], b: EvalResult[A]): EvalResult[A] = 
  a.map2(b)((a,b) => a * b)

Division by Zero

What we have at this point is a nice implementation of Numeric that uses Either's for error handling, which in turn is built on Applicative. Let's see how easy it is to add new errors and capabilities to the expression evaluator.

enum EvalError {
  case InvalidSymboName
  case SymbolNotFound
  case DivisionByZero
}

First we add a new error type DivisionByZero. The next thing we need is for Numeric to have a concept of whether a number is zero or not. Remember that we can implement Numeric for many different types and not all of them represent zero the same way. We can therefore add an isZero predicate to the type class.

def isZero(a: T): Boolean

Next every instance of Numeric needs an implementation of it, so for exapmle in the Int instance we have the following.

def isZero(a: Int) = a == 0

The implementation for Numeric Either let's us write the isZero for any value in an either as long as that value has a numeric instance of its own.

given evalResultNumeric[A: Numeric]: Numeric[Either[EvalError, A]] with {
  def isZero(a: EvalResult[A]): Boolean = {
    a match {
      case Right(a) if summon[Numeric[A]].isZero(a) => true
      case _ => false
    }
  }

Finally we can implement the division operator for Numeric Either like this.

def div(a: EvalResult[A], b: EvalResult[A]): EvalResult[A] = {
  if isZero(b) then
    Left(EvalError.DivisionByZero)
  else 
    a.map2(b)(_ / _)
}

Wrap up

That's all for now, I hope you enjoyed this post and video. Please contact me using the methods above with any questions, suggestions or corrections!

Handling Errors functionally with Scala 3

Wed, 06 Jan 2021 00:00:00 +0000

Introduction

This is a companion blog my fifth Functional Justin YouTube video which you can find here: https://youtu.be/wNVQ75KM8-4

If you're new to the series I'm exploring Scala 3 and functional programming using a simple expression evaluator, and adding features to it every week. Most of the videos are coffee break sized (10-15 minutes) but this one took a bit longer as I needed more time to explain the concepts. Next time will probably be back down to the more bite-sized format.

In video 1, https://youtu.be/J01u_Dmrx5U, I showed how you can use Scala 3 features like Context Functions to pass context around. The eval function below is an expression evaluator that takes expressions of type Exp, returns a result of type Int and has an implicit environment Env which is a symbol table of values.

type WithEnv = Env ?=> Int

def eval(exp: Exp): WithEnv =
  exp match
    case Var(id) => handleVar(id)
    case Val(value) => value
    case Add(l,r) => handleAdd(l,r)

def handleVar(s: String): WithEnv =
  summon[Env].get(s).get

This is a nice example of context functions, but not so good an example of a pure functional program, let's see why.

Purity

What is a pure function? In short it has three properties… it is a total function, it is deterministic and it has no side effects.

A total function has an answer (of a fixed type) for everything. Our expression evaluator is a total function because every expression you put in can be evaluated and returns a value. Now let's say we had a divide function, and you can pass in a divisor that is zero and the answer is infinity. That is not representable by Int so the function is not total. We can only throw an error at this point.

By deterministic, we mean that the evaluator gives the same answer for every input expression, which may seem self-evident, but imagine if we had a random number command. When used it would return different answers every time and the program would not be deterministic.

Finally, by no side effects, we mean the program does nothing impure. It is not going to print to the screen, send an email, or throw errors.

Is the expression evaluator code pure?

If your nose wrinkled when you saw this code above summon[Env].get(s).get then you're probably an experienced Scala programmer who knows that you should not call get on an option.

What's happening there is a symbol table look up. First I 'summon' the symbol table (see video 1 to understand context functions and where the symbol table is coming from), then I look up the symbol using get. This returns an Option because the symbol may be missing!

I hopefully pointed out at the time that you shouldn't call get on an Option in serious code because it will throw an error. That means that as a program that can throw errors, our program is not pure.

Handling errors with Either

Since the 90s the Haskell folk have been dealing with impurity by wrapping it up in data types that describe the effects, and manipulating them with type classes. If that sounds hopelessly abstract, then fear not, in practise it's quite simple and we will fix our problems with a few lines of code.

Firstly let's look at what we mean by a data type… it is usually a higher kinded type that "contains" things of another type that you can define at compile time.

Scala's Either is a great example of a data type. It encodes the concept of errors. Our pure code does not deal with errors but we can still encode errors by wrapping them as follows.

val e1: Either[String, Int] = Right(10)
val e2: Either[String, Int] = Left("Oops")

By encoding our values like this we can represent a computation that has succeeded as a Right value, and a computation that has failed with some error as a Left value.

What this means is we can no longer apply pure functions these values directly. That is sort of the point. What we wanted to do was isolate pure functions from having to deal with errors at all. So how do we operate on Eithers? Well you are probably familiar with the map function, and that can be used to apply a pure function to an either!

val e3 = e1.map(a => a + 1)
// Right(11)
val e4 = e2.map(a => a + 1)
// Left("Oops")

Categorically Speaking

You may not really think of it as Category Theory, but whenever you map an Either you are using Functors!

The Haskell documentation is a nice place to learn about Functors. If you think of a normal pure function as a mapping of values from A to B, a Functor can map values that have been embellished, or wrapped in some special data type.

https://wiki.haskell.org/Functor

Helpfully, the kind folks behind Scala 3 have added how to implement type classes to their documentation. We can use that a starting point to build our own Functor and then make an instance that works with Eithers.

trait Functor[F[_]]:
  extension [A, B](x: F[A])
    def ffmap(f: A => B): F[B]

This is all we need to define a Functor type class that can extend supported types with a map function. Note that I've added an f to differentiate the function from the built in map. Then I added another f by mistake, don't tell anyone, they might not notice!

Before we can use this against an Either we need to implement an instance of the typeclass. Remember that Functor needs a type of kind F[_]. It has one "type hole". Either has two, which is not going to work, so let's start by specialising to Either with only a fixed error type of String.

First we make a type alias that reduces the Either to one unknown type, the computation result type A.

Next we provide an implementation of ffmap that does the work of mapping our pure function over an Either.

Note that this is roughly the same as the pure function. Instead of A => B we are mapping F[A] => F[B] where F is the Either.

type StringEither[A] = Either[String, A]

given Functor[StringEither] with
  extension [A, B](x: StringEither[A])
    def ffmap(f: A => B): StringEither[B] = {
      x match {
        case Right(a) => Right(f(a))
        case Left(err) => Left(err)
      }
    }

Let's try it out.

val e1: Either[String, Int] = Right(10)
val e2: Either[String, Int] = Left("Oops")

val e3 = e1.ffmap(a => a + 1) // Right(11)
val e4 = e2.ffmap(a => a + 1) // Left("Oops")

We can now apply pure functions to Eithers with String error types. Where we want to get to is to be able to apply pure functions to Either[Error,Numeric[A]] so we're not quite there yet.

The first problem is that we can't handle the Error type that I want to use in my expression evaluator, we can only handle String. Well we can just make another instance of Functor for Either[Error,A]?

Well, yes we could, but how about we make a generic instance of Functor for all Eithers?

To do that we need to use type lambdas. These were available in Scala 2 but are greatly simplified in Scala 3.

https://dotty.epfl.ch/docs/reference/new-types/type-lambdas.html

Here's the new instance for Functor with some notable changes.

1: given eitherFunctor[E]: Functor[[A] =>> Either[E, A]] with
2:   extension [A, B](x: Either[E,A])
3:       def ffmap(f: A => B): Either[E,B] = {
4:         x match {
5:           case Right(a) => Right(f(a))
6:           case Left(err) => Left(err)
7:         }
8:       }

Line 1 is where the action is. First note that we named the given instance eitherFunctor. Our previous instance had no name. You can leave the name out, but it's not recommended, especially for libraries, since it makes the code easier to work with. See also that the instance itself takes parameter E which will represent our error type.

Next the instance of Functor is for the type [A] =>> Either[E, A] which is our type lambda. It means please give me a type that has a single parameter A but that will be substituted into the Either[E,A] in a way that is similar to how parameters are substituted into a lambda function.

val e1: Either[String, Int] = Right(10)
val e2: Either[Int, Int] = Left(1)

val e3 = e1.ffmap(a => a + 1) // Right(11)
val e4 = e2.ffmap(a => a + 1) // Left(1)

Now we can map over any type of Either! As you can see in the first case the pure function mapped over the A. In the second case the pure function was not executed and the error value is simply passed along.

Functor Laws

Next time we'll look at the Functor laws and show that our code obeys them.

Wrap up

I hope you enjoyed this blog and/or video. Please share, like or subscribe and help me spread this content to those that may find it useful.

For further info

Source code

https://github.com/justinhj/evalexample

https://dotty.epfl.ch/docs/reference/new-types/type-lambdas.html https://dotty.epfl.ch/docs/reference/contextual/extension-methods.html https://dotty.epfl.ch/docs/reference/contextual/type-classes.html

Type classes with Scala 2

Sun, 06 Dec 2020 00:00:00 +0000

Photo by Crissy Jarvis on Unsplash

Make your own Numeric type class

This is the companion blog for a Functional Justin video which you can find here https://youtu.be/pJFfXhZlR5o. This is a series of coffee-break sized videos that each explore a topic in the world of Scala Functional Programming.

In this video I talk about Type classes and here is the blog version for those that would rather read.

What are type classes

Here are some good references for reading about type classes in Scala (they can of course be implemented in other functional programming languages).

Sam Halliday has a great introduction to data types and type classes in his book Functional Programming for Mortals. https://leanpub.com/fpmortals/read#leanpub-auto-data-and-functionality

This idea of separating data from functions can also be found in the Cats functional programming library, with the library itself being organised into distinct sections for data types and type classes. https://typelevel.org/cats/typeclasses.html https://typelevel.org/cats/datatypes

My former colleague and functional programming advocate Francis Toth also has a nice blog on this topic. https://contramap.dev/2020/04/09/typeclasses.html

In my own words, type classes are not explicitly part of the Scala language, but are rather a pattern of implementation that enable you to define behaviours which can then be implemented for data types. Data types are values like strings and numbers, collections like lists and sets and "higher kinded types" like IO, ZIO and so on.

Type classes do not have to be implemented a certain way, but in Cats, Scalaz and other libraries you will find them implemented just as they are here, but additional layers of complexity to manage things like stack safety and take advantage of tools to generate boilerplate code.

In general type classes consist of:

A trait that has one, or sometimes more, type parameters.
It contains one or more abstract methods. These define behaviours that must be implemented for each instance.
Also you will find generalized methods that are built using the abstract ones.
Typically a type class contains no data

Numeric

In the video I implemented the Numeric type class. What is Numeric? It is a type class you'll find in the Scala standard library and is used to build a generic representation of numbers.

https://www.scala-lang.org/api/current/scala/math/Numeric.html

What I develop here is just enough that we can use it for the expression evaluator in the first video (which only requires add and multiply operations).

You can find the complete code here

https://github.com/justinhj/evalexample/blob/master/src/main/scala/Scala2Numeric.scala

Let's look at the pieces one by one…

trait Numeric[T] {
  def Add(a: T, b: T): T
  def Mul(a: T, b: T): T

  def square(a: T): T = mul(a, a)
}

Here we define the trait which decribes the core of our type class; what it can do, and what you need to implement if you want your data type to be an instance of this typeclass.

Add and Mul are abstract, whilst the square function is derived or generalized and does not need to be implemented when making new supported instances.

Instances

Once you have the interface for your type class you can define instances for various data types. Here we define the instance for Long.

implicit val numericLong: Numeric[Long] = new Numeric[Long] {
  def add(a: Long, b: Long): Long = a + b
  def mul(a: Long, b: Long): Long = a * b
}

Note that this is implicit to make it easier to use our instances with other types, taking advantage of the fact that users can import the definititions and then use them in their code.

Here's an example function that is written using the Numeric type class as a parameter. This function displays what we call ad-hoc polymorphism, in that this function does not know all of the instances that exist for the input type T, nor does it need to. All it knows is that to do its work it needs an instance of Numeric[T] so that it can access the add method.

def sumList[T](ts: List[T])(implicit numeric: Numeric[T]): T = {
  ts.reduce((a, b) => numeric.Add(a,b))
}

Now we can sum a list of any T where T has a Numeric instance. Here we use the Int instance (not shown).

val l1 = List(1, 2, 3, 4)
val sum = sumList(l1)

println(s"sum of int list is $sum")
// sum of int list is 10

Improving the ergonomics

So that's all you need to build type classes but we can take a couple of steps to make things more ergonomic. For one we can ditch the implicit parameter and instead make use of [[https://docs.scala-lang.org/tutorials/FAQ/context-bounds.html]context bounds]]. This makes things clearer for the caller of the function.

def sumList[T : Numeric](ts: List[T]): T = {
  val numeric = implicitly[Numeric[T]]
  ts.reduce((a, b) => numeric.add(a,b))
}

Note that we have to summon the implicit Numeric instance using implicitly. This is a function in the standard library which takes advantage of the way context bounds work: the context bound Numeric specifies that there is an implicit Numeric instance in scope, but there is no named parameter as before. The implicitly function lets us access that implicit in a succinct way.

def implicitly[T](implicit e: T): T = e

So using context bounds helps a little with the use of type classes, the next step is to use implicit conversions so that we take the functions in our type class and make them look like ordinary methods on the data type.

object ops {

  implicit class NumericOps[T](a: T)(implicit numeric: Numeric[T]) {
    def add(b: T): T = numeric.add(a, b)
    def mul(b: T): T = numeric.mul(a, b)

    def +(b: T): T = add(b)
    def *(b: T): T = mul(b)
  }

}

Now if we import ops we can take advantage of the implicit conversion from type T to type NumericOps[T] to give us syntax like below.

val s1 = "abcd"
val s2 = "efgh"
val product = s1 * s2
println(s"product $product")
// product aeafagahbebfbgbhcecfcgchdedfdgdh

So you can see that by implementing a somewhat goofy instance of Numeric for string (given below) we now have the ability to use the multiplication operator on it as if it was a regular number.

implicit val stringNumeric: Numeric[String] = new Numeric[String] {
    def add(a: String, b: String): String = a + b

    def mul(a: String, b: String): String = for (
      as <- a;
      bs <- b;
      s <- as.toString ++ bs.toString) yield s
  }

Note that while this implentation of string arithmetic is not very rigorous and just for fun, there's nothing to stop you from implementing Numeric for data types that do have well defined arithmetic operations such as Roman Numerals.

Coherence

Type class coherence is an important concept I'll leave you with. This is a guideline in place to keep programs easy to reason about. It's a good practice to keep common instances together with your type classes so that users can easily find them, and don't duplicate the work. It's also important that you don't try to make multiple instances and let the users select one depending on their needs. The reason for that is the behaviour of your program can change profoundly when you do this, and that's terrible. It means you can't take advantage of local reasoning, one of the benefits of functional programming. You would need to be very careful with imports to make sure you are using the instance you think you are.

Final words

If you're coming from Java or similar OOP language you may recognise some of this as the adapter pattern (except with Scala implicits). Type class traits also have similarities to Go interfaces, although the type class pattern goes a bit beyond them in scope.

I would be amiss not to mention Haskell here, which has type classes implemented as a first-class language construct, and for some people the over-use of type classes in Scala is somewhat of an anti-pattern. We will see in future videos that the pattern will be greatly simplified in Scala 3 however.

References

Typelevel Cats functional programming library documentation https://typelevel.org/cats/

Functional Programming for Mortals https://leanpub.com/fpmortals

Scala 3 Context Functions

Wed, 18 Nov 2020 00:00:00 +0000

Some Scala 3 Things: Context functions, Enums and significant whitespace

Updated: for Scala 3.0.0-M3 `as` keyword was removed

This is the companion blog for my first Functional Justin video which you can find here https://youtu.be/J01u_Dmrx5U. I spend around 15 minutes adding some Scala 3 (formerly Dotty) features to an Scala 2 program.

The program itself builds a simple Algebraic Data Type (ADT) to represent a simple arithmetic expressions. We can then build expressions in this algebra and evaluate it using an eval function using pattern matching…

sealed trait Exp
case class Val(value: Int) extends Exp
case class Add(left: Exp, right: Exp) extends Exp
case class Mul(left: Exp, right: Exp) extends Exp
case class Var(identifier: String) extends Exp

Now given an expression like Mul(Var("z"), Add(Val(30), Mul(Var("x"), Var("y")))) I'd like to be able to recursively traverse it and calculate a final Int value at the end.

Val represents an Int value, whilst Add and Mul take care of addition and multiplication. You could go ahead and add more functions. Var is interesting because it takes an a string identifier (i.e., a variable name) and will look it up in an environment. The environment is represented a Scala map of String to Int.

type Env = Map[String, Int]

For the eval function we just use a pattern match to dispatch to functions that handle each particular operation. These handler functions and eval are mutally recursive, and note that every function has to have the Env passed to it as an implicit parameter, yet only Var needs it. This will be important later.

Here's the eval function and handlers.

def eval(exp: Exp)(implicit env : Env): Int = {
  exp match {
    case Var(id) => handleVar(id)
    case Val(value) => value
    case Add(l,r) => handleAdd(l,r)
    case Mul(l,r) => handleMul(l,r)
  }
}

def handleAdd(l: Exp, r: Exp)(implicit env : Env) = eval(l) + eval(r)
def handleMul(l: Exp, r: Exp)(implicit env : Env) = eval(l) * eval(r)
def handleVar(s: String)(implicit env: Env) = env.getOrElse(s, 0)

Note that we could have inlined these functions in eval, but it a larger example it's important to break things out to keep things managable.

That is all the implementation we need, and all that remains is to create an expression, create an environment (declared implicit so Scala knows to include it as an implicit when eval is called) and print the result of evaluating the expression.

val exp1 : Exp = Mul(Var("z"), Add(Val(30), Mul(Var("x"), Var("y"))))

implicit val env : Env = Map("x" -> 17, "y" -> 10, "z" -> 2)
val eval1 = eval(exp1)

println(s"Eval exp gives $eval1")

You can compile and run the code to see this working. The code is here. https://github.com/justinhj/evalexample/blob/master/src/main/scala/Scala2Eval.scala

Fun with Enum

Scala enums have been improved greatly. For one they are very simple to create and use just as in other languages.

enum StatusCode:
   case OK, TimedOut, Error

Here we've defined three enums that have ordinal values 0 to 2. You can access the ordinal value with the .ordinal method, convert ordinal values to Enums using .fromOrdinal and convert Strings to enums (assuming they match) with .valueOf.

println(s"Ordinal value of StatusCode.Error is ${StatusCode.Error.ordinal}")
println(s"StatusCode from ordinal 1 is ${StatusCode.fromOrdinal(1)}")
println(s"StatusCode from string OK is ${StatusCode.valueOf("OK")}")

// Ordinal value of StatusCode.Error is 2
// StatusCode from ordinal 1 is TimedOut
// StatusCode from string OK is OK

You can also add your own parameters and definitions to enums. The underlying ordinal values are still there. For example you could encode Http Status codes as follows.

enum HttpStatusCode(code: Int) {
  case OK extends HttpStatusCode(200)
  case NotModified extends HttpStatusCode(304)
  case Forbidden extends HttpStatusCode(404)

  def isSuccess = code >= 200 && code < 300
}

Scala 3 team also took the opportunity to make Enums a more concise notation for ADTs and GADTs. For our purposes that means we can simply the definition of Exp as follows.

enum Exp {
  case Val(value: Int) extends Exp
  case Add(left: Exp, right: Exp) extends Exp
  case Var(identifier: String) extends Exp
}

In fact you can further simplify to the following (you could also remove the braces).

enum Exp {
  case Val(value: Int)
  case Add(left: Exp, right: Exp)
  case Var(identifier: String)
}

Explicit implicits

A focus of the Scala 3 team is to help beginners access the language and in particular simplifying implicits. There are many subtle changes here but two obvious ones are that you now have different keywords for implicit parameters and creating implicit instances. In our code this means that when we supply the implicit symbol table to eval we now use the new given syntax instead of implicit.

implicit val env : Env = Map("x" -> 17, "y" -> 10, "z" -> 2)

becomes…

given envMap: Env = Map("x" -> 7, "y" -> 6, "z" -> 22)

Similarly, the method parameters now no longer use the implicit keyword and instead you prefix the parameter name with using.

def eval(exp: Exp)(implicit env : Env): Int

becomes…

def eval(exp: Exp)(using env : Env): Int

You don't have to change your Scala 2 code at this point, it is still compatible, but for new code and in the long term you should gradually eliminate implicit.

Context Functions

Last and not at all least are context functions. This gives us one more opportunity to remove boiler plate from the eval code. When you create a regular function value it has a type like Function1[A,B]. In other words it is a function that takes a value A and returns vale of type B. Context Functions are a new function value type (this is synthesized by the compiler so you won't see it your code), with an input and an output type. The difference is that the input is understood to be provided implicitly.

Let's make this more concrete. Assume you have a function that needs an ExecutionContext. We can make a Context Function type that will take an implicit execution context and return some paramaterized type T.

type Executable[T] = ExecutionContext ?=> T

How would that be used in a real program? Let's say you have some deeply nested function (f4 in the code below) and it is only down at that level you need the implicit execution context. Without implicit parameters you'd add the ExecutionContext parameter to every single function call all the way down and then have to take care to pass it along. With Scala 2 implicits you still have to declare the parameter but you can make it implicit and avoid the burden of manually passing it along.

With Scala 3 you can define the function to be of type Executable[T] and then we don't need to even name the implicit parameter, we just know that it will be included automatically all the way down. Here is a complete example.

import scala.concurrent.{Future, ExecutionContext, Await}
import scala.concurrent.duration._
import scala.language.postfixOps

object Executable extends App {

  type Executable[T] = ExecutionContext ?=> T

  def f1(n: Int): Executable[Future[Int]] = f2(n + 1)
  def f2(n: Int): Executable[Future[Int]] = f3(n + 1)
  def f3(n: Int): Executable[Future[Int]] = f4(n + 1)
  def f4(n: Int): Executable[Future[Int]] = {
    val ex = summon[ExecutionContext]
    Future {
      println(s"Hi from the future! n is $n")
      n
    }
  }

  {
    given ec: ExecutionContext = scala.concurrent.ExecutionContext.global
    Await.result(f1(10), 1 second)
    // Hi from the future! n is 13
  }

}

Context functions reduce boilerplate when dealing with implicit parameters in deeply nested code. We can apply this technique to our eval function so that the symbol table itself is the implicit piece of context.

type WithEnv = Env ?=> Int

def eval(exp: Exp): WithEnv =
  exp match {
    case Var(id) => handleVar(id)
    case Val(value) => value
    case Add(l,r) => handleAdd(l,r)
  }

def handleAdd(l: Exp, r: Exp): WithEnv = eval(l) + eval(r)

def handleVar(s: String): WithEnv =
  val env = summon[Env]
  env.getOrElse(s, 0)

You can take a look at the final Scala 3 version of the code here.

https://github.com/justinhj/evalexample/blob/master/src/main/scala/Scala3Eval.scala

Final notes

Of all the new features in Scala 3, I found Context Functions of most interest because of Martin Odersky's blog from 2016 https://www.scala-lang.org/blog/2016/12/07/implicit-function-types.html where this intriguing quote appears near the end. (Context functions were initially known as implicit functions).

There are many interesting connections with category theory to explore here. On the one hand, implicit functions are used for tasks that are sometimes covered with monads such as the reader monad. There’s an argument to be made that implicits have better composability than monads and why that is.

On the other hand, it turns out that implicit functions can also be given a co-monadic interpretation, and the interplay between monads and comonads is very interesting in its own right.

But these discussions will have to wait for another time, as this blog post is already too long.

Somewhat of a Fermat's last theorem moment there, and I am also interested in how we can represent concepts, that are currently implemented in libraries which model category theory, using vanilla Scala 3 or alternative representations.

References

https://en.wikiquote.org/wiki/Pierre_de_Fermat

https://dotty.epfl.ch/docs/reference/enums/enums.html https://dotty.epfl.ch/docs/reference/enums/adts.html

http://dotty.epfl.ch/docs/reference/other-new-features/indentation.html

https://dotty.epfl.ch/docs/reference/contextual/givens.html https://dotty.epfl.ch/docs/reference/contextual/using-clauses.html

https://dotty.epfl.ch/docs/reference/contextual/context-functions.html

Foundations and Applications of Implicit Function Types https://infoscience.epfl.ch/record/229878/files/simplicitly_1.pdf

http://recurse.se/2019/09/implicit-functions-in-scala-3/