Stories by Jeffrey Rennie on Medium

Dear C# Developers, Just #@!&ing Learn JavaScript Already

Jeffrey Rennie — Mon, 29 Sep 2025 18:14:43 GMT

Website user is ready to slap you upside the head for not learning JavaScript

I get it. I too have dipped my toes into the cesspool of JavaScript and recoiled in horror.

React is tangled mess, and every new version of React brings a new tangled mess of breaking changes. Yesterday I followed the official instructions to create a React app with TypeScript and got a compile error. I asked Gemini for help, and it suggested a mess of changes to configuration files that I’ll never get right. It was hopeless and I gave up.

NPM and everything built with NPM is a house of cards built on top of a minefield. I work on some websites built with Vue, and every time I upgrade one little dependency, I spend a day unbreaking everything that broke as a result of the upgrade. Dependabot reports new vulnerabilities in my Vue projects’ dependencies at least once a week. Most of Dependabot’s patches fail to compile.

Running JavaScript on the server was a bad idea 15 years ago, and that hasn’t changed.

I won’t sweep these issues under the rug and attempt to persuade you to adopt a JavaScript framework. On the other hand, your refusal to learn JavaScript is causing you and your users more pain than just #@!&ing learning JavaScript. Refusing to learn JavasScript makes about as much sense as building websites and refusing to learn HTML.

If every C# developer were half as skilled in JavaScript as they were in C#, then ASP.NET Core MVC would be sufficient to build any enterprise app. There would be no need for Blazor, and the Web would be a better place if Blazor never existed.

The trouble with Blazor

Blazor is a combination of two seriously flawed and disparate technologies that, when combined, leave the programmer with more problems than solutions.

A user suffering from a website built with Blazor

The first technology is Blazor Web Assembly. It compiles C# code into Web Assembly and runs it in the browser. It’s a fine idea in theory, but in practice, it punishes a website’s users. On a mobile network, users spend over 3 seconds downloading about 10 MB of .wasm and .dll files to interact with the a Blazor web page. 10 MB of data transfer just cost a user in Malawi roughly 1% of their monthly salary. And decompressing the files and then compiling them from web assembly into machine code drained their phone’s battery. Also, Blazor Web Assembly apps are laggy and memory hungry. Of the dozens of JavaScript frameworks available today, you literally can’t choose one slower or more memory hungry than Blazor. The average device browsing the internet today is a 2-year old Android phone that cost less than $200. Blazor apps struggle on such devices.

Blazor Web assembly will not only punish your users, it will also punish you when it’s time to debug a misbehaving website. I’m not talking about running the application in your debugging environment. I’m talking about when a tester hands you their laptop and shows you a bug in the production website that only happens on that laptop, so you open the browser’s debugging tools and see… absolutely nothing useful. Good luck!

The second technology is Blazor Server. It ships a small JavaScript library to the browser which opens a WebSocket back to the server. At least it doesn’t drain users’ batteries or bank accounts, but users will still suffer when there is a hiccup in that WebSocket connection to the server. Maybe the server crashed or restarted, or the user’s phone switched cell towers, or was routed to a different server in the server farm. A user was filling out page 9 of a 10-page form, and now they have to start over on page one. They’re not happy.

And like Blazor Web Assembly, Blazor Server will punish you when it comes time to debug. If you’re lucky enough to see messages passed on the WebSocket (I didn’t), you’ll have to decode the binary BlazorPack protocol. Good luck!

Blazor Server punishes another innocent party spared by Blazor Web Assembly: Site Reliability Engineers. They can’t simply shut down an underutilized Blazor server because the few users still connected to that server would see the website completely freeze. This makes regular maintenance like rolling out new versions of your website, scaling up and down to meet load, and moving web servers from one machine to another much more complicated.

If Blazor is a train wreck and JavaScript frameworks are hot garbage, then what’s a C# programmer to do?

Learn some JavaScript. Just the basics will take you a long way.

There’s good news. JavaScript in the browser in 2025 is not the crumbling Tower of Babel it was in 2009. Internet Explorer is long gone. Microsoft Edge, Google Chrome, Firefox and Safari all conform to web standards. Modules prevent global namespace pollution and other headaches.

To demonstrate how powerful and easy it is to add a little JavaScript, let’s consider the default demo app you get when you run dotnet new blazor.

I rebuilt this application as an MVC app and reproduced all Blazor’s interactive features with only 6 lines of JavaScript. Let’s explore the app.

First, Blazor’s components are nothing special. I used MVC’s partial views instead of Blazor’s components. Let’s compare.

Blazor:


    
    @Title

    
        Please take our
        brief survey
    
    and tell us what you think.


@code {
    // Demonstrates how a parent component can supply parameters
    [Parameter]
    public string? Title { get; set; }
}

MVC:

@model string


    
    @Model

    
        Please take our
        brief survey
    
    and tell us what you think.

The differences between these two source files are so small they’re not even worth discussing. I see no reason to prefer the Blazor component over the MVC partial view.

Now let’s examine the 3-bars button in the top right corner, that when clicked, displays the menu.

Blazor implemented the menu with the following code. Unbelievably, it performs a network round trip to the server just to open the menu. If the network request is delayed by 2 seconds, then the user will wait 2 seconds to see the menu drop down. More likely, they’ll give up and try your competitor’s website.


    
        BlazorApp
        
    




    
        
            
                 Home
            
        

        
            
                 Counter
            
        

        
            
                 Fetch data
            
        

    



@code {
    private bool collapseNavMenu = true;

    private string? NavMenuCssClass => collapseNavMenu ? "collapse" : null;

    private void ToggleNavMenu()
    {
        collapseNavMenu = !collapseNavMenu;
    }
}

Here’s the MVC version. It actually doesn’t need any JavaScript to implement the drop down when clicked. Even when the network is unavailable, the menu drops down immediately.


    
        BlazorApp
        
            
                
            

        
    






    
        
            
                 Home
            
        

        
            
                 Counter
            
        

        
            
                 Fetch data
            
        

    



@functions {
    string classForPath(string path) {
        return path == Context.Request.Path ? "active" : "";
    }
}

It employs a handy feature of HTML and CSS: the state of a checkbox can control the appearance of another element. Here’s the relevant CSS:

#toggle-menu:checked ~ #nav-menu {
    display: block;
}

Next, let’s re-implement the Counter with a little JavaScript:

First, Blazor’s implementation:

@page "/counter"

Counter

Counter


Current count: @currentCount




@code {
    private int currentCount = 0;

    private void IncrementCount()
    {
        currentCount++;
    }
}

And now, the MVC version:

@{
    ViewData["Title"] = "Counter";
}

Counter


Current count: 0

See type=”module” on the script tag? That confines the scope of count to the namespace bounded by . count never enters the global namespace. This avoids name collisions that caused many headaches before type=”module” became a thing.

Also, scripts with type=”module” are always deferred. They don’t get evaluated until the document is completely parsed. This means there’s no need to put them in a special @section Scripts {} block.

If you’re a C# programmer and you see something confusing in the JavaScript code, please leave a comment.

Again, the MVC version has the advantage that the count will increment immediately with each click, no matter what condition the network is in.

Finally, let’s examine the Weather Forecasts page:

Here’s Blazor’s implementation.

@page "/fetchdata"

Weather forecast

@using BlazorApp.Data
@inject WeatherForecastService ForecastService

Weather forecast


This component demonstrates fetching data from a service.


@if (forecasts == null)
{
    Loading...

}
else
{
    









            @foreach (var forecast in forecasts)
            {






            }

                                    Date                 Temp. (C)                 Temp. (F)                 Summary             
        
                                            @forecast.Date.ToShortDateString()                     @forecast.TemperatureC                     @forecast.TemperatureF                     @forecast.Summary                 
        
    

}

@code {
    private WeatherForecast[]? forecasts;

    protected override async Task OnInitializedAsync()
    {
        forecasts = await ForecastService.GetForecastAsync(DateTime.Now);
        StateHasChanged();
    }
}

Date	Temp. (C)	Temp. (F)	Summary
@forecast.Date.ToShortDateString()	@forecast.TemperatureC	@forecast.TemperatureF	@forecast.Summary

MVC’s implementation is split across two handlers, /FetchData and /Forecasts. First, the view for /FetchData:

@{
    ViewData["Title"] = "Weather forecast";
}

Weather forecast


This component demonstrates fetching data from a service.



    Loading...

The JavaScript code fetches /Forecasts from the server, and replaces the Loading… place holder. This will work even if the /Forecasts request arrives at a different server than the preceding /FetchData request. In fact, it will usually work even when the two servers are running different versions of the website. /Forecasts’ view requires no JavaScript:

@model MVCApp.Data.WeatherForecast[];

@{
    Layout = "_NoLayout";
}











        @foreach (var forecast in @Model)
        {






        }

                        Date             Temp. (C)             Temp. (F)             Summary         
    
                                @forecast.Date.ToShortDateString()                 @forecast.TemperatureC                 @forecast.TemperatureF                 @forecast.Summary

Date	Temp. (C)	Temp. (F)	Summary
@forecast.Date.ToShortDateString()	@forecast.TemperatureC	@forecast.TemperatureF	@forecast.Summary

And we’re done. That’s it. We’ve reproduced every interactive feature of the Blazor default application with only 6 lines of JavaScript.

The benefits are many. I added less than 1k of JavaScript to the website instead of 10MB of wasm. The UI responds immediately instead of waiting for a round trip from the server. Debugging with only the browser is straight-forward. I see plain HTML in the network tab, and I can step through each line of JavaScript I wrote. I can deploy a new version of my app without waiting for all users to disconnect from running instances.

Some readers may have heard alarm bells when they saw element.innerHtml = await response.text();in the code above. There’s no reason for alarm because the response came from a .cshtml file (a View) which did proper HTML escaping. Fetching a View and assigning the response to .innerHtml is no more dangerous than simply providing a link to the same URL.

Indeed, assigning .innerHtml to View responses is a simple way to transform yourself from a backend engineer to a full-stack engineer. No hydration or dehydration is necessary. The network tab always displays readable HTML. If you like this technique, then you’ll love HTMX. HTMX makes it easy to dynamically update a page with View responses without writing any JavaScript, but more about that later.

A Hello World application and a code base of 100,000 lines of JavaScript are two very different beasts.

Two very different beasts

Reading 10 lines of JavaScript is rarely difficult. Managing a project with 100,000 lines of JavaScript is never easy.

JavaScript modules, as I demonstrated above, are the first step towards corralling complexity. In short, they work a lot like .cs files and using statements. They don’t pollute the global namespace. Importing a JavaScript module multiple times results in it getting evaluated only once. One module can selectively import the symbols it needs from another module. A module can rename symbols as it imports them.

TypeScript tames the JavaScript beast.

Microsoft (actually, the same person who designed C#) invented TypeScript, so I’m surprised they don’t spend more energy promoting it to C# developers. They publish the Nuget package Microsoft.TypeScript.MSBuild which compiles TypeScript files in a .csproj automatically. Microsoft also publishes instructions on how to add TypeScript to you MVC project. The instructions make it easy to add TypeScript to your project without running an npm command. Thank you Microsoft!

The benefits of using a type-safe language are well known to C# developers. Building an application with TypeScript instead of JavaScript means that far more mistakes will be found at compile time rather than runtime. The benefits are really valuable when working with a large team and a large code base.

Let’s peek at what the /counter implementation looks like with TypeScript. First, Counter.cshtml:

@{
    ViewData["Title"] = "Counter";
}

Counter


Current count: 0

And Counter.ts:

let count = 0;
document.getElementById("countBtn").onclick =
    () => document.getElementById("countSpan").innerText = String(++count);

And here’s my favorite part. Below is Counter.js, the code compiled from Counter.ts above and served to the browser. This is the code I step through when a tester hands me their laptop with a misbehaving website:

let count = 0;
document.getElementById("countBtn").onclick =
    () => document.getElementById("countSpan").innerText = String(++count);
//# sourceMappingURL=Counter.js.map

Aside from comments and white space, it looks identical to the original TypeScript. There’s no magic. When it comes time to debug an app in production, I don’t even need a source map.

One thing missing from Microsoft’s TypeScript tools for ASP.NET core is hot reloading. I find writing code without hot reloading to be frustrating, so I built MVCAppWithTypeScript to perform hot reloading with esbuild. I recommend using it as a template if you’d like to add TypeScript to your ASP.NET core MVC project.

JavaScript isn’t Hard.

I learned C#, JavaScript, Rust, and F# in that order. It took me about 3 times as long to learn each of Rust and F# than it took to learn JavaScript. It’s not hard. Yes, there are lots of quirks in JavaScript, but TypeScript disambiguates some of those quirks, and I’d rather live with the quirks than make my users and myself suffer Blazor.

In conclusion

Before building a project with Blazor, I recommend just @#!&ing learning JavaScript. It’s doable. It’s not the nightmare it used to be. TypeScript will convert some run-time mysteries of JavaScript into highly readable compile-time errors.

Also, feel free to ignore the Next Great JavaScript framework that comes out next week. It’s trap. It will work great on day 1, but 2 months and 3 versions later, you’ll regret ever hearing about it.

HTMX is another way to dynamically update a page without the flaws of Blazor and without writing a line of JavaScript. I published a Udemy course on how to add HTMX to your ASP.NET core MVC Application. Use the code NEVER-BLAZOR before October 25, 2025 to receive a 20% discount.

Dear C# Developers, Just #@!&ing Learn JavaScript Already was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

I built the same software 3 times, then Rust showed me a better way

Jeffrey Rennie — Tue, 22 Jul 2025 16:30:49 GMT

I build software that manipulates PDF documents. Over 2 decades, I have built such software many different ways:

With Python using PLY.
With C++ using Flex and Bison.
With C++ building a recursive-descent parser.
With Rust building a recursive-descent parser.

The Rust implementation was 3 times faster than the C++ implementations because Rust showed me a new way to represent PDFs in memory.

When building the fourth application in Rust, I tried to represent the PDF document in memory the same way I did in C++, but Rust wouldn’t let me. No matter what I tried, Rust reported errors with lifetimes and mutability. I spent about a week trying to convince the Rust compiler to let my code compile, but it wouldn’t yield. Finally, I gave up and redesigned the in-memory representation of PDF documents to conform to Rust’s rules. The result was a massive boost in performance and reduced memory consumption.

To be clear, I don’t think any of the performance differences between the C++ and Rust applications were due to the languages themselves. Had I represented the internal PDF structure in C++ the same way I had in Rust, I’d bet a dollar that the C++ code would have been faster than the Rust code. I had just never thought to write the C++ code that way.

PDF Document Internals

The internal structure of a PDF Document is complex. The specification for PDF version 1.7 is 756 pages long, and that doesn’t even describe lots of supporting technology needed to fully decode a PDF document like a full JPEG decoder, multiple font file decoders, compression algorithms and more.

However, the core skeleton of a PDF Document is actually quite simple, and not much different from JSON. Consider the following JSON representation of a family tree:

https://medium.com/media/481b615144dac8d372e07a1bbd0b8c4c/href

I’ll transform this JSON into something a little more like a PDF document, and then explain the transformations:

https://medium.com/media/0a6cfca122cabc9f68a8081d33f6cdc1/href

The transformations explained:

id fields have been removed, and replaced with a unique object number. Id was defined by the implicit JSON schema, but the object number is defined by the PDF specification.
Links to other objects are represented by a new symbol: R-object-number.
A look up table at the end of the file lists the first line number for each object. This makes it quick and easy to follow a link from one object to another.

How would you represent this file format in memory, knowing that most PDF documents are too large to fit into memory, and a document and its objects will only be accessed from a single thread?

In C++, I represented the document like this:

https://medium.com/media/b7a1009b61247c075e6f09d341f798f8/href

This code followed object oriented design principles. Every object type, such as string, array, link, etc. was implemented with its own class derived from Object. Each class knew how to write itself to a PDF file. Every object was allocated on the heap. Reference counts safely kept each object in memory as long as it was needed. Circular references weren’t actually a problem for reasons that are outside the scope of this article.

Let’s tally the operations needed to destroy the following object:

  {
    "name": "Michael Doe",
    "birthday": [1995, 3, 10],
    "father": R-0,
    "mother": R-1
  }

free 8 Objects: 3 numbers + 2 links + 1 array + 1 string + 1 dictionary
decrement the doc’s reference count 8 times
decrement objects’ reference counts 7 times
free 5 strings
free 1 vector
free 1 hash_map

That’s a total of 15 frees and 13 reference counts decremented. Objects may be scattered across the heap so cache locality is likely low, negatively impacting performance. In this context, there’s a code branch immediately after decrement. Code branches also negatively impact performance.

This C++ code could not be directly translated to Rust because the Rust does not allow willy-nilly pointers like this.

Look at the C++ code and try to answer a simple question: which lives longer, a document or an object in the document’s cache? It’s impossible to know just by looking at the code I provided. Rust requires an answer to this question or it will refuse to compile the code.

One way to maybe satisfy Rust’s compiler would be to use Rc<>, which is a smart pointer type that counts references. I was new to Rust when I was writing this code. I didn’t know or look for a reference-counting smart pointer type, because I thought a reference-counting smart pointer would throw away half the benefits of Rust’s borrow checker. I’d be sacrificing compile-time guarantees for runtime enforcement; it would have been no better than assert statements in C++. At that point, why even write the code in Rust? It would have been easier to write the code in C++.

Also, it would have been possible to implement something like the Object base class in Rust by using traits, but I chose to use an enum instead. Enums in Rust are a lot like C++’s std::variant<>, but Rust provides syntax to make working with enums easy and less error prone. Anyways, Rust’s enums were shiny and new to me and I wanted to try them. Let’s look at the Rust implementation:

https://medium.com/media/cd8f963eab7a06a86d6559bc84424934/href

Rust’s lookup() function returns a copy of the object from the cache, whereas C++’s lookup() function returns a pointer to the object in the cache. I thought copying objects would substantially and negatively impact performance, but benchmarks and profilers showed me the performance benefits of using an enum far outweighed the cost of copying objects. Because objects are no longer modified in place, I added a set() function to update the document with changes to objects.

Compared to the C++ code, the Rust code allocates far fewer blocks on the heap.

Let’s tally the operations needed to destroy the same object:

{
    "name": "Michael Doe",
    "birthday": [1995, 3, 10],
    "father": R-0,
    "mother": R-1
  }

free 1 Vec<>
free 1 HashMap<>
free 5 strings

For a total of 7 frees and 0 reference count decrements. Compare that to C++’s 15 frees and 13 reference count decrements.

These changes alone made my Rust program perform more than twice as fast as the same program implemented in C++.

I was so inspired by the performance benefits of reducing heap allocations that I switched the string implementation in the Rust code to a type that stores short strings in-place without allocating any memory. Only longer strings require heap allocations. So in the example above, keys like “name” and “birthday” require no heap allocations. Only longer strings like “Michael Doe” require a heap allocation. That reduces the tally for destroying the object above to:

free 1 Vec<>
free 1 HashMap<>
free 1 string

So 3 frees and 0 reference count decrements. Compare that to C++’s 15 frees and 13 reference count decrements.

The drastic reduction in allocations and reference counts and better cache locality made my Rust implementation about 3 times faster than my C++ implementation.

Rust is not faster than C++

It’s possible and not too difficult to optimize the C++ so it performs like my Rust code. In fact, I’d bet that with all the same optimizations applied, the C++ code would be faster. That’s not the point.

The point of this article is: the obvious, convenient way to write the code in Rust also turned out to be optimal for performance. That’s amazing.

I wrote the C++ code around 2005, when std::variant<> didn’t yet exist, std::string didn’t do small string optimization, and OOP was the accepted wisdom. I haven’t written C++ intensively in about 10 years. Has the accepted wisdom of C++ programmers changed since then?

Also, is something like Rust’s enums available in your favorite programming language? Do you think your favorite programming language could match Rust’s performance on this task?

I built the same software 3 times, then Rust showed me a better way was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

git rebase or git merge? Neither.

Jeffrey Rennie — Thu, 19 Jun 2025 00:48:16 GMT

I just watched some people argue about this, and I didn’t hear anyone describe my favorite solution, so I’ll describe it here. It leaves a perfect trail of breadcrumbs so if I make mistake, I can always rollback. Hopefully, it saves you some time and pain.

In this scenario, I’ve been working in my feature branch for two weeks. Of course, lots of changes have been committed to the main branch while I’ve been working in my feature branch. Now it’s time to pull changes from main.

> git status
On branch feature
nothing to commit, working tree clean
> git checkout main
> git pull
> git checkout -b feature2
> git merge --squash feature

And then I resume my work in the feature2 branch.

Why this is better than rebase or merge.

The git history is perfectly linear, thanks to--squash.
Changes are never overwritten or lost.
Resolving conflicts the wrong way can be undone later.

And yes, this is a serious flaw in git.

I hate the fact that git provides at least 3 ways to do the most common task of any source control system: pull down recent changes from the main branch. The git command line is hostile to humans. I haven’t found anything better, though.

Why Functional Programming Languages Will Never Be Mainstream

Jeffrey Rennie — Thu, 02 Jan 2025 16:38:02 GMT

Functional programming has been the next big thing for the past 65 years, so why do all functional programming languages together not exceed 5% of mind share according to IEEE, Tiobe, etc? This article describes two reasons why functional programming languages are more difficult than the rest, and how that prevents them from gaining wider adoption.

Climbing Mount Lambda

During 2024’s Advent of Code, I completed 39 of the 50 puzzles using F#. F# describes itself as functional-first, and it’s really well suited for solving puzzles like these. It was fun and it was a great way to learn F#. My solutions were, on average, fewer lines of code than other solutions posted in other languages. I also found my F# solutions to be more readable, but I think every programmer finds their own code to be more readable.

However, even after weeks of practice with F#, I sometimes struggled to write code for reasons that were specific to functional programming languages, and this wasn’t my first time using a functional programming language.

The first computer science course I took in college in the 1990s was taught in Scheme. Scheme is a simpler version of Lisp. Lisp is the first functional programming language; it was first released in 1960.

In 2020, I started building a new web app, and I spent a few weeks evaluating Elm. Elm is a functional language that runs in the browser. I even wrote a little hang man game using Elm. I ultimately chose other technologies to build my app, but I liked Elm.

So at a minimum, Advent of Code was my third dive into a functional programming language. As I solved puzzles with F#, two reasons became apparent as to why functional programming languages are harder than the rest.

Reason 1: Writing a loop is a lot harder.

Loops are absolutely essential to code. The first line of code I ever wrote was 10 PRINT “JEFF IS KING” and the second line was 20 GOTO 10. So a loop was literally the second line of code I ever wrote. A programming language that makes it tricky to write a simple loop is going to struggle to find an audience.

Haskell, Roc, Elm, Clojure, Elixir and OCaml don’t provide the familiar while and for loop language constructs that make writing loops easy in other languages. With these functional languages, you have two choices: recursion or “higher level” functions.

Recursion is hard. Don’t think it’s hard? Try explaining it to someone who has never written a line of code. Now try explaining loops. When I tried this experiment, my listener quickly understood loops but couldn’t grasp recursion. Still don’t think recursion is harder than writing a loop? Please leave a comment because you’re the first programmer I’ve met who thinks so. I must have written 300+ recursive functions over my 30 year career and I still find it easier to write loops. I like to avoid recursion when feasible, and in functional languages that means using higher level functions.

To write loops without recursion in a functional programming language, you have to master about 50 higher level functions. Compare that to most other languages where you only have to master 2 or 3 language constructs (for, while, foreach) to write a loop.

Here are some higher level functions provided by F#. The ones in bold were used in my Advent of Code solutions.

allPairs, average, averageBy, choose, chunkBySize, collect, compareWith, contains, countBy, except, exists, filter, find, findBack, findIndex, findIndexBack, fold, foldBack, forall, groupBy, countBy, iter, iteri, map, mapFold, mapFoldBack, mapi, max, maxBy, min, minBy, pairwise, pick, reduce, reduceBack, rev, scan, scanBack, skip, sum, sumBy, tryFind, tryFindBack, tryFindIndex, tryFindIndexBack, tryPick, unfold, where, windowed, zip.

You may want to ignore the higher level functions and stick with recursion, but that’s not practical. Chances are, your teammates will use higher level functions and you’ll need to read their code. And when your teammates review your code, they’ll ask you to replace your recursive functions with calls to higher level functions. It’s a reasonable request because code that calls higher level functions is usually easier to read than recursive functions. Also, code that calls higher level functions is usually a line or two shorter, and functional programmers love to play code golf. So, you’re stuck having to learn about 50 functions compared to any non-functional language in which you’d have to learn only 3 looping constructs. Applying some questionable math, that makes functional programming languages 15 times more difficult to learn.

Even while programming solo, I found myself drawn towards higher level functions. Remarkably, my progression mirrored the five stages of grief:

Stage 1: Denial. I denied the existence of recursion and higher level functions and kept writing for and while loops. F# is a bit unusual among functional programming languages because it provides for and while loop constructs, so I used them.

Stage 2: Anger. F# provides for and while, but it doesn’t provide break, continue or return. This made writing loops frustrating. For example, on day 21, I wanted to write this code:

https://medium.com/media/c093b179f854bbeb6418f92aea29328b/href

This code failed to compile because F# doesn’t have a return statement. To get the code to compile, I had to write:

https://medium.com/media/86b64be88c107186c5ca19652ac968d3/href

I felt angry. I had to introduce additional variables and logic just to exit the loop, because F# didn’t provide a return statement. It felt like the designers of F# were intentionally making my life more difficult than it needed to be.

Stage 3: Bargaining. I bargained with F# by switching from loops to recursion. With recursion, I could effectively break out of loops early.

https://medium.com/media/6e4b29c5f5dd539da57c575978ab4dcb/href

Stage 4: Depression. I was disappointed that switching to recursion hadn’t provided any advantages. The recursive code avoids mutable state, and some people would call that a win, but I don’t, because my original function with a loop was already a pure function. The recursive code is no fewer lines of code than the while loop above, so that didn’t feel like a win either.

Stage 5: Acceptance. I finally concluded that to get the most out of F#, I’d have to take the time to study and learn 50 higher level functions.

Even after practicing F# for over 60 hours, and taking time to study every higher level function, it still took me 20 minutes to transform moveIsLegal into code that calls a higher level function:

https://medium.com/media/8fdf093d0d8db3cf7f7c5d1f3b1aa2d6/href

And this, far more than the other reason I discuss below, is why people struggle with functional programming languages. It took me a total of 30 minutes to write moveIsLegal in functional form compared to 10 minutes in procedural form. This was day 21 in the Advent of Code. I had spent at least 60 hours practicing F# and I was still struggling to write a loop.

Reason 2: Misleading error messages.

When I made mistakes writing F#, it was very hard to understand the compiler’s error messages. The error messages weren’t wrong, they just weren’t helpful, and sometimes they were even misleading. My experience was similar with Elm in 2020, and I believe it would be similar with Haskell, Roc, and OCaml, because they also implement type inferencing and currying.

Type inferencing means that you rarely have to declare types of function arguments, function return values, or local variables, because the compiler figures out the types from context. When the code is correct, type inferencing is really nice because the code contains pure logic; it’s not cluttered with type information. However, when the code fails to compile, the compiler has fewer clues as to what the code is attempting to do, and therefore struggles to provide a useful error message. Here’s an example:

https://medium.com/media/9b8fa22ca226886e836302804ea288a9/href

What was my mistake? The comma on line 11 should be a semicolon. The 5 errors reported by the F# compiler gave me no clue to the true mistake. I frequently copied code into ChatGPT and asked it what was wrong. More often than not, ChatGPT helped. In this case, ChatGPT’s second guess provided the clue I needed to fix my mistake. But copying code into ChatGPT isn’t always an option. Many organizations wisely prohibit copying source code into ChatGPT. Even ChatGPT will tell you why you shouldn’t copy code into ChatGPT.

Currying is another language feature that is very nice when you get it right, and very difficult to figure out when you make a mistake. Imagine a function f that takes two arguments, f(a, b). What happens if you try to call f with only one argument f(a)? In most programming languages, the compiler will report an error. In languages that curry, compilers won’t report an error, at least not where you made the mistake. In languages that curry, f(a) returns another function that takes a single argument b. In other words let g = f(a); let n = g(b); evaluates to the same result as let n = f(a, b);. This means that when you accidentally pass too few arguments to a function, the compiler never reports an error at the call site. Instead, it reports an error somewhere else. Here’s another mistake I made:

https://medium.com/media/b041f2f2727613303da10354b6035932/href

What was my mistake? I forgot to pass one more argument to Seq.scan on line 12. Could you deduce that from the error messages?

Non-Conclusions

I’m not claiming that functional programming languages are fundamentally flawed or unsuitable for any purpose. They have lots of great features found in few other languages. In my experience, when I write code in a functional language, I write fewer lines of code and fewer defects. If someone wanted to pay me to build a financial application with .NET, my first question would be, “Can I use F#?”

Also, I’m not claiming that other programming languages won’t find success by adding features borrowed from functional programming languages. Rust has immutable state and tagged unions. JavaScript arrays have methods like filter(), map(), and reduce(). Lots of books have been published with titles like Functional Programming in C++/JavaScript/Go/etc. These other languages don’t suffer from the 2 Reasons I described above, so importing functional language features won’t hurt their adoption.

Conclusions

I claim that functional programming languages are harder than the rest, and the additional level of difficulty will prevent them from becoming mainstream.

I also claim that functional language features arriving in procedural and object-orient languages won’t help functional languages become mainstream. I see the endpoint in the evolution of programming languages as a hybrid that weaves together the best ideas from all paradigms. I don’t see a march towards functional programming.

Maybe people who develop and maintain functional programming languages don’t want their languages to see greater adoption. But for those advocating greater adoption, I humbly offer the following advice.

First, build simple looping constructs into the language like for, while, foreach, and don’t forget continue, break, and return. If that means introducing mutable state in some limited way, then so be it. Functions with internal mutable state can still be pure.

Second, improve the compiler’s error messages. Maybe some AI built into the compiler would better diagnose mistakes and provide better error messages.

Third, I have heard a few high-profile speakers claim that a functional programming language is no more difficult than procedural programming languages. This message is counter productive. Here’s what happens when someone hears that message and tries a functional programming language for the first time: after struggling for a few hours, they feel dumb, and they distrust everything else you say. No one likes to feel dumb, and they resent you and your language for making them feel dumb. Their struggle has taught them functional programming is harder. You claimed it wasn’t. How many of your other claims are false? Do programmers really make fewer mistakes with functional languages, or is that just another false claim?

Instead of claiming functional programming languages aren’t more difficult, admit that they are more difficult, but show how the benefits outweigh the costs. The F# home page gets this half right by focusing on the benefits, but nowhere does it prepare the reader for a long, frustrating learning journey.

Slow, rough road ahead.

Final Doubts and Questions

Maybe I’m wrong. Maybe a boom in the popularity of functional programming languages is right around the corner. But that begs the question, what makes tomorrow different from the past 65 years? If not for the extra difficulty with loops and compiler errors, then what exactly has kept functional programming languages from becoming mainstream? I look forward to reading your reasons in the comments.

Why Functional Programming Languages Will Never Be Mainstream was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

I wish Go were a Better Programming Language

Jeffrey Rennie — Sat, 02 Nov 2024 16:47:37 GMT

I recently wrote and deployed my first Go application.

Over a 30 year career, I’ve written applications in C, C++, Visual Basic, Objective-C, Typescript, Python, Tcl, PHP, Java, C#, Rust, and probably some other languages I’ve forgotten.

I just wrote and deployed my first Go application. It’s a little web service. There’s not much to it: about 1000 lines of code that translate incoming HTTP requests into arguments for a command line tool, then pipe the output of the command line tool into HTTP responses. This is a perfect fit for Go, and I don’t regret choosing it. However, the experience made me wish Go were a better programming language.

I’ll start with the positive and proceed to the negative.

The Positive

Go’s standard library is fantastic. With other languages like Typescript and Python, my apps have a few direct dependencies which in turn bring in hundreds of indirect dependencies. I probably spend about 10% of my coding time upgrading dependencies because a security vulnerability was discovered in an older version or the author decided to drop support for version 2 and everyone has to migrate to version 3. And I have never once run npm audit and seen a clean result.

By contrast, Go’s standard library provided a web server that supports HTTP2. That meant drastically fewer dependencies than same app built with Rust or Typescript. Also, I searched for Go’s vulnerability database and found that security vulnerabilities in the standard library are rare, and usually only deny service. These are way less scary than the vulnerabilities I routinely see in JavaScript dependencies.

Templ is a dream. Why don’t all templating libraries work this way? I don’t need layouts, bodies, blocks, and the other confusing abstractions I see in other templating solutions. I’m looking at you, ASP.NET. All I need are type-safe functions and data. Templ succeeds in not making the solution more complicated than the problem.

I love Goroutines. I’ve been a fan of light-weight threads since the 90s. They’re the best way to implement parallel processing. I’m mystified that languages born in the 90s and later like Java, Rust, and C# haven’t adopted them. Async Rust is a mess. Async Javascript is error prone. I’ve written Typescript code that forgot to await an asynchronous function, and the result was a long debugging session for me. With Goroutines, the flow of the code matches the flow of the logic in my head, and there’s no performance burden of an operating system thread.

Two more benefits of Go that were critical for my application were fast cold start times and low memory usage. Go lived up to my expectations on all these points.

The Surprisingly Neutral

I’m surprised that I didn’t hate Go’s (lack of) error handling. Yes, roughly 1 of every 5 lines of my code is if nil != err {. However, delivering an accurate, useful error message to the user is always difficult, even with exceptions or Rust’s Result type.

Also, when I look at this code again in a few months, I suspect I’ll appreciate the if nil != err { statements in ways I don’t today. Error propagation and handling is difficult to understand in unfamiliar code written in a language that throws exceptions.

The Negative

During development, my application misbehaved and crashed in ways that would have been caught at compile time with other programming languages, specifically Typescript, Rust, and C#. I probably made other mistakes that I haven’t discovered yet and will eventually be discovered by my users. That’s a bad experience for them and me.

I was burned by code-as-strings. Code-as-strings are string literals in the source code that actually behave as code.

I followed the official Go documentation and wrote this line of code: mux.HandleFunc(“GET /form”, formHandler) I was very surprised when a fetch for /form didn’t execute the formHandler. Instead, it executed the default handler, the same as fetching /. How could this be? After about an hour of debugging and searching, I learned that I had installed Go 1.21, but I was reading documentation for Go 1.23. HandleFunc() only started started supporting method specifiers (the GET in my string) in version 1.22.

I blame myself for mismatching the versions of the go compiler and the documentation. I blame the Go standard library authors for not catching my mistake at compile time. It would have been easy to catch it at compile time. Instead of enhancing the string passed to HandleFunc() in version 1.22, they could have added a new function like this: mux.HandleMethodFunc(“GET”, “/form”, formHandler) Then my error would have been caught at compile time, because the 1.21 compiler would have complained that HandleMethodFunc() doesn’t exist.

It’s nearly impossible for the compiler to catch mistakes in code-as-strings, and they’re used lots of places in Go, for example in serializing structs to Json:

type LogEntry struct {
 Severity string `json:"severity"`
 Message  string `jsan:"message"`
}

The Go compiler provides no clue that I have made a mistake in the code above. Other programming languages catch and report this mistake at compile time, because they don’t encode code as strings. They encode code as code.

I dereferenced nil and Go panicked. Tony Hoare called null his billion-dollar mistake. I guess the Go language designers wanted to one-up Mr. Hoare and invented nil. Not only did my code dereference nil on multiple occasions, I didn’t even understand when a value could be nil. Take this function, for example:

func IsInternal(e HttpError) bool {
  ...

Can e be nil? I can’t tell unless I go look at the definition of HttpError. If it’s a struct, then no, it can’t be nil. If it’s an interface, then yes, it can be nil.

If variable p is nil, can I call p.foo()? Maybe. It might panic, it might not. The only way to know is to check the definition of foo(). And as code is maintained, someone may change foo()’s definition so it doesn’t panic on nil, and leave behind a hundred call sites all checking if p == nil {. The ambiguity over whether a variable can be nil probably triggers more accidents than a broken traffic light.

I also made mistakes with nil that didn’t panic. I realized I had written a Go anti-pattern only after watching Francesc Compoy’s talk on understanding nil for the second time. I had written code like this:

func OhNo() error {
 var httperr *http_error = nil
 return httperr
}

func TestOhNo(t *testing.T) {
 if OhNo() == nil {
  fmt.Println(("It's nil."))
 } else {
  fmt.Println("No it isn't.")
 }
}

I was surprised when TestOhNo() printed “No it isn’t.” This is a language quirk that Go programmers learn quickly, I hope. I expect to make mistakes like this when learning a new language. But I blame Go for not catching this mistake at compile time. This bug was lurking in my code and I got lucky and watched Mr. Compoy’s talk and figured out my mistake. Otherwise, I would have only discovered it when a user reported my application behaving incorrectly.

By the way, go vet caught none of the issues I described above.

Conclusion

I make too many mistakes to make Go my first-choice programming language. I have come to depend on the Rust and Typescript compilers catching my mistakes. My users unknowingly appreciate it because they never see those mistakes. And one benefit I enjoy is writing fewer tests: no need to write a test for when a parameter is null if the parameter can never be null.

Even with this experience, there are still some, but not many applications for which I would choose Go. For nearly every programming language, I can imagine an application where it would be the ideal language to use.

But in the end, I feel disappointed by Go. The Go runtime is excellent. Go is so promising. I just wish Go were a better programming language. I wish it caught more mistakes at compile time instead of letting my users discover them at run time. Other languages do it; I expect no less from my next language.

5 Reasons Why We Moved to Google Cloud Run from Firebase Functions

Jeffrey Rennie — Sun, 06 Oct 2024 16:16:54 GMT

Firebase Functions provide an easy path for programmers to quickly deploy a backend service for their websites. But as the website grows, it grows more difficult to maintain Firebase Functions. In this post, I break down our rationale for moving to Cloud Run into 5 reasons.

1. Firebase Functions lack repeatable deploy and rollback.

Deploying a Firebase Function means deploying JavaScript code, which in our case was transpiled from Typescript. Firebase functions don’t provide a way to archive the transpiled JavaScript code that we deployed.

When rolling back to an earlier version, deploying freshly re-transpiled JavaScript was risky. We had to check out the old source Typescript, recompile and redeploy, and hope that the build environment hadn’t changed since the code was first deployed. Perhaps we upgraded our version of npm or the Typescript compiler in the meantime and the code would fail to compile or behave differently. That’s bad.

A great feature of Firebase is that it allows you to deploy a subset of functions. If you only change the code for one function, you can redeploy just that one function.

A terrible flaw of Firebase is that it allows you to deploy a subset of functions. If you’re running 10 functions, they may have come from 10 different versions of the same code base or different code bases. It becomes very difficult to know exactly which code is running, and therefore which version to roll back to when there’s an issue.

We could build our own system to archive the JavaScript we had deployed, but Google Cloud Run provides a ready-made solution. With Cloud Run, we’re running Docker images archived in Google Artifact Registry. It’s trivial to switch back to an earlier Docker image, and we can control which Docker image is running with Terraform.

2. Firebase Functions lack certainty of runtime environment.

Exactly which version of NodeJs does a Firebase Function run in? What happens when Google decides to stop supporting NodeJs version Q and we’re still running it? What happens when they upgrade from version Q.01 to version Q.02?

We no longer have to worry about the answers to these questions because with Cloud Run, we choose and own the runtime environment inside the Docker image. Google can’t push us to the next version if we don’t want to move to it.

3. With Cloud Run, we can use any programming language.

Firebase Functions can only be written in NodeJs or Python. Cloud Run services can be written in any programming language, and invoke any binary that fits into a Docker image.

Here are some reasons to choose a stack besides NodeJs or Python:

Your entire team are Ruby experts.
Performance is critical, so you need a faster programming language.
You need to manipulate images with ImageMagick.

In practice, all our functions are still running the original NodeJs code, but it’s nice to have these options if we need them.

4. With Cloud Run, there is less for attackers to exploit.

Firebase Functions’ service account has over 8000 privileges by default. Every unnecessary privilege granted to a service account gives an attacker more power if they find an exploit in your service.

Firebase Functions run as the App Engine default service account, and we found no way to change that. We could try removing privileges from that account one by one, but would other parts of our service break? I’m not sure.

Good security practice recommends starting from zero privileges and only adding them as necessary. With Cloud Run, we created a service account with zero privileges and added fewer than 10 privileges it needed to run correctly.

5. Migrating to Cloud Run wasn’t hard.

The Firebase client library invokes Firebase Functions via a simple HTTP protocol that took about 30 minutes to reverse engineer using only a web browser. It wasn’t hard to update the code running in Firebase Functions to run in a Cloud Run Service and speak the same protocol. It was even easier to update the client code to connect to a new host.

If you’d like to see a post about how the Firebase Functions protocol works and how to implement it, please leave a comment below.

Which web server starts up the fastest? Java, Python, C#, NodeJS, or Go?

Jeffrey Rennie — Fri, 10 May 2024 16:46:59 GMT

An experiment measures how long 5 programming languages take to start up a web server.

When your website is too slow, users leave. Maybe the home page takes too long to load. Maybe the effects of a button click take too long to see. Either way, a three second delay may trigger users to abandon your website find another, and your website makes less money.

Cold starts are one source of delay for web applications. A cold start for a web server refers to the process of initializing and launching the server from a state where it has not been running or is starting up from scratch. This typically involves tasks such as compiling source code or virtual machine code, establishing connections to databases or other services, and preparing to handle incoming requests from clients. During a cold start, the server may experience delays as it initializes, configures, and prepares itself to handle incoming traffic, which can impact response times for the first few requests.

Cold starts are uncommon when web applications run on persistent (virtual) machines like in Google Compute Engine, AWS EC2, or Azure Virtual Machines. With these platforms, cold starts happen only when a new version of the web application is deployed, or a machine is replaced or added to the pool of web servers. Otherwise, the web application may run for weeks without a restart, so very few users will experience a delay due to a cold start.

Cold start performance has grown more important with the adoption of serverless platforms like Google Cloud Run, AWS AppRunner, and Azure Container Apps. These platforms create and teardown instances of your web application in response to fluctuations in web traffic. The benefit is that you don’t pay for excess virtual machines when your website’s traffic is low. The drawback is many more users will experience delays due to cold starts, because the platforms frequently launch new instances of your web application when traffic increases.

Therefore, understanding how a programming language impacts your web application’s cold start time is more important than ever. It should never be the sole reason for choosing a programming language, but it’s essential data for making an informed decision.

We ran an experiment to measure cold start times of five different programming languages when run inside Docker containers on Google Cloud Run. All the source code is available here.

Here are the measured response times for fetching a simple “Hello world” page from web servers built with the 5 languages. All times are in milliseconds.

https://medium.com/media/b12be233bb3fb8d1b75fb70fe2ffae9b/href

Go is the clear winner. Go’s response time at the 90th percentile is faster than all the other language’s 25th percentile. In this experiment, Go is more than 10 times faster than Java.

Surprisingly, the size of the docker image doesn’t seem to correlate with the cold start time. The two fastest languages, Go and NodeJS, have the smallest and largest docker images.

However, a “Hello world” application is of limited predictive value, because real web applications are much more complicated.

For the second experiment, we built more realistic web pages. The new web page runs a database query, and renders the result in the response. In this case, we used Firestore for the database. Running the database query entails loading the libraries to talk to the database, opening a connection to the database, authenticating, relaying the query, parsing the result, and rendering the result. This is much more similar to a real web application. Here are the results:

https://medium.com/media/3866f3b0472e5f7449070a423735a0ce/href

Compared to the “Hello world” example, response times grew slower by about 50%, except for Go. Python and C# swapped positions, but otherwise the rankings stayed the same.

All the raw data is recorded in this sheet.

It’s worth repeating: performance in general and cold start times in particular should never be the sole reason to choose a programming language when building a web application. However, ignoring performance also comes with the risk of slow websites that turn users away.

I hope this post helps you make an informed decision when picking a programming language for you next web application.

Special thanks to Taiga Narusawa who wrote the code and collected the data for this experiment.

Why I Avoid Using Hash Tables

Jeffrey Rennie — Tue, 05 Mar 2024 23:56:05 GMT

Most of the programming languages I have used provide two dictionary types: one implemented with a hash table, and another implemented with a search tree. Here are some I’ve encountered:

https://medium.com/media/8b8f44fb84f04d4dfdd962bb55959253/href

It’s well established that hash tables are faster, and all the evidence I’ve seen agrees. So you may be surprised to learn that when I write code, I avoid hash tables. Instead, I use search trees and I only switch to hash tables if I have evidence (a profiler’s report) that switching is necessary. My reasons follow.

Hash tables summon chaos into your program.

Have you ever tried debugging an issue that happens 1 out of 100 tries? Or only happens in production, but not test environments? Or only happens on one machine that you don’t have access to? If you haven’t, then you must be new here; welcome to software engineering.

These rarely occurring issues are difficult and frustrating to track down and debug. They’re the result of non-deterministic behavior, which is a polite word for chaos.

There are lots of sources of chaos. Some examples are the computer’s clock, input and output, memory allocation patterns, and hash tables.

How do hash tables summon chaos? Consider the following code which populates two hash tables with the same Products, and then tests the hash tables for equality:

record struct Product(int ProductId, string VendorName);

var a = new Dictionary {
    { new Product { ProductId = 1, VendorName = "Contoso" }, 100m },
    { new Product { ProductId = 2, VendorName = "Fabrikam" }, 150m }
};
var b = new Dictionary {
    { new Product { ProductId = 2, VendorName = "Fabrikam" }, 150m },
    { new Product { ProductId = 1, VendorName = "Contoso" }, 100m }
};

Console.WriteLine(".Equals() {0}", a.Equals(b));
Console.WriteLine(".SequenceEquals() {0}", a.SequenceEqual(b));

What will this code print?

The correct answers are Line 1: False and Line 2: impossible to know.

.Equals() always returns False because C# is C#. It’s not what I hoped for, but at least it’s consistent.

.SequenceEquals() enumerates all the items in the dictionaries and compares each element. They may be enumerated in the same order, or they may not be. When they’re enumerated in the same order, the code will print True. When they’re not, the code will print False. The docs say:

For purposes of enumeration, each item in the dictionary is treated as a KeyValuePair structure representing a value and its key. The order in which the items are returned is undefined.

Undefined behavior is chaos. I don’t want chaos in my software. The better I can predict how my code will behave, the less time I’ll spend debugging it.

In practice, how does this chaos cause harm? Imagine I write a function and a test. The function dumps a hash table to a CSV file, and the test confirms the resulting CSV file looks as expected.

static void WritePricesToCsv(Dictionary prices,
  string filename)
{
    using (var writer = new StreamWriter(filename))
    {
        foreach (var price in prices)
        {
            writer.WriteLine("{0},{1},{2}", price.Key.ProductId,
              price.Key.VendorName, price.Value);
        }
    }
}

[Test]
public void TestWritePricesToCsv()
{
    // Arrange
    var prices = new Dictionary
    {
        { new Product { ProductId = 1, VendorName = "Vendor1" }, 100m },
        { new Product { ProductId = 2, VendorName = "Vendor2" }, 200m }
    };
    var filename = "test.csv";
    var expectedContent = "1,Vendor1,100\r\n2,Vendor2,200\r\n";

    // Act
    WritePricesToCsv(prices, filename);

    // Assert
    var actualContent = File.ReadAllText(filename);
    Assert.AreEqual(expectedContent, actualContent);

    // Cleanup
    File.Delete(filename);
}

I merge the code and the test passes for the next 10 years. Then I leave the project, some poor sap takes ownership of my code and the test starts failing because the foreach loop spontaneously starts listing the products in reverse order. It’s 100% allowed to do that because the documentation says the order of enumeration is undefined. To make matters worse, the test happens to fail only when the test runs on the CI/CD machine. It always passes on the poor sap’s machine. It’ll probably take the poor sap a day to find the issue. What a frustrating waste of time.

So sometimes some tests fail. That’s not the end of the world. But it can get much worse. The next example is a lesson I learned the hard way.

Imagine I’m building a microservice, and my microservice calls a backend Price Server periodically to fetch the latest prices.

The code looks something like:

void RefreshPrices(Dictionary prices) {
    var request = new FetchPricesRequest();
    foreach (var price in prices) {
        request.Add(price.Key);
    }
    var response = FetchPrices(request);
    ...

Now imagine there’s a million instances of my microservice all fetching requests from the same Price Server. Because the load is so heavy, I add a caching proxy between my microservice and the Price Server.

The proxy serves 99% of the requests from its cache. If all the requests missed the cache and hit the Price Server directly, they’d overwhelm it.

Everything is running smoothly in production. Then, one day, for no apparent reason, the caching proxy which was serving 99% of requests from its cache instead starts serving 2% of requests from its cache and forwarding the other 98% of requests to the Price Server. The Price Server is overwhelmed and crashes.

Why did the caching proxy suddenly fail to find requests in its cache? Because even though all the FetchPricesRequests contained the same Products, they contained them in different orders. To the proxy, requests with the Products in different orders are different requests. A request with products [a, b] can’t be served from a cache entry with products [b, a]. The proxy server must forward the request to the Prices Server.

Had I used a SortedDictionary (search tree) instead of Dictionary (hash table), the failures above would never have happened. Requests with the same lists of products would always carry those products in the same order, and therefore be served from the cache.

Hash tables are vulnerable to denial of service (DoS) attacks.

One type of DoS attack on hash tables is known as a “collision attack.”

Hash tables use a hash function to map keys to indices in an array (buckets). Ideally, each unique key should map to a unique index, but due to the finite size of the array and the practically infinite number of possible keys, collisions happen.

A collision happens when two or more keys are hashed to the same index. The hash table in the diagram above has a single collision at index 4 between the items C and D.

In the worst case scenario, all the keys hash to the same index. The performance of a hash table look up degrades from O(1) constant time to O(n) linear time. The hash table performs no better than a linear search in an array. The diagram below shows a hash table in this bad state.

An attacker attempting a DoS attack tries to get my hash table into this worst case scenario.

Imagine my web service allows customers to define new products. They provide the product name, and I keep the product information in a hash table with the product name as the key. An attacker can contrive a list of one million product names that all yield the same index when passed to my hash function. It will take some guess work for the attacker to figure out which hashing algorithm I use for the string. Lucky for the attacker, I used the default string hash function built into C#, which the attacker figures out very quickly. The attacker then begins adding one million products with colliding names to my web service. This causes my web service to burn 100% CPU doing linear searches and insertions into the hash table. My web service is too busy to serve my other customers’ requests. The DoS attack succeeds.

Had I chosen a search tree instead of a hash table, this attack would have had minimal effect on my service. The performance of a look up in a search tree never exceeds O(log(n)). My web service would have continued to serve other customers while the attacker defined a million new products.

Hash tables leak memory.

Hash tables leak memory when you remove items from them. To demonstrate, I wrote some code that fills a hash table with 100,000 items, then clears the table so it contains 0 items, and prints the size of heap memory after each step.

void FillPrices(IDictionary prices)
{
    // Print the initial heap size.
    var startingMemory = GC.GetTotalMemory(true) / 1024;
    Console.WriteLine("initial memory: {0}", startingMemory);
    var now = DateTime.Now;
    
    // Fill the prices dictionary with 100,000 products.
    for (int i = 0; i < 100000; i++)
    {
        prices.Add(new Product { ProductId = i, VendorName = RandomName() }, i);
    }

    // Print the heap size after filling the dictionary.
    var memory = GC.GetTotalMemory(true) / 1024;
    Console.WriteLine("memory for {0} items: +{1}", prices.Count,
        memory - startingMemory);

    // Clear the dictionary and print the heap size.
    prices.Clear();
    var elapsed = DateTime.Now - now;
    memory = GC.GetTotalMemory(true) / 1024;
    Console.WriteLine("memory for {0} items: +{1}",
        prices.Count, memory - startingMemory);

    // Refill it one more time and print the heap size.
    for (int i = 0; i < 100000; i++)
    {
        prices.Add(new Product { ProductId = i, VendorName = RandomName() }, i);
    }
    memory = GC.GetTotalMemory(true) / 1024;
    Console.WriteLine("memory for {0} items: +{1}", prices.Count,
        memory - startingMemory);

    // Print the time it took to fill and clear the dictionary.
    Console.WriteLine("elapsed: {0}", elapsed);
}

I ran the code and here’s the output:

initial memory: 73
memory for 100000 items: +10658
memory for 0 items: +6752
memory for 100000 items: +10659
elapsed: 00:00:00.1015928

I’m not surprised to see the hash table consumes 10 MB when it contains 100,000 items. I am surprised to see the hash table still consuming 6 MB when it contains 0 items. To be fair, the memory isn’t lost forever. It does get reused when more items are added to the Dictionary. But will my application actually refill the dictionary or just waste space with an empty dictionary? Answering that question requires a thorough investigation of all the code paths that touch the dictionary, and I’m lazy and I don’t want to do that.

Compare this to performing the same operations on a SortedDictionary (search tree) instead of a Dictionary:

initial memory: 10732
memory for 100000 items: +10937
memory for 0 items: +0
memory for 100000 items: +10937
elapsed: 00:00:00.2382486

After clearing the SortedDictionary, the garbage collector was able to reclaim all memory. I frequently work on web assembly applications where memory is constrained, so SortedDictionary’s tidy memory habits are much more attractive to me.

Why do hash tables that contain 0 items consume so much memory? Earlier I described how hash tables use a hash function to map keys to indices in an array (buckets). As items are added to the hash table, buckets are added and the array grows. As items are removed from the hash table, buckets are dropped but the array never shrinks. In theory, the hash table could shrink the array, but in practice, I haven’t seen that happen. 100 Go Mistakes and How to Avoid Them specifically calls out this issue as #28. Have you seen a hash table actually shrink its array? Please leave a comment and tell me where.

But hash tables are faster!

Yes, they are, and the elapsed lines in the print outs above support that claim.

In the applications I’ve worked on, the slightly slower performance of a search tree has never been visible in a profiler. Something else, a compression algorithm for example, has always been burning up so much CPU that the performance of the search tree had no impact. Your mileage may vary.

Conclusion

If I’m trying to win a contest on leetcode.com, I’ll choose a hash table because it’s faster. But if I’m writing code in any other context, I’ll choose a search tree so I can avoid the problems discussed above. A search tree always enumerates its items in sorted order, isn’t vulnerable to DoS attacks, and cleans up memory when items are removed.

There are ways to mitigate these problems when using a hash table, but those mitigations require diligence. I have a limited supply of diligence in a day, and I’d rather spend my diligence solving my customers’ problems than spend it solving problems created by my chosen data structure.

I hope this article helps you make more informed decisions when choosing between hash tables and search trees. If you think it will, please click the clap icon below.

The source code I used to measure memory and performance is available on GitHub.

Convert an ASP.NET Website into a SPA Using HTMX

Jeffrey Rennie — Sat, 16 Sep 2023 20:11:55 GMT

Turn your ASP.NET MVC/Razor Website into a SPA Without Blazor

Should you rewrite your ASP.NET MVC/Razor website with Blazor? No. Blazor solves one problem and introduces three more. Blazor punishes your users with large, slow web-assembly packages and punishes users who have JavaScript disabled because your website becomes unusable. Blazor punishes you by requiring you to squander precious time rewriting your website and by making it much harder to debug production issues, because blazor websites rely on websockets or web assembly.

Instead of Blazor, save yourself time, money, and stress by using HTMX to transform your website into single-page application (SPA). With HTMX, you and your users will suffer none of the punishments of Blazor listed above.

In this article, I’ll demonstrate how to use HTMX to transform a legacy ASP.NET website into an interactive Single-Page Application by changing 10 lines of code, none of them JavaScript. I’ll describe the advantages of HTMX along the way.

This technique works for ASP.NET websites that follow the standard _Layout.cshtml file pattern, which looks something like this:




    
    @ViewData["Title"] - Web Application
    


    
    
        
            @RenderBody()

Sample Application

To generate a sample ASP.NET website for this demonstration, I randotnet new webappwith the fictional business name Wayoprise. I used ASP.NET 6, but this technique works with older and newer versions of ASP.NET too. The source code is on GitHub and the website is running on Google Cloud Run.

When I visited the Wayoprise home page and clicked Privacy, the browser briefly blanked the screen and then loaded the Privacy page. It looked mostly the same as the previous Home page. The only thing that changed was about 100 bytes inside the

tag. However, I saw this in the browser’s network trace:

This is a lot of wasted effort for a 100-byte change. For starters, 2.63kB was transferred, more than 10 times the size of the actual change. The browser didn’t download the JavaScript files because they were cached, but it still had to re-evaluate them in the context in new of the new page. This wasted time and energy on the server, and time and energy in the browser.

My goal was to avoid all the wasted effort. When I clicked the Privacy link, I wanted to see the following network trace:

Ideally, I wanted the bytes transferred to be only the new content inside the

tag:

Privacy Policy

Use this page to detail your site's privacy policy.

A Brief Introduction to HTMX

HTMX is a lightweight (44k, 14k gzipped) JavaScript library that simplifies web development by enabling server-side features to be added to HTML elements with minimal code. It allows you to enhance traditional server-rendered web applications by adding client-side interactivity without the need for complex JavaScript frameworks. HTMX can progressively enhance existing HTML by specifying behavior via attributes, such as hx-get, hx-post, or hx-trigger, to fetch or send data to the server, update the DOM, and trigger events.

Adding HTMX to the Sample Application

To add HTMX to the application, I downloaded the HTMX Javascript library with the following command:

curl -L "https://unpkg.com/htmx.org@1.9.5" > wwwroot/lib/htmx.js

And I added one line of code to _Layout.cshtml’s scripts section:

And then HTMX was fully installed into my ASP.NET project. The changes were recorded in this git diff.

Updating the Sample Application to Use HTMX

With HTMX installed, it was time to put it to work.

First, I wanted to tell HTMX, “when a user clicks a link in the body, don’t fetch a whole page. Instead, fetch a fragment of HTML and replace the contents of the

tag with the response.” Here’s how I did that.

I edited _Layout.cshtml and added hx-boost and hx-target to the body tag:

The HTMX website explains hx-boost:

The hx-boost attribute allows you to “boost” normal anchors and form tags to use AJAX instead. This has the nice fallback that, if the user does not have javascript enabled, the site will continue to work.

For anchor tags, clicking on the anchor will issue a GET request to the url specified in the href and will push the url so that a history entry is created.

In the code sample above, hx-boost=true tells HTMX to boost all the links in the body. When a user clicks a boosted link, HTMX uses AJAX to fetch the URL. hx-target’s argument is any CSS selector. hx-target=main tells HTMX to replace the

tag with the response. Effectively, I told HTMX to replace the content returned by RenderBody() with the content returned by a new call to RenderBody().

Although I had told HTMX which element to fetch and replace, I hadn’t updated the application to return a fragment of HTML code. A fetch to /Privacy would still return a whole page.

So, I updated the top of _Layout.cshtml to return a fragment of HTML when the request was coming from an HTMX boosted link:

@if (ViewContext.HttpContext.Request.Headers["HX-Boosted"].Contains("true")) {
    @RenderBody()
} else {

...

HTMX adds an HX-Boosted: true header to its fetches. That made it easy to detect whether a request was coming from an HTMX boosted link or a regular link. With this code change, _Layout.cshtml responded with only the fragment of HTML returned byRenderBody()for HTMX-boosted links.

There was still one potential pitfall lurking in this code. A browser could cache the response of a request with HX-Boosted: true, and serve it from the cache for a request lacking HX-Boosted: true. That would result in a broken page. To tell the browser not to make that mistake, I added one more line of code:

 @{ ViewContext.HttpContext.Response.Headers.Add("Vary", "HX-Boosted"); }

All these changes were recorded in this git diff.

I compiled and ran the code, and the network trace looked exactly like what I wanted:

No JavaScript or CSS was refetched or retrieved from the cache. Only the new HTML fragment was transferred.

However, I found a bug. After I clicked the privacy link, the url in the browser’s address bar was updated with the /Privacy path, but the title in the browser tab still said “Home page — Wayoprise” when it should have been updated to say “Privacy Policy — Wayoprise.”

Fortunately, HTMX provides a way to update the title at the same time it updates the body, by using the hx-swap-oob attribute. The HTMX website describes hx-swap-oob:

The hx-swap-oob attribute allows you to specify that some content in a response should be swapped into the DOM somewhere other than the target, that is “Out of Band”. This allows you to piggy back updates to other element updates on a response.

To update the title, I inserted a new tag into the fragment of code returned for HTMX boosted links:</p><pre>@if (ViewContext.HttpContext.Request.Headers["HX-Boosted"].Contains("true")) {<br> <title hx-swap-oob=title>@ViewData["Title"] - Wayoprise
@RenderBody()
} else {
...

All these changes were recorded in this git diff.

I recompiled and ran the website, and the HTMX-boosted web site rendered exactly like the original website. The response to a request for /Privacy looked like this:

Privacy Policy

Use this page to detail your site's privacy policy.

The final result is running in Google Cloud.

Success!

I converted this ASP.NET web site into a Single-Page application without writing a single line of JavaScript! Users now enjoy a faster, seamless website on every device, I’ve reduced my web hosting bills because my web servers handle fewer requests and transfer fewer bytes, and I have a code base that’s easier to understand than one built with Blazor or a JavaScript front-end framework. The source code for the sample is posted to GitHub.

HTMX is very powerful, and this article only began to explore the ways it can be applied to ASP.NET applications. Please clap if you found this article useful. Please clap twice if you’d like to see more examples of integrating HTMX with ASP.NET.

Other Reasons Why Rust Is My Favorite Programming Language

Jeffrey Rennie — Thu, 24 Jun 2021 18:52:10 GMT

There’s a lot of hype surrounding Rust now. Some features dominate the headlines:

Rust is as fast as C or C++.
Rust is memory safe.

Rust programmers also enjoy great documentation and a great package manager, which no doubt have contributed to Rust’s growing popularity.

There are a few more features of the Rust language that I thoroughly enjoy, but I haven’t seen discussed much. I’ve been coding in Rust for about a year, and in that time it has become my favorite language among the dozens I know.

Here are three less-talked-about language features that make programming in Rust a joy:

There’s no null.
Powerful macros.
Precise control over mutability.

There’s also a bonus feature I’m saving for the end, that only makes sense after explaining the other features.

The Rust cheat sheet quickly explains some of Rust’s unique syntax in the code samples that follow.

Let’s dive deeper into each feature.

There’s no null

Null is probably the single greatest mistake in programming language design. Tony Hoare called it his “billion-dollar mistake.”

Have you worked on a software project that never threw a NullPointerException or dereferenced null and crashed? If your project was written in Rust, Swift, Elm, or Haskell, then maybe you have. Otherwise, the chances are slim to null (pun intended).

Dereferencing null will bring a program to a screeching halt. To cope with the constant danger and avoid crashing their software, programmers write code like this:

https://medium.com/media/e11074565af20e5efeadc8c410700b60/href

This code is so dominated by null checks that it’s hard to understand what the function actually does. The poor programmer wrote more lines of code to cope with the possibility that an argument may be null than they wrote lines of code to solve the problem they wanted to solve. What’s more, programmers have read this null checking code so many times that they’ve trained themselves to ignore it. Did you see the bug I introduced in the code above?

Because Rust doesn’t have null, the sample code above could be written in Rust something like this:

https://medium.com/media/a8b802dd02dea2c673512ae369157b6f/href

The Rust code doesn’t check for nulls because neither argument can be null. Any attempt to pass null would be caught by the compiler and reported as a type mismatch. Now that I know Rust, I cringe when I see null checking in other languages.

But of course, null can be very useful. Sometimes it’s convenient for classes or structures to optionally hold values, and for functions to optionally return values. For that purpose, Rust provides the Option type. An instance of Option contains either None or Some(T) but never both.

Let’s consider a structure that represents a url like “http://www.google.com/”. Urls optionally contain a port number like this: “http://www.google.com:80/”. In Rust, we may represent a parsed Url with the following struct:

https://medium.com/media/67993576f1cf634aa15b90a87bb0a363/href

Imagine we want to write a method called incr_port() which increments the port number in a url; we’ll invoke the method like this url.incr_port();.

Naively, we might write the method like this:

https://medium.com/media/2a075b9e9b69307cf0516a2a2df44317/href

The Rust compiler rejects incr_port() above because port is not a u16; it’s an Option. The compiler prevents what would have been a latent null pointer exception in other languages.

14 |      self.port += 1;
   |      ---------^^^^^
   |      |
   |      cannot use `+=` on type `std::option::Option`

To increment the port, the code must first confirm that port has some value.

https://medium.com/media/1c810af2d303f2ab4f0a26ead237fd85/href

With Option, Rust provides optional values when we need them, without the possibility of crashing the program. More importantly, Rust saves the programmer from the chore of null checking every argument and object.

Powerful macros

Let’s continue with the Url struct above. I’ll add one line of code before the Url struct, and then discuss all the good things this one line does:

https://medium.com/media/1220047433728f045ca119e82b8d9dff/href

By adding this one line of code,

#[derive(Eq, PartialEq, Ord, PartialOrd, Clone, Hash, Default, Debug)]

I can now compare two Urls for equality like this:

url == url2

I can compare them for ordering too. The PartialOrd & Ord macros compare members of the struct in the order in which they appear in the struct.

url > url2

Which means I can put Urls into sorted containers:

let mut sorted_urls = BTreeSet::new();
sorted_urls.insert(url);

I can clone a Url to get an exact copy:

let url2 = url.clone();

I can insert Urls into hash containers:

let mut urls = HashSet::new();
urls.insert(url4);

I can create a default Url, kind of like invoking a parameterless constructor in other languages:

let url = Url::default();

And, I can print a pretty debug string for a Url:

println!(“{:?}”, url);

Output: Url { protocol: “http”, host_name: “www.google.com", port: Some(80), path: “” }

That’s a ton of functionality to pack into one line code! In other languages, there are ways to implement the same behavior: code generators, post processors, IDEs, etc. Some languages use introspection. However, I like Rust’s solution best because no additional tools are needed, there’s no additional code for me or my peers to review and maintain, and type errors will be caught at compile time.

These macros all behave like you expect. For example Ord, short for ordering or ordered, compares each member of the struct in order. This may trip people up while refactoring code, because changing the order of the members of the struct will change the sort order. To avoid that, I could implement std::cmp::PartialOrd and tailor it to my needs.

Third-party libraries can also take advantage of the #[derive()] macro. The serde library is Rust’s de facto standard for serializing/deserializing structs into/from more than a dozen formats. Let’s add some code to allow a Url to be serialized to and deserialized from json. After importing the serde library, I add two more attributes to the #[derive()] macro:

#[derive(…, Serialize, Deserialize)]

And now I can serialize and deserialize Urls from json.

let text = serde_json::to_string(&url)?;
let url2: Url = serde_json::from_str(&text)?;

Parse errors are immediately returned to the calling function via the ? operator.

The #[derive()] macro added a ton of functionality to my Url struct by adding only two symbols to my code! Had Url contained a type that could not be serialized to json, the compiler would have reported an error.

With great power comes great responsibility. If you maintained a C or C++ code base in the 1990s, then you probably saw people do terrible things with macros. The C/C++ preprocessor’s macros can replace any symbol with any text. People used that power to create code that looked nothing like C, and was very hard to understand and maintain. Here’s a real example from the era:

https://medium.com/media/f88c751484cac1c251a7025db0cc235f/href

This code actually defines a method, and builds an if-else ladder in the body, but you’d never know unless you dissected the macros. Examining this code in the debugger is also confusing. Here’s what BEGIN_MESSAGE_MAP() looks like:

https://medium.com/media/c4568960d8db81510332efbba7eaa483/href

Unsurprisingly, there was a backlash against macros for creating this mess. The languages born from this experience intentionally lacked macros: namely Java and Go.

But I think those languages threw the baby out with the bath water. The issue wasn’t macros in general; the issue was the C/C++ preprocessor. After using dozens of libraries in Rust, I have yet to see anything as mysterious as BEGIN_MESSAGE_MAP(). In fact, I’ve only used macros from two libraries: serde and anyhow.

I see a few reasons why programmers have been more disciplined with macros in Rust than in other languages:

Rust provides const functions, which can replace macros in many cases. For example, you could write a const sqrt() function to calculate the square root of a constant at compile time.
With Rust macros, you can’t super-globally replace one symbol with any text you want. For example, this is impossible: #define PLUS +
In Rust, macros must expand to complete expressions, statements, items, types, or patterns. They can’t expand to half a function. They can’t have unbalanced parenthesis, brackets or braces. The BEGIN_MESSAGE_MAP() macro above would be impossible in Rust because it has unbalanced braces.
Defining a macro in Rust is much more complicated than with the C/C++ preprocessor. Learning how to write a macro in Rust is like learning a whole new programming language. You can’t just #define max(a, b) ((a) < (b) ? (b) : (a)). If something can be expressed any way besides a macro, Rust programmers like to choose the other way.

Precise Control over Mutability

Rust gives the programmer precise control over mutability. Mutable data can be modified. Immutable data cannot be modified.

Immutable data is great for everyone who has to read code, be they human or machine. It’s far easier to reason about immutable data, it’s easier for the compiler to optimize code that uses immutable data, and immutable data is always thread safe.

Consider this sample code that uses the Url sample from above and demonstrates why immutable data makes code easier to reason about.

https://medium.com/media/00e25d7f508915b7a970ec018d93d241/href

Because url is passed to fetch() via an immutable reference, it’s impossible for fetch() to modify the contents of url¹. Therefore, it’s easy to predict the value of url through each iteration of the loop. I don’t have to inspect fetch()’s code or even look at its argument types to confirm that it doesn’t modify url, because I can see the argument is passed as an immutable reference: &url. To enable fetch() to modify url, I’d have to change the calling code to pass url as a mutable reference, like this: fetch(&mut url).

At the same time, I don’t have to make url immutable in the context of the whole loop to make it immutable during the call to fetch(). I still can still modify url by calling url.incr_port() within the loop.

Passing url as an immutable reference also provides guarantees to fetch(). It’s not only impossible for fetch() to modify url, it’s impossible for anything else to modify url while fetch() has an immutable reference to url. No other thread can modify url while fetch() is executing. That makes writing fetch() easier.

I say that Rust gives the programmer precise control over mutability, because Rust doesn’t entangle mutability with other concepts. The programmer can choose for data to be mutable or immutable regardless of whether the data is allocated on the stack or on the heap, regardless of whether it’s passed by reference or by value, and regardless of whether it’s a struct or a primitive type like an integer.

Without precise control over mutability like Rust, other languages have introduced a variety of patterns and language features to provide some form of immutability. Some examples are the builder pattern, read-only interfaces, and multiple aggregate types each with their own mutability rules like class vs struct vs record. Using these other solutions requires evaluating trade offs or writing more code than would be required in Rust.

Bonus Feature: Rust is Expressive

Rust has been described as an expressive language; expressive is a very abstract term. What is expressiveness and why is it useful?

In an expressive language, it’s possible to introduce new ideas that the language designers never dreamed of without changing the syntax of the language and without boilerplate code. The fact that many concepts were incorporated purely via a library rather than syntax is evidence that Rust is expressive.

For example, the Option type is not built into the language; it’s just a standard library. Other languages required dedicated syntax to encode optional values, but with Rust, the existing language features allowed programmers to express optional values efficiently with Option; no changes to the language were necessary.

Another example of Rust implementing a concept with a standard library rather than syntax is Rust’s From and Into traits, which allow programmers to write custom conversions from one type to another. Other languages added operator syntax to give programmers the ability to define custom type conversions. With Rust, the existing language features were sufficient. From and Into are ordinary traits.

This all sounds great for language designers, but how could it be practical for the rest of us?

Consider this function:

https://medium.com/media/a5da6d0cf8b33150678fad4d52389d77/href

The io::Result<()> return type can contain exactly one of the following: an Error value, or the empty value () signifying success.

Imagine implementing this function. To make sure we always reset the file pointer to its original position, it sure would be convenient to have something like try {} finally {} found in other programming languages. Then, we could simply call f.seek(original_offset) in the finally block. But Rust doesn’t have exceptions, and it doesn’t have try {} finally {}.

To cope with the absence of try {} finally {}, I wrote a macro that behaves the same way. First, some disclaimers:

This is the first macro I’ve ever written, so it surely can be improved.
This is the first macro I’ve ever written, because I haven’t had the need to write a macro.
This macro is for demonstration purposes only. When I actually had the same need in production code many months ago, I wrote plain Rust code.
Because Rust has a built-in macro called try!(), I had to name my macro something different, so I named it tryf!().

Enough disclaimers. Here’s what the function implementation looks like with my tryf!() macro:

https://medium.com/media/d6fd67a00b2bf3a707950bd44f9546e1/href

The syntax is a bit different, but if you’re familiar with languages with exceptions, I hope it’s obvious what this code does. That’s the beauty of an expressive language. I can take ideas that are foreign to the language itself and express them efficiently in Rust.

Because Rust is expressive, I’m confident that I’ll be able to efficiently express a great variety of my own ideas in code, with mistakes caught at compile time, and without boilerplate code.

Conclusion

Of course, I also like Rust for the same reasons as so many others: similar speed to C/C++, memory safe, great docs, great package manager, runs in the browser via WebAssembly, etc. But I thought Rust also had some hidden gems, so I wrote this post.

If you’re considering adopting Rust for work or hobby, I hope this post has given you some useful insights to help you make a more informed decision. To begin learning Rust, I recommend starting with the book Rust by Example.

Acknowledgements

Thank you to the following people who no doubt have different opinions about their favorite languages, but generously reviewed drafts of this post.

Foot Notes

[1] Sometimes, you may need to add a reference count or the like to an otherwise immutable struct. For situations like these, Rust provides Cell, Rc, and more.

Other Reasons Why Rust Is My Favorite Programming Language was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.