Whisper.cpp is an ML model which interprets spoken words and outputs written words and can be run on a Mac laptop. Simon Willison frequently refers to his use of MacWhisper on his blog, so I thought I’d try my hand at a command-line approach to transcribing a YouTube video.

Whisper.cpp is easy to setup but there’s a learning curve. You need:

1. ggml-base.en.bin placed in models/ subdirectory
2. ggml-metal.metal which is a C++ file
3. Your audio in WAV format at 16kHz

$ # Step 1, setup Whisper.cpp
$ brew install whisper-cpp # or install for your machine type
$ mkdir models
$ wget -O models/ggml-base.en.bin https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.en.bin?download=true
$ wget -O ggml-metal.metal https://github.com/ggerganov/llama.cpp/raw/master/ggml-metal.metal

Now that Whisper.cpp is ready to run, you need to get your input material. In my case, I got a webm video which had an Opus audio stream. This required some careful but simple conversion:

$ # Step 2, get your input
$ youtube-dl --extract-audio --audio-format best [yt url]
$ ffmpeg -i file_from_youtube_dl_execution.opus -ar 16000 -vn prepared_input.wav

Now you have your input and Whisper.cpp is setup, you can generate your transcript. You can run it just like whisper-cpp prepared_input.wav and copy the output from stdout, or you can pass dictate how it’s output: txt, csv, srt, vtt, lrc, and json.

$ # Step 3, extract your transcript
$ whisper-cpp --output-txt prepared_input.wav
$ less prepared_input.wav.txt # read your transcript!

You can supply as many --output-<fmt> flags as you would like and it will output all formats you request. Pretty neat!

Some of it isn’t though. What I have favorited (or starred, or recommended) is not lost to time. With the release of playlists in Overcast, I could see all my starred episodes. The data was somewhere! I emailed Marco, its creator and developer, to ask if he could offer some kind of export mechanism for this from the production app. I also signed up for the TestFlight and submitted a feature request through the beta app, but I didn’t hear back.

I ruminated on this topic for a few months until a few nights ago. I wanted to look up a starred episode, but got frustrated scrolling through the UI. I realized that it would be more efficient to scroll on my Mac, so I opened Overcast on the Mac. The idea struck me instantly: Marco has mentioned before that he uses sqlite. Surely I could find the database file on my Mac?

After some hunting, I found it: oc.db in ~/Library/Containers/[App UUID]/Data/Documents. I held my breath and opened it with sqlite3. It worked! Now, how do I find the starred data? Turns out that Marco has indicated this with a field called OCFeedItem.userRecommendedTime. When a user stars an episode, the time is taken and that’s written to their database and to the servers and used for Overcast’s recommendation engine.

I wrote some SQL to pull out the episodes and piped it through Simon Willison’s excellent sqlite-utils:

$ sqlite-utils  ~/Library/Containers/.../Data/Documents/oc.db "$(<~/Downloads/overcast_export_recommended_episodes.sql)"

:tada: I finally had the data I was looking for. A full list of every episode that I have ever starred as beautiful JSON, going back to 2017.

I decided to publish my favorites since the reason to star them is to remember them and to share them with the world. You can view the code if you want to emulate this. If you use my script, consider joining me in subscribing to Overcast Premium so that Marco can continue to offer Overcast on the Mac.

I’m extraordinarily excited to finally have access to my data. With the recent rise of federated social networks and the talk of owning one’s data (or at least a copy!), I am put at ease that if Overcast went out of business tomorrow, I would still have a historical record of my favorite podcast episodes to share with friends and strangers alike.

My thanks to Marco for allowing Overcast to run on the Mac so that I could grab a copy of my data, and for guiding me through generation of the Overcast.fm permalinks

I have 3,855 Instapaper bookmarks collected in the over 10 years since I signed up for the service. That’s a lot of reading! The beauty of Instapaper is that it saves a text version of these webpages to its database, allowing you to preserve the original words of the article before it (usually) disappears from the Internet. Instapaper is also a great learning tool, since it allows you to highlight and notate your articles as well.

Instapaper has changed hands a number of times over the years. Initially written by Marco Arment, it was sold to Betaworks in 2013 and sold later to Pinterest in 2016, then became its own company. As with any tech product, you don’t know if what they offer today will persist into the future. Apps more famous and well-loved than Instapaper have been shut down over the years, so it’s entirely possible that Instapaper could have the same fate someday. What if Instapaper shut down? Where would that leave me? If I have the full list of all my bookmarks, highlights, and notes, then it would be fine if Instapaper shut down. So, I set out to see if I could make that happen.

parkr/instapaper-archive is the culmination of this work to date. A simple Go program reads from the Instapaper API and from the instapaper-export.csv file generated from user Settings and generates a Jekyll site. (Since the Instapaper API only allows listing up to 500 articles per folder, it was necessary to combine the CSV export with the API results to get a full picture.) It downloads highlights today (not notes yet). It preserves whether an article was favorited (denoted by a star):

But preserving my data is just the start. What could I learn from my archive? I could do all sorts of interesting analysis with it. I can go back and see what I was interested in, say, 2013 and how that compares to my present interests. I can more easily find that one article that shaped my understanding of this or that topic. Most importantly, I own my own data – if Instapaper folded tomorrow, I would not lose a decade’s work.

If you use instapaper-archive, I’d be interested to hear how your experience goes in an issue on the repo. In the age of closed gardens and locked-up data, it always feels great to truly own your data.

I love this little ScanSnap S1100 for its miniature size and ease of use. Unfortunately it comes with bloated software which doesn’t quite meet the privacy trust threshold for me. Since I like the scanner but don’t like the first-party software, I figured this was a perfect opportunity to see if I could get scanimage to work.

Some Googling later, I found the SANE project, where “SANE” stands for “Scanner Access Now Easy.” As a macOS user, I immediately searched Homebrew, where I found sane-backends, a package which would ostensibly give me the scanimage binary and the backend firmware needed for my scanner.

With my scanner plugged in, I ran scanimage but got the error scanimage: no SANE devices found. Huh? My scanner was definitely plugged in. System Report on the Mac told me so. The sane-backends package comes with a helpful binary called sane-find-scanner, but I got another error there too: could not fetch string descriptor: Pipe error. Not very helpful.

Back to Google, to figure out why the program couldn’t find my scanner. A number of forums referenced these mysterious .nal files. I decided to see if I had this .nal file in the sane-backends installation. Unfortunately I did not. There’s a file called etc/sane.d/epjitsu.conf which referenced a .nal file for the Fujifilm S1100 I had plugged in, but the file it referenced didn’t exist. That’s odd – I would have expected “sane-backends” to include the backends for the scanners.

A bit more Googling and I found a GitHub repository which contained the necessary files and I cloned it into the directory where the epjitsu.conf file was expecting it to be. I re-ran sane-find-scanner and – hooray! – it registered. I ran scanimage and finally got something!

With scanimage successfully working, I decided to modify Julia’s scan.sh script for my needs. I decided to use tiff as the output format and use --batch-prompt so that I could feed the ScanSnap (the default batch behavior expects you have a feeder scanner which pulls in each page one at a time without human intervention). I ran my custom script and – oops! - no convert binary in sight. This binary is provided by the imagemagick package, so back to Homebrew to install that. Once that was sorted, I successfully made my tiff scans and converted them into a PDF.

I chose TIFF for its quality. When scanning a document, I want it to be crisp to promote readability and better quality if it gets printed from these scans. Unfortunately, TIFF files can be massive. A single page was just over 1MB, and somehow convert created a PDF file which was 348 MB large – huge when the input was only 40 1MB files. Luckily Preview on the Mac has a way to fix this. Open the file, then choose “File -> Export…”. Under the “Quartz Filter” drop-down, there’s an option to choose “Black & White” or “Grey Tone”. Choose either and export. (I found that choosing “Reduce File Size” irreparably reduced the quality, YMMV.)

And finally, the saga was complete. I had my 40-page PDF at a reasonable size! I hope this is useful to others who don’t want to install gigabytes of software from their scanner’s vendor but can’t find a way past the default sane-backends install. Clearly you need to BYOD: bring your own driver. Once you have that in place, you’re golden!

Downloading and setting up the runner is fast and easy. You can add them to a single repo or to whole organizations. Unfortunately, there’s not currently a means of setting a self-hosted runner for a user account – users can only add them to individual repos.

DISCLAIMER: Security on self-hosted runners is up to you. Every workflow you execute on your machine has all the access your user does unless you isolate the runner properly. Proceed at your own risk!

Use case 1: Full-text Instapaper archive

Instapaper has been around over 10 years now, and I have been a user for almost all of that time. I have thousands of saved articles with highlights and notes that I don’t want to lose. I developed a Go program to pull down my saved articles and generate a Jekyll site: each article is a post with its content from the Instapaper Full-text API endpoint, and each article has 2 data files, one for metadata like “did I star this?” and one with a list of my highlights. A script builds the Jekyll site, then syncs it to a home server where it is served with nginx. Since the home server is running Tailscale, I can easily go to https://hostname/instapaper-archive to view every article I have ever saved.

All this works well, but it requires that I remember to run this script on my laptop when I have read some articles on Instapaper, and it is closely tied to state on my laptop. Logs are not persisted anywhere. It has all the problems you’d expect with a personal program you’re running by hand. So, how can I automate this? A self-hosted GitHub Actions runner, of course!

GitHub Actions allows you to run workflows on a schedule. Using Cron syntax, you can schedule a workflow to run as often as every 5 minutes. The beauty of a self-hosted runner is that I control the environment: I control the network, I control the disk, and I control the workflow. (There’s a way to fix the network bit by using tailscale/github-action to setup an ephemeral node, but why eat into my Actions minutes when I can use an idle computer in my home instead?)

I setup the workflow to run the archiver, run jekyll, then rsync the resulting site to my home server for later browsing. Since the archiver works incrementally, I setup a directory outside the working directory that I could write to. This allows me to more easily maintain state between runs.

The key pieces of this workflow are:

on:
  schedule:
  # Run every day at 5:30 and 17:30 UTC.
  - cron: '30 5,17 * * *'

and

runs-on: self-hosted

If you have a number of self-hosted Actions runners on your repo, you can tag them with whatever you like and target just a subset:

# Finds runners tagged with both "self-hosted" and "mytag"
runs-on: [self-hosted, mytag]

It’s nice to have this kind of scheduled work automated for me running on a computer I trust. But is there anything more critical you can use self-hosted runners for?

Use case 2: Scheduled Puppet runs

As a self-professed nerd working in the server space for a few years, I maintain a Puppet repo to configure my personal servers. It’s worked well for me for over 5 years now. I have a cron job on each host that runs puppet apply. This seems to work, but there’s no transparency and it wastes CPU by running every 5 minutes. If I ran a self-hosted runner on each host instead, I could run Puppet on push to my main branch, instead of waiting for the cron job to run, and I could see the puppet run’s logs much more easily.

I gave this a try on my home server first (since it’s easier to recover a host you have physical access to), and it worked like a charm. I configured the runner as root (eek!) and installed a systemd service with ./svc.sh install && ./svc.sh start. I wrote a workflow to pull the latest changes and run puppet apply in my checkout of the repo on each host, and it worked like a charm.

Since I have multiple hosts, I decided I needed to tag them with their hostname so I could target a job to each host. Each host’s puppet apply is thus independent so if there’s a problem on one host, it doesn’t affect the functioning of puppet on the others.

In order to run on push, I added:

on:
  push:
    branches: [main]

and I created a job for that host:

jobs:
  homeserver:
    runs-on: [self-hosted, homeserver]
    …
    steps: …

I pushed this up and it all worked! Changes to the Puppet repo were applied directly to the host by the self-hosted runner. Next, I added a self-hosted runner to my cloud VM and specified a second job in the YAML:

jobs:
  homeserver:
    runs-on: [self-hosted, homeserver]
    …
    steps: …
  cloudvm:
    runs-on: [self-hosted, cloudvm]
    …
    steps: …

This worked like a charm. Now I can stop wasting CPU with my cron job and I can easily see the result of any change I make in the GitHub UI.

Conclusion

Self-hosted runners are pretty magical. You can test your code with the computer under your desk! You can apply system configuration changes with a simple Git push! You can run all sorts of code whenever you want! So far I’m really impressed with how these runners “just work” out of the box. What are you using self-hosted runners for?

I discovered a project called brew-gem which simplifies the gem install process for gems which are system-wide. It installs the gem and its dependencies in an isolated directory and creates an executable stub in /usr/local/bin. This means you can brew gem install jekyll and have Jekyll and all its dependencies installed without rvm/rbenv or any other manager! No sudo needed, either.

Apple bundles Ruby on its Macs, but Homebrew also supplies Ruby so you can pass the flag --homebrew-ruby to use Homebrew’s Ruby (i.e. from brew install ruby) or pass --system-ruby to use the system Ruby version. By default, brew gem uses the Homebrew Ruby version if it’s installed.

Next time you need a utility written as a Ruby gem, try brew gem install!

Recently, I have felt like I never have a powerful-enough machine. Running Docker for Mac brings even a $2000 or $3000 MacBook Pro (Intel) to its knees. Issues persist with Apple’s silicon as well, given that it’s relatively new so the software hasn’t fully caught up yet.

My most recent team at GitHub worked on an internal app. The app was made up of a series of different services. For development, we used Docker Compose to run a dev database, a Rails app, a webpack compiler, a background worker, and probably one or two things I have now forgotten. Running 4 or 5 services in development in Docker for Mac which were all polling for filesystem events from the host into a VM and into cgroups was a mess. After I upgraded to a machine with 64GB of RAM, this workflow became much better, but it was still a huge battery drain and would make the CPU red hot.

We were early experimenters of GitHub Codespaces. Codespaces natively supports Docker Compose, so setting things up was relatively easy. Once Codespaces became generally available, I migrated to use it. Every day, I woke my computer from sleep and reconnected to my Codespace in the browser. As I worked, I felt more productive than ever. Changes were instantly reflected. It felt like I had this supercomputer that I could access from anywhere.

A few months ago I began working at Google. Google is famous for its one monolithic codebase of billions of lines of code. No single machine could accomodate this amount of code, so they invented CitC, a cloud-based file system overlay of the monolithic codebase. Many Googlers have physical workstations at their office desks, but pandemic hires are instead given a virtual machine in the cloud. These machines are given the workstation treatment. I’m able to remotely access mine through a browser window as though it were a virtual monitor for the machine. The immense power of these VMs gives developers the ability to develop massive applications with relative ease. My experience at Google has only enhanced my delight for coding in the cloud.

Coding in the cloud can be cheaper too. Assuming approximately 246 working days per year (261 work days minus 15 days for vacation), a $3,000 machine would cost you roughly $4.06 per day of work over 3 years. Some companies refresh machines every 2 years, which raises that cost to $6.10/day. Codespaces currently costs $0.72/hour for 8 cores, and it automatically shuts off and stops billing for vCPU after 30 minutes. A normal 8 hour work day includes meetings and other non-coding work, so let’s assume you’re actively coding for about 5 hours per day, costing the business $3.60/day or roughly $2656.89 for the same 3 year working period. This is cheaper than your physical machine! They can give you a lightweight, inexpensive computer and you can do absolutely everything in the cloud. If you leave after 1 or 2 years, the company won’t need to spend the money for your Codespace, thus spending less on you than if they had given you a big, beefy machine during onboarding.

I’m sure there are many specific edge-cases I didn’t account for which might sway the financials one way or the other, but this remains true: working in the cloud allows scalability and flexibility in today’s fast-paced world, and offers true developer delight.

I found a wonderful blog post entitled “Even faster bash startup”, in which the author tackled this same problem.

They started with the hyperfine utility to benchmark the login:

$ hyperfine -i 'bash -l'
Benchmark #1: bash -l
  Time (mean ± σ):      1.242 s ±  0.089 s    [User: 648.1 ms, System: 559.0 ms]
  Range (min … max):    1.129 s …  1.415 s    10 runs

Almost 1.5 seconds for bash to login on my machine. But how can you determine what exactly is taking up this time?

The real “Eureka!” moment from this post is under the title “death by a thousand cuts”:

My .bashrc is split into several smaller parts in ~/.bashrc.d, so I can profile these and see if anything stands out.

Genius. I followed suit and created a ~/.bashrc.d directory (saved in my dotfiles repo of course) and split up the contents of my .bashrc file into discrete files. My ~/.bashrc file became largely setting up the benchmarking and sourcing other files. I could then run __bashrc_bench=1 . ~/.bashrc:

$ __bashrc_bench=1 . ~/.bashrc
/Users/parker/.bashrc.d/10_env.sh: 0.001
/Users/parker/.bashrc.d/20_history.sh: 0.000
/Users/parker/.bashrc.d/30_prompt.sh: 0.000
/Users/parker/.bashrc.d/40_completion.sh /usr/local/opt/fzf/shell/key-bindings.bash: 0.001
/Users/parker/.bashrc.d/40_completion.sh: 0.001
/Users/parker/.bashrc.d/41_brew_completion.sh /usr/local/etc/bash_completion.d/brew: 0.005
/Users/parker/.bashrc.d/41_brew_completion.sh /usr/local/etc/bash_completion.d/cheat.bash: 0.000
/Users/parker/.bashrc.d/41_brew_completion.sh /usr/local/etc/bash_completion.d/fd.bash: 0.001
/Users/parker/.bashrc.d/41_brew_completion.sh /usr/local/etc/bash_completion.d/gh: 0.015
/Users/parker/.bashrc.d/41_brew_completion.sh /usr/local/etc/bash_completion.d/git-completion.bash: 0.010
/Users/parker/.bashrc.d/41_brew_completion.sh /usr/local/etc/bash_completion.d/git-prompt.sh: 0.002
/Users/parker/.bashrc.d/41_brew_completion.sh /usr/local/etc/bash_completion.d/rg.bash: 0.001
/Users/parker/.bashrc.d/41_brew_completion.sh /usr/local/etc/bash_completion.d/youtube-dl.bash-completion: 0.000
/Users/parker/.bashrc.d/41_brew_completion.sh /usr/local/share/autojump/autojump.bash: 0.013
/Users/parker/.bashrc.d/41_brew_completion.sh: 0.050
/Users/parker/.bashrc.d/50_aliases.sh: 0.004
/Users/parker/.bashrc.d/60_functions.sh: 0.000
/Users/parker/.bashrc.d/70_git_dotfiles.sh: 0.000
/Users/parker/.bashrc.d/80_fzf.sh: 0.000
/Users/parker/.bashrc.d/81_rbenv.sh /usr/local/opt/rbenv/completions/rbenv.bash: 0.000
/Users/parker/.bashrc.d/81_rbenv.sh: 1.068
/Users/parker/.bashrc.d/82_iterm2.sh: 0.000
/Users/parker/.bashrc.d/99_localrc.sh: 0.000

This is just from today. 15ms on the gh auto-completion, 13ms on the autojump completion, and… 1068ms on rbenv? It’s not from the rbenv bash completion, so it must be something in the script itself. This is almost 85% of the total bash boot time that we found earlier with hyperfine!

I recently found myself discussing the nuances of building openssl for rbenv ruby installations on macOS, and one of the maintainers of the ruby-build installation utility had recommended using the Homebrew-managed openssl, so I added the following line to my 81_rbenv.sh file:

export RUBY_CONFIGURE_OPTS="--with-openssl-dir=$(brew --prefix openssl)"

Hm, exporting a variable is extremely fast. How long does brew --prefix openssl take?

$ hyperfine 'brew --prefix openssl'
Benchmark #1: brew --prefix openssl
  Time (mean ± σ):      1.063 s ±  0.019 s    [User: 561.5 ms, System: 478.6 ms]
  Range (min … max):    1.032 s …  1.094 s    10 runs

Yikes. That entire script’s time is just running brew! I wonder: could we hardcode this instead of calling brew?

$ brew --prefix openssl
/usr/local/opt/openssl@1.1
$ ls -l /usr/local/opt/openssl*
lrwxr-xr-x  1 parker  admin  28 Apr  1 22:03 /usr/local/opt/openssl -> ../Cellar/openssl@1.1/1.1.1k
lrwxr-xr-x  1 parker  admin  28 Apr  1 22:03 /usr/local/opt/openssl@1.1 -> ../Cellar/openssl@1.1/1.1.1k

Yes! It looks like I could use either of those two paths instead. In order to be modular, it’s best to use a variable. How about $HOMEBREW_PREFIX? Thus, the line becomes:

export HOMEBREW_PREFIX="${HOMEBREW_PREFIX:="/usr/local"}"
export RUBY_CONFIGURE_OPTS="--with-openssl-dir=${HOMEBREW_PREFIX}/opt/openssl"

Does that fix our issue?

$ hyperfine -i 'bash -l'
Benchmark #1: bash -l
  Time (mean ± σ):      49.4 ms ±  10.2 ms    [User: 32.0 ms, System: 14.4 ms]
  Range (min … max):    42.0 ms … 117.3 ms    62 runs

50ms? Yes, that’s significantly better than 1200ms! This is a huge success, with many thanks to ~/.bashrc.d for making the benchmarking of each snippet so much easier.

Do you have a complicated ~/.bashrc or ~/.zshrc? Consider following this same method! It makes tracking down those pesky slow start-up scripts so much easier. Reusing this benchmarking code in bash is easy too – the combination of $TIMEFORMAT and time makes this a simple proposition. Profiling bash scripts of all shapes and sizes has never been so easy!

How do Rubyists fix this? They use a Ruby version manager.

There are many Ruby version managers. My preference is rbenv, but rvm, chruby, and asdf are all good options.

Getting setup with rbenv is easy on a Mac (full installation instructions):

# 1. Install rbenv & ruby-build
$ brew install rbenv ruby-build
# 2. Load rbenv into your shell environment persistently (.bashrc for bash, .zshrc for zsh)
$ echo 'eval $(rbenv init -)' >> ~/.bashrc && . ~/.bashrc # for bash
$ echo 'eval $(rbenv init -)' >> ~/.zshrc && . ~/.zshrc # for zsh
# 3. List versions available to install! Choose one.
$ rbenv install --list
# 4. Install that version!
$ rbenv install 2.7.3
# 5. Set that version as the new default on your machine.
$ rbenv global 2.7.3
# Override for a specific directory by running `rbenv local <new_version>`

This installs this version of ruby into $HOME/.rbenv/versions. Since this is in your home directory, there are no permissions issues. Gems are installed inside each individual version (see gem env GEM_PATH for the exact paths) so it’s much easier to install.

Running rbenv global <version> means that your shell will refer to that version by default. Running gem install in any directory will install gems using that version, and running ruby <file> or a gem executable will use that Ruby version.

Different projects may have different Ruby versions. Rbenv makes specifying which Ruby version you want easy: inside your project’s root directory, run rbenv local <version> (it will prompt you to install the version if it’s not present). This is really helpful when you need to be switching between versions but don’t want to remember. It will write out a .ruby-version file so be sure to commit that to your source control!

Want to delete a version you’re not using anymore? Easy: rm -r ~/.rbenv/versions/<version>.

Using rbenv makes your life easier! I have been happily using rbenv for many years now and can easily install and use gems and manage ruby versions with ease.

When setting up gpg (v2) to automatically sign your commit messages, you may run into an error:

error: gpg failed to sign the data
fatal: failed to write commit object

If you re-run your git commit command with GIT_TRACE=1, you can see it running gpg --status-fd=2 -bsau <your-signing-key> but failing.

The cause of this is the pinentry process. By default gpg uses the pinentry program which requires a TTY and access to the shell’s stdin. This doesn’t appear to function properly on a Mac. Instead, you should use pinentry-mac.

$ brew install pinentry-mac
$ echo "pinentry-program /usr/local/bin/pinentry-mac" >> ~/.gnupg/gpg-agent.conf
$ gpgconf --kill gpg-agent

Re-run your git commit command and you should see a GUI pop-up to request your GPG key’s passphrase. Enter it and the commit will succeed!

You can setup automatic signing with the following commands:

$ git config --global user.signingkey <key_id>
$ git config --global commit.sign true
$ git config --global commit.gpgSign true

Make sure to upload your GPG key to your GitHub account:

$ gpg --armor --export <key_id>

Happy signing!

My first workout with Fitness+ was over the holiday break, shortly after it was first released. We couldn’t attend class at our normal gym and needed a good sweat. We’ve been huge fans of Fitness Blender for years, using their free YouTube videos to get a good sweat in while away from home. Since 3 months of free Fitness+ came with the purchase of an Apple Watch Series 6, I though we’d give it a go.

Our first workout was a HIIT class with Bakari. It was a burner! We really liked the movements that Bakari picked out, and enjoyed his joyful approach to pushing through discomfort. Each move had a modification to reduce impact, which made everything all the more accessible. Don’t want to jump today? No worries, here’s a low-impact modification. We tried workouts of all durations, with our favorites being longer for that really satisfying burn. HIIT from Fitness+ checked all our boxes and left us sore and energized.

Like thousands of households across the U.S., we purchased a rowing erg this year for our home. I gave a 20-minute workout with Anja a try, rowing away to some “Upbeat Anthems” and the encouraging coaching from Anja. The structure was on point and you can scale your effort up and down depending on what you’re looking for. Two thumbs up here.

Next, I tried yoga. As someone who’s only dipped his toe into yoga in the past, this felt fairly challenging to me. Some trainers move through poses too quickly and can leave an inexperienced person frustrated. Others offer a more measured approach. I generally pick shorter sessions in yoga to help ensure it’s a good experience.

Most recently, we tried dance! What a thrill. We laughed the whole time, trying desperately to stay in sync with the instructor. These are workouts you’d want to do over and over to get the most out of them unless you’re an experienced dancer. We were lost half the time! Every time the instructor said “4 steps,” we were ready for 4 steps but then he took 4 steps backward. Things like that trip up casual consumers.

Time to Walk was recently released and immediately stuck itself at the top of my Workouts app on Apple Watch, which is extremely annoying. The list of workout types (e.g. HIIT, Rowing, Outdoor Run, etc) are ranked by how much you use them – except now “Time to Walk” takes top spot even though I’ve never used it. This goes against the muscle memory I have of always tapping the top spot since it’s my most-often-used workout. No longer. I am surprised Apple would make such a stupid move – frankly, I don’t want to see that if I don’t use it.

Fitness+ has been a great addition to our fitness regimen, and especially shines on days we can’t attend the gym. I’d recommend it to anyone in the Apple ecosystem who is looking to sweat a little!

github-change-default-branch is a command-line tool which walks through all repos from a user or organization on GitHub and prompts you to ask whether you’d like to change the default branch to a new name. You can change the target name with a flag:

$ github-change-default-branch -login=parkr -new-name=main
parkr/auto-reply - :loop: Handle GitHub webhooks and manage issues on your repositories. Currently runs @jekyllbot.
  https://github.com/parkr/auto-reply
  change default branch from "master" to "main"? (y/n) >

If you choose to change the default branch, the program will do the following:

1. Add a new ref with the given name at the same commit SHA as the current default branch
2. Update the default branch setting for the repo to point to the new ref
3. Delete the old default branch ref from the repo

To install into $GOPATH/bin:

$ go get github.com/parkr/github-utils/cmd/github-change-default-branch

Changing the name of the default branch for your repos has never been easier, so now there’s no excuse to keep the current insensitively-named default.

Amazon Prime is a collection of services you subscribe to for a monthly or yearly fee. This includes video, music, and free fast delivery. Investors are always looking to Prime membership numbers as a sign of the strength of Amazon’s business.

Amazon has abused its market power and great wealth at great cost to its workers. This has been apparent for years, but truly came into full clarity during the COVID-19 crisis. In its fulfillment centers, workers are not being protected. Workers are being told they will have to forfeit all pay if they get sick (as a result of their work or not)¹. The people who turn our orders into packages at our door are being squarely screwed in this crisis², while Amazon rakes in profits and market share³.

Too many people are hurting.

There are myriad other ways to buy the goods that Amazon sells. I’ll be buying from small businesses directly for all my shopping that would have previously been on Amazon. In light of COVID-19, I can get everything online from many local businesses.

It’s often said that we vote with our dollars. I, for one, will be voting for something new. Join others⁴,⁵ who are voting “no” to Amazon’s exploitation.

The fit out of the box is comfortable. These buds stay in your ears. Whether racing down the stairwell with your pup or going out for a jog, you won’t have to think about them falling out. They started getting a bit looser after a few miles of running each time I smiled at a passerby and needed to be pushed farther into my ear more frequently.

The vent prevents you from feeling too clogged and only hearing your haggard breath during a workout. This was my primary complaint with the Jaybird Vista: every breath, every step, and the rushing wind would all scream through the music during a workout. This vent makes these buds so much more practical for me.

Noise cancellation is insanely good. When you put the first earbud in, you won’t notice any cancellation. This detail prevents that “clogged-up” feeling you might get with an earplug in just one ear. Once you put in the second earbud, the world fades away. Transparency is very good and very easy to activate when someone wants to speak to you.

The sound quality is great. I didn’t really mind the AirPods since so much other sound leaked in. With more isolated sound, these buds are impressive in their wide range.

Charging case is oddly shaped but not too dissimilar to the regular AirPods case. Having wireless Qi charging by default is a game changer.

The controls are pretty tough to use. I have pretty skinny fingers, and getting a hold of the stem to squeeze it without dislodging the earbud is still something I still haven’t figured out. In this way I like the original AirPods better. But the squeeze approach offers short and long press options, which increase the possibilities. This improves the usefulness of the buds.

The look is sleek. I have smaller ears and they just fade away.

The price is insanely high. We really shouldn’t be spending so much on headphones. Since I use these so often though and my original AirPods are starting to have connection issues, I figured I’d give them a shot. The whole true wireless headphone market is overpriced.

Overall, I am very happy with the AirPods Pro.

So I have this great golang server which connects to a remote host over SSH. Not so bad so far, golang.org/x/crypto/ssh makes this pretty easy. The hard part is that the normal tools we use to generate SSH keys are not compatible with this crypto/ssh library.

My goal was to have a private key, PEM encoded, and encrypted with a passphrase. This would be loaded by my Go code. Then, I wanted to generate a public key for that private key that was in that funky authorized_keys SSH form of key-type key-value key-comment.

Using ssh-keygen gets you a PEM-encoded private key and a properly encoded public key, but when you try to load the private key into the Go library, it chokes on the passphrase. The trick is that several important headers are missing which tell the Go library that it’s encrypted.

The Solution

# To generate the private key, run the following:
$ openssl genrsa -des3 -out private.pem 4096

Now you have your private key. Now you need to generate the public key.

# To generate the *public* key from your private key, run the following:
$ openssl rsa -in private.pem -outform PEM -pubout -out public.pem

Now you have a PEM format for your public key. Nice! This can’t be used with SSH’s authorized_keys file though, so we’ll have to do one more conversion:

# To generate the ssh-rsa public key format, run the following:
$ ssh-keygen -f public.pem -i -mPKCS8 > id_rsa.pub

Now you have a public key in the format OpenSSH expects.

• private.pem - your encrypted, PEM-encoded private key
• id_rsa.pub - your OpenSSH-compatible public key

That seems to do the trick for me, and SSH connections are working great between my Go service and my servers. :tada: :sparkles:

Good news! You can take charge of your DNS configuration with Git, GitHub and OctoDNS.

OctoDNS is a library for managing your DNS, which ships with a series of commands for syncing, dumping, and reporting. OctoDNS was originally developed at GitHub to aid in tracking DNS across our domains and in helping integrate DNS management into our normal tools, like Git, our deployment stack, and so on.

OctoDNS operates kind of like rsync: you specify a source (usually YAML files) and specify a destination or destinations (DNS providers) and ask it to sync. Any configurations present in the source configuration (YAML) will sync to the destination (DNS providers). I haven’t tried, but theoretically you can even sync from one DNS provider to another and skip YAML altogether! But the point of this blog post is to share how we manage DNS with Git & GitHub so we’ll be using YAML.

Getting Started

1. Configuration

We’ll create our repository and dump a zone to get started. Create a repository on GitHub, initialize with a readme, and clone:

~$ git clone https://github.com/YOURUSERNAME/dns
~$ cd dns

Now, make a config directory and specify the source & destination for your configurations. At GitHub we name this file after its environment. We’ll be fancy and call our environment “production.”

In this example, I’ll use Cloudflare as my DNS provider but you can use any other supported providers. The records they support depend on their API capabilities, so make sure to check that your provider supports creating the types of records you want. In this file, we’ll use the built-in environment variable expansion. This means any octodns-* commands will need to be run with these variables set.

~/dns$ mkdir config
~/dns$ cat > config/production.yaml <<EOF
---
providers:
  config:
    class: octodns.provider.yaml.YamlProvider
    directory: ./config
  cloudflare:
    class: octodns.provider.cloudflare.CloudflareProvider
    email: env/CLOUDFLARE_EMAIL
    token: env/CLOUDFLARE_TOKEN

zones:
  example.com.:
    sources:
      - config
    targets:
      - cloudflare
EOF

Ok, so the configuration specifies two things: the providers, and the zones.

The providers are your sources and destinations for the DNS records. Here, we’re specifying a provider called config which reads the YAML from ./config. We also specify a provider called cloudflare which uses Cloudflare’s API and requires the CLOUDFLARE_EMAIL and CLOUDFLARE_TOKEN environment variables.

The zones are your domains! You can specify unique sources and targets for each. “Target” is our destination, and you can specify as many as you wish! They must be configured in the providers dictionary to use them.

2. Dump your zone

Now that you have your configuration file in place, you can run OctoDNS! First, we’ll dump the zone you added to your configuration file. OctoDNS will create a new YAML file with all the records it could find in the target you specify.

We’ll use Docker to make installation a little simpler, but if you’re into managing your own Python virtualenv and such, feel free to do that. The OctoDNS repository’s readme has great documentation on running it straight on your host.

$ docker run -it --rm \
    -e CLOUDFLARE_EMAIL=yourlogin@gmail.com \
    -e CLOUDFLARE_TOKEN=yourtoken \
    -v $(pwd):/opt/dns \
    parkr/octodns:v0.9.4 \
    octodns-dump \
    --config-file /opt/dns/config/production.yaml \
    --output-dir /opt/dns/config \
    "example.com." \
    cloudflare

This will print out a YAML file in config/example.com.yaml, containing your DNS entries (comments my own):

---
# These are the top-level DNS records for example.com.
# This specifies:
#    1. ALIAS example.com to subdomainarecords.example.com
#    2. MX example.com to mxa.emailprovider.com at priority 10 and mxb.emailprovider.com at priority 10
#    3. TXT example.com with "v=spf1..."
? ''
: - ttl: 300
    type: ALIAS
    value: subdomainarecords.example.com.
  - ttl: 300
    type: MX
    values:
    - exchange: mxa.emailprovider.com.
      preference: 10
    - exchange: mxb.emailprovider.com.
      preference: 10
  - ttl: 300
    type: TXT
    value: v=spf1 include:emailprovider.com ~all
# Here is an example of a CNAME record, subdomaincname.example.com.
# It points back to example.com.
subdomaincname:
  ttl: 300
  type: CNAME
  value: example.com.
# And here is an example of an A record with corresponding IPv6 AAAA record,
# subdomainarecords.example.com.
subdomainarecords:
- ttl: 300
  type: A
  value: 100.100.100.100
- ttl: 300
  type: AAAA
  value: 2001:2001:2001:2001:2001:2001:2001:2001

Whew! This is great! Now we can commit this file and propose a change.

3. Syncing your zones

Let’s make a change. Let’s modify the A records for subdomainarecords.example.com. To do this, we’ll update the YAML:

subdomainarecords:
- ttl: 300
  type: A
  value: 300.300.300.300
- ttl: 300
  type: AAAA
  value: 4001:4001:4001:4001:4001:4001:4001:4001

Now we’ll test our changes with octodns-sync:

$ docker run -it --rm \
    -e CLOUDFLARE_EMAIL=yourlogin@gmail.com \
    -e CLOUDFLARE_TOKEN=yourtoken \
    -v $(pwd):/opt/dns \
    parkr/octodns:v0.9.4 \
    octodns-sync --config-file /opt/dns/config/production.yaml

It will print out some beautiful output showing the before and after picture (it will NOT apply the changes yet):

********************************************************************************
* example.com.
********************************************************************************
* cloudflare (CloudflareProvider)
*   Update
*     <AaaaRecord AAAA 300, subdomainarecords.example.com., ['2001:2001:2001:2001:2001:2001:2001:2001']> ->
*     <AaaaRecord AAAA 300, subdomainarecords.example.com., ['4001:4001:4001:4001:4001:4001:4001:4001']> (config)
*   Update
*     <ARecord A 300, subdomainarecords.example.com., ['100.100.100.100']> ->
*     <ARecord A 300, subdomainarecords.example.com., ['300.300.300.300']> (config)
*   Summary: Creates=0, Updates=2, Deletes=0, Existing Records=6
********************************************************************************

Now, let’s apply our changes! To do this, add the --doit flag to our octodns-sync command:

$ docker run -it --rm \
    -e CLOUDFLARE_EMAIL=yourlogin@gmail.com \
    -e CLOUDFLARE_TOKEN=yourtoken \
    -v $(pwd):/opt/dns \
    parkr/octodns:v0.9.4 \
    octodns-sync --config-file /opt/dns/config/production.yaml --doit

You’ll see CloudflareProvider[cloudflare] apply: making changes when it’s making your changes.

Tada! Wait for the records to expire (300 seconds) and dig for your new records!

4. Make it go with GitHub Flow

Now, let’s propose a change using a PR workflow. This is especially important when you’re managing the DNS

1. Create a new branch and check it out
2. Modify a record and commit the changes
3. Push the changes and submit a PR
4. Run octodns-sync without --doit and leave a comment with the output to show the changes you’re making
5. Get approval
6. Run octodns-sync with --doit to apply the changes
7. Verify the records are present in DNS and merge the PR

:tada: GitHub Flow helps make managing your DNS much easier.

See it in action

Want to see it in action? I am managing my own DNS at parkr/dns, as well as the DNS for Jekyll at jekyll/dns.

I hope this helps illustrate how OctoDNS can take the pain away from managing DNS manually in a web portal.

Their most significant claim (and the biggest reason to switch) is that it’s much faster than other domain name servers. I live in Northern Virginia, so my normal internet browsing is ridiculously low-latency. When you live tens of miles from one of the largest data center hotspots in the U.S., you can expect quick response times. If I ping github.com for example, I get a response in about 5ms. OpenDNS is about ~7-8ms on average response times. So I figured I would measure 1.1.1.1:

64 bytes from 1.1.1.1: icmp_seq=5997 ttl=58 time=4.470 ms
64 bytes from 1.1.1.1: icmp_seq=5998 ttl=58 time=5.055 ms
64 bytes from 1.1.1.1: icmp_seq=5999 ttl=58 time=5.089 ms
64 bytes from 1.1.1.1: icmp_seq=6000 ttl=58 time=5.987 ms
^C
--- 1.1.1.1 ping statistics ---
6001 packets transmitted, 6001 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 2.496/6.336/313.273/9.857 ms

6.3ms on average is pretty darn good. This measurement was taken ~9-10pm on a Sunday night, so you might imagine added latency due to streaming video. I left this ping running for 1 hour and 40 minutes, and kept on using my computer like I normally do. There were some odd spikes to 175ms, and even one to 313ms (!), but overall the performance was stellar. 6.3ms plus or minus 9.9ms isn’t so bad. (In a much shorter test, OpenDNS’s 208.67.222.222 was 7.7ms plus or minus 11ms, so it is indeed faster).

Reading through some of the comments on the above blog post, you’ll see that some folks are having issues connecting. ISPs (even Comcast in Nashville, apparently) are blocking traffic to 1.1.1.1 (and sometimes the alternative address 1.0.0.1), which is a bummer. YMMV, so before switching over, be sure to test that you can connect. They also have IPv6 addresses:

$ host 1dot1dot1dot1.cloudflare-dns.com
1dot1dot1dot1.cloudflare-dns.com has address 1.0.0.1
1dot1dot1dot1.cloudflare-dns.com has address 1.1.1.1
1dot1dot1dot1.cloudflare-dns.com has IPv6 address 2606:4700:4700::1001
1dot1dot1dot1.cloudflare-dns.com has IPv6 address 2606:4700:4700::1111

This DNS was announced yesterday and while it seems like a great addition to the growing list of DNS alternatives, I think I’ll let others take the leap for a few months before diving in. Shaving 1.1ms on average off my DNS response times likely won’t make a huge difference in my day-to-day browsing. I hope Cloudflare’s new DNS is successful, for the benefit of the whole internet.

When I was just graduating from college and had a steady income, I finally decided it was time to have my own cloud server. Rackspace was the easiest choice back then, and I spun up a server with CentOS 6. Being new to server administration, I spent a long time organizing the packages and apps I installed. I ran a few Ruby web apps and hosted my static site built with Jekyll. I even went through the arduous process of manually configuring postfix to receive email for me on this host. Needless to say, I learned a lot.

GitHub switched to Debian in the last year and I figured I would migrate my server to Debian, too. During my internship at 6Wunderkinder / Wunderlist, I saw first-hand how nice immutable deploys were. The concept was fairly novel for a Ruby on Rails shop: with each deploy, spin up brand new VM’s, install your code, add them to the load balancer, and spin down the old ones. This is very much how Kubernetes rolling deploys happen today. After several years of my CentOS server running and collecting cruft, I decided to follow in 6Wunderkinder’s footsteps and spun up a fresh Debian server.

In the interim, I had, like millions of other developers, become enammored with Docker. The idea of packaging up code and dependencies in a language-agnostic container was immediately appealing. I began experimenting with running my apps inside Docker containers, and proxying traffic from nginx to the Docker containers. After a while, I found a system I liked: run the docker container in an upstart script, proxy traffic received by nginx to the docker container. When I wanted to update the app, I pushed a new revision of the Docker image, modified my upstart script to use the new tag, and ran restart <app> on my VM.

Over time, I made some improvements to this: I migrated all my apps from upstart to systemd when moving to Debian, and I generated nginx configurations for static and proxy sites instead of manually creating them.

In the last month or so, I decided that spending $35/mo on a VM didn’t make much sense when there were equivalent computing environments for much less money. I signed up for Linode, which offers 1GB VM’s for $5/mo, and had to migrate all my apps again. Time for a configuration management system! I had never much liked Chef, so I chose to give Puppet a try. What a massive difference this change has made.

I began by setting up my development environment: creating my user, adding my SSH public keys, and cloning my dotfiles. This was a fairly straight-forward process. I pushed up this new code to a repository. In order to get started with Puppet on the new host, I had to install Puppet. I went with v4.8, cloned down my repository to /etc/puppet, and ran puppet apply. It worked! I setup a script/apply bash script and set it up to run every hour.

The next piece was to setup hieradata and a parker::app resource which I could use to setup my above flow. First piece was the docker image. I installed the puppetlabs-docker Puppet module (using puppet-librarian, of course!), and used the docker::image resource to download the image of my choosing. Next, I created a systemd::unit_file for my app. The unit file is based on a common template using docker run and writing all environment variables to a file and using the --env-file flag when running the docker container. It exposes the server to a port of my choosing. I used the camptocamp-systemd Puppet module to get systemd::systemctl::daemon_reload, which I notify when a systemd unit file changes. Last, I used the service resource to ensure the service is running when I apply my puppet catalog.

This made this very easy! I setup a new .pp file for each of the servers I wanted to run. For those which used a database, I used the puppetlabs-mysql module and used the mysql::db resource to create a database, user, and password specific to each app. If I needed any other resources (for example, a specific unix user for the app), then I could add these in this .pp file.

To install each app, I used hieradata. Hiera is basically the greatest thing ever, and makes managing your puppet code so much nicer. All my parameters to each class go in the hiera YAML file for each host. And at the top, I specify a classes array, which essentially includes each class. When I want a new app, I create the class for it, and include it in this list for the host I want to install it on.

I also run a few static sites from this server (like this one!) and wanted to set those up, too. I created a site resource and created a new class for each static site I wanted to host. These are simple webroot nginx configurations so all I do is create directories, symlink a file in sites-available to sites-enabled, and let my jekyll-build-server handle the creation of the site.

This has been working great so far! Setting up new apps and static sites is super simple now. I’m in the process of both manging nginx with this (I compile manually using the versions of nginx, zlib, openssl, and pcre that I want), and setting up my home NUC to use this, too. I have a bunch of little command-line utilities which do my bidding and it’d be nice to get those all installed via this new method, too.

One open question I have is managing secrets. I just write these in plain text in my hieradata YAML file, which isn’t ideal. Anyone with the proper unix permissions who gets onto my server can see all my credentials. One thing I’ve done is setup a separate user on GitHub.com with access to my static sites and other repos of mine and use personal access tokens for that user instead of for my user. This helps reduce the likelihood that a security breach on my servers would result in any meaningful access to private code that I didn’t explicitly allow my server to access. Using GitHub Apps seems like a better next step, but I haven’t quite gotten there. If you have any recommendations for how I can better organize this, please drop me an email!

Managing my servers with Puppet has been a great pleasure so far and I’m really pleased with how it’s helped make managing my servers even easier. :sparkles:

Our typical solution to this is something like:

def sort_sparse_property(objects, property)
  objects.sort do |apple, orange|
    apple_value = apple.public_send(property)
    orange_value = orange.public_send(property)

    if !apple_value.nil? && orange_value.nil?
      -1
    elsif apple_value.nil? && !orange_value.nil?
      1
    else
      apple_value <=> orange_value
    end
  end
end

Indeed, we did that in Jekyll::Filters#sort_input. It works fine, but it’s pretty slow. The speed difference between Enumerable#sort and Enumerable#sort_by is well-documented: sort_by wins every time. But why?

Well, I read the documentation for Enumerable#sort_by all the way through to the end. Lo, and behold! It walks right through an optimization problems using sort! The key problem is that the sort block is allocating objects during every call. That’s a huge waste of allocations, and if you have ever worked on production systems written in Ruby, you know object allocations are one of the key reasons for slowness.

To reduce object allocations, the Ruby documentation suggests something called a “Schwartzian transform”:

A more efficient technique is to cache the sort keys (modification times in this case) before the sort. Perl users often call this approach a Schwartzian transform, after Randal Schwartz. We construct a temporary array, where each element is an array containing our sort key along with the filename. We sort this array, and then extract the filename from the result.

This is exactly what sort_by does internally.

Holy smokes! So going back to our example above, it’s suggesting we first extract the property for each object, then we sort, then we pull the object out again. Let’s give that a try:

def sort_sparse_property(objects, property)
  objects.map { |object| [object.public_send(property), object] }
    .sort do |apple, orange|
      # apple & orange are now an arrays of [value, object]
      # Use the value cached in this array to compare!
      if !apple.first.nil? && orange.first.nil?
        -1
      elsif apple.first.nil? && !orange.first.nil?
        1
      else
        apple.first <=> orange.first
      end
    end
    .map { |object_info| object_info.last }
end

The first call to #map creates an array for each object with the value we want to compare paired with the object itself. The call to #sort performs the sort as usual, but this time it’s using the cached value. The last #map extracts the object from the sorted array of [value, object] pairs.

I wrote a benchmark for Jekyll using Jekyll::Document objects and ran it for 2 properties: one which is sparsely available (i.e. only on a few objects), and one which is more fully available (i.e. on all but a small number of objects) and the results were shocking.

For a sparsely available property, the Schwartzian transform was 7-8x faster. And for the more fully available property, the Schwarzian transform was 1.75x faster. Those are monumental savings!

It seems like a huge waste to create a new array and a new object for every item in the original Enumerable, but as we have just proven, it is far more efficient due to the number of times the sort block is called.

Next time you implement a custom sort block, remember the Schwartzian transform!

Ask any open source maintainer what their job description is, and you’ll likely hear about reviewing patches sent from community contributors as a major responsibility of a maintainer. Maintainers are gatekeepers: they decide what code makes it into the project and what code doesn’t. Part of this is flat-out rejecting proposals (“This feature will not be accepted because it does not fit into the philosophy of the project.”) and part of this is working with contributors to get the functionality proposed up to code style, functionality, and testing standards of the rest of the project. Maintainers are making decisions non-stop about the product, style, functionality, and testing quality of a contribution.

Decision fatigue is defined as:

the deteriorating quality of decisions made by an individual after a long session of decision making.

When I first heard of this, I immediately related to it. Decision fatigue affects me greatly as a maintainer and has become a bigger problem for me since I began working as a programmer full-time.

In college, my primary focus was on doing homework and studying. The decisions I made were essentially around scheduling when to study. My free time seemed endless in some ways, since I didn’t have a particularly rigid schedule. Rehearsals, classes, studying, socializing. I wasn’t making qualitative decisions about whether what I was learning deserved to be in my brain; school doesn’t really give you that choice once you’ve selected which classes to take. I worked on Jekyll in between classes and in the evenings. It was an escape from the monotonous work of completing problem sets.

After graduation, I took a great job at VSCO, a start-up near and dear to my heart. I had used VSCOCam since the early days of the Grid and was really excited to be joining the start-up world. Jekyll was experiencing a renaissance. We were merging contributions, releasing more often, and growing the project to meet the evolving needs of web developers. It was in a good place, so I focused on making sure I could be successful at VSCO. As my focus shifted, I began reviewing fewer issues and contributions and I found myself punting on discussions all the time. I would save the link somewhere and “get back to it later.” Issues and pull requests started hanging for weeks, then months. It was getting harder and harder to determine the merits of a contribution. Every holiday, when I had a few days off work, reviews would start feeling a bit easier and I would chug through some contributions, feeling guilty that I had let them stall for so long. Once I met my girlfriend, my holidays were spent with her instead of on my computer. Slowly, I spent less and less time on Jekyll.

When I began working at GitHub on Pages, we were turning Ben Balter’s 20% project back into a full product with a full-time staff. We were cleaning up old technical debt, we made massive architectural changes, and started working on a roadmap for new features. I had been hired with the understanding that I’d be able to dedicate some of my time to Jekyll. It was an integral part of Pages after all, and I could continue building on the renaissance with more resources behind me. I put some more work into Jekyll, but the problems I could solve within GitHub beckoned. Brad Fitzpatrick wrote about this about people joining Google:

Prior to joining Google I always joked that Google was the black hole that swallowed up open source programmers. I’d see awesome, productive hackers join Google and then hear little to nothing from them afterwards. … Many open source programmers are just programmers. They like working on fun, hard problems, whether on open source or otherwise. … The Google development environment is so nice. … It’s very easy to hack on anything, anywhere and submit patches to anybody …

All this goes to say: fun, hard, new-to-me problems existed at work and I got to work on them with some of the best minds in the start-up world. The development environment at GitHub is really nice. Building a feature into the main app is easy, testing is full of niceties I never had elsewhere, and deployment is just 1 command to Hubot. I began to slowly disappear from Jekyll into this wonderland of Ruby and smart people at GitHub. When I was in college, I didn’t know a single person who was familiar with the command-line until Senior year. Students learn to program Java & Python in IDE’s, and even our classes about net infrastructure revolved around the AWS web UI. Joining the Jekyll community my Freshman year helped me learn practical development skills (like source control!) from really smart people. GitHub was that, but full-time, Chat, and great centralized tooling. The things I had joined the Jekyll community to learn I had learned, and was spending all my time soaking up the nuggets of wisdom from my colleagues, disappearing from the open source community for long lengths of time.

Some wisdom I have picked up has come in the form of advice from Ben Balter and Mike McQuaid pertaining to running an open source community. Ben had the idea of affinity teams & affinity team captains and the idea of setting up a philosophy document as guiding principles for the development of Jekyll. Mike suggested I write maintainer-focused documentation for existing and future Jekyll maintainers. I did that too. Clearly, communication and delegation were two aspects of running an open source community that I still hadn’t learned well.

Every time I take a look at the list of PR’s, I want to just close the vast majority of them because they don’t seem useful to me. I have lost my broader lens of users’ needs, and we haven’t documented our core use-cases anywhere. When I see a contribution, I don’t know how to evaulate it. We have process docs, but nothing stating how to evaluate feature requests. Do you ask the user to ship it as a plugin? How do you know if it should be in Jekyll core? Should we ship a new plugin with this functionality? Does this break a relied-on behavior? This laundry-list of questions spirals out of control until I close my browser tab.

Recently, I have been considering what it would look like to pass on the torch of lead maintainer to another member of the Jekyll community. Our affinity team captains are really excellent and are passionate about making Jekyll great. But I’m afraid if I leave, the new maintainers won’t feel like they can make decisions because they’ll feel the same paralysis.

Is this paralysis, coupled with the pressure of a project that needs work if it will survive, the first step in burning out?

I don’t know the answers here, and I don’t know when I’ll find them. All I know is that I need to start asking myself how I can reduce decision fatigue. The Jekyll community deserves better than an absentee lead maintainer. If you have any ideas or resources for dealing with decision fatigue, feel free to email me. Or, if you also suffer from this and just want to reach out to commiserate, I’m all ears. I’d be really curious to hear your thoughts.