set_trace_func

Ruby HTTP Clients in 2025

2025-02-27T00:00:00+00:00

At the dayjob, we do payments, both credit card and bank transfers. Over the years, I’ve done over a dozen integrations to support the banks our customers use, using a mix of HTTP and SFTP.

The latest one we had was working with a bank that used an HTTP API to submit transactions, but was extremely picky about how the request was made. We needed to make an HTTP call with:

a custom Authorization header format (literally Basic auth but with a different name)
a custom (nonstandard) header
a multipart form data field with a required content-type
a separate multipart file upload

After massaging our request enough to get it working with Net::HTTP, I thought I’d give some other options a shot for comparison.

There was a blog post by honeyryderchuck a bit over a year ago that did a big shootout, and I’m running through the subset of those that I’ve actually used: Net::HTTP, Faraday, HTTP, and HTTPX. (I’ve also used HTTParty in past years, but it’s not what I’d recommend for this much request complexity.)

First, some boilerplate - sample data, and a common class stub and usage example:

class HttpClient
  def initialize(auth)
    @host = 'example.com'
    @path = '/api/path'
    @auth = auth
  end

  def passcode
    Base64.encode64("#{@auth[:user]}:#{@auth[:pass]}").chomp
  end

  def post(criteria, transactions_csv)
    # ...
  end
end

auth = {user: 'user', pass: 'pass'}
client = HttpClient.new(auth)

criteria = { process_date: "#{Date.today.strftime('%Y%m%d')}" }
transactions_csv = <<~CSV.gsub("\n", "\r\n")
  E,C,001,99001,09400313371,10000,TABC00001001,ACME Corp
  E,C,002,99002,09400313372,20000,TABC00001002,John Doe
  E,C,003,99003,09400313373,30000,TABC00001003,Jane Doe
CSV
client.post(criteria, transactions_csv).body

This sets up a class to hold the URL and authentication, and a generic helper for the Authorization header. Then we set up our form data blob (to be rendered to JSON) and file data - note that Net::HTTP expects an actual File object for the file upload (and Faraday wrapping Net::HTTP needs the same), but in this example we’d be generating it live from database records and so we expect to wrap it in a StringIO.

Net::HTTP

Our baseline implementation is Net::HTTP. To get multipart form support, we need the multipart-post gem, and require 'net/http/post/multipart'.

As part of our API experimentation we also tried http-form_data, which is part of the HTTP project, but that needed a bunch of boilerplate to get set up, but multipart-post winds up being fairly clean. It’s still Net::HTTP though, which means suffering through its old-school object-oriented API. The only really exciting thing to call out here is the magic :parts header that multipart-post supports to let us provide the content-type for the formdata part of the payload.

def post_net_http(criteria, transactions_csv)
  form_data = {
    criteria: criteria.to_json,
    transactions: UploadIO.new(
      StringIO.new(transactions_csv),
      'text/plain', 'transactions.csv'
    )
  }

  headers = {
    'Authorization' => "Passcode #{passcode}",
    'filetype'      => 'STD',
    :parts          => {
      criteria: {'Content-Type' => 'application/json'},
    }
  }

  request = Net::HTTP::Post::Multipart.new(@path, form_data, headers)

  http = Net::HTTP.new(@host, 443).tap do |http|
    http.use_ssl = true
  end
  http.request(request)
end

Faraday

As mentioned, Faraday is a wrapper around Net::HTTP and so winds up looking fairly similar. It needs a separate faraday-multipart gem for multipart support, but doesn’t need special hand-holding to manage part content-types. Also, actually setting up the connection object is much more readable. My biggest gripe is headers being provided as an argument to new, and not being supported by conn.request :headers or something.

def post_faraday(criteria, transactions_csv)
  form_data = {
    criteria: Faraday::Multipart::ParamPart.new(
      criteria.to_json,
      'application/json'
    ),
    transactions: Faraday::Multipart::FilePart.new(
      StringIO.new(transactions_csv),
      'text/plain',
      'transactions.csv'
    )
  }

  headers = {
    'filetype' => 'STD'
  }

  http = Faraday.new("https://#{@host}", headers:) do |conn|
    conn.request :authorization, 'Passcode', passcode
    conn.request :multipart
  end

  http.post(@path, form_data)
end

HTTP (the gem)

For HTTP The Gem, despite being a separate gem at least the multipart form support is tagged as a dependency so it’s available by default. It’s a slight step down from the others in that you need to juggle two different classes while assembling the form data, but otherwise it’s nearly identical to Faraday. On the plus side, it’s smart enough to handle a String file part as well as an IO, and the HTTP gem chained API calls are always pleasant to work with.

def post_http(criteria, transactions_csv)
  form = HTTP::FormData::Multipart.new({
    criteria: HTTP::FormData::Part.new(
      criteria.to_json,
      content_type: 'application/json'
    ),
    transactions: HTTP::FormData::Part.new(
      transactions_csv,
      content_type: 'text/plain',
      filename: 'transactions.csv'
    )
  })

  http = HTTP
    .auth("Passcode #{passcode}")
    .headers('filetype' => 'STD')

  http.post("https://#{@host}#{@path}", form:)
end

HTTPX

Finally, HTTPX: no separate gem for multipart, it’s supported out of the box. No wrapper classes, just a nested hash. Same readable chained API as HTTP, though the with method feels incongruous.

def post_httpx(criteria, transactions_csv)
  form = {
    criteria: {
      content_type: 'application/json',
      body: criteria.to_json
    },
    transactions: {
      content_type: 'text/plain',
      filename: 'transactions.csv',
      body: transactions_csv
    }
  }

  http = HTTPX
    .plugin(:auth)
    .authorization("Passcode #{passcode}")
    .with(headers: {'filetype' => 'STD'})

  http.post("https://#{@host}#{@path}", form:)
end

Thoughts

Net::HTTP is a gnarly mess of an API, and depends on another gem so I can’t even really say “it’s available and built-in.” Do not like.

Faraday is a bit verbose, but building the multipart section is highly orthogonal and easy to see what’s going on.

HTTP is very similar to Faraday for multipart - simpler in that there’s only one Part class, but needs a Multipart wrapper at toplevel instead of just a Hash. I’m a huge fan of the rest of its API for actually making requests, though, and is my overall preference for overall usability and readability. (Also, it’s pretty straightforward to write a wrapper that builds HTTP FormData objects out of the same simple hash HTTPX wants.)

HTTPX is a bit worse API than HTTP (with(headers: ...) being nonobvious) but gets bonus points for being pure Ruby. Lack of support by vcr for mocking tests (as of this writing) is a big bummer though - if not for that it’d get my top nod.

Git Archaeology

2020-04-22T00:00:00+00:00

Recently at work we passed a major milestone on our codebase, and I wanted to see if I could run some analysis over time on authorship and see how long some early contributors’ work stuck around in the product.

A bit of experimentation and random googling left me with these three scripts.

The first is dig.sh, which will accept the path to the git repository to analyze (because I’m making a few dozen files, and don’t want to dirty the primary repo), a date to process, and optionally a commit to analyze. If a specific commit is not provided, it’ll look up the first commit before the provided date.

How this looks in practice is something like ./dig.sh ../my_repo 2011-04-01. Because we run a monorepo now, and have merged a few external repos together, this sometimes didn’t pick up a proper mainline commit, I occasionally had to come back on a manual pass and touch it up: ./dig.sh ../my_repo 2011-04-01 4dbdd82.

The actual meat of the script is the last 3 lines, which gets a recursive directory listing as of the commit in question, filters to files that match a provided pattern, runs git blame across all of them, and counts the number of entries for each author.

#!/bin/bash

# ARGV: repo_path, date, commit

export DIR=`pwd`
cd $1

export DATE=$2

if [ -n "$3" ]; then
  export COMMIT=$3
else
  export COMMIT=$(git rev-list -1 --before="$DATE" master)
fi

export PATTERN='\t(app|lib)/.*\.(rb|js|erb|haml)$'

git ls-tree -r $COMMIT | egrep -o "$PATTERN" | while read f; do
  git blame -w -M -C -C --line-porcelain $COMMIT -- $f;
done | egrep -a '^author ' | sort | uniq -c > $DIR/$DATE.txt

Next up is dig_all.sh, which is just a barebones orchestration script. Like the above, you provide a repo path and branch, and it will sequentially run through history. Due to the growth in the repo over time, early years would take under 10 minutes/month to process, but months this year were running over 3 hours. Thus the start/end year arguments, so I could run a few scripts simultaneously - it’s impressively not disk-bound on my mac, I did some testing and could run 3 at once without significantly slowing the runtime.

#!/bin/bash

# ARGV: repo_path, HEAD_branch, start_year, end_year

for year in $(seq $3 $4); do
  for month in $(seq -w 1 12); do
    export DATE=$year-$month-01
    export COMMIT=$(cd $1 && git rev-list -1 --before="$DATE" $2)
    if [ -n "$COMMIT" ]; then
      echo $DATE $(uptime | cut -d' ' -f 1)
      ./dig.sh $1 $DATE $COMMIT
    fi
  done
done

Once we’ve got all the data, I wanted to make a bar chart race out of it. massage.rb to the rescue, collating the raw data, and then outputting a CSV.

#!/usr/bin/env ruby

require 'csv'
require 'pp'

data = {}
Dir['*.txt'].each do |file|
  File.readlines(file).each do |line|
    date, _ = file.split('.')
    count, name = line.strip.split(' author ')

    data[[name, date]] = count
  end
end

names = data.keys.map(&:first).uniq.sort
dates = data.keys.map(&:last).uniq.sort

out = []
out << [''] + dates.map{|d| Date.parse(d).strftime("%b %Y") }
names.each do |name|
  shortname = name.match(/^([^ ]+..)/)[1]
  out << [shortname + '.'] + dates.map{ |d| data[[name, d]] || "" }
end

puts out.map(&:to_csv)

Et voila (names anonymized to protect the guilty):

Git Rebase by Example

2017-06-20T00:00:00+00:00

Today at work, I determined that my current work in progress branch was going to want to be merged into production ahead of our next scheduled merge/deploy of master. To make it easier to merge into our production branch for a hot-fix deploy, I want to have the branch based off production instead of master.

So, I could do this manually:

$ git reset HEAD~ # move HEAD commit back to staging
$ git stash save # move uncommited changes to stash
# repeat until all my changes are in stash
$ git checkout production
$ git stash pop
$ git commit ...
# again, repeat until complete

However, this is a pain in the neck because it’s all manual, and it loses the commit metadata (timestamps, but also author info if you’re moving commits from multiple people). Also, because I prefer small merges to big merges, I’ve merged master into my branch a few times, and that means deciding to commit or wipe commits.

Instead, let’s git do all the work, and use git rebase to move my commits to their new home, and also filter out those merges.

Before getting started, we should backup my branch, by just doing a git checkout -b wip-rebase. Then, to confirm exactly what my changes were we can run git cherry -v origin/master. This will list all the commits on the current branch that are not on the specified branch.

Then, I want to start by getting rid of all my extra merges, and filter the branch to just these outbound changes. For this we’ll do an interactive rebase: git rebase -i acfad~, where acfad is the commit hash at the top of my change list (you can also just use a visual git browser or whatever to find the base commit for your changes). This will pop open an editor, and let me decide exactly how I want to handle each commit.

This is a powerful tool, and all its options are beyond the scope of this article, but it will let you do anything from editing a commit message, to combining or reordering commits, to executing shell commands between specific commits.

Here, we’re just dropping commits, so we can scroll down the editor here and any commit that wasn’t listed in our git cherry output we can change the pick to drop. Or, even easier, delete the line.

Now save and quit the editor, and git will do its job. We can verify with another git cherry -v origin/master and compare output now with earlier output. Note that our oldest commits are unchanged, but the commit hash for the newer ones is changed. This is because the hash of a commit depends on the hashes of its parents, and by deleting merged commits from our history we are of course unable to use them as parents anymore. Also, looking directly at the git log we can see that deleting all commits from one branch of a merge commit will also delete the merge itself. For our purposes, as long as the commit messages in both git cherry outputs are the same, then we’re doing fine.

Now that the busy work is behind us, we can get to the actual goal of moving the commits. The command for this will be git rebase --onto origin/production acfad~ – again we’re referring to the parent of the first commit in our branch.

This is the part that works exactly as I described above, git is “rewinding head to replay your work on top of it”. As with any merge (applying a commit in a changed context) you can expect to see some conflicts and can resolve them as usual. However when done, instead of commiting we’ll move on with git rebase --continue. If at any time things get confused and you make a mistake, git rebase --abort will roll you back to your branch as it was before rebasing.

Once you’re done with the conflicts, git will finalize the rebase behind the scenes, and leave you back at your current branch having updated the root parent, and touched all the commits on the way down. We can confirm this with a final git cherry -v origin/master.

Now is a good time to run specs (or let CI work that for you) and do whatever other testing you need to confirm everything functions as it should.

Building an Object Graph in Rails

2015-06-04T00:00:00+00:00

I was needing to do some object cleanup in our rails app the other day, and purge some malformed objects, so I put together a quick script using some ActiveRecord reflection to walk the object chain.

# Squelch SQL logs, if you're running from rails console
ActiveRecord::Base.logger.level = 1

# Output formatters. Trailing ':' on both, plus the staggered indent
# of 4n and 4n+2 makes the output valid YAML, if automated analysis
# is called for.
def puts_node(node, indent)
  puts "    "*indent + node.class.name + "#" + node.id.to_s + ":"
end
def puts_assoc(assoc, indent)
  puts "    "*indent + "  " + assoc.to_s + ":"
end

def puts_tree(node, seen=[], indent=0)
  puts_node(node, indent)

  unless seen.include? node
    # seen maintains a list of nodes to avoid mutual recursion
    seen << node
    node.class.reflections.keys.each do |assoc|
      # To see all locations an object is referenced, get rid of the
      # "- seen" here. I only cared about which objects were present
      # anywhere in the tree, so this was fine.
      associated = Array(node.send(assoc)) - seen
      next if associated.empty?

      # Print an entry for the association, then recurse
      puts_assoc(assoc, indent)
      associated.each do |subnode|
        puts_tree(subnode, seen, indent+1)
      end
    end
  end

  # Also outputs a footer listing all seen objects once, take it or leave it
  if indent.zero?
    puts
    seen.each {|node| puts_node(node, indent)}
  end

  nil # return nil to avoid flooding terminal in rails console
end

# Usage: call with the root node for the object graph
puts_tree(User.find(42))

Worked like a charm, and made it easy to compare my bad object with other good ones. Just be careful of any global objects (a common shared subscription package, for instance, that has_many :users) that could lead to traversing your entire database, or logging associations that could overwhelm your output on older or heavily used objects. Subtracting a blacklist from reflection keys on line 18 would do the trick there.

No More Bundle Exec

2012-09-06T00:00:00+00:00

Update 2023: Just use direnv with layout ruby.

Bundler is pretty darn good. Installing all your gems globally sucks. bundle install --path does a great job of fixing that but it means you need to bundle exec any shell commands you want to run, which again sucks. There are lots of attempts to fix this, but they’re all fairly convoluted.

I’m a fan of simpler solutions wherever possible. I use zsh as my shell, which has a handler you can hook into if the command you’re trying to run is not found. It’s a simple matter to hook that into a custom shell script from your ~/.zshrc:

function command_not_found_handler() {
    ~/bin/command-not-found $*
}

I know bash supports this kind of handler (Ubuntu uses it to provide command helpers for not-yet-installed programs) but I don’t know the exact details. Alas, my favorite shell ever, fish, only provides the executable to its corresponding helper, so while it can suggest an alternate command, it can’t auto-correct it.

My script happens to be in Ruby, but it could just as easily be a standard shell script as all I’m doing is some file existence tests:

#!/usr/bin/env ruby

# ARGV is the entire command we wanted to run, but we
# really only care about the actual executable for fallbacks
command = ARGV.first

def run(cmd)
  $stderr.puts "Running #{cmd.inspect} instead"
  system(cmd)
end

case
when File.exist?("./.bundle/config") && File.exist?("./bin/#{command}")
  run("bundle exec #{ARGV.join(' ')}")

else
  exit 127
end

Now, as long as you’re being sure to bundle install --binstubs it should Just Work. And because it only functions if you’re in a directory that’s been bundled, you don’t run into the security risks that you would by trying to get ./bin added to your $PATH directly.

Lastly, the case statement instead of an if is a bit redundant in the simple case above, I’ve actually got a few more filters for things like isolate and git - don’t forget to quote anything that might need space literals:

# Paste git repo url to clone it
when command =~ /^git(@|:\/\/).*\.git$/
  run("git clone #{command.inspect}")

# paste compressed url to download+extract it
when command =~ /^(?:ftp|https?):\/\/.+\.t(?:ar\.)?gz$/
  run("curl #{command.inspect} | tar xzv")

when File.exist?("./tmp/isolate/ruby-1.8/bin/#{command}")
  run("rake isolate:sh['#{ARGV.join(' ')}']")

Joining Git Repositoires

2012-05-15T00:00:00+00:00

At work, we provide an API for our app and maintain web-based documentation for said API. We originally had the documentation in a separate git repo, but as it makes much more sense to maintain the docs directly alongside the code it documents we wanted to merge the two repositories. This was done in two steps.

Moving files

First, we need to prep the docs repo such that the content is in a reasonable location, rather than the root directory. This is done fairly easily with git filter-branch (zsh):

% mkdir -p doc/api_docs
% git checkout -b for_transplant # work in a branch for safety
% for file in <files/dirs to move>;
    do git filter-branch --tree-filter \
      "test -e $file && mv $file doc/api_docs || echo skip" \
      HEAD;
    done

This goes through out commit history, and runs through each commit moving old content into a subdirectory. It’s slow in that it does a full history pass for each file, but I didn’t care to figure out how to move everything except the docs directory.

Merging the repos

First, for clarity, let’s make a new branch based off of the original commit in our destination repo:

% git log --oneline | tail -n 1
e7c9feb Initial commit
% git checkout e7c9feb
% git checokut -b doc_import

First, prepare the main repo for the incoming transplant by cloning a bare copy (git refuses to pull an external repo into a normal checkout).

% git clone --bare git@github.com:example/myapp.git myapp-bare

We can then pull in the commit objects from the other repository:

% cd myapp-bare
% git fetch -f ../api_doc_site for_transplant:api_docs

The -f tells git to ignore the different initial commits, and then we explicitly specify the remote and local branches to move commits to. You might need to resolve a merge conflict at this point, if for example both repositories committed a different .gitignore in their initial commit.

We can now go back to our regular repo and pull those branches in:

% cd ../myapp
% git remote add bare ../myapp-bare
% git pull bare api_docs

Finally, merging that branch into master gets things all up-to-date, and we can start unified work while maintaining the full original history of the documentation.

Parsing JSON in SQL

2011-11-19T00:00:00+00:00

The Problem: You have a database column with some data serialzed as JSON in it that you’d like to pull out into its own column to index it.

The Solution: Run a data migration to pull the value out. Table has 5 million rows and you don’t want to round trip all that data through ActiveRecord? Just parse the json directly with some SQL:

def json(key, field='params')
  key_json = "\"#{key}\":"

  # key start/end locations, including ""
  k_a = "LOCATE('#{key_json}', #{field})"
  k_z = "LOCATE('\"', #{field}, #{k_a}+1)" # this is terminating "

  # is there a space after colons?
  spad = "IF(LOCATE('\": ', #{field}), 1, 0)"

  # is value a string?
  val_string = "LOCATE(CONCAT('#{key_json}', IF(#{spad},' ',''), '\"'), #{field}, #{k_a})"
  qpad = "IF(#{val_string}, 1, 0)"

  # value start/end locations, excluding "" if present
  v_a = "(#{k_z}+1 + 1 + #{spad} + #{qpad})" # 1 for colon, spad for optional space, qpad for possible quote

  end_if_string = "LOCATE('\"', #{field}, #{v_a})"
  end_if_not_string = "IF(LOCATE(',', #{field}, #{v_a}), LOCATE(',', #{field}, #{v_a}), LOCATE('}', #{field}, #{v_a}))"

  v_z = "IF(#{val_string}, #{end_if_string}, #{end_if_not_string})"

  value_string = "SUBSTRING(#{field} FROM #{v_a} FOR (#{v_z} - #{v_a}))"
  "IF(#{k_a}, #{value_string}, NULL)"
end

up do
  execute "
    UPDATE model_table
    SET status = #{json('status')}
  "
end

The generated sql looks pretty gnarly but mysql ran through it stupidly fast. I shudder to think how long it’d take activerecord to load and update each record individually.

Isolating Rails

2011-01-19T00:00:00+00:00

Rails 3 is now very friendly with regards to dropping Bundler support, only loading it if it’s installed and a Gemfile exists. Since Isolate is so awesome, I thought I’d just drop a quick script in here to convert an existing Rails app to use Isolate instead of Bundler.

#!/usr/bin/env ruby

require 'fileutils'

File.open("Isolate", 'w') do |isolate|
  File.readlines("Gemfile").each do |line|
    next if line =~ /^\w*#/
    next if line =~ /^source/
    next if line =~ /^\w*$/

    line.sub!(/, :require.*(,|$)/, '\1')
    line.sub!(/^([ \t#]*)group/, '\1environment')
    
    if line =~ /:git/
      line = "# Don't use git, build it as a gem\n# " + line
    end

    isolate.puts line
  end
end

File.open("config/boot.rb", 'a') do |boot|
  boot.puts
  boot.puts("require 'isolate/now'")
end

FileUtils.rm('Gemfile')

This should convert an existing Gemfile to an Isolate file, remove the Gemfile (so that rails won’t try to load it), and update the app to load Isolate appropriately.

I’m basically only guessing that the group/environment setup is correct, so if anyone has any corrections to this let me know and I’ll update it.

Migrating Disqus

2009-12-22T00:00:00+00:00

In changing this blog over to jekyll, my urls changed (there’s now a trailing slash). Easy enough to tell google about it, just set up redirects, but there’s no easy way to tell Disqus about it so my comments migrate over.

The good news is that it’s pretty straightforward using their API, the only bad news is that I can’t delete the new threads auto-generated for the new urls, so I’m just moving them out of the way.

I’m using the HTTParty gem to wrap API access, like so:

require 'rubygems'
require 'httparty'
require 'json'

class Disqus
  include HTTParty
  base_uri 'disqus.com'
  format :json

  def initialize(key, version='1.1')
    @key = key
    @version = version
  end

  def auth
    {:user_api_key => @key, :api_version => @version}
  end

  def get(action, opts={})
    result = self.class.get("/api/#{action}/", :query => opts.merge(auth))
    result["message"]
  end

  def post(action, opts={})
    result = self.class.post("/api/#{action}/", :body => opts.merge(auth).to_params)
    p result
    result["message"]
  end
end

Do note that I’m adding trailing slashes to the api calls to avoid a redirect. Doesn’t matter for the GET, but the redirect on POST was causing issues.

With this in hand, I’m grabbing my forum, looping through the threads, and renaming any that have comments (a whopping 3 of them).

key = "secret" # get yours at http://disqus.com/api/get_my_key/
disqus = Disqus.new(key)

forum = disqus.get(:get_forum_list).first # I just have one
forum_api_key = disqus.get(:get_forum_api_key, :forum_id => forum["id"])

start = 0 # manual pagination, eww
loop do
  threads = disqus.get(:get_thread_list, :forum_id => forum["id"], :start => start)
  break if threads.empty?

  threads.each do |thread|
    posts = disqus.get(:get_thread_posts, :thread_id => thread["id"])
    next if !posts.empty?
    
    target_url = thread["url"]+"/"

    # There's another thread in the way...
    if other_thread = disqus.get(
      :get_thread_by_url, 
      :forum_api_key => forum_api_key,
      :url => target_url
    )
      # free up the url we want to use
      disqus.post(
        :update_thread,
        :forum_api_key => forum_api_key,
        :thread_id => other_thread["id"],
        :url => target_url + 'old'
      )
    end

    # update thread url
    disqus.post(
      :update_thread,
      :forum_api_key => forum_api_key,
      :thread_id => thread["id"],
      :url => target_url
    )
  end

  start += 25
end

Et voilà, old comments are in the right place now.

Jekyll: Custom Liquid Tags

2009-12-04T00:00:00+00:00

The base install of Jekyll at the moment doesn’t let you run any arbitrary ruby code. This is so that they can use it for github pages and not need to worry about making a super-secure sandbox just to generate some HTML.

Unfortunately, that means we’re out of luck for creating custom liquid filters. The most annoying deficiency for me is tags. The way the default liquid map filter works isn’t friendly with @site.tags, so to generate my Tags page I had to do some really crazy stuff with capture:

<div id="articles">
  <table>
    {% for tag_ in @site.tags %}
      {% capture tag %}{{ tag_ | first }}{% endcapture %}
      <tr><th>{{ tag }}</th>
          <th><a name="{{ tag }}" class="anchor">&nbsp;</th></tr>
      {% for post in @site.posts %}
        {% if post.tags contains tag %}
          <tr><td>{{ post.date | date: '%b %e, %Y' }}</td>
              <td><a href="{{ post.url }}">{{ post.title }}</a></td></tr>
        {% endif %}
      {% endfor %}
    {% endfor %}
  </table>
</div>

Fortunately, it wasn’t to hard to make a fork, and in my fork I added a super simple code loading option. Now, I can add a quick extension in _lib/filters.rb like so:

module Jekyll
  module Filters
    def keys(input)
      input.keys
    end

    def tagged(input, tag)
      input.select{|post| post.tags.include? tag}
    end
  end
end

Now tags.html looks like this:

<div id="articles">
  <table>
    {% for tag in @site.tags|keys|sort %}
      <tr><th>{{ tag }}</th>
          <th><a name="{{ tag }}" class="anchor">&nbsp;</th></tr>
      {% for post in @site.posts|tagged:tag %}
        <tr><td>{{ post.date | date: '%b %e, %Y' }}</td>
            <td><a href="{{ post.url }}">{{ post.title }}</a></td></tr>
      {% endfor %}
    {% endfor %}
  </table>
</div>

There’s a bit of trickery there that liquid doesn’t document very well on lines 3 and 5 - in the second half of the for block you can chain filters on the collection you’re iterating over. The short format used is something along the lines of collection|filter:arg,arg,arg|filter...

Similarly, I had some ugly code in my regular archive page to group by year and put headings in:

{% for post in site.posts %}
  {% unless post.next %}
    <tr><th>{{ post.date | date: '%Y' }}</th><th>&nbsp;</th></tr>
  {% else %}
    {% capture year %}{{ post.date | date: '%Y' }}{% endcapture %}
    {% capture nyear %}{{ post.next.date | date: '%Y' }}{% endcapture %}
    {% if year != nyear %}
      <tr><th>{{ post.date | date: '%Y' }}</th><th>&nbsp;</th></tr>
    {% endif %}
  {% endunless %}

  ...
{% endfor %}

Now, a few extra liquid filters later, it looks like this:

{% for post in site.posts %}
  {% if post|last_of_year? %}
    <tr><th>{{ post.date | date: '%Y' }}</th><th>&nbsp;</th></tr>
  {% endif %}
  
  ...
{% endfor %}

If you want to get easy extensions in your own project, rather than maintaining Yet Another Jekyll Fork, please vote up my merge request on github.

As a side note, blogging about liquid is a pain. The least pain I’ve found so far is to use liquid to output the leading open brace for all tags. Looks like garbage in my text editor, but it gets the job done:

{{'{'}}{ post.title }}

Blogging about blogging about liquid (as above) I leave as an exercise to the reader.