Why

A few years ago, I moved my webserver (what runs this site, my PDS, and lots of other services) to CentOS Stream back when that was still viable, then Rocky Linux as the new replacement when Stream sort of died off. Is it perfect, is it the right way to do things? No, but it’s what I prefer at this point. ¹

The Problem: part one

The default Bluesky PDS is intended for being deployed on Ubuntu 20.04/22.04. It will fail to deploy on anything other than those two distributions (even 24.04 which also works but that’s neither here nor there).

Solutioning a problem

While I was in the testing phase of deploying my PDS, I was using an Ubuntu server that I could successfully deploy using the installer. I already use Caddy though, so the compose file having Caddy dockerized is not ideal. Fixing that was a pretty easy cleanup to get a proper pds.env file out of it, since there’s no canonical source outside of the installer. Now I had a relatively standard PDS deployment using Docker, but using the Caddy service I use for everything else on my server.

Best Practices?

I have opinions on the way Bluesky PBLLC has made the systemd unit for the PDS in GitHub. It deploys as a oneshot, so journalctl is not super useful, systemctl will never show it as running, and it is generally just hard to set it as a dependency or a dependent of any other services due to it being oneshot.

Fixing this took me on a rabbit hole / detour on how to do it “better”.²

Enter Sandman Podman

I had been watching my coworkers explore Podman a bit at work, and looking into it, it had the perfect thing for my needs, the ability to “systemd-ize” a container workload in a way that made sense for both the container, and for systemd. I also took this opportunity to set up litestream for backups as well.

Now that everything’s mostly stable, I can finally document all my configuration for anyone else to use.

Under the hood it’s using Quadlets, so all of the unit files we’ll create in /etc/containers/systemd.

First we create a pod to contain everything and publish the port:

[Pod]
Volume=pds.volume:/pds
PublishPort=127.0.0.1:3000:3000

Then we slot in a volume:

[Unit]
Description=Bluesky PDS Volume

[Volume]
Label=app=pds

Now that we’ve got everything organized, we can bring in the PDS container itself:

[Unit]
Description=Bluesky Personal Data Server service
Before=caddy.service

[Container]
Label=app=pds
Image=ghcr.io/bluesky-social/pds:0.4
AutoUpdate=registry
Pod=pds.pod
EnvironmentFile=/opt/pds/pds.env

[Install]
WantedBy=multi-user.target default.target

I make this container boot before my Caddy service which brings up all the associated resources as part of the quadlet.

We then make the appropriate changes to the Caddyfile:

[...]
*.at.jkp.sh, at.jkp.sh {
        tls {
                on_demand
        }
        reverse_proxy http://localhost:3000
}
[...]

Now I get autoupdates for my PDS without needing to use the watchtower container along with logs that feed right into systemd. Additionally, my PDS storage itself ends up abstracted away for me in a podman volume, rather than being scattered in a random non-standard place on my VPS. In short, it does basically exactly what I want it to do, just the way I want it, with some learnings along the way.

I wanted to go the extra mile though and make sure it matched the theme of the site. After a bit of digging I discovered that SVG supports media queries, so all the fun light/dark mode detection we can do with CSS on the web, also works in your favicon (at least assuming you’re using a browser that supports SVG favicons. ¹)

After vectorizing my work, I added some classes to the paths, put it all in a single SVG like so:

    [...]
    <path d="m213 87h138v140h-89v86h-49zm49 90h39v-40h-39z" fill="#eaedf3" fill-rule="evenodd"
            class="dark"></path>
    </svg>
    <style>
        @media (prefers-color-scheme: light) { .dark { display:none; } }
        @media (prefers-color-scheme: dark) { .light { display:none; } }
    </style>

All I have to do is treat it like any other SVG favicon and it just works.

Sadly, it doesn’t work in Safari, so Safari will always have a light mode favicon. Not the end of the world, just a bit of a bummer.

You open up Netscape Navigator, type in http://www.nytimes.com and your DSL connection chugs for a second as more kilobits than a computer from the 80s would know what to do with fly into your shiny new Dell.

After all of that though, you know you’re at the New York Times’ website, since you went there right from their physical paper. There wasn’t a middleman, or anyone telling you that they were in fact the Gray Lady, you just did a little bit of leg work.

Time Warp

Now it’s 2010, and you want to see what the Times has to say about your favorite political topic this week. You could head to their website, grab the paper from the guy on the corner, or for your first time you could open up this new website called Twitter, and there’d just be a handful of words written by the Times, summarizing a story, and you wouldn’t have to go anywhere. How do you know if it’s actually the Times? Well Twitter thought of that for you, and they added a little blue checkmark next to their account for you. ¹

Now you can trust that @nytimes is the Times on Twitter, right? Right? Well for a while you used to be able to, but eventually it all fell apart. And in my opinion, that’s a large part of why Bluesky is popping off (or “has the juice” as the kids say) as Twitter / X slowly begins the doom spiral that any site sees in the future as they become no longer relevant.

The “problem”

So Bluesky / ATProto is created to decentralize Twitter (originally, then is spun off entirely pre Musk acquisition), and a primary tenet of both Bluesky PBLLC (the corporation) and ATProto (the protocol itself) is to ensure that long term, the ecosystem is protected from any actor in the space from turning hostile. There’s no “special sauce” that Bluesky PBLLC has (other than chat/DMs but we’ll set that aside for the time being) that is exclusive to them.

Every part of the ecosystem then needs to be designed to be hostile-proof, from where user data lives, to how users consume and create data. This goes as far as how user identities are stored and referenced. There’s been a lot of work put in to ensure that in the long run, you are totally in control of your identity, not Bluesky PBLLC.

So the “problem” here is that Bluesky doesn’t have a concept of verification, at least not the way Twitter had. There is no “central arbiter” of trust, because how can you have a central arbiter of trust when the entire ecosystem is designed to avoid that single point of failure?

Instead all the “verification” is done with domain names, and there’s been more than a few folks upset / weirded out / annoyed that there isn’t as much verification as there was on Twitter.

The more I think about it though, the less I think it is a problem, and more us as a social media using society not remembering the olden days.

Why this matters

We’ve spent the last 10-15 years forgetting that the web is a protocol, not a platform. Mike Masnick touched on this in Protocols, not Platforms (and in fact is now on the board of Bluesky PBLLC). In the example of the 90s, there wasn’t a platform saying whether or not the NYTimes on the web was “verified”, there was just a little bit of legwork to check and crosscheck that indeed this website was theirs. There was no middleman, just a protocol.

However, in the next time warp, we’re instead relying on these centralized parties to decide who is and isn’t verified, and shifts us to treating the web as a platform, not a protocol. People don’t browse the web, they use Facebook, they use Twitter.

The concept of domain handles, verification, the whole nine on how Bluesky handles this is the first step on getting us as an internet-using society back to protocols.

A domain is the most ownership of anything you can have on the web. If a new ATProto social site opens up, you know that jack.is there is also me, just like you know it’s my Bluesky account, just like you know it’s the site you’re reading right now. I own my identity, not Instagram, or Twitter, or Bluesky, and I think that means something more than a little check in a circle that gives me the good feelings.

Emily’s method was great, but required hardcoding the atproto URI into the post itself. That’s a little hard to do with my deployment and post method, where the RSS feed is generated, and then that triggers an Echofeed (Not Sponsored) that posts to Bluesky.

Because of this, I wanted something better. Ideally, I could have it just find the right post without me having to worry about it. Thanks to the power of the Bluesky API, I’m able to do that!

The special sauce is right here in getPostAndThreadData. It runs a search for the Echofeed emoji I use for all my posts (📝), looks for my author DID, as well as the page URL that would have been posted. It then limits it to 1, and grabs that URI. Once we’ve got a URI, we then pass it to Emily’s code like normal along with some styling and design tweaks.

Is it perfect? Not yet, but now it’s just fine tuning some CSS styling (and also getting React to be chunked a little better).

const getPostAndThreadData = async (
  slug: string,
  setThread,
  setUri,
  setError,
) => {
  const agent = new Agent("https://public.api.bsky.app");
  const assembledUrl = "https://jack.is/posts/" + slug.toString();
  try {
    const response = await agent.app.bsky.feed.searchPosts({
      q: "📝",
      author: "did:plc:cwdkf4xxjpznceembuuspt3d",
      sort: "latest",
      limit: 1,
      url: assembledUrl,
    });
    const uri = response.data.posts[0].uri;
    setUri(uri);
    try {
      const thread = await getPostThread(uri);
      setThread(thread);
    } catch (err) {
      setError("Error loading comments");
    }
  } catch (err) {
    setError(err);
  }
};

In the words of Brennan Lee Mulligan: “GET IN THE COMMENTS!”

Tinylytics

My current provider of some of the juicy info (and provider of the fancy kudos) buttons. It gets the job done, but I’d love to see more info.

Goatcounter

Offers more information, has a sweet craigslist-era aesthetic, and could be self hosted if I want.

Umami

My current front runner in all honesty. Has everything I want, and then some, and is being selfhosted by me right now. Does what I need it to do and I think will win.

If I switch however, I’m going to have to reimplement my upvote buttons somehow but that’s a problem for future me.

Update

Umami self-hosted won.

Now I can have my blog and my PDS for Bluesky all in one place. With that said, I still haven’t moved my Bluesky account to my PDS just yet, but it’s ready to go for when I’m ready to push the button.

The Vibes

Twitter, and I mean Twitter before it turned to hell in a handbasket, had vibes from various circles. You had Tech Twitter, you had Weird Twitter², etc. Even with all of that though, the vibes and memes were still often in sync because everyone was using the same app, and had at least some of the same trending topics they were aware of. Milkshake Duck, bean dad. There was always a main character, and often you didn’t really want to be the main character.

For better or for worse, Mastodon doesn’t always have those vibes in my opinion. It’s often a little bit stuffy. I’ve found Tech Twitter there in many ways, but Weird Twitter never seemed to migrate there, even when it was first popping off before Bluesky was a thing.

Bluesky though has all your circles, and is picking up at a much faster than Mastodon ever did. You’ve got your dril, you’ve got your darth, you’ve got your WeRateDogs. Having one of those on Mastodon was a challenge, and I don’t think there was ever a time even if all three of those accounts had an Mastodon / AP presence at all, were they ever posting at the same time.

So the vibes are there for Bluesky like Old Twitter the way Mastodon just isn’t. But what about the tech?

The Tech

Now I was an avid ActivityPub defender (and still am to some extent) when ATProto originally launched, and was feeling like Bluesky was too “corporate”, and kind of didn’t look into it much, and only begrudgingly joined.

Only relatively recently did I do a deeper dive into the pros and cons of ATProto, and I’ve sort of turned around on how I feel about the two.

I tried to run my own Mastodon server one time, and to be honest it Kinda Sucked. Mastodon simply doesn’t scale well down to one or two users. It’s fine if you’re on a larger server as you can share the load (shoutout Adam for omg.lol and the associated Mastodon server social.lol), but the effort I had to go to to even try to get Mastodon working on a server that already had services on it and it was not fun.

Mastodon also has some odd protocol design decisions that make life harder for small web, which seems oddly counter-intuitive. The most obvious one (and one that I’ve talked about in the past)³ is the fact that any rich link previews for any posted link (the OpenGraph info) is fetched by every single ActivityPub instance that sees that post. In other words, if you’re followed by 1,000 users on 1,000 separate Mastodon / ActivityPub instances, and you post a link to your neat little blog, in the span of a few seconds, you’re going to get 1,000 requests to your website. Is this necessary? Not really, but it’s the way the protocol has been designed, and continues to stay that way. There’s even a cute name for it, a Mastodon stampede⁴. This just isn’t sustainable.

ActivityPub is also aggressively chatty when it comes to the actual communication between servers. Migrating between servers is basically just like shoving a HTTP 302 at your followers and hoping for the best. You don’t get to move your posts, just your followers and following. Your username doesn’t stay the same either.

ATProto on the other hand is built on layers of abstraction, so your handle is just a reference to a permanent identifier. Change your handle, no problem. Want to move your account entirely from one PDS (Personal Data Server) to another? No problem. The protocol has support for it from the beginning. Rather than sending a bunch of pushes when you post something, your PDS just adds a new post to your PDS, and the network aggregates and displays to clients. It just feels cleaner. Spinning up a PDS compared to a Mastodon server is like night and day. I was up and running with my self-hosted test user in about 10 minutes and that was with having to tweak the install script slightly to meet my preferences.

The Summation

So which one should win? It’s not that one can win and the other shouldn’t, they each have different design goals. However, I’ve come around on which one is (at least currently), more scalable, and I think it’s ATProto. From a social perspective, Bluesky seems to be winning the normal user demographic, which Mastodon always had trouble getting users to adopt.

As for my self-hosted PDS, I’m still deciding whether or not to move my current Bluesky hosted account to it. There’s currently no import path back to Bluesky (not because of protocol reasons, but because of giant account reasons), and that makes me feel like this is a permanent commitment if I do. We’ll see I guess.

Author Attribution on Fediverse

Thanks to Robb posting it, I now have my fancy fediverse author tags configured, so it shows attribution properly when my post is posted.

It’s simple, but it’s neat.

Tri-state dark mode selector

And I don’t mean the tri-state area. ¹

The Astro them I use, Astro Paper supports both light and dark mode. However, once you click the selector, it switches from using system selected (prefers-color-scheme) to the user selected option and doesn’t allow a way of switching back. I didn’t love this, so I added a third option.

Now the theme picker rotates between system, light, and dark. Pick whichever theme suits your fancy, no matter what your operating system tells your browser. Or pick system and let your operating system be the boss. You decide.

In other words, sorry for your RSS reader getting confused again if it sees all my posts as new entries again since the guid changed. We should be good now though, with less breaking of your readers.

For example, if you use Caddy, it might look something like this:

        @fedi {
                header_regexp User-Agent (http\.rb/\S+\s\(Mastodon|Friendica|Pleroma\s|Akkoma\s|Misskey/|gotosocial)
        }

        # do something with the detected user agent that's cache-y

Since this blog is static, I just tank the Mastodon hits as necessary, but something to keep in mind if you’re hosting your own thing.

It’s time for the regularly scheduled blog updates.

Yesterday (metaphorically speaking)

The prior setup was using Hugo and deployed to either Netlify, Cloudflare Pages or my own box. This was great, but after thinking through the Small Web ™️ movement some more I thought it made more sense to move to something in-house.

Today

The site now uses Astro and deploys right to my server. For deploying, it’s a two-step process. All commits on main on GitHub are checked. After everything checks out, I can trigger a release in GitHub to deploy to my server.

Here’s part of one workflow:

name: Check pushes
on:
  push:
    branches:
      - main

Here’s the other:

name: Deploy
on:
  release:
    types: [published]

DNS Sidenote

I’ve also switched to using Desec for my DNS hosting, it fits my needs nicely, is free, and enforces DNSSEC.

Hopefully the first of multiple blog posts to come.

Unfortunately, as I was finishing up this post, the post announcing Octopress 3.0 comes out. While I had known it was coming, I wasn’t expecting it to be so soon. Obviously this post will be less useful, but it should still be informative. I’ve included what I think will need to be modified to function on 3.0 at the bottom.

Table of contents

My Site Flow

• Local
• Private Github Repo
• Built and Deployed using TravisCI to a DigitalOcean droplet

I’ve got a drafts workflow using a couple tasks I found online¹ and then promptly modified:

## -- Misc Configs -- ##

drafts_dir      = "_drafts" # directory for draft files

# usage rake new_draft[my-new-draft] or rake new_draft['my new draft'] #
desc "Begin a new draft in #{source_dir}/#{drafts_dir}"
task :new_draft, :title do |t, args|
if args.title
title = args.title
else
title = get_stdin("Enter a title for your post: ")
end
raise "### You haven't set anything up yet. First run `rake install` to set up an Octopress theme." unless File.directory?(source_dir)
mkdir_p "#{source_dir}/#{drafts_dir}"
filename = "#{source_dir}/#{drafts_dir}/#{title.to_url}.#{new_post_ext}"
if File.exist?(filename)
abort("rake aborted!") if ask("#{filename} already exists. Do you want to overwrite?", ['y', 'n']) == 'n'
end
puts "Creating new draft: #{filename}"
open(filename, 'w') do |post|
post.puts "---"
post.puts "layout: post"
post.puts "title: \"#{title.gsub(/&/,'&')}\""
post.puts "comments: true"
post.puts "#placeholder"
post.puts "categories: "
post.puts "---"
end
end

# usage rake publish_draft
desc "Select a draft to publish from #{source_dir}/#{drafts_dir} on the current date."
task :publish_draft do
drafts_path = "#{source_dir}/#{drafts_dir}"
drafts = Dir.glob("#{drafts_path}/*.#{new_post_ext}")
drafts.each_with_index do |draft, index|
begin
content = File.read(draft)
if content =~ /\A(---\s*\n.*?\n?)^(---\s*$\n?)/m
data = YAML.load($1)
end
rescue => e
puts "Error reading file #{draft}: #{e.message}"
rescue SyntaxError => e
puts "YAML Exception reading #{draft}: #{e.message}"
end
puts "  [#{index}]  #{data['title']}"
end
puts "Publish which draft? "
answer = STDIN.gets.chomp
if /\d+/.match(answer) and not drafts[answer.to_i].nil?
mkdir_p "#{source_dir}/#{posts_dir}"
source = drafts[answer.to_i]
filename = source.gsub(/#{drafts_path}\//, '')
dest = "#{source_dir}/#{posts_dir}/#{Time.now.strftime('%Y-%m-%d')}-#{filename}"
puts "Publishing post to: #{dest}"
File.open(source) { |source_file|
  contents = source_file.read
  contents.gsub!(/^#placeholder$/, "date: #{Time.now.strftime('%Y-%m-%d %H:%M %z')}")
  File.open(dest, "w+") { |f| f.write(contents) }
}
FileUtils.rm(source)
else
puts "Index not found!"
end
end

For this to work using rake preview, you’ll need to modify your preview task slightly, add the flag --drafts to the jekyll call.

system "compass compile --css-dir #{source_dir}/stylesheets" unless File.exist?("#{source_dir}/stylesheets/screen.css")
jekyllPid = Process.spawn({"OCTOPRESS_ENV"=>"preview"}, "jekyll build --watch --drafts")
compassPid = Process.spawn("compass watch")

The Build Process

Personally, I don’t want to have to deal with building and uploading my site. Instead I use Travis-CI targeted to a specific Github repo branch, and some modified rsync scripts.

language: ruby
cache: bundler

rvm:
  - 1.9.3
branches:
only:
  - master

before_install:
  - echo -e "Host *\n\tStrictHostKeyChecking no\n" >> ~/.ssh/config
  - echo -e "$DEPLOY_PRIVATE_KEY" > ~/.ssh/id_rsa
  - chmod 600 ~/.ssh/id_rsa
  - eval `ssh-agent -s`
  - ssh-add ~/.ssh/id_rsa

script:
  - bundle exec rake integrate #move any stashed posts to source
  - bundle exec rake generate #generate static to /public
  - bundle exec rake deploy #rsync /public to server

Rsync Setup

You’ll need to configure your deployment account’s SSH private key in Travis-CI by replacing all new lines with \n, and then escaping that to \\n, so that it ends up being one line with \\n in between each original line.² (NOTE: Make sure the key was generated without ssh-agent running, and set a blank passphrase.) Next, go to your Travis-CI settings, create a new environment variable, name it whatever you want, I used $DEPLOY_PRIVATE_KEY, then paste in your one-line private key into the variable section. Now that it’s a environment variable in all Travis workers your builds will use, the before_install tasks will disable strict host key checking (requires input, which Travis doesn’t allow), and then echo the contents of DEPLOY_PRIVATE_KEY to id_rsa, then add that key to the ssh-agent, in my case, allowing the worker to connect using the user travis-ci to my droplet.

If the user you’re connecting with in your Rakefile’s settings isn’t the owner of your specific directories on your hosting server, rsync will probably fail to sync properly. In that case, you’ll need to add --omit-dir-times to rsync-args in the Rakefile.

After all of this, you should be good to go. If you’re using the master deploy method, push your site up to master, and it should build properly on Travis, and deploy to your rsync destination.

Octopress 3.0

For the most part, as far as I can tell, these modifications will mostly still work for 3.0. The drafts changes will no longer be needed anymore, as the Jekyll drafts workflow is now using a separate drafts folder. The .travis.yml will need to be tweaked slightly, as well as the _config.yml having an exclude added to keep Jekyll running on Travis-CI from trying to build the templates in /vendor, but other than that, this post is mostly forwards compatible to Octopress 3.0.

How massive is it? More than 700kB.

It’s kinda a lot, and as it is I had to tweak my Astro config to make Vite not warn about this:

  vite: {
    optimizeDeps: {
      exclude: ["@resvg/resvg-js"],
    },
    build: {
      chunkSizeWarningLimit: 768,
      // TODO find a better way to handle this
      // where "this" means the bsky react import chunk size
    },
  },

I wasn’t a huge fan of this for a few reasons. One, that was a lot of unneeded JS being delivered that I didn’t need, and I had to modify a warning setting which seemed less than ideal.

I had seen @atcute/client by Mary float across my feed, and poked at it a bit, but only after I figured out how to make unauthenticated requests with it did it become clear how much better it was going to make my bundles.

Only took a little bit of modifying the work I had done using Emily Liu’s example to instead use atcute:

const rpc = new XRPC({
  handler: simpleFetchHandler({ service: "https://public.api.bsky.app" }),
});

const agent = new Agent("https://public.api.bsky.app");
const assembledUrl = "https://jack.is/posts/" + slug.toString();

const response = await agent.app.bsky.feed.searchPosts(...);
const response = await rpc.get("app.bsky.feed.searchPosts", {
  params: {
    q: "📝",
    author: "did:plc:cwdkf4xxjpznceembuuspt3d",
    sort: "latest",
    limit: 1,
    url: assembledUrl,
  },
});

The unexpected benefit here is that @atproto/api didn’t actually have all the helper functions Emily needed for it to work, so there was a direct call to one of the xrpc endpoints. Now that everything just uses rpc.get, it’s a lot more consistent and easier to follow.

I did have to add a few //@ts-expect-error sprinkled around, but they’re all guarded by typeguards so I’m not too worried.

Now instead of 700kB, my BlueskyComments.js is only 10kB. A vast improvement.

Thanks again to Emily for posting their code to begin with, and Mary for an excellent suite of tooling that’s much more compact than the full ATProto API.

I wanted to like Atom, and I was one of those stubborn ones who had been putting off trying VS Code for probably years and continued to use Atom and putting up with the subpar performance, even on a relatively overpowered computer.

A few weeks back though, I decided to bite the bullet and give VS Code a fair try, and after using it for just a few days, I didn’t see myself going back to Atom anytime soon. There’s just so much more performance in Code compared to Atom (which confuses me because they’re both Electron), and included in that performance is more features included standard, instead of needing 100 extensions to get what I want out of it.

Looking at the writing on the wall, I don’t foresee Atom lasting too much longer, I haven’t seen a new feature added to Atom since right around the time Atom Teletype came out, and then that’s been pretty much it since then.

Sadly this means my theme that I’ve spent the better part of five years tweaking and modifying will end up slowly dying, but given that Pubster really isn’t primarily my work when you boil it down, I don’t feel too bad about it.

Hello Code

So what’s my Code look like now?

I’ve started using Iosevka as my programming font with it configured to more or less match Pragmatica Pro’s font features, and I’m really enjoying it so far. I’ve tried a bunch of different fonts, and had tried Iosevka in the past and wasn’t sure how I felt about it, but now that it’s got ligature support and decent hinting, I’m a fan. Additionally, for plain text editing (markdown, plaintext, commit messages), I switch over to Iosevka Sparkle which is designed for prose use using settings.json overrides for specific languages:

"[markdown]": {
  "editor.defaultFormatter": "esbenp.prettier-vscode",
  "editor.fontFamily": "Iosevka Sparkle, Iosevka"
}

As for the color scheme itself, I decided to give Monokai Pro a try, and I’m really loving it. The theme itself is nice, and the file icon themes it comes with are also fantastic and clear without being obnoxious. While it does cost a few bucks, for how nice it looks I think it’s definitely worth it.

The problem

As mentioned in Bluesky Comments, my posting workflow doesn’t allow me to know the Bluesky post ahead of time. So I thought, well I’ll just use the search API to find the post URL based on my echofeed post. That worked pretty well, but then I saw this skeet:

bryan newbold (⛱️ sabbatical edition) @bnewbold.net

please don't use the bsky "search" endpoints as a generic API or for automation. it is not designed for that, is an expensive waste/abuse of resources, and we will block by IP or user agent. searches should come from direct human queries in the moment

January 19, 2025 at 7:08 PM UTC

My first reaction was “Oh it’ll be fine this is just limited use and there was another post that says it’ll be fine for limited use”. It was fine, but after a day or so after Bryan’s post, I started getting CORS errors from my search function, and I figured that was that, and I needed to do this a better way, and maybe learn something along the way.

The research

I had seen Jetstream in my research prior to now, and figured some sort of database would probably be the best bet, where it’s just a single lookup, rather than every browser of the site having to do the search (even if it wasn’t blocked).

My initial thought was using Typescript for a server API, and there’s definitely still benefits to there, being a robust ecosystem of ATProto packages. I however wanted to Think Different™ so I decided to write it in Rust, both because I could, and also because I wanted to learn a bit of Rust “systems” programming, since I’ve been spending a lot of my day in Typescript.

After doing some very detailed Kagi searches ¹, I discovered Rocket and determined it was actually perfect for my needs. Simple declarations of handler functions, and it’s nice and type-safe. What’s not to like.

It’s Rust time

With just a bit of boilerplate code, I can spin up the API handler needed for the metadata info for my front-end.

#[get("/")]
fn index() -> &'static str {
    "at-comments database API server"
}

#[get("/<slug>")]
fn post_meta(slug: &str) -> Result<Json<models::Post>, NotFound<String>> {
    match db::post_meta(slug) {
        Ok(post) => Ok(Json(post)),
        Err(_) => Err(NotFound("Resource was not found.".to_string())),
    }
}

#[launch]
async fn rocket() -> _ {
    // setup websocket stuff
    tokio::spawn(post_listener::subscribe_posts());
    // setup server to respond
    rocket::build()
        .mount("/", routes![index])
        .mount("/", routes![post_meta])
}

There’s two important parts here in the main server, and I’ve labeled them A and B.

Part A: Metadata endpoint

We’ll say for the sake of argument that somehow the web server has the info for a given post already (since we’ll discuss the Jetstream and DB in part B), so part A is the thing that looks up info for a given post. One of the nice things about Rocket is that it has lots of type guards, so all I need to do is specify that it should be expecting a string input in the path, and it’s going to return a Result<Json<models::Post>, NotFound<String>>. We call out to db::post_meta(slug) which checks the database using the post slug, and returns the Post object. Since Post is tagged as a Serialize struct in db.rs, Rocket can just convert it to JSON using library code and safely return it. If it’s not found, then it just returns a 404 and we move on with our day.

The comments code now just needs to make a request to the endpoint to get the Bluesky post rkey:

const getPostAndThreadData = async (
  slug: string,
  setThread: (thread: AppBskyFeedDefs.ThreadViewPost) => void,
  setUri: (uri: string) => void,
  setError: (errorString: string, error: string) => void
) => {
  const url = `https://meta.jack.is/${slug}`;
  const meta = (await ky.get(url).json()) as Meta;

After getting the metadata, it just acts like normal.

Part B: Websockets and databases

So how do we get the post info to put in a database for the API route to have what it needs? Bluesky Jetstream of course. Using jetstream-oxide, as well as the patches from cyypherus, we can now listen to the Jetstream and filter for just my posts. This actually ends up being more robust than the old search API, as here I can restrict my filter to “just posts with 📝 as the first character”, which couldn’t easily be done using the search API.

Using Diesel the websocket loop then inserts into a postgres database with all the info needed for the front-end to find the post.

Next steps

There are one or two improvements I’d like to make here, primarily around the database handling. It’s not using connection pooling at the moment, but figuring out how Rocket handles all of that (and passing a connection to an arbitrary tokio function) was out of scope for this MVP.

In case you’re curious, you can find the code here: at-comments-listener. It’s primarily designed for me, so there’s a lot of stuff in there that’s hard coded, but maybe someone will find it interesting.