Sickness Update

I finally got through the rough week of dealing with the Flu, and used that down time to catch up with TV shows I was behind on. Finished watching Paradise, and caught up with Daredevil: Born Again and Invincible. I also started reading Could Should Might Don’t, my first non-fiction read in a long time. So far I’m not completely sold on the whole premise, but the topics the author chooses to break down are well presented.

Games!

I’ve also picked up some of my half-completed games while I was sick. I’m almost done beating Chrono Gear: Warden of Time. I also resumed playing Horizon Forbidden West but the clunky combat pissed me off again. I love the story too much to let this detract from my enjoyment of it, so I opted to cheese my way using a trainer mod for a boss that was genuinely anti-fun. I’m not no-clipping around the place, just using invulnerability to get past a fight that feels more annoying than skill based. I’ve only used it once so far, maybe in the future I’ll get too lazy and start using it for all bosses, who’s to say. Either way I’d rather enjoy the game this way than not play it at all.

Writing

In my Flu-stricken stupor I completely forgot about April Cools this year, and had to scramble at the last minute to put a post together. Admittedly, I tried to cheat my way out of the spirit of the thing by having an LLM convert my rough outline to a post but I found the writing to be so genuinely sickening to read that I threw it all away and just tanked the effort of writing through a fever to get this out.

Work and Career

I got the increment I was expecting at work, which was nice. I took most days off while recovering from the Flu so not a whole lot got done, but I helped grease things along wherever I could manage to spare some time to review the documentation changes we were working on.

Miscellaneous Tech Things

I’m planning to redo my room layout to open up some space in the center following Yash’s example, so I had to get a multi-plug extension for my extension so I could neatly tuck away all the power cables that go into my — extension-ception. I didn’t know this was a thing!

I also worked a bit on Claw, my Android app for Lobsters. I removed a data migration path that had done its job in the past two releases, and added the very initial bits of login support in the app. Eventually the goal is for the app to be able to let you upvote and save posts directly to Lobsters instead of just the local store, and to maybe let you reply to comments/posts if I take on the daunting task of handling WYSIWYG Markdown input.

People!

The weekend was packed with a lot of socializing. IndieWebClub on Saturday was a discussion focused session and we traded a lot of hot takes about editors, LLMs, writing tools and everything in between. It was a great time! Sadly I was not able to finish my writing draft within the allotted time, but I hope to have it out by next week.

On Sunday Yash and I met up with Anunay and a friend of his for lunch at 23rd Street Pizza which was rather underwhelming. We meandered around a fair bit after that for some dessert and much needed steps, then got back to our place to chill for a bit. Aforementioned friend introduced us to Flip 7, which was a hilarious time with just 4 players and we fully lost track of time until it had been 3 hours. This game made me fully understand how gambling addictions come to be. The temptation to got for just one more flip is too damn high!

The food section

For this week I will choose to be lazy and delegate the responsibility for capturing our week’s food to my roommate Yash, who is far more diligent about this than I can ever hope to be. Check out his week note!

This is a post for April Cools, encouraging people to break out of their standard fare and write something new.

For the longest time I’ve not had a particularly great inclination for cooking. I had lived at or very close to my home for practically all my life, I was pampered (spoiled) and was never really encouraged to pick it up. That had to change when I moved to Bengaluru as part of being acqui-hired by Cloudflare in early 2025.

I moved into a 2 bedroom apartment with old friend and current colleague Yash, and we decided to get a home cook. Having been in a hotel for a month or so while house-hunting we had landed pretty squarely on not wanting to be dependent on food delivery. Through some neighbourly drama that is not worth unpacking, we had to fire our home cook pretty quickly and we found ourselves in the kitchen with a strong desire to not starve. We started with the basics: simple curries with chapattis. Chickpeas, black beans, paneer (cottage cheese), and potatoes were pretty common.

The progress was honestly pretty slow. We started with using precooked chapattis sold in the supermarket for a few weeks before shame really set in. We bought a food processor (neither of us had the temperament for kneading dough by hand), and started making our own chapattis at home. We made a very early experimental foray into a Taro root (called “Arbi” in Hindi) based curry, a favorite of Yash’, which was an absolute nightmare to prepare and really killed the vibe on gastronomical experimentation for a while. Eventually we started doing pastas, fried rice, pulao, even a chicken curry where we learned the hard way why you’re supposed to thaw chicken 6 hours in advance (it was edible).

My mother, who had kept me out of the kitchen all this time, found herself quite concerned for the well being of her pampered buffoon. For maybe 6 months I had to send a picture of my dinner every night so she could rest assured that I had a home cooked meal and wasn’t pounding McChickens three times a day. My photo gallery in the past year is probably 40% dinner plates by count. I’m not gonna pretend I took all of those to satisfy her and not to gloat to myself, but she definitely is the reason it started :)

I was already part of the air fryer cult back at home, so when my friend Subhrajyoti was moving out to the Netherlands I happily snapped up his air fryer. This unlocked a whole new dimension in the kitchen, and since then we haven’t gone a week without some type of frozen food in our freezer. The convenience is simply unbeatable, though Yash has not been a very big user of it so far. Some day he’ll see the light.

Yash and I had been big fans of a hummus bowl sold by a food delivery service that was our staple breakfast at office for a while, and when they discontinued it I decided to finally look into how much effort it was to just make hummus myself. Turns out, not much! Hummus is just cooked chickpeas blended into a paste with maybe 5 ingredients added to it. I used the Christmas <> New Year’s week to try it out and ended up with a genuinely delicious batch.

The success of this experiment made me try this 3 more times to increasing levels of success, and also make my own falafels a couple of times. Still haven’t nailed that one, but got very close the last time!

Yash also got us into making burritos ourselves, a repeat of the story with hummus but more driven by California Burrito being expensive rather than it disappearing altogether. It makes for something we can prep easily, lasts a couple of days, is quite healthy and very filling. Some highlights from the first burrito misadventures, before I resigned to just doing burrito bowls and skipping the embarrassment of not being able to wrap a tortilla very well.

I bought a veggie chopper that lets me just make big piles of nutritious salads, topped with some air fried chicken sausages or whatever and just have an exceptionally filling dinner in under 20 minutes. Completely game changer for the days when I’m feeling lazy, has helped immensely with my struggle to lose weight.

When I started going to the IndieWebClub events late last year, I picked up the habit of publishing weeknotes. These have gradually started featuring food more prominently as I’ve gotten comfortable with simply trying things in the kitchen. Being able to navigate the kitchen with confidence has given me so much freedom that I previously thought to be out of my reach. Excited to see what other cuisines I’ll pick up this year!

Boring tech stuff

I’ve been trying out superpowers, which takes gaslighting-via-markdown (otherwise known as Agent Skills) to a logical extreme, forcing LLMs to behave like a very disciplined senior engineer.

I used this to change a longstanding misfeature in Claw that caused it to mark comments as unread when you first view a post, which is quite redundant. superpowers spent a solid hour on this, during which I also manually reviewed a spec and an implementation plan, but the result is quite exhaustive and has robust test coverage which I definitely wouldn’t have bothered with.

I also let it loose on our work codebase by feeding it a list of prior issues we had faced in a particularly brittle component and had it refactor the code to follow the popular “functional core, imperative shell” pattern and then unit test the core to prevent regressions along the bug categories we’ve been chasing for quite a while. That refactor ran even longer, but ultimately the result was again rather impressive. LLMs are (for the most part) really good at fixing tests, and superpowers forces the LLM to follow a red-green-refactor cycle for all changes which means it’s always trying to fix tests.

I finally finished running the full gauntlet of webmention.rocks tests against my WebMentions server. It exposed a few gaps in the implementation but I’m at 100% compliance now, the only remaining issue is now with my site not returning HTTP 410 for deleted posts because I don’t really know how to make it happen.

Boring not-tech stuff

I finished reading Project Hail Mary, super good read. I hadn’t realized Andy Weir also wrote The Martian! The book was great, the movie’s promised to be even better but good lord has it been hard to find a way to watch it with the lack of IMAX screens.

The keyboard I had ordered last week arrived, completing the new set :)

I hadn’t spotted our Metro station cat in a while, so it was nice to come across it again.

Health woes

I caught a fever on Thursday at the office which was starting to feel concerning right away, and by night had gotten bad enough that I had to arrange to see a doctor in the morning. Got diagnosed with a Flu and got a big dose of medicines to work through the weekend and see the doctor again on Monday. Was not expecting the throat problems, even my damn uvula had swollen and I could barely drink or breath. Super rough 3 days so far, the fever’s settled down now but I still have a very sore mouth and have trouble eating and swallowing. Pain

And finally, the food

Before I got absolutely mauled by the Flu I did do some cooking, which mostly was burrito prep that lasted us two days.

I had some chicken breasts sitting in the freezer that I wanted to finish off, so I used some of the rice from the large batch we made for the burrito bowl meal prep and made a basic tomato and cream curry for the chicken which came out pretty delicious for being basically 20 minutes of work

Hopefully the next weeknote will find me healthier :(

As I mentioned last week, I watched Iron Lung on Monday. Only the premium cinemas were running shows so I had to buy an expensive ticket at PVR Director’s Cut, and got to watch it in a reclining chair that was basically a bed. Loved the movie, Markiplier did great work as both director and actor. Hopefully we can get a couple more of these authentic video game recreations from individual creators before the suits at Hollywood figure out how to ruin them even more.

I was back in the office this week, work was mostly CI and suffering. On Wednesday I got caught out in the hailstorm on my way back from the office. Thankfully I didn’t catch anything from that, but I did have to figure out how to wash my shoes which was surprisingly easy — thanks Skechers.

Toys

I had been saving up to gift myself a new keyboard and mouse for my birthday, and ended up making it an early gift by placing the orders this week. I went with the Wobkey Rainey75 keyboard and a VXE R1 Pro Max mouse. The mouse arrived today, surprisingly in white rather than the purple/lavender I thought I had chosen. Clearly I made a mistake during the order, but I’m quite happy with it regardless. It’s been a long time since I had a wireless mouse and it’s such a massive difference from my wired Razer Viper Mini. Also nice to have a working scroll wheel again :')

I also bought myself a microphone arm so I could get a bit more desk space, and struggled through a few bouts of stupidity before realizing that I needed an additional screw to plug my Blue Yeti into the standard 3/8 inch mounts that feature in the microphone arms. I ordered said screw from Amazon, set it all up today and when I went to put the old desk stand in the box I realized that the screw actually came with the microphone and I am just a one-of-a-kind dumbass.

I had been bracing for pricing changes for my Netcup VPS, and it finally arrived this week. I run a pretty small ARM VPS at Netcup so my costs are only going from 5.84 EUR to 6.92 EUR, a hit I can take for the time being. Really unfortunate that the unprofitable companies propping up the AI bubble have so much access to free capital that they’re destabilizing the whole industry.

Books

I installed Calibre-Web last week, and spent most of this week doing manual housekeeping on the 40 or so books I own. Cleaning up metadata, syncing read states, setting up shelves — a lot of frankly pointless work that will hopefully make life easier in the long run. At IndieWebClub I tried to give Tanvi her own account on my instance, but hadn’t realized that we’d have to share the library. I’ve set up a shelf for my books so we can hopefully rely on that to work around the problem.

With my reading goal quite lagging, I finally finished off The Wizard Hunters that I started reading on February 6. I usually get through a book in 2 to 3 weeks, but this was slow both because it was technically two books in one and hence around 700 pages in length, and because I was just not reading as often the past few weeks. I want to read Project Hail Mary before I watch the movie, so hopefully reading this in the next week will put me back on track for my goal of reading 24 books a year.

Writing

At IndieWebClub this week I finished off and published the post about securing Calibre-Web behind Caddy and Pocket ID.

And finally, the food

As I promised last week, with the missing Cloudflare R2 integration in my CMS no longer being an excuse for not showing off the food I keep talking about, here is some of the food I made this week.

I had some chicken breast lying around in the freezer and an overwhelming desire to consume it, so I threw together a pretty simple pasta sauce using tomato and cream, cooked the chicken in it and then tossed it with pasta and some spices. I used the air fryer to quickly toast some sourdough bread I had and that was dinner and the next day’s breakfast.

Another night I ordered some nachos with my groceries so I put together a simple salad to go with the nachos. Apparently this is called “cowboy caviar” in the US of A. I made this on a whim so I didn’t have kidney beans, but I did get some feta cheese and it came out very nice. Bright flavors, super satiating, lots of fiber. What’s not to like?

Yash was back home on Saturday so we went back to our classic rajma (kidney beans) gravy which I supplemented with a side of raita.

Requirements

• Caddy with the caddy-security plugin
• Pocket ID
• Patience.

Pocket ID and caddy-security setup

Follow the Caddy guide here to set up an OIDC client and the caddy-security configuration in your Caddyfile. This gets you 90% of the way, but due to recent regressions in caddy-security you’ll need to make some tweaks.

First of all, in the oauth identity provider block, add this line:

trust login redirect uri domain exact ${app.domain} path prefix /

Replace ${app.domain} with the domain to the service you are securing.

The guide also assumes you will re-use the same caddy-security authentication portal for all your services which is fine, but I prefer to have each OIDC client be isolated on a service level instead of just having a generic caddy-security one. I’ll explain the basic changes first then dive into the NixOS-specific stuff I did for my own deployment.

security {
  # Rename the provider from generic to the service name
  oauth identity provider calibreweb {
    delay_start 3
    # Give this its own realm
    realm calibreweb
    # This is the OIDC provider type, make sure this stays `generic`
    driver generic
    # Service-scoped secrets from the Pocket ID OIDC client setup
    client_id {$CALIBRE_WEB_POCKET_ID_CLIENT_ID}
    client_secret {$CALIBRE_WEB_POCKET_ID_CLIENT_SECRET}
    scopes openid email profile
    base_auth_url https://auth.msfjarvis.dev
    metadata_url https://auth.msfjarvis.dev/.well-known/openid-configuration
  }

   # Rename the authentication portal as well
  authentication portal calibreweb_portal {
    crypto default token lifetime 3600
    # Use the new name for the identity provider defined above
    enable identity provider calibreweb
    trust login redirect uri domain exact books.msfjarvis.dev path prefix /
    cookie insecure off
    cookie domain books.msfjarvis.dev
    transform user {
      # The new realm name
      match realm calibreweb
      action add role user
    }
  }

  # Per-service configuration
  authorization policy calibreweb_policy {
    set auth url /caddy-security/oauth2/calibreweb
    allow roles user
    inject headers with claims
  }
}

With this setup you should be able to then copy-paste the code block above for a new service and just replace the name and URLs. Do note that the security block is globally unique, and only the contents inside it are meant to be duplicated.

Sidebar: NixOS version

People not using NixOS can ignore this section

On NixOS I already have a Caddy module that applies a set of defaults, so I retrofitted generating the security block into it. I added an option to let individual services configure the inputs via module options:

pocketIdApplications = mkOption {
  type = types.attrsOf (
    types.submodule {
      options = {
        domain = mkOption {
          type = types.str;
          description = "Domain of the proxied service";
        };
        clientIdEnvVar = mkOption {
          type = types.str;
          description = "Environment variable name containing the client ID";
        };
        clientSecretEnvVar = mkOption {
          type = types.str;
          description = "Environment variable name containing the client secret";
        };
      };
    }
  );
  default = { };
  description = "Applications to protect with Pocket ID OIDC via caddy-security";
};

Then in the services.caddy.globalConfig option, you loop over the value of this to generate the config we wrote above.

${lib.optionalString (config.services.caddy.pocketIdApplications != { }) ''
  security {
    ${lib.concatStringsSep "\n" (
      lib.mapAttrsToList (name: app: ''
        oauth identity provider ${name} {
          delay_start 3
          realm ${name}
          driver generic
          client_id {${app.clientIdEnvVar}}
          client_secret {${app.clientSecretEnvVar}}
          scopes openid email profile
          base_auth_url https://auth.msfjarvis.dev
          metadata_url https://auth.msfjarvis.dev/.well-known/openid-configuration
        }

        authentication portal ${name}_portal {
          crypto default token lifetime 3600
          enable identity provider ${name}
          trust login redirect uri domain exact ${app.domain} path prefix /
          cookie insecure off
          cookie domain ${app.domain}
          transform user {
            match realm ${name}
            action add role user
          }
        }

        authorization policy ${name}_policy {
          set auth url /caddy-security/oauth2/${name}
          allow roles user
          inject headers with claims
        }
      '') config.services.caddy.pocketIdApplications
    )}
  }
''}

With all this in place, you can then configure this alongside (in my case) Calibre-Web and have all the stuff above get generated without having to think too much about it:

sops.secrets.calibre-web-caddy-env = {
  sopsFile = lib.snowfall.fs.get-file "secrets/calibre-web.env";
  format = "dotenv";
  owner = config.services.caddy.user;
  restartUnits = [ "caddy.service" ];
};
services.caddy.pocketIdApplications."calibreweb" = {
  domain = cfg.domain;
  clientIdEnvVar = "$CALIBRE_WEB_POCKET_ID_CLIENT_ID";
  clientSecretEnvVar = "$CALIBRE_WEB_POCKET_ID_CLIENT_SECRET";
};
systemd.services.caddy.serviceConfig.EnvironmentFile = [
  config.sops.secrets.calibre-web-caddy-env.path
];

Caddy routing setup

Usually just throwing a reverse_proxy localhost:<port> gets you all the way with Caddy, but Calibre-Web specifically requires some extra configuration that serves for a nice demonstration. We need to treat different paths in different ways, so we’ll use the Caddy handle directive to create mutually exclusive routing paths for them.

caddy-security adds a /caddy-security path in your sites that should always require authentication, so we simply route it to our portal without any changes.

handle /caddy-security/* {
  route {
    authenticate with calibreweb_portal
  }
}

Caddy lets you label a set of paths or with the @name syntax, this is just here for convenience. Calibre Web exposes OPDS and Kobo-specific endpoints for use with your devices that likely can’t do OIDC, so we allow those to bypass the authentication requirements. The transport buffer size was taken from https://github.com/janeczku/calibre-web/issues/1891 after I faced the same sync issues.

@integrations {
  path /opds /opds/* /kobo /kobo/*
}
handle @integrations {
  reverse_proxy localhost:8080 {
    header_up X-Scheme https
    transport http {
      read_buffer 1024k
      write_buffer 1024k
    }
  }
}

This is the catch-all route for everything else that doesn’t need special treatment. This only differs from the one above in requiring authorization.

handle {
  route {
    authorize with calibreweb_policy
    reverse_proxy localhost:8080 {
      header_up X-Scheme https
      transport http {
        read_buffer 1024k
        write_buffer 1024k
      }
    }
  }
}

The full configuration then becomes this:

handle /caddy-security/* {
  route {
    authenticate with calibreweb_portal
  }
}
@integrations {
  path /opds /opds/* /kobo /kobo/*
}
handle @integrations {
  reverse_proxy localhost:8080 {
    header_up X-Scheme https
    transport http {
      read_buffer 1024k
      write_buffer 1024k
    }
  }
}
handle {
  route {
    authorize with calibreweb_policy
    reverse_proxy localhost:8080 {
      header_up X-Scheme https
      transport http {
        read_buffer 1024k
        write_buffer 1024k
      }
    }
  }
}

Sidebar: Authentication vs Authorization

This is something that took too long for me to hammer into my brain, so I’ll reproduce it here as well for others who may have had the same question but didn’t feel confident asking it.

• Authentication: Verifying your identity. This answers the question of who the user is.
• Authorization: Verifying your access. This answers the question of whether the previously identified user should be allowed through to a specific service.

We require authentication for the caddy-security portal because you should be able to at least see that you have an account. The rest of the service requires authorization so only people who should be allowed access to the Calibre-Web instance are able to log in.

And that’s the long and short of it! There was a lot of one-time effort in getting this working and I had to dig through a lot of GitHub issues to find bits and pieces that had to be tweaked, but I’m quite happy with how this all came together. I now have Calibre-Web syncing to my Kobo eReader, and no longer have to struggle with a split desktop library back at home and here in Bengaluru.

As hinted above, I was back home this week! My friend Rohan was getting married and I couldn’t imagine not being there, especially when our whole friend group was getting together. It was a great time, he sounded genuinely excited and happy which is all I care for. I got to see a lot of friends I haven’t met in person for over a year now, and it was very nice catching up and being a little rowdy over some drinks :)

While at home I also took the opportunity to hack some more on the homelab server and try out microvm.nix to enable running more than one instance of things on the same machine which has historically been a problem on NixOS. I intended to use all that preparatory work to try out Booklore in a microVM since it uses MariaDB which I had no plans to run on my host, but that fell through when my suspicions around the quality issues in the v2 release were validated by the revelation that the developer has let Claude run rampant over what was honestly a pretty decent bit of software. The following meltdown from the developer cemented my decision to stay away from it entirely. Instead, I set myself up with Calibre-Web.

It was quite straightforward, and most of my time tinkering with it actually went into setting up forward authentication using Caddy and Pocket ID. The end result of it can be found here, and I intend to write a post about it next week so others have to suffer a little less than I had to :')

At home, I do my gaming on Linux where I ran into a peculiar bug involving Discord and Tailscale, which didn’t let me join any voice calls. I investigated this a bit online, and apparently it was reported already and the fix was to hide the tailscale0 interface from the Discord client?? That was also how I found about firejail, and like most things there was already a NixOS module to make the setup easy.

On Saturday I flew back home via a morning flight, which is weirdly the first time I realized that the flight between Bengaluru and Hindon is closer to 3 hours than 2. Always thought otherwise, don’t know why.

I found out about IzzyOnDroid’s AI policy yesterday and since I’ve written before about using LLMs for the development of Claw, I filed an issue to ask for its removal from the repository. I massively appreciate the work they do for the FOSS ecosystem for Android (having donated a not-insignificant amount to their OpenCollective through Android Password Store) and I would hate to take that for granted and knowingly ignore their policies. There hasn’t been a decision made yet on it, but I’ve provided all the necessary context to aid the process.

Swapnil and I went to watch Boong right before I sat down to write this weeknote. Very cute and heartwarming, really loved it. It also made me realize how much I’ve missed going to the movies this past month, so I’m remedying it by watching Iron Lung tomorrow.

More meta stuff: I’ve previously talked about how my CMS does not have an integration with Cloudflare R2 as an excuse to not attach pictures of my food, which obviously meant that the feature got shipped so I have no reason to be lazy anymore. Prepare for diabolical salads next week!

My phone woes continued into this week, with the Pixel 8 seemingly forgetting my home Wi-Fi network entirely and refusing to connect to it. For two days I had it connect to my spare phone’s hotspot for Wi-Fi. Just like last week, this problem also magically resolved itself in two days and things went back to normal. I hate software even more.

I finally had to go bug my landlord to get some maintenance work done in the flat when my ceiling fan broke down, not a fun time in 32 degree Celsius weather. That took entirely too many phone calls than one would expect, but I managed to take the opportunity and get other stuff fixed around the apartment as well. Hopefully that’ll be the end of this.

At work I kept fighting with CI related things, which were made slower by the fact that I had to reach out to a different team to help with reviews since nobody in mine was working. Eventually I think I’ve got it all working but what a colossal waste of time it has been throughout this week. We had a minor scare with the Android SDK from a user report about crashes when attempting to join a meeting but thankfully we were able to debug it and identify that the problem wasn’t as widespread as we initially suspected and required a very specific setup to trigger. I fixed the root cause and cut a release, I’ll probably write a blog post about it once I’ve completed the internal incident report.

I had to fly back home via an early flight on the 8th and hadn’t packed for it yet so I was debating if I should go to IndieWebClub but thankfully I did end up going, the topics on the agenda were great and a lot of thoughtful discussion happened that I enjoyed participating in.

I packed for the trip at a definitely-not-too-late 8 PM, stayed up till 2 AM playing games with friends while waiting for my taxi and managed to reach the airport in an astonishing 45 minutes. All the airport staff was still there but the foot traffic was maybe a fifth of what I usually see in Kempegowda airport, so I basically speedran through check-in and security. Time from taxi to departure gate, maybe 30 minutes.

And that was kind of it! I’m back home in Ghaziabad for a week to attend a friend’s wedding and will be back in Bengaluru next weekend.

Warning: Feelings

My roommate Yash is back with his parents for a couple of weeks, which has left me alone in the house and quite severely exposed my delusion of being a loner. I’ll probably write a separate post about this at some point, but I’ve been quite overwhelmed with feelings of loneliness which aren’t helped at all by me being a general homebody so this week has been quite rough, and forced me to start training myself to address the things that feed into it. I ended up finally utilizing the free therapy options offered at Cloudflare, and make sure to have at least one cleaning related task on my daily to-do to ensure I do not spiral into my room becoming a depression nest (shoutout Geega my goat). This has helped so far, and will likely be plenty to tide me over until I myself go back to my family at the end of next week. I also reached out to my colleagues at work and have scheduled a day of working together out of a cafe since our teams agreed to work from home for the next week due to most people taking off for Holi.

I promise I’ll be okay, it’s a bit of a sad boy funk, I’ll get over it.

Onward to the rest of the weeknote!

• I had to email the customer support for one my credit cards when I realized I needed to use my registered email, and had helpfully used one of DuckDuckGo’s disposable aliases for the job 😅. Figuring that one out was thankfully pretty quick, and has been put down in a note for future Harsh.
• Amogh helped me figure out how to be able to pay for stuff on Steam (the aforementioned credit card being at fault) again, so any irresponsible gaming purchases I talk about in the coming weeks are his fault.
• Last week I talked about adding WebMentions support to this site, this week I kept iterating on it and it now properly extracts comment text when available and correctly attributes to the real user instead of the brid.gy relay.
• I shipped a new release of Claw this week, incorporating some UI feedback from Abhinav and my first attempt at a (mostly) LLM generated feature: Temporary tag blocks. I had to get my hands dirty at the end anyway because the LLM just completely forgot about data migration and integrity, but having it do 80% of the work while I was at the gym is pretty cool. I also have to admit that the UI flow that Claude came up with is miles better than anything that had been floating around in my brain. I don’t know much it helped, but I used this “skill” as part of the prompt.
• Someone on the payroll team at Cloudflare probably discovered Idempotence this week as everyone at our office got paid twice :P
  • Fret not, the extra money is already back with Cloudflare. Cheers to a brief taste of unearned wealth, the closest I will come to sharing an emotion with a billionaire.
• While trying to make that bank transfer I realized that my SIM had stopped working altogether, suspiciously after the Android 17 Beta 2 upgrade. Mild panicking ensued, and I had to add an additional side quest for a duplicate or a new SIM the next morning when I was meeting Swapnil and Mayank for breakfast.
  • The boys helpfully accompanied me to the Jio store where I got the unfortunate news that the SIM owner has to be present in-person for a duplicate (which happened to be my mother), so I went to get a new Airtel SIM instead which was a relatively quick affair.
  • Of course right after that when I get home, the Jio SIM starts working again so all of the struggle was for naught. I fucking hate software sometimes.
• My weekly NixOS upgrades were basically impossible this week, there was a bad Linux regression, and home-manager also has a severe bug so I just reverted everything and will deal with it next week.

In case anyone is missing the food talk, this week was rather light on the experimentation front due to the aforementioned sad boy hours. I mostly ate salads and air fried a chicken breast or two a couple of times. I did try air fried potato wedges one night and they came out pretty nice, albeit slightly dry since I got the temperatures wrong. Lesson learned!

Now, onward to the real content. A lot of time was spent in the kitchen this week, with Yash and me continuing our burrito escapades. On Monday, I decided to try air-frying some tortilla chips to see how it would go, and despite the obvious problems caused by overcrowding the rather tiny fryer basket, I’d consider it a success overall. Definitely a good option for dinner in a pinch.

A couple of days later, we went back in for even more burritos, building on our previous experience. The beans got a more generous round of seasoning, the rice was cooked and cooled ahead of time, and we were more mindful about how much we tried to shove into each burrito. This was remarkably successful, with both of us managing to create fully wrapped burritos by the end of the ordeal. I opted for a veggie-heavy burrito bowl with my remaining portions instead of wrestling with the tortilla again, which felt like the smarter choice. Throwing in some air-fried chicken sausages elevated the whole situation, but there was a definite lack of carbs, which I’ll have to keep in mind for the future.

On Friday, I had zero energy to cook, so I just chopped up the remaining lettuce along with some onions, tomatoes, and cucumbers, tossed it all in some sauces we had lying around, and pretended it was a gourmet salad. 10/10, would do again.

Yash is back home for a couple of weeks, which means I’m left to fend for myself in terms of sustenance. So I’m making a U-turn back to the Mediterranean phase of my life and already have big batches of falafel and hummus sitting in my fridge. This time I did remember to use the appropriate amount of greens and—shocked Pikachu face—it’s actually come out way nicer than my last mishap. Who knew following the recipe was the way?

With all the food talk out of the way, the boys and I have been getting back into Helldivers 2, and things have been going (not) great. Super fun, though!

Relatedly, my laptop is back to its fortnightly cycle of throttling at the most inopportune moments, which has made gaming frustrating every so often. Since Windows is fairly out of my wheelhouse, I decided to commit the sin of letting an LLM replace the usual experience of stumbling through outdated forum articles. I had it run a gamut of debugging scripts to try and diagnose the problem. Everything that came out of this multi-day rigmarole is archived in this Git repository. I’m not yet 100% confident that what Claude Sonnet 4.5 identified as the problem has truly resolved the issue, but it’s been fine for the past day or so.

Continuing the theme of tying things into the next entry, I have more LLM nonsense to share! I use jtx Board to manage my TODOs via CalDAV and wanted a simple way to show them in my Glance dashboard, which supports custom APIs—but unsurprisingly, only JSON. I had Claude try to write a simple proxy that could massage the CalDAV response into a suitable JSON format that I could plug into Glance. On its first pass, it took the credentials I had explicitly provided in a separate gitignored file for integration tests and put them into config.example.yml. When I asked it to remove them, it also included them in the LLM response text itself, causing them to be leaked twice instead. It also tried to claim that integration tests were passing when they obviously weren’t. Somewhere along the way, it lost the ability to issue write tool calls, which resulted in hilarious scenarios where it would attempt a write that never went through, run a test that still obviously failed, and then crash out about it. Fun for the whole family.

IndieWebClub #20 was this weekend! I had missed the last two installments because I was out of Bengaluru, so I was super excited to catch up with people again. I met a bunch of new folks, had some great conversations—just a really good time. I would wholeheartedly recommend it to anyone who wants to have their own website, regardless of their technical expertise. You can get any level of help from the nicest, most patient people and become part of a small but growing community. It’s really special to me! The next one’s on the 7th of March—mark your calendars 🙂

And finally, this site now supports WebMentions! The server-side of this is built on top of the Cloudflare Developer Platform, leveraging Workers, the D1 SQLite database and Workers Queues. It was a nice way to finally dive back into Cloudflare’s compute offerings since I had last used them in 2022 and I was quite pleased with how much better everything is now. A lot more things are now possible to run on Workers than before, and the local development experience is really good now. I’ll probably build some more things on Workers in the future, and hopefully write about them.

I was finally able to put in a replacement request for the defective Pixel Watch 4 I had purchased a couple of weeks ago now that I was in Bengaluru and delivery was a more straightforward affair. Finding the chat option for this was an absolute nightmare trying to navigate Google’s endless mazes but once I was in there it was a pretty easy process. Blue Dart’s speed was quite the savior, and I got the replacement device back in just 3 days.

Now that I have a more capable fitness tracker I’ve been trying to be more mindful about staying active, which has been going less than stellar because I had been away from the gym for nearly a month so building back my endurance and stamina is gonna take a few more days. Definitely going to try and push for 10K steps a day again.

On Wednesday I fell on my way back from the gym by getting my foot stuck in an uneven footpath, which left me with some light but painful bruises. Thankfully they seem to have buffed out over the week, and the only permanent damage is going to be to my phone case :')

On Friday, my roommate Yash and I decided to try out making burritos which were quite scuffed but ultimately rather delicious. The 8 inch burrito wraps that we got ended up being rather small, and the absence of any burrito wrapping skill between the two of us did not help the situation. Would like to try it again sometime if we can find bigger wraps, or maybe make our own.

Valentine’s day was spent enjoying my own company and working on some homelab stuff I had been putting off. My Git server is constantly being spammed by Chinese and Vietnamese spammers, which I tried to mitigate unsuccessfully. Will take another stab at it soon.

I also put out a new release of Claw since it had been a month since the previous one, and it seems to have not exploded right away so that’s a relief.

To start off the week, I finally got my first Balatro win off an extremely lucky draw of jokers – one which gave “mult” equal to the sell value of all jokers and another that gained sell value each turn. I also hit an extremely lucky spectral card that replaced a whole hand with jesters. The seed for that run was A7IBENU9, in case anyone wants to show me up :P

I also introduced Balatro to my sister who got the hang of the Poker aspect of it, but doesn’t quite get the Roguelike part yet.

Outside Balatro and the occasional ceremony for the wedding I was attending, I occupied my time in trying to finish reading the minor behemoth that was Book of Ile-Rien. When I finally got done with it, I realized I had basically every Martha Wells book on the Kobo except the sequels to the Ile-Rien series which I wanted to read next :‘D

The eBook edition was not available on the Kobo India store so I had to get it from Barnes & Noble, then figure out how to transfer it to the Kobo which also became a quick note on the blog.

The flight back to Bengaluru was a Boeing which didn’t quite inspire confidence, and the rather rough turbulence in the last 15 minutes of the journey had me genuinely fearing for my life but I made it down in one piece. I had to wait a while for the 3:30 PM bus back to Indiranagar but it was made pleasant by the AC lounge in the basement bus pickup area of the Kempegowda Airport.

Earlier in the week I had discovered (via IndieWebify) about the existence of the h-entry standard, and the site implements it now.

I was planning to be at IndieWebClub #19 today, having been away for the last 2 iterations of it but I woke up with the sniffles and didn’t have the energy for it. Hopefully next time 🤞

The most interesting part of my week was me getting involved in some Discourse™ over a Cloudflare blog post that made dubious claims about a vibe coded Matrix implementation that was written to run on Workers. Needless to say I will not be exercising my “Disclaimer: I’m a CF employee but do not speak for the company” rights in the future.

On a positive note despite the biting cold and fog I’ve been able to get a solid 3-4 hours in the sun every day which were often translated into heavenly naps. To put it in perspective how rare it is for me to sleep out of turn, I’ve owned my Amazfit tracker for 2 years and this week is when I first discovered it tracks naps. During all the sun bathing I also managed to get through the first half of The Book of Ile-Rien.

And that’s kinda it! Having little to no access to the Internet has been kind of driving me crazy but I preloaded the Kobo with enough material to have no excuse to not just be reading, so that’s what I’ve been using my time for. The entirety of next week is going to be wrapped up in the festivities for the wedding I’m here for so I expect to have another light weeknote which will thankfully be written from Bengaluru :)

• I started reading The Book of Ile-Rien, without realizing that the copy I bought is actually a combined version of The Element of Fire and Death of the Necromancer so I have a 900+ page behemoth on my hand that is a departure from the science fiction fare I usually consume. In six chapters I’m up to 17 unique words describing medieval things that I had to look up so overall this has been quite the experience.

• The Pixel Watch I ordered last week came in on Tuesday and by Thursday its display had already developed horrid discoloration that forced me to put it in for an RMA and get a replacement. It’s particularly bad timing since I will be traveling to my hometown soon for a marriage and then head back to Bengaluru, so I tried to have it be picked up from here and get it delivered in Bengaluru. Unfortunately the support executive I talked to got the wires crossed on that and scheduled the pickup from Bengaluru 🥲. Still trying to get that sorted.

• We had a fair amount of thunder and rains on Friday which immediately knocked out both me and the power grid. The power came back 7-8 hours later but I continue to have the worst cold.

Being away from work for a couple of weeks now hasn’t been as isolating as I would’ve expected which says good things about the team, but it has still been quite the drudgery being stuck inside what with the cold and the rain. I already miss the fabled Bengaluru weather…

• When I was leaving Bangalore, I tried taking a KIA BengaWalk electric bus to the airport but it never showed up so I ended up in an expensive taxi instead. My experience with the bus network was pretty good when I used it last time to return from the airport so this was quite disappointing.
• Obsidian updated this week to use rounded elements everywhere which I have mixed feelings about. It hasn’t felt too disruptive this week but I also didn’t use it all that much due to being mostly offline, so we’ll see in the next couple of weeks what side of the fence I end up on.
• Finished reading Shift on the flight to Varanasi.
• Immediately picked up Dust once I reached the village.
• The generally poor mobile network at the village exposed a failure condition for the home screen widget in my app Claw, so I took a crack at having Copilot implement a caching feature to make the updates tolerant of transiently bad network. It took some prodding to make it all work, but in the end I was mostly satisfied with its ability to copy existing code [PR].
• The day before we were supposed to head back home, I got roped into assembling wedding invites with my Dad which ended up taking most of our time but it was nice to catch up with Dad for an extended period that I usually don’t get the opportunity for.
• We ended up splitting our drive back home with a pit stop in Allahabad at a family friend, and I ashamedly used that time to catch up with work. I had completely forgotten about the Q1 kickoff meetings so there’s a couple of hours of recordings to watch when I’m back next week.
• We left bright and early for the drive back which was rather rough with the heavy fog. I used the time in the car to finish off reading Dust, which might’ve been the fastest I’ve gotten through 300+ pages. The Silo series is written in quite a cliffhanger-heavy style where every chapter keeps you hooked, so I found it hard to put down. No complaints though, I loved all 3 books.
• My Amazfit fitness tracker fell apart again so I finally ordered the Pixel Watch 4 I had been eyeing, with a sturdy case to protect it from my neglect.

• I had to shave my beard because of some skin irritation that I am medicating for, which apparently was enough to confuse my landlord into wondering who I was.
• As I am in a quite Mediterranean time of my life, another batch of hummus was created, this time with a lot of more spices and it came out quite delicious indeed.
• My phone has had the “pink line of dead pixels” syndrome for a while but it finally became enough of a problem for me to get it repaired, and in the process Android’s Repair Mode bugged out and forced me to wipe my device to get it out of repair mode. Quite the annoyance to put onto a person already in the dumps, made worse by a Firefox bug that caused passkeys to not work.
• The aforementioned data wipe also exposed a bug in Claw which prevented auto backup from working, so at least something good came out of me losing 3 months of saved posts since I had last backed things up manually.
• Willi Geiger, the head of the Cloudflare Media team was visiting the Bengaluru office this week. He took a session explaining the Media business as well as 1:1 meetings with every one of us who works in the Media team. Was nice having him around, shadowing how we do things in the Bengaluru office.
• Watched Avatar Fire and Ash with a couple of friends, as usual it’s a great spectacle with a flimsy story. Hollywood, keep giving this guy money to justify the existence of IMAX.
• Last week I had ordered some accessories for the IKEA Pegboard which arrived on Saturday, and I was finally able to fill out the thing and put up some of the stuff that was cluttering my desk.
• Yash and I have been experimenting on the cooking front to start making more things than our usual 4 dishes and made some baingan bharta on Saturday. Definitely burnt some of my fingertips off while roasting and peeling the eggplants but completely worth it.

Why does it need 108 gigabytes to store 102K page views? I wish I could tell you. One of the developers of Plausible replied to my grumbling explaining that it was likely ClickHouse collecting a ton of logs which was the real storage hog rather than the data itself. By that point I had already finished migrating off to Umami and deleted the offending ClickHouse data so I couldn’t verify the hypothesis.

The migration itself was quite simple so I’ll reproduce the steps below for any interested people looking for migration steps.

• I set up the NixOS module for Umami, and enabled it on a fresh subdomain to run it alongside Plausible during the migration.
  • Umami has this weird default of creating an admin account with the `admin:umami` credentials so make sure the first thing you do is change this password. I wish they would add OIDC support already so I could stop relying on this.
• I logged into Plausible and following their docs, obtained a ZIP file of all my data till date.
• Umami does not have any first-party support for importing data, so I had to use a third-party Python script that converted my ZIP file of CSVs from Plausible into a 5 megabyte .sql file that I could import into the Umami database table. The conversion worked for me in the first try, which was quite the relief.
• With the SQL file in hand, I was able to easily import it into the running Umami instance by running sudo -u umami psql -U umami -d umami < plausible_migration.sql. This worked flawlessly as well, and I was able to confirm on the Umami web interface that my data had been successfully imported.
• In my website, I switched the JS script to the one I got from Umami, and updated the Content Security Policy headers to match.
• Once this was deployed, I quickly clicked through the site and was able to see my session in Umami and confirm it was working.

And that’s kind of it! After this I turned off Plausible and was able to delete the ClickHouse directory and reclaim the nearly half of my 256 GB disk it had been keeping hostage. Umami’s lack of OIDC and easy import options is definitely a sore point, but with how easy the whole thing ended up being for me I can live with it for the time being. Interestingly, Plausible only lasted 8 months before I moved out of it. I should’ve been more wary of ClickHouse than I was of MySQL when I chose Plausible over Matomo :)

• I started reading the next part of the Silo series: Shift. With the constant power cuts happening here in Indiranagar, this book on my Kobo has been the only thing I can do reliably without the threat of an arbitrary interruption.

• Roomie and I met up with Fenil this week for some tea and snacks, was lovely catching up and yapping about stuff. Hopefully we can make it a regular thing!

LLMs are an extremely divisive topic so I hesitated for a while before writing this post but reading this blog from Joan Westenberg convinced me to not worry too much about what I put on my own blog.

For the longest time I’ve found LLMs to be quite underwhelming. I tried all the new and fancy models on and off, first via GitHub Copilot which I get for free for being an active open source maintainer, and then with Windsurf at work. The sycophancy was annoying, the training data was always outdated, and the models were yet to learn how to accept they had no fucking idea about the thing I asked.

That only came to change very recently, with OpenCode. We have access to it at work via the Cloudflare AI Gateway and the latest Anthropic models.

OpenCode is quite capable by default, with sensible defaults and a functional UI to go with it. What really unlocked its capabilities for me was oh-my-opencode, a plugin that supercharges OpenCode by giving it the capabilities to launch background tasks via “agents” that enhance its capabilities across specific dimensions such as online research, UI design and system design. It adds integrations with the Language Server Protocol to ensure edits don’t introduce diagnostic errors. It includes a comment checker to prevent the typical annoyance of over-commenting code and giving it the distinct LLM smell. It bundles a set of curated MCP servers to enhance its ability to do research: Exa to do web searches, Context7 to find up-to-date documentation, and Grep to efficiently search through code on GitHub. All of these complicated bells and whistles come together to become a cohesive tool that can tackle both easy and difficult programming tasks with a very high degree of reliability.

I’ve only started using OpenCode outside work in the past week or two, but the value add is undeniable. Here’s a non-exhaustive list of things I used OpenCode for in Claw, with varying degrees of complexity and impact.

• It fixed a crash that has hit almost every user at some point in the past two years.
• It helped me add Sentry instrumentation for the app’s database which was previously missing and I kept struggling to understand the right APIs for it.
• It helped write an end-to-end UI testing suite that ensured common flows didn’t break during my rowdy refactoring.
• It fixed another crash that hit users of the app’s home screen widget
• It fixed yet another crash caused by not handling process death correctly (I can already hear the groans of Android devs worldwide)
• It fixed my poorly written deep linking implementation
• It spent an hour taking apart the internals of the AndroidX Paging3 library to fix a performance regression caused by the first fix.

None of these are things that would be impossible for me to figure out, but they sure as hell wouldn’t have been done in the span of two weeks. I don’t really subscribe to the idea of fully phoning it in, the so called “vibecoding” aspect of using an LLM as a coding assistant. In every case where the LLM has solved a bug, I’ve learned something new from the fix and even went beyond in many cases because of the leg up I got by using the LLM.

With the Sentry MCP server, I can spin up an OpenCode session, say “Investigate the root cause for Sentry issue <id> and include references in the summary” and go off to do something else. When I get back, I have a better understanding of the issue in question, I can use the references to do my own research and tell the LLM to implement one of potentially multiple ways to fix the bug. In many situations, writing the actual code is just not the more fun part of the problem.

That said, I do want to clarify that I still find writing code to be enjoyable or fulfilling in most cases even with an LLM in the picture. This pull request was mostly me doing a bulk find -> replace across the codebase, something an LLM would excel at. However, I believe I am more capable of learning about Metro’s capabilities ahead of time and spotting opportunities to replace existing constructs with Metro-native implementations, so I did it by hand and manually reviewed each file.

Just a few months ago, I would not have believed that I could use LLMs to write any code I intended to actually maintain. Similar to Simon Willison’s tools.simonwillison.net, I had also set myself up with a small repo and a public page that hosted those tools. These things still have the default LLM sheen to them, and I gave up on even having them follow a personalized code style. For all definitions of the phrase, it is vibe coding. They solve simple problems for me, and will likely never break or need an update.

Despite my changed opinions about LLMs as programming assistants, I am vehemently against the anthropomorphizing of LLM-based chatbots. They do very real harm, and should be very aggressively regulated. I hold an even stronger belief that image and video generation models should not be available to the general public at all. The fact that humans can no longer trust even their eyes is deeply disturbing to me. It has been proven that these technologies will actively be misused for deceit and in the case of Elon Musk’s xAI, sexual harassment of minors.

This is close to devolving into rambling, so I’ll end it here. I don’t intend to form a crippling reliance on LLMs anytime soon, nor start paying for them. As long as GitHub is willing to continue wasting money on giving me access to LLMs for using their website, I’ll use them for my personal projects. When they stop, I’ll just go back to the good old elbow grease that has got me this far. If I can write Java over SSH into nano, I think I can still use a search engine by hand.

I’m writing this at 9 PM on a Sunday which for a weeknote feels like turning in an assignment at 11:59 PM on the due date, going forward I’ll try to block a chunk of time earlier in the day to work on this. The Obsidian setup I created to help with these weeknotes is still coming in quite handy, makes me quite happy that I chose to migrate out of Logseq earlier this year.

• I finished my personal reading goal of 24 books this year, being tracked here. Next year I wanna blend in a bit more non-fiction into this, and hopefully raise the count by a few more.
• Continuing on the book front, I finished the last two short stories from the Murderbot series: Home: Habitat, Range, Niche, Territory and Rapport: Friendship, Solidarity, Communion, Empathy, and started reading Wool.
• There was an alumni dinner for Obvious on Wednesday where we ended up with 20+ people at Idylll. Met a lot of people from before my time at Obvious, and caught up with the ones I did work with. Super fun experience, was very glad that I could make it.
• My Telegram account got banned for no particular reason, likely by mass reporting by someone I pissed off and forgot about? I’ve reconvened on Discord for now but it’s clearly a problem that my primary communication platform can just evaporate overnight. I can set something up for myself but moving over friends is gonna be the real pain.
• I watched Dhurandhar on Thursday with a couple of friends and it was great! Bit more gory than I expected from Bollywood and the CGI was a little iffy but the story and acting more than make up for it.
• I tried getting back into Hollow Knight but the combat is not really working for me, so I’ll let the roguelike fever die down for a bit and pick up Control or Horizon Forbidden West which I’ve been sitting on for a while.
• At work I’ve mostly been busy with the Kotlin 2.3.0 upgrade which has a lot performance improvements and additions relevant to our needs. I was able to get everything to work, but unfortunately can’t ship it without a compatible SKIE release. Hopefully soon!
• I finally added search to this blog! It’s something I had wanted for a long while and being at IndieWebClub finally motivated me to execute on it. I’m pretty happy with the result, but I intend to customize the UI and search result ranking a bit since I saw some instances of tag pages being prioritized over the actual blog post using the tag.

I was at droidcon India this Saturday and got the chance to meet a bunch of people I hadn’t seen in a while and attend some really great talks. It was a great experience and definitely motivated me to come out for meetups and events a bit more and be less of a loner :P

Before I start walking through my day, I would like to extend an apology to my friends who accompanied me to the conference and the Android developer community at large. I might be the singular dumbass who would be at an event where both Adit Lal and Jitin Sharma were speaking and plan an itinerary that included neither of their talks. I am extremely sorry and will never live this down.

We arrived at the Conrad Bengaluru a little before 9 AM to an already bustling space and settled into the Fireside Chat with Google’s DevRel Team. Lots of talk about AI as expected, but also the topic of passkeys and Credential Manager potentially doing away with the ridiculous OTP authentication flows used by banks here in India (🤞 one can hope).

After a short break we attended our first talk, Suresh Maidaragi speaking about Building Mobile Apps at Scale with Kotlin Multiplatform. It was a case study on their experience at Physics Wallah of unifying a lagging iOS app with their higher development velocity Android app via KMP. I was interested in learning some new things here as my work has involved a Kotlin Multiplatform powered SDK for ~2 years now, but was left a little disappointed as the talk was rather shallow on technical details. I was rather surprised horrified to hear that their app takes 2 hours to build in their release CI with their current setup!

Another short break and then one of my highlights for the day: Rahul Ravikumar’s “A Busy Android App Developer’s Guide to Perfetto”. Inspired by Lalit Maganti’s recent writing I had been teaching myself how to use Perfetto to debug some performance issues at work, so this was perfect for me and the colleagues I had dragged along to the conference (I joke, they came willingly. I think). This talk was very information-dense, I learned a lot about navigating Perfetto and making sense of its seemingly endless UI components. We got a sneak peek at the in-development Tracing 2.0 library and all the problems it’s solving from Tracing 1. Especially exciting was context propagation, which can connect parent coroutines to their children in the Perfetto UI allowing you to effectively trace multistep async operations.

The next talk I attended was Himanshu Singh speaking about “Building Android Open Source Libraries: Managing Public APIs with Intention”. Another talk I found to be rather underwhelming, as the synopsis in the agenda made me assume this would carry a little more technical content than it ended up with. Himanshu mostly spoke about the social aspects of starting and managing open source projects in general before mentions of Kotlin BCV and release automation towards the end.

After the talk we had a lunch break, and I got to catch up with Pratul Kalia who used to be my CTO at Obvious. He’s now building Tramline to make mobile releases as streamlined as deploying to the web. I’m a happy solo user of the product, but am otherwise not paid to shill it here :P

We skipped the 1:30 PM to 2:00 PM time slot since and just hung out since none of the talks particularly appealed to our small group of 4.

At 2:00 PM we headed to Sid Patil and Rutvik Bhatt’s talk: “The Art of Re-architecture: Lessons from the Trenches at Foodpanda & Delivery Hero”. As someone currently in the weeds of planning a re-architecture myself, their deep insights on how to evaluate the desired outcomes and engaging stakeholders of an engineering investment like this were very helpful. The mantra of “Refactor for today, Re-architect for tomorrow” was particularly inspiring as the resident refactorman of my team 👌

Next up was Rajesh Hadiya with “Build Kotlin Compiler Plugins for Production level Android Apps”. Despite the extremely dense topic he did a great job at making it fun and approachable. I’ve tried to solve a logging-related problem earlier last year with a Kotlin compiler plugin so I was pretty familiar with the pain involved in dealing with the Kotlin IR and the various compiler backends. Rajesh was able to explain the compiler fundamentals that enable the various frontend/backend compiler plugins work the way they do in a succinct manner.

We skipped the 3:45 PM to 04:05 PM slot as well due to nothing particularly appealing to the group.

Omkar Tenkale’s talk titled “Building the Coroutine Framework from Scratch” fully delivered on the premise. We started by talking about the distinctions between kotlinx.coroutines and kotlin.coroutines, then fully throwing away the former and just relying on the standard library + compiler intrinsics to rebuild the primitives we’ve come to rely on from kotlinx.coroutines such as runBlocking, launch, Dispatchers , GlobalScope and such. The talk was packed to the gills with low level details yet was very approachable and I left the room a lot more knowledgeable and appreciative of kotlinx.coroutines! Omkar’s was easily my second favorite talk here after Rahul’s Perfetto deep dive.

The last talk we attended was Nikheel Vishwas Savant from Meta talking about their experience building up a “Cross-Platform Bluetooth Architecture for Android, iOS, and Embedded Devices” for the Meta RayBan glasses. It was expected yet disappointing that we didn’t see any code in this talk, but there were a lot of great nuggets of information that I expect to take back to my work on RealtimeKit.

And that was it for droidcon! By 5:30 PM we were all pretty exhausted so I bid my goodbyes to the people staying back and explored the nearby Church Street area with a couple of friends before heading home.

This was my first proper con-style event in a long time (the last time I was at one was Google Developer Days back in 2017!) so I was somewhat nervous but it ended up being a great time. I missed a few people who were also there which was sad, but now that I’m in Bengaluru full time I hope to take more opportunities to show up at meetups and see people more than once every 3 years :)

Installing caddy-tailscale for NixOS

As of the latest NixOS release, the Caddy package in nixpkgs supports including plugins directly which greatly simplifies this step. As an example, here’s how I include the caddy-tailscale plugin with the excellent caddy-defender that lets me drop traffic from AI scrapers more safely.

You can also keep the package definition in a separate file like this, and use this script to keep it up-to-date. DISCLAIMER: The script was vibe coded by Claude, all I’ve done is checked it for obviously silly shit.

   services.caddy = {
     enable = true;
+    package =
+      pkgs.caddy.withPlugins {
+        plugins = [
+          "github.com/jasonlovesdoggo/caddy-defender@v0.8.5"
+          "github.com/tailscale/caddy-tailscale@v0.0.0-20250207163903-69a970c84556"
+        ];
+        hash = "sha256-z+zj3rfXbyxldRjO1yoLD77ACRWEAofzMDiZe/bHAqw=";
+     }
     globalConfig = ''
       servers {

This will swap out the caddy binary in your NixOS configuration with one that has the caddy-tailscale module patched in. You will also need to provide a Tailscale authkey to allow the caddy-tailscale module to interact with the Tailscale API and provision new nodes in your Tailnet. This can be done by setting the TS_AUTHKEY environment variable in your Caddy service configuration. Here’s an example of how to do so using sops-nix:

sops.secrets.services-tsauthkey-env = {
  sopsFile = ./secrets/tailscale.env;
  owner = config.services.caddy.user;
  format = "dotenv";
};
services.caddy.environmentFile = config.sops.secrets.services-tsauthkey-env.path;

Create the secrets/tailscale.env file with sops, the contents should look something like this:

TS_AUTHKEY=tskey-auth-something-something

Setting up your first private service

Now we’re ready to rumble. To showcase the full capabilities of caddy-tailscale, we’re going to set up Grafana with proxy authentication and let Tailscale handle the authentication for us. Here’s a simple example (N.B. $TAILNET_NAME should be replaced with your Tailscale network name.):

services.caddy.virtualHosts = {
  "https://${config.services.grafana.settings.server.domain}" = {
    extraConfig = ''
      # This will create a new node at grafana.$TAILNET_NAME.ts.net
      bind tailscale/grafana
      # Enables the Tailscale authentication provider
      tailscale_auth
      # Forwards your Tailscale user ID to Grafana. Usually your email address.
      reverse_proxy ${config.services.grafana.settings.server.http_addr}:${toString config.services.grafana.settings.server.http_port} {
        header_up X-Webauth-User {http.auth.user.tailscale_user}
      }
    '';
  };
};

services.grafana = {
  enable = true;
  settings = {
    "auth.proxy" = {
      # Enable proxy-based authentication
      enabled = true;
      # Automatically create new accounts for people who access the service
      auto_sign_up = true;
      # Do not store login cookies, instead always relying on proxy authentication
      enable_login_token = false;
    };
    server = {
      domain = "grafana.$TAILNET_NAME.ts.net";
      http_addr = "127.0.0.1";
      http_port = 2342;
    };
  };
};

This configuration will set up a new Grafana instance at https://grafana.$TAILNET_NAME.ts.net that is only accessible to members of your Tailscale network. When you visit the URL, your Tailscale identity will be used to log you into your existing Grafana account, or sign you up if this is your first time visiting the service.

Caveats

While this approach works great for most needs, there are a few caveats to be aware of:

• Each service created this way registers as a new device in your Tailnet. Make sure to remain aware of the devices limit on your account.
• Since each “virtual” device being created via this is technically backed by your physical machine, the “Services” tab in your Tailscale dashboard will end up with duplicate entries of services that are running on the same machine. For example, I run 5 caddy-tailscale services on my server, which results in 6 entries of the SSH service being listed. This may be a deal breaker to you, depending on how you use the Tailscale dashboard.

Conclusion

Despite its experimental status, caddy-tailscale is pretty powerful and exposes enough functionality to cover most basic use cases. I’ve been pretty happy running it in my homelab for the past couple months and I’m excited to see how it evolves in the future. If you have any questions or corrections, please let me know either in the comments below or on Mastodon.

GitHub has been getting increasingly unpleasant for the past few years, but them officially announcing its demise as an independent entity and joining Microsoft’s “CoreAI” division was the straw that broke the camel’s back and made me want to use my own Git forge more actively. Thus the Forgejo migration shot up in my TODO list, and ended up being a rather involved process.

This post is gonna have some rambling and going around in circles, but the TL;DR is that I ended up having to migrate individual accounts instead of the whole instance data by using a tool that I ironically generated using GitHub Copilot (the code is available on my Git server).

Attempt 1: The naive way

Forgejo initially used to support migrating data from Gitea pretty easily, but the increased friction of having to maintain a full fork against a moving target forced them to drop support for these migration paths. I of course am stupid and was not aware that I had missed my window, so I had to make the unpleasant discovery that simply swapping the latest Gitea for latest Forgejo will not work anymore. The NixOS manual’s migration section suggested that this was possible, but it’s actually missing crucial information about version requirements that was added later.

My naive attempt almost caused me to irreversibly corrupt my instance data, so I was more careful with future attempts and used a backup each time.

Attempt 2: The gitea dump command

This looked like it had some potential until I realized the lack of a companion gitea restore functionality. The generated dump is a big ZIP file of your entire state directory, along with the SQL commands to recreate your database with whatever database engine you were using. The recommended way to use this dump is to unzip it and then replay the SQL commands into sqlite (my current database) to build up the state directory. This approach kind of felt like a dead-end already but I gave it a good try anyway.

I had also chosen to run Forgejo using PostgreSQL instead of SQLite, since I was already using it with a bunch of other software running on this machine and had proper backup strategies in place. This meant I had to reach out for pgloader to import the SQLite-compatible dump into Postgres which worked out pretty well. Running pgloader sqlite:///var/lib/gitea/data/gitea.db 'postgresql://forgejo:@/forgejo?host=/run/postgresql' as root was able to successfully migrate the database to Postgres.

Unfortunately this didn’t work due to the previously discussed version incompatibilities, and Forgejo rejected this database I had created for it. At this point I had lost a couple hours on this and constant electricity problems at my place had additionally caused my patience to run thin. I decided to cut my losses and re-deployed Forgejo with the same settings as my Gitea server, deciding that I would figure out a way to just copy my account data over since the instance configuration was managed by NixOS anyway (for the most part).

Attempt 3: Vibe coding deployed somewhat effectively

I’m not generally a big believer in the AI hype. I have yet to pay for any of these tools, between GitHub and my employer I get plenty of access to top of the line models that are supposedly reinventing my field of work every 3 months. LLMs have yet to meaningfully help me at work, and the only times I’ve gotten real value out of them is by getting them to write one-off things like this that I am glad to have but would likely never invest the time to upskill into and build myself.

I hooked up the free license to GitHub Copilot that I get for satisfying some criteria of “popular open source maintainer” into Zed and wrote out a simple README file describing what I wanted out of the tool and had it go to town. The end result of this was an unnecessarily abstracted Go project (I doubt anybody would use that directory structure for a project this small) that looked like it would do the job. You can find the source code for this here.

According to a comment on this post, organizations may not be migrated correctly by this script. I do not wish to change the code to an untested version so be prepared to do some manual fixes to address that deficiency. I would also accept a patch from someone who does end up fixing the script for themselves.

Now onto actually running this tool. For this to work, it would require both my new Forgejo server and my old Gitea server to be up at the same time. First hurdle: conflicting ports. This was solved pretty easily.

I screwed up here by running Forgejo on the primary domain and deploying the old Gitea server into a new Tailscale-based service. This caused multiple failures:

Inability to log into the Gitea instance.

I could no longer sign into Gitea since I had 2FA enabled using Passkeys which require the domains to match up.

To make up for the inability to log into my Gitea server to create an access token, I just used the gitea CLI instead.

sudo -i su - gitea
# To get the actual path to the Gitea binary, which weirdly isn't installed for the gitea user?
systemctl cat gitea.service | grep ExecStart=
/nix/store/foo-bar-baz-gitea-1.23.1/bin/gitea admin -w $(pwd) user generate-access-token --token-name forgejo-migration --scopes "read:repository,read:user" --raw

On the forgejo side, a similar dance ensued but this time to actually create my account since this was a fresh start.

sudo -i su - forgejo
systemctl cat forgejo.service | grep ExecStart=
# yes the actual CLI seems to still be available as gitea
/nix/store/foo-bar-baz-forgejo-10.0.0/bin/gitea admin user create -w $(pwd) --username msfjarvis --email me@msfjarvis.dev --admin --access-token --access-token-name forgejo-migration --access-token-scopes "read:user,write:repository"

With access tokens in hand, I ran the tool and hit my second problem.

Tailscale ACLs bite me in the ass

The way Tailscale’s networking shebang works meant that the server running this isolated Gitea service couldn’t connect to it via the network without some tweaks to my ACL policies. I was not feeling like I had this extra debugging in me, so I opted to just temporarily hijack another subdomain pointing to this server for the purpose and resumed from there.

Mirrors of GitHub private repos did not work

Back when I first created mirrors of all my stuff I had also done so for my private repos and just forgot about it. When the tool tried to migrate them they obviously failed to mirror since I wasn’t providing any access tokens for GitHub. I went back to Zed and had Claude add a retry for this and pull an access token from the $GITHUB_TOKEN environment variable.

When I ran the tool again it was able to successfully recreate all the repositories from my Gitea server.

Conclusion

Overall this migration was worthwhile for me, both for documenting it for others in my position as well as for finally being free of Gitea. I trust the people in charge significantly more, and they continue to deliver great work driven in part by them actually using their own software at scale.

May 10th update: Powered by a lack of sleep and extreme fatigue I cooked up a far simpler solution that is mentioned at the end of the post.

Setting the scene

I’m a very happy user of the Glance dashboard, and make use of it multiple times a day.

As a NixOS user, I have a very popular problem of wanting to iterate on my Glance configuration without having to rebuild my whole system every time to see the change.

One way I’ve seen this addressed (please message me on @msfjarvis@androiddev.social if you can find the actual blog post) is to have some sort of “dev” switch that generates a writable version of your configuration using some home-manager tricks that you can poke at before encoding it into Nix. Turning off the aforementioned “dev” mode will make the configuration go through the more usual Nix ways and give you your declarative system back.

I think that idea is cool! I also think that idea kinda sucks when the configuration is in a “language” you despise, like YAML. Which is where I am!

Coming up with a solution

At the end of the day, the problem I have is that there are too many intermediate steps between me writing Nix and it being converted into YAML. So let’s break it down!

Phase 1: Pull out the Nix bits I need to a standalone file that I can muck around with: done

Now, we need a way to convert this Nix code into a YAML file that Glance can ingest. I came across a relatively new project that solves exactly this: nix-converter.

Running nix-converter --from-nix -f path/to/file.nix -l yaml outputs YAML to stdout, ready to be shoved wherever you desire. This worked almost flawlessly, except an easily worked-around issue:

{
  # This generates the following (incorrect) YAML
  # foo: "baz"
  foo.bar = "baz";
  # This way of declaraing the attrsets works around the
  # issue and generates the following:
  # foo:
  #   bar: baz
  foo = {
    bar = "baz";
  };
}

Glance will automatically reload itself if the config file changes, which greatly simplifies the setup. The only remaining bit we have is to automatically generate this YAML every time we make a change to the Nix file, so that we can freely switch between editor and web browser to see changes without taking a trip to the terminal.

I solved this using watchexec and a tiny Bash script that I stuffed into a devshell command:

out_conf=$(mktemp)
${lib.getExe pkgs.watchexec} -r -w "${glanceConf}" -- "${lib.getExe pkgs.nix-converter} --from-nix -f ${glanceConf} -l yaml > $out_conf" &
${lib.getExe pkgs.glance} -config "$out_conf"

This is a mix of regular Bash and Nix which will make sense to Nix users, but for everyone else: ${lib.getExe pkgs.watchexec} gives you a path of the form /nix/store/store-hash-watchexec-version/bin/watchexec which is the binary you can execute. $glanceConf is a Nix variable containing a string path to the glance.nix file we pulled out earlier.

All put together it looks like this. If I need to edit my Glance config, I will go into the glance devShell, boot up Zed, run the dev command in my terminal and enjoy a fully automated iteration experience.

Caveats

This setup is not even remotely perfect, nix-generator has bugs and discrepancies compared to the Nix to YAML implementation inside Nixpkgs. Not all constructs that work fine with Nixpkgs work that well with nix-generator, complicating development.

May 10th update

Out of the blue I had a minor crisis looking at the kludge I had previously put together and realized there was actually a simpler way to be generating this YAML code without having to forego the niceties of Nixpkgs. The end result of that is this commit, which I will explain below.

The bulk of the work is now handled by a bash script, which I have annotated as comments below to explain what’s going on.

#!/usr/bin/env bash

# EXPR is a Nix expression that we will be evaluating 
# and building to obtain our final YAML
EXPR="
let
  # Load nixpkgs from the search path because it's the easiest way to do it.
  pkgs = import <nixpkgs> { };
  # This is the same code we use in our Glance module to generate the YAML
  settingsFormat = pkgs.formats.yaml { };
  settingsFile = settingsFormat.generate \"glance.yaml\" (import ./${1:?});
in
# settingsFile returns a derivation that the `nix build` below will, well, build.
\"\${settingsFile}\"
"

nix build \
  # Prevent Nix from trying to check binary caches each time we build
  --option substitute false \
  # Feed the nixpkgs input from your flakes registry to the search path
  --impure -I nixpkgs=flake:nixpkgs \
  # Pass in our expression from above that will emit the derivation to be built
  --expr "${EXPR}"

# ./result is the default nix3-build output path
glance -config ./result

So while I was rather satisfied with the free ride I had gotten from Simple Analytics till now, having to pay a rather eye-watering 15 USD every month for it was stretching the otherwise $0 USD budget for my blog. The site itself is hosted by Netlify, but I have a small server from netcup which was suitable to take on the load of running a self-hosted analytics server that I could switch to.

Choosing the analytics service

Like the absolute chump I am, my first visit was to awesome-selfhosted’s analytics page and from there, the only options that stuck out were Plausible and Matomo.

I will be honest here, the reason to not choose Matomo was very superficial - I didn’t want to be running PHP and MySQL. Plausible is an Elixir app and thus not much better as far as me understanding the language goes, but I was reasonably more confident about its choice of Clickhouse and Postgres as the backing data stores.

Getting set up

My server runs NixOS, and thankfully there is a module for Plausible already so setup was very simple and all I had to add was this:

services.caddy.virtualHosts = {
  "https://stats.msfjarvis.dev" = let
    p = config.services.plausible.server;
  in
  {
    extraConfig = ''
      reverse_proxy ${p.listenAddress}:${toString p.port}
    '';
  };
};

sops.secrets.plausible-secret = {
  sopsFile = lib.snowfall.fs.get-file "secrets/plausible.yaml";
};
services.plausible = {
  enable = true;
  server = {
    baseUrl = "https://stats.msfjarvis.dev";
    secretKeybaseFile = config.sops.secrets.plausible-secret.path;
  };
};

The only hiccup here was me apparently not knowing what the hell is 64 bytes supposed to mean and failing to fill in a long enough string into the secretKeybaseFile, which is how it came to end with brother-man-please-work-what-the-fuck-man during testing after which I swapped it out for a 20 word passphrase generated by Bitwarden.

Once everything was running, I set up an admin account and replaced the SimpleAnalytics JS snippet on my website with the one Plausible gave me. Notable difference here was a lack of tracking for users who disable JS, so that’s a net improvement to user privacy that I was too naive to think about in 2020.

Migrating existing data from Simple Analytics

To their credit, Simple Analytics made this very easy and the export option is at the top of their settings page. My only complaint is about their range selection feature, I found the calendar widget very un-intuitive and the input field didn’t let me just fill in the date manually.

Once I had the 45 mB CSV in hand, I looked around for someone else’s homework to copy so I didn’t have to write the migration tool myself. I came across this repo which looked like it would do the job but I had no plans to install NodeJS, so I had gpt-4o (via my free, GitHub-provided Copilot license) rewrite it as a single-file Python script that I could hack on more comfortably. That version is available here, with some additional changes described below.

I ran the script and got a handful of CSVs that Plausible was happy to import, after which I realized that the location data was completely missing. Indeed, my hopes of not having to write any code were going to remain so.

I pulled up the Plausible CSV import docs, got the headers it was expecting for the location CSV and was able to piece together the data relatively easily. One big omission was Simple Analytics not recording cities for page visits, which Plausible had a field for. After figuring out what it was expecting, I filled in zeroes for that column.

I have a big gripe with Plausible’s non-existent error reporting here. When my newly generated CSV failed to import, Plausible simply put up an ⚠️ sign in the list of imports and offered no logs. After digging through their GitHub issues I discovered that the errors had to be pulled out from a Postgres table which I was not very impressed by. For others who come by here with the same problem: it’s in the errors column in the oban_jobs table of the plausible database.

The final step was making the data public like it was on Simple Analytics, so my humble blog’s popularity or lack thereof can continue to be scrutinized by anyone.

Wrapping up

All things considered this took me approximately an hour to get done, most of which was spent digging through Plausible’s GitHub issues and re-learning how to use the psql CLI. I’m quite pleased with how it turned out in the end!

Thanks to Tanvi and Abhinav for answering some questions regarding Plausible.

You start with 3 cards in your hand, and draw one each turn. Each card has an energy that decides what it costs to play, and a power that adds to the total to determine which player wins that location. You start at 1 energy and gain +1 energy every turn.

The game has a competitive mode, and I am horrendous at it. Over the course of the 2 years or so I’ve played this game I’ve made multiple excuses as to why I am so unlucky on the ranked ladder but ultimately I just play the game wrong and then blame my opponents. Like a true gamer.

Last month I migrated my personal journal and knowledge base to Obsidian. On a whim, I decided that it might be a fun project to build a system to track and visualize statistics about my Snap games to try and objectively answer the question - why am I so bad?

Laying down the foundation

To get started with analysis, I first need to figure out what data to collect and how.

For a typical game, the stuff you want to record is the following:

• Your deck: The game has many archetypes, some more reliably effective than others
• Opponent’s deck: If they are playing a deck that hard counters or just tends to always beat yours
• Locations: Some locations benefit certain decks and can heavily influence the result
• Outcome: Whether you won, lost or retreated
• Cubes: You lose or gain cubes depending on the outcome and the number of players who “snapped”

Taking all this into account I settled on the following template that I would use for each individual game I recorded.

---
date: {{date}}
time: {{time}}
type: snap-game
result: win/loss/retreat
opponent_name:
opponent_deck:
my_deck:
locations:
  - location1
  - location2
  - location3
cubes: # Use positive for gains (e.g., +4), negative for losses (e.g., -2)
notes:
---

## Game Details

### Key Moments

### Lessons Learned

The YAML frontmatter is what I use to power my dashboards, and the Markdown underneath is unstructured data to record how the game actually progressed so I can also have a play-by-play if I fumbled a good play or managed to cook something clever.

Obsidian provides a very serviceable GUI to edit frontmatter fields which makes the job of filling it in pretty easy.

I fed this page into the Obsidian Templates plugin which takes care of filling in the date and time automatically when I create a new page based on this template.

Building the visualization

For this purpose I leveraged the excellent Obsidian DataView plugin which provides an SQL-like query language that can introspect the Markdown documents in a standard Obsidian Vault and surface structured data from it.

As an example, here’s the code for the dashboard that is in my daily note template to see the current day’s games.

TABLE WITHOUT ID
	length(rows) as "Total Games",
	length(filter(rows, (x) => x.result = "win")) as Wins,
	length(filter(rows, (x) => x.result = "loss")) as Losses,
	length(filter(rows, (x) => x.result = "retreat")) as Retreats,
	sum(map(rows.cubes, (x) => number(x))) as "Net Cubes"
FROM "snap-games"
WHERE type = "snap-game"
	AND date = date("{{date}}")
FLATTEN file.link as FileLink
GROUP BY null

It is pretty self-explanatory, but essentially this walks through every page in my Vault that has its type field set to snap-game and today’s date, then collates the win/loss record from the other YAML frontmatter fields and makes a nice and simple table out of it.

This can also be extended to cover the past week

TABLE
	date as Date,
	time as Time,
	result as Result,
	my_deck as "My Deck",
	opponent_deck as "Opponent's Deck",
	cubes as "Cubes Δ"
FROM "snap-games"
WHERE type = "snap-game"
	AND date >= date(today) - dur(7 days)
SORT date desc, time desc

I can see what decks I played and how they performed relative to each other

TABLE
	length(rows) as Games,
	round(length(filter(rows, (x) => x.result = "win")) / length(rows) * 100, 1) + "%" as "Win Rate",
	sum(map(rows.cubes, (x) => number(x))) as "Net Cubes"
FROM "snap-games"
WHERE type = "snap-game"
	AND date >= date(today) - dur(7 days)
GROUP BY my_deck
SORT length(rows) desc

I can also see which locations favor me the most

TABLE
	length(rows) as "Times Encountered",
	round(length(filter(rows, (x) => x.result = "win")) / length(rows) * 100, 1) + "%" as "Win Rate",
	sum(map(rows.cubes, (x) => number(x))) as "Net Cubes"
FROM "snap-games"
WHERE type = "snap-game"
	AND date >= date(today) - dur(7 days)
FLATTEN locations
GROUP BY locations
SORT sum(map(rows.cubes, (x) => number(x))) desc

Did it help?

I don’t know!

Work getting hectic meant I had little time to actually play and the re-launch of the High Voltage mode took up most of that.

I did play a tiny amount of games when I first built this out, and did pick up on some of my tendencies that make me lose games but ultimately it’s a very small sample size. The new season of Snap will start tomorrow so hopefully I’ll be able to get back into it and actually figure out if this whole shebang was worth it.

• I used internal links of [[this format]] pretty liberally in my Logseq graph as a tag system, which will need migration to Obsidian’s tags mechanism.
• Some of my pages refer to daily journal pages using a humanized date variant which Logseq automatically converted to proper links, which does not work in Obsidian.
• There seems to be no automatic tagger for Obsidian similar to how logseq-automatic-linker works.

Migrating dates and tags to Obsidian

I haven’t found a solution for automated tagging yet, but inspired by Simon Willison’s post about how he uses LLMs for code I decided to have GitHub Copilot write out a Python script to perform the migration of Logseq-isms to the appropriate Obsidian-native patterns.

The choice of Python was for multiple reasons: it’s fast enough for the job, I understand it well, and it can get much farther without needing third-party dependencies.

There wasn’t admittedly a whole lot to this process, since all I was doing is reviewing and running some 100 odd lines of Python then hard resetting my repo after it did the wrong thing. My initial prompt was this:

Write a Python script using only the standard library to process an Obsidian
vault and migrate Logseq patterns to Obsidian's systems in the markdown files.
Below is a list of changes to make:

1. Convert any `[[Thing]]` style links to use Obsidian-native tags by adding a
`#` prefix, resulting in `#Thing`. If there are spaces in the link text,
convert them to underscores.

2. If a link's text matches the date format `May 8th, 2024`, replace its text
by changing the date format to YYYY-MM-DD. In the given example, the output
should be `[[2024-05-08]]`.

The first output from this was unsurprisingly buggy, the script converted [[May 13th, 2024]] to #2024-05-13 instead of [[2024-05-13]]. The second iteration did not resolve the bug either, but in the third one progress was made.

This was where Copilot impressed me a bit. After running the third iteration and finding a bug in the results, I told the chatbot that the date Jun 8th, 2024 was not being converted and it was able to quickly clock that this was caused by my usage of May in the initial prompt which made it look only for full month names, while Jun is obviously abbreviated. I’m not sure if I would’ve figured that one out right away, but it was a good way to also learn more about the Python standard library’s date formatting facilities.

The final script is available here, with individual revisions so you can step through the changes made by Copilot as I prompted it.

Getting set up on mobile

This was the bit I was more excited for, because one major advantage of using Obsidian on mobile that I didn’t realize earlier was that the same plugins Just Work™️. As far as I could tell, Logseq did not have support for plugins on mobile which mas a little annoying because how I interacted with the app changed across platforms even though there was no technical reason for it to. In Obsidian everything works out-of-the-box since plugins get vendored into your vault so you can actually modify and review the plugin code you’re running.

My existing workflow on Android involved manually committing and pushing changes to my graph/vault via MGit which was too many clicks in the MGit app and was getting a little frustrating over time. I asked on the Fediverse if someone was aware of a newer Git client for Android and very quickly discovered PuppyGit.

Once I had removed Logseq and imported my existing vault into Obsidian (I’ve opted to re-use my existing Git repository), I followed PuppyGit’s guide to setting up automatic syncing with Obsidian which worked flawlessly. It works by mapping a list of apps to a list of repositories that they interact with, and using an accessibility service monitors when apps are launched or closed, upon which it would pull remote changes to the repos or commit and push respectively.

The Git plugin I was using for my desktop didn’t work on Android since it required a git binary, but they helpfully include an option to disable it only for the current device which let me delegate to PuppyGit on mobile with no issues.

A week into it I find myself much happier using Obsidian than I ever was with Logseq, and I’ve even started using it for more things beyond journaling such as taking meeting notes and organizing my tasks which I found too unintuitive to achieve with Logseq. Logseq excels at a daily journaling tool but once my needs started expanding it was pretty clear that Obsidian really is the all-rounder I needed.

Logseq has had a long-standing problem with the version of Electron used by it being EOL, which finally prompted its removal from Nixpkgs on security grounds this week and accelerating my migration to Obsidian. The below is a short list of things I did to make Obsidian more comfortable as a daily journaling system.

Moving over content

I made the following changes to the files in my Logseq graph when moving to the new Obsidian Vault

• assets and pages got copied over as-is
• The journals folder got renamed to Daily since I wanna do notes for longer durations as well, and all the files inside renamed to replace underscores with hyphens since it looked like that was more common in Obsidian.

Plugins

I enabled the following core plugins:

• Daily Notes: Automates creation of daily notes
• Templates: Allows you to create reusable templates
• Backlinks: Shows references to the current note

Looking around other people’s experience with this migration path I also found the following community plugins:

• Periodic Notes: Enhanced daily/weekly/monthly notes
• Natural Language Dates: Type dates like “tomorrow” or “next Monday”
• Tasks: Advanced task management
• Dataview: Query and display info from your notes
• Templater: More powerful templating than the core plugin
• Git: Self-managed sync for vaults

Side note: I greatly appreciate the fact that Obsidian defaults to vendoring plugins in the Vault itself so if you’re tracking the .obsidian folder in Git, you get a fully reproducible Vault regardless of what machine you open it on.

Configuration changes

In Settings → Daily Notes:

• Date format: YYYY-MM-DD
• New file location: Daily/
• Open daily note on startup: Enabled

In Settings → Hotkeys:

• Open today’s daily note: Alt+D
• Create new note: Ctrl+N
• Toggle edit/preview mode: Ctrl+E
• Search in all notes: Ctrl+Shift+F

Pending changes

What I’ve done so far is just the result of 30 minutes of looking around, and there is still a bunch of currently broken stuff that needs resolving.

• I used internal links of [[this format]] pretty liberally in my Logseq graph as a tag system, which will need migration to Obsidian’s tags mechanism.

• Some of my pages refer to daily journal pages using a humanized date variant which Logseq automatically converted to proper links, which does not work in Obsidian.

• There seems to be no automatic tagger for Obsidian similar to how logseq-automatic-linker works.

Despite these issues, I think the move to Obsidian will end up being a net positive to my note taking process. Please let me know on Mastodon or the comments here if you have any tips to improve things.

nix3-flake-metadata

The “new” (read: experimental) Nix CLI has a bunch of useful commands one of which is nix flake metadata that dumps a bunch of information about a flake, most notably, a tree-like representation of your inputs. It makes it significantly easier to find duplicate inputs and avoid the problem of “1000 instances of Nixpkgs”. nix-melt can also assist with this, though for this purpose I find the output from the Nix CLI to be easier to grok than navigating the nix-melt TUI.

NixOS module system

The module system is behind a lot of what makes NixOS and Nixpkgs so powerful for composing reproducible systems. The nix.dev reference includes a great tutorial that is a great read regardless of the complexity level of what you are trying to achieve.

Updating your custom packages from a Git branch

This isn’t directly related to Nix itself but to nix-update which is a very popular tool for keeping Nix packages up-to-date'

You can pass the --version=branch argument to update the package from the default branch instead of the latest tag/release for an Arch-style -git package. Specify --version=branch=branch_name if you want to use a non-default branch instead.

The Nix language

It is possible to replace a call to builtins.map with nixpkgs.lib.catAttrs if the transformation is just pulling out a field. For example: builtins.map (item: item.field) list can instead just be catAttrs "field" list.

Building packages from a REPL

In a Nix REPL you can run :bl inputs.nixpkgs.legacyPackages.aarch64-linux.pkgs.attic-server to build the derivation and create a repl-result-out symlink in your current working directory. You can also run :bl nixosConfigurations.ryzenbox.pkgs.attic-client if you wish to build the package from your NixOS configuration’s instance of Nixpkgs.

Avoiding mysterious errors with fixed-output derivations

For many languages Nixpkgs’ builders vendor dependencies in a fixed-output derivation (FOD) which will give seemingly random errors if the contents change but its name doesn’t, since the old copy of the FOD will continue to be used in the later build stages. For example, in a package using buildGoModule, updating the version but not changing the vendorHash will cause a new FOD to be created but the build will continue with the FOD of the previous version.

To avoid these problems when doing manual updates, make sure to change the relevant cargoHash/vendorHash to lib.fakeHash to cause the new FOD to be picked up which will also give you the correct hash to fill in the field.

Tailscale exit nodes on NixOS

Tailscale is a mesh VPN that includes a feature to run certain endpoints as exit nodes, allowing traffic from other devices in the VPN to be routed from that machine.

On NixOS, the relevant firewall knobs for it are exposed under the services.tailscale.useRoutingFeatures option which is required to run an exit node on a machine as well as allow a machine to use a different exit node. Set this to server if you want to run the specific machine as an exit node, and to client if you want it to be able to use exit nodes in your tailnet.

Dual booting Windows using systemd-boot

NixOS includes options within its systemd-boot module that allow configuring boot entries for Windows without requiring manual intervention of copying over EFIs and what not. There are an initial few steps involved to get the relevant values, which are documented in the option reference.

Nix’s builtins.toString

The toString builtin in Nix handles Booleans rather unintuitively (even if it makes sense for the shell interpolation use-case) so here’s the docs for you to go read just in case something else is a surprise.

This post assumes basic familiarity with Fly.io, the flyctl CLI and Nix.

Fly.io is one of the many players in the “We run your app close to your users” space. I’ve used their services for just over a couple years to run my personal instance of linkleaner. Recently I ran into a combination of a platform outage and localized issue that necessitated recreating the app from scratch, and I decided to try doing so locally rather than my usual CI-based deployment system.

Deployments with Fly.io usually start with a Docker/OCI image. Fly.io requires this image to either exist in their own OCI registry at registry.fly.io, or any other publicly accessible registry. The common way to do this is to use a Dockerfile and build the image locally, then use their flyctl CLI to upload that image from your local Docker registry to registry.fly.io and then deploy the app.

Since I use Nix as my Docker image builder (check out this talk that explains why one would prefer it over Docker itself), I do not run Docker locally and had no plans to start now. This guide is mostly the result of that stubbornness so I intend to keep it short.

Setting up

The main ingredient of this is the skopeo CLI that allows one to interact with remote OCI registries. We’ll replace the Fly.io Docker integration steps with skopeo copy that allows copying Docker archives directly to a registry.

You’re also going to need a Fly.io token to authenticate with registry.fly.io. With flyctl installed you can run fly tokens create deploy to get an auth token usable for the purpose. I stored mine in a FLY_AUTH_TOKEN environment variable that will be referenced later.

Building and deploying

First we need to build the image. With Nix it is as easy as running nix build .#container (since I declare the OCI image as a package in my Nix configuration) which will generate a symlink at ./result pointing to a Docker image tarball.

➜ ls -l result
result ⇒ /nix/store/h0jgv00fg68d7gsaadd11ydpk6wxxn8w-docker-image-linkleaner.tar.gz

Now to the actual deployment: assuming your app is already set up (fly apps create), you just need to copy this image over to their registry and trigger a new deployment. skopeo makes this rather easy.

skopeo \
  # Remove the need for a policy file since we only need this for a one off thing
  --insecure-policy \
  copy \
  # The source argument, which is a docker image archive at `./result`
  docker-archive:./result \
  # The destination argument, the `build.image` value from your fly.toml
  docker://registry.fly.io/linkleaner:latest \
  # Injecting the previously generated credentials
  --dest-creds x:"$FLY_AUTH_TOKEN" \
  # Docker v2.2 manifest format: https://containers.gitbook.io/build-containers-the-hard-way#registry-format-oci-image-manifest
  # Probably not necessary but I just wanted to be sure
  --format v2s2

This will print a bunch of stuff while it’s doing its thing and upon success you have now placed your organically built, Docker-free OCI image into the Fly.io registry.

And that’s more or less it! You can now proceed with the usual Fly.io deployment process of running flyctl deploy and it’ll be able to find the image and run your application:

➜ flyctl deploy
==> Verifying app config
Validating /home/msfjarvis/code/linkleaner/fly.toml
✓ Configuration is valid
--> Verified app config
==> Building image
Searching for image 'registry.fly.io/linkleaner:latest' remotely...
image found: img_3mno4wk809624k18

Watch your deployment at https://fly.io/apps/linkleaner/monitoring

-------
Updating existing machines in 'linkleaner' with rolling strategy

-------
 ✔ [1/2] Cleared lease for 0801937b607358
 ✔ [2/2] Cleared lease for 9080155dfe0de8
-------

GIT_EDITOR variable

While Git will usually pick the $EDITOR environment variable to present an edit UI for commit messages and rebase to-dos, it also supports its own $GIT_EDITOR variable that can be used to set a Git-specific editor. This functionality can be leveraged from integrated terminals inside IDEs to allow writing commit message from the IDE without having to use their entire Git UI.

git add

• git add -u will stage changes to all tracked files and leave untracked ones alone.
• git add -N will stage a file without any of its changes. You can treat that as an “intention” to create a file as well as making it easier to use git diff to see what is being added without needing the --cached flag every time.

Git submodules

Probably the biggest pain in the ass in all of Git land. Every time I’ve had to use a submodule, I’ve disliked the experience. Here’s how to properly get rid of a submodule in case you’re switching to git-subrepo or a different solution.

git submodule deinit -f path/to/module
rm -rf .git/modules/path/to/module
git config -f .gitmodules --remove-section submodule.path/to/module
git add .gitmodules
git rm --cached path/to/module

In order, this does the following:

1. deinit removes the checkout of the submodule and removes it from .git/config
2. Deletes the bare repository Git creates of the submodule which is used to generate the actual checkout
3. Edits your .gitmodules file to remove the submodule entry
4. Stages the changes to .gitmodules
5. Deletes the submodule fully from the index

At this point you can commit the changes and hopefully never deal with a submodule again :D

Git credential helpers

While trying to configure pass-git-helper to authenticate with my personal Git forge I ended up learning a bit about how Git interacts with credential helpers

Git credential helpers work via writing and reading from stdin and stdout which helped me debug an issue I was having with pass-git-helper. To get a credential helper to spit out the password, it needs to be called with get as the first argument and pass in some basic key=value data. The interaction looks somewhat like given below, with < indicating the stdin of the credential helper and > indicating its stdout.

# for URL https://git.msfjarvis.dev/msfjarvis/super-private
$ pass-git-helper get
< protocol=https
< host=git.msfjarvis.dev
< path=msfjarvis/super-private
> protocol=https
> host=git.msfjarvis.dev
> username=msfjarvis
> password=hunter2

Maintaining a fork in the presence of fetch.pruneTags

Enabling fetch.pruneTags causes my Nixpkgs Git checkout to constantly delete and fetch tags, since my fork is missing most tags that upstream has so they keep getting pruned when updating from origin and get re-created when fetching upstream.

One solution suggested for this was setting remotes.<name>.tagOpt = "--no-tags" but that didn’t do the job for me.

The thing that worked was to conditionally disable pruneTags for just the origin remote so it would not try to clear out the tags pulled from upstream. Achieved by running git config --local remotes.origin.pruneTags false.

This is a post for the April Cools Club which encourages people to break away from the typical cringiness of April Fools and do things you don’t normally do.

To set the stage, every screenshot you will see going forward is gonna be Minecraft. I just love the game and I have had such fun with it for the past 5 years that it feels remiss to not share every so often (which I do these days at @msfjarvis@mstdn.games). These are gonna be in order from oldest to newest, and I’ll try to annotate each screenshot with dates, alt text and the relevant anecdote as I remember them but honestly a bunch of this is just goofy shit I happened to capture.

Just a bridge, really

This bridge is in a Minecraft world that Sasikanth and I started in the second half of 2022, and built entirely by him next to our starter base. After Sasi kinda moved on from playing on the server (as Minecraft players inevitably do, myself included) I copied the world and started using it as my singleplayer world and I still play on it to this day.

An unlikely friendship

I don’t think I really remember where this is from, but if I had to guess it was the village I and Sasikanth discovered and promptly lay ruin to which today happens to be my full time base.

The start of the storage room

After commandeering the aforemention village I decided to lay roots next door, and this is basically the start of my storage room. The basic design is still the same but it has like, walls and stuff now.

I am a dwarf, and I’m digging a hole

Honestly not much to say, I was in a bit of a slump with my mental health and decided the best use of my mushy brain was to dig down a whole chunk to eventually build a slime farm.

Did I drain this Ocean Monument or did it drain me?

I didn’t play much for a month or two so I decided to pick up a somewhat involved project to get me back in the swing of things, which happened to be a Guardian farm for the prismarine family of blocks. I’ll let the other screenshots paint the picture, but suffice to say I had under-estimated the scope of this 😬

When this was finally done I genuinely used it like 4 times total, turns out I’m not really a prismarine guy so that’s a couple weeks I am not getting back.

Getting real personal with a Warden

To break up the monotony of placing sand for the Guardian farm I paid a visit to a near by Ancient City and ended up a little too close to a Warden, which did eventually kill me.

The sea shanty era

I took another couple months off and then set up a small server with a handful of friends to mess around on a fresh world with relative newbies to the game. A lot of chaos ensued, but a large chunk of the time I spent on the server ended up being me and Yash just boating across oceans looking for anything mildly interesting while in a voice call with our friends. It was fun times, but as always people’s interest dwindled down and I shut the server down a couple weeks later.

Excavating the Nether

I wanted Ancient Debris for a project and decided to just get a whole load of it at once, which resulted in this set of TNT-filled tunnels. Here’s the damage all the TNT did:

My first Sniffers

When the Sniffers were added to the game I just had to get them, and obviously then I created a farm for the seeds they “sniff” up.

The End Ring Project

I envisioned a continous ring of Prismarine walkways around all the End gateways to be a cool way to get to them rather than just pillaring up but I biffed the “circle” so many times that it’s kinda stalled at the moment.

All Trimmed Up

I went on a quest to obtain every single armor trim and enough Netherite to create armor to put them on, which in total probably took 20-odd hours including all the Diamond and Ancient Debris mining as well as actually locating all the trims. My Minecraft closet has more variety than my IRL one which feels like cause for concern.

Bonus randomness

The Minecraft x FaZe collab

For some reason Minecraft likes to spawn Nether Fossils that look way too much like the [FaZe Clan] logo and I apparently have a bunch of them screenshotted, so here they are:

The impossible portal

I manually prune unused chunks before Minecraft updates in order to let them regenerate with the updated terrain and accidentally sliced off this portal as it happened to be on a chunk boundary. It’s all the way back at my starter base so it does not see much use, but it’s there.

The end…?

I don’t know why I picked this to do of all the things I could have for April Cools. However, reliving all the memories of fun times I had with my friends and even by myself were worth the pain it was to go through 800 odd screenshots and find moderately interesting things :)

Hopefully I have something cooler for next year and that I actually give myself more than 4 hours to write it up.

Preface

This post is going to assume some things about you and your project, but you should be fine even if these aren’t true for you.

• You’re somewhat comfortable mucking around with Gradle
• Your project is using Gradle 8.7, the latest as of writing

If you’re stuck on a lower version of Gradle, you will hit this bug with the code samples in the post. Replacing all calls to includeGroupAndSubgroups with includeGroupByRegex can let you work around it temporarily (Note the addition of the .* at the end):

- includeGroupAndSubgroups("com.example")
+ includeGroupByRegex("com.example.*")

Obtaining a baseline

To get an idea for how long it actually takes your project to fetch its dependencies and to establish a baseline to compare improvements again, we can leverage Android Studio’s relatively new Downloads Info view to see how many network requests are being made and how many of those are failing and contributing to our slower build. Gradle has a --refresh-dependencies flag which ignores its existing cache of downloaded dependencies and redownloads them from the remote repositories which will allow us to get consistent results, barring network and disk fluctuations.

In Android Studio, create a new run configuration for Gradle’s in-built dependencies task that will resolve all configurations and give us a more representative number. The --refresh-dependencies flag will force a full re-download to ensure caches do not affect our benchmarks:

When you run this task, you’ll see the Build tool window at the bottom get populated with logs of Gradle downloading dependencies and the Downloads info tab will start accumulating the statistics for it.

How Gradle fetches your dependencies

The Gradle documentation for dependency resolution explains in some depth how the version conflict resolution and caching systems work, but that’s pretty jargon-heavy and too much of a detour from what we’re really here to do so I’ll just Spark Notes™️ it and move on to more fun stuff.

• Gradle requires declaring repositories where dependencies are fetched from.
• Dependencies are looked up in each repository, in declaration order, until they are found in one.
• Gradle makes a lot of network requests as part of this lookup, and it is in our interest to reduce them.

A quick attempt at optimisation

In the previous section I started off by mentioning repositories, which define where Gradle will look for dependencies. The repositories setup for a typical Android project might look something like this:

// settings.gradle.kts
pluginManagement {
  repositories {
    mavenCentral()
    google()
  }
}

dependencyResolutionManagement {
  repositories {
    maven("https://jitpack.io") { name = "JitPack" }
    google()
    mavenCentral()
    maven("https://oss.sonatype.org/content/repositories/snapshots/") {
      name = "Sonatype snapshots
    }
  }
}

This tells Gradle that you want plugin dependencies to be looked up from Maven Central and gMaven, and for all other dependencies to come from JitPack, gMaven, Maven Central, or the Maven Central snapshots repository; in that order. The first simple tweak you can make is to reorder these based on how many of your dependencies you expect to come from what repository.

For example, in a typical Android build most of your plugin classpath will be dominated by the Android Gradle Plugin (AGP), so you’d want gMaven to be come first so that Gradle does not waste time trying to find AGP on Maven Central.

 pluginManagement {
   repositories {
-    mavenCentral()
     google()
+    mavenCentral()
   }
 }

For the dependencyResolutionManagement block, JitPack is very likely to only host one or two of the dependencies you require, while most of them will be coming from gMaven and Maven Central, so shifting JitPack to the very end will significantly reduce the number of failed requests.

 dependencyResolutionManagement {
   repositories {
-    maven("https://jitpack.io") { name = "JitPack" }
     google()
     mavenCentral()
+    maven("https://jitpack.io") { name = "JitPack" }
     maven("https://oss.sonatype.org/content/repositories/snapshots/") {
       name = "Sonatype snapshots
     }
   }
 }

Going for zero

With the minor changes made above we have already significantly improved on our failed requests metric, but why stop at good when we can have perfect.

Gradle’s repositories APIs also support the notion of specifying the expected “contents” of individual repositories, which tells Gradle what groups of dependencies are supposed to be available in what repositories. This allows it to prevent redundant network requests and significantly boosts sync performance.

These filters can be of two types:

• Declaring that certain artifacts can only be resolved from certain repositories: exclusiveContent
• Declaring that a repository only contains certain artifacts: content

The difference is subtle, but should become clearer shortly as we start hacking on our setup.

For our plugins block, we want only gMaven to supply AGP and everything else can come from Maven Central. Here’s how to achieve that:

 // settings.gradle.kts
 pluginManagement {
   repositories {
-    google()
+    exclusiveContent { // First type of filter
+      forRepository { google() } // Specify the repository this applies to
+      filter { // Start specifying what dependencies are *only* found in this repo
+        includeGroupAndSubgroups("androidx")
+        includeGroupAndSubgroups("com.android")
+        includeGroup("com.google.testing.platform")
+      }
+    }
     mavenCentral()
   }
 }

For other dependencies that are governed by the dependencyResolutionManagement block, the setup is similar. To demonstrate the usage of the second kind of filter, we’re introducing an additional constraint: assume the build relies on the Jetpack Compose Compiler, and we go back and forth between stable and pre-release builds of it. The pre-release builds can only be obtained from androidx.dev, while the stable builds only exist on gMaven. If we tried to use exclusiveContent here, it would make Gradle only check one of the declared repositories for the artifact and fail if it doesn’t find it there. To allow this fallback, we instead use a content filter as follows.

 dependencyResolutionManagement {
   repositories {
-    google()
+    google {
+      content {
+        includeGroupAndSubgroups("androidx")
+        includeGroupAndSubgroups("com.android")
+        includeGroupAndSubgroups("com.google")
+      }
+    }
+    maven("https://androidx.dev/storage/compose-compiler/repository") {
+      name = "Compose Compiler Snapshots"
+      content { includeGroup("androidx.compose.compiler") }
+    }
     mavenCentral()
     maven("https://jitpack.io") { name = "JitPack" }
     maven("https://oss.sonatype.org/content/repositories/snapshots/") {
       name = "Sonatype snapshots
     }
   }
 }

This setup tells Gradle the specific artifacts present in these repositories but does not enforce any restrictions on which repository said artifacts can come from. Now, if I use a pre-release version of the Compose Compiler, Gradle will first try to look it up in gMaven and then fall back to the androidx.dev repository.

In the above example we also see JitPack being mentioned, which we only wish to use for a specific dependency that’s unavailable elsewhere. The exclusiveContent filter is precisely for this use case:

dependencyResolutionManagement {
  repositories {
    google {
      content {
        includeGroupAndSubgroups("androidx")
        includeGroupAndSubgroups("com.android")
        includeGroupAndSubgroups("com.google")
      }
    }
    maven("https://androidx.dev/storage/compose-compiler/repository") {
      name = "Compose Compiler Snapshots"
      content { includeGroup("androidx.compose.compiler") }
    }
    mavenCentral()
-    maven("https://jitpack.io") { name = "JitPack" }
+    exclusiveContent {
+      forRepository { maven("https://jitpack.io") { name = "JitPack" } }
+      filter { includeGroup("com.github.requery") }
+    }
    maven("https://oss.sonatype.org/content/repositories/snapshots/")
  }
}

The Sonatype OSS snapshots repository is only intended to be used for snapshot releases of dependencies we’d otherwise source from Maven Central, so we can indicate to Gradle to only search for snapshots in there with a mavenContent directive:

dependencyResolutionManagement {
  repositories {
    google {
      content {
        includeGroupAndSubgroups("androidx")
        includeGroupAndSubgroups("com.android")
        includeGroupAndSubgroups("com.google")
      }
    }
    maven("https://androidx.dev/storage/compose-compiler/repository") {
      name = "Compose Compiler Snapshots"
      content { includeGroup("androidx.compose.compiler") }
    }
    mavenCentral()
    maven("https://jitpack.io")
    exclusiveContent {
      forRepository { maven("https://jitpack.io") { name = "JitPack" } }
      filter { includeGroup("com.github.requery") }
    }
    maven("https://oss.sonatype.org/content/repositories/snapshots/") {
      name = "Sonatype Snapshots"
+     mavenContent {
+       snapshotsOnly()
+     }
    }
  }
}

Bonus section: Kotlin Multiplatform

If you’re working with Kotlin Multiplatform, these directions will sadly not cover all the dependencies being fetched during your build. There’s a YouTrack issue (KT-51379) that you can subscribe to for updates on this, but in the mean time here’s the missing bits:

dependencyResolutionManagement {
  repositories {
    // workaround for https://youtrack.jetbrains.com/issue/KT-51379
    exclusiveContent {
      forRepository {
        ivy("https://download.jetbrains.com/kotlin/native/builds") {
          name = "Kotlin Native"
          patternLayout {
            listOf(
                "macos-x86_64",
                "macos-aarch64",
                "osx-x86_64",
                "osx-aarch64",
                "linux-x86_64",
                "windows-x86_64",
              )
              .forEach { os ->
                listOf("dev", "releases").forEach { stage ->
                  artifact("$stage/[revision]/$os/[artifact]-[revision].[ext]")
                }
              }
          }
          metadataSources { artifact() }
        }
      }
      filter { includeModuleByRegex(".*", ".*kotlin-native-prebuilt.*") }
    }
    exclusiveContent {
      forRepository {
        ivy("https://nodejs.org/dist/") {
          name = "Node Distributions at $url"
          patternLayout { artifact("v[revision]/[artifact](-v[revision]-[classifier]).[ext]") }
          metadataSources { artifact() }
          content { includeModule("org.nodejs", "node") }
        }
      }
      filter { includeGroup("org.nodejs") }
    }
    exclusiveContent {
      forRepository {
        ivy("https://github.com/yarnpkg/yarn/releases/download") {
          name = "Yarn Distributions at $url"
          patternLayout { artifact("v[revision]/[artifact](-v[revision]).[ext]") }
          metadataSources { artifact() }
          content { includeModule("com.yarnpkg", "yarn") }
        }
      }
      filter { includeGroup("com.yarnpkg") }
    }
  }
}

If you’re not using a JavaScript target in your project, it should be safe to skip the NodeJS and Yarn repositories but it’s probably easier to keep it configured ahead of time in case you adopt JavaScript in the future.

Conclusion

The exact percentage improvement you can expect can vary depending on how many dependencies you have as well as how many repositories were previously declared and in what order, but you should most definitely see a noticeable difference consistently. These are the before and after numbers for a project I optimised for my day job.

Before

After

Like and subscribe, and hit that notification bell so you don’t miss my next post some time within this decade (hopefully).

I’m collecting some tips here about “fancy” things I’ve done using Renovate that may be helpful to other folks. You’ll be able to find more details about all of these in their very high quality docs at docs.renovatebot.com.

Disabling updates for individual packages

There are times where you’re sticking with an older version of a package (temporarily or otherwise) and you just don’t want to see PRs bumping it, wasting CI resources for an upgrade that will probably fail and is definitely not going to be merged. Renovate offers a convenient way to do this:

{
  "packageRules": [
    {
      "managers": ["gradle"],
      "packagePatterns": ["^com.squareup.okhttp3"],
      "enabled": false
    }
  ]
}

Grouping updates together

Renovate already includes preset configurations for monorepos that publish multiple packages with identical versions, but you can also easily add more of your own. As an example, here’s how you can combine updates of the serde crate and its derive macro.

{
  "packageRules": [
    {
      "managers": ["cargo"],
      "matchPackagePatterns": ["serde", "serde_derive"],
      "groupName": "serde"
    }
  ]
}

Set a semver range for upgrades

Sometimes there are cases where you may need to set an upper bound on a package dependency to avoid breaking changes or regressions. Renovate offers intuitive support for the same.

{
  "packageRules": [
    {
      "matchPackageNames": ["com.android.tools.build:gradle"],
      "allowedVersions": "<=7.4.0"
    }
  ]
}

Supporting non-standard dependency declarations

Dependency versions are sometimes specified without their package names, for example in config files. These cannot be automatically detected by Renovate, but you can use a regular expression to teach it how to identify these dependencies.

For example, you can specify the version of Hugo to build your Netlify site with in the netlify.toml file in your repository.

[build.environment]
HUGO_VERSION = "0.109.0"

This is how the relevant configuration might look like with Renovate

{
  "regexManagers": [
    {
      "description": "Update Hugo version in Netlify config",
      "fileMatch": [".toml$"],
      "matchStrings": ["HUGO_VERSION = \"(?<currentValue>.*?)\""],
      "depNameTemplate": "gohugoio/hugo",
      "datasourceTemplate": "github-releases"
    }
  ]
}

You can read more about Regex Managers here.

Making your GitHub Actions usage more secure

According to GitHub’s official recommendations, you should be using exact commit SHAs instead of tags for third-party actions. However, this is a pain to do manually. Instead, allow Renovate to manage it for you!

{
  "extends": [
    "config:base",
    ":dependencyDashboard",
    "helpers:pinGitHubActionDigests"
  ]
}

Automatically merging compatible updates

Every person with a JavaScript project has definitely loved getting 20 PRs from Dependabot about arbitrary transitive dependencies that they didn’t even realise they had. With Renovate, that pain can also be automated away if you have a robust enough test suite to permit automatic merging of minor updates.

{
  "automergeType": "branch",
  "packageRules": [
    {
      "description": "Automerge non-major updates",
      "matchUpdateTypes": ["minor", "patch", "digest", "lockFileMaintenance"],
      "automerge": true
    }
  ]
}

With this configuration, Renovate will push compatible updates to renovate/$depName branches and merge it automatically to your main branch if CI runs on the branch and passes. To make that happen, you will also need to update your GitHub Actions workflows.

 name: Run tests
 on:
   pull_request:
     branches:
       - main
+  push:
+    branches:
+      - renovate/**

Closing notes

This list currently consists exclusively of things I’ve used in my own projects. There is way more you can achieve with Renovate, and I recommend going through the docs at docs.renovatebot.com to find any useful knobs for the language ecosystem you wish to use it with. If you come across something interesting not covered here, let me know either below or on Mastodon at @msfjarvis@androiddev.social!

Ever since discovering nix(3) flake check from crane (wonderful tool btw, highly recommend it if you’re building Rust things), I’ve wanted to be able to quickly write my own flake checks. Unfortunately, as with everything Nix, dummy-friendly documentation was hard to come by so I started trying out a bunch of things until I ended up with something that worked, which I’ll share below.

The premise

I had been using a basic shell script with a nix-shell shebang for a while to run formatters on my scripts repo and while it worked, nix-shell startup is fairly slow and it just wasn’t cutting it for me. So I decided to try porting it to nix flake check which would benefit from evaluation caching and be faster while removing the overhead of nix-shell from the utility script.

The thing you’re here for

Like everything in Nix, the checks needed to be derivations that Nix will build and run the respective checkPhase of. So naively, I put together this to run the alejandra Nix formatter, shfmt to format shell scripts and shellcheck to lint them:

outputs = {
  self,
  nixpkgs,
  flake-utils,
}:
  flake-utils.lib.eachDefaultSystem (system: let
    pkgs = import nixpkgs {inherit system;};
    files = pkgs.lib.concatStringsSep " " [
      # Individual shell scripts from the repository
    ];
    fmt-check = pkgs.stdenv.mkDerivation {
      name = "fmt-check";
      src = ./.;
      doCheck = true;
      nativeBuildInputs = with pkgs; [alejandra shellcheck shfmt];
      checkPhase = ''
        shfmt -d -s -i 2 -ci ${files}
        alejandra -c .
        shellcheck -x ${files}
      '';
    };
  in {
    checks = {inherit fmt-check;};
  });

I needed a space separated list of my shell scripts to pass to shfmt and shellcheck, so I used a library function from nixpkgs called concatStringsSep that takes a list, and concatenates it together with the given separator. That’s the files binding declared in the snippet above.

Here I ran into my first problem: Nix expects every derivation to generate an output which meant this doesn’t actually build.

➜ nix flake check
error: flake attribute 'checks.fmt-check.outPath' is not a derivation

There’s been some discussion about this but the TL;DR is that mkDerivation must produce an output. So I tried to cheat around this requirement by faking an output.

diff --git flake.nix flake.nix
index b7fef3b99110..a531a30ad88e 100644
--- flake.nix
+++ flake.nix
@@ -18,6 +18,7 @@
       ];
       fmt-check = pkgs.stdenv.mkDerivation {
         name = "fmt-check";
+        dontBuild = true;
         src = ./.;
         doCheck = true;
         nativeBuildInputs = with pkgs; [alejandra shellcheck shfmt];
         checkPhase = ''
@@ -25,6 +26,11 @@
           alejandra -c .
           shellcheck -x ${files}
         '';
+        installPhase = ''
+          mkdir "$out"
+        '';
       };
     in {
       checks = {inherit fmt-check;};

dontBuild does exactly what you’d think and makes Nix not execute the buildPhase of the derivation, and the mkdir $out in the installPhase generates the output directory Nix was looking for which is still valid even if completely empty.

You can make this slightly faster by using a smaller stdenv that won’t pull in a compiler toolchain or be rebuilt when said toolchain is updated:

diff --git flake.nix flake.nix
index 7ce7a2ba80f8..b69db13fbc6d 100644
--- flake.nix
+++ flake.nix
@@ -16,7 +16,7 @@
       files = pkgs.lib.concatStringsSep " " [
         # bunch of shell scripts since I didn't have an extension I could glob against
       ];
-      fmt-check = pkgs.stdenv.mkDerivation {
+      fmt-check = pkgs.stdenvNoCC.mkDerivation {
         name = "fmt-check";
         dontBuild = true;
         src = ./.;

The final result

This is what the flake looked like for me after all this

{
  description = "A very basic flake";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
    flake-utils.url = "github:numtide/flake-utils/main";
  };

  outputs = {
    self,
    nixpkgs,
    flake-utils,
  }:
    flake-utils.lib.eachDefaultSystem (system: let
      pkgs = import nixpkgs {inherit system;};
      files = pkgs.lib.concatStringsSep " " [
        # Individual shell scripts from the repository
      ];
      fmt-check = pkgs.stdenvNoCC.mkDerivation {
        name = "fmt-check";
        dontBuild = true;
        src = ./.;
        doCheck = true;
        nativeBuildInputs = with pkgs; [alejandra shellcheck shfmt];
        checkPhase = ''
          shfmt -d -s -i 2 -ci ${files}
          alejandra -c .
          shellcheck -x ${files}
        '';
        installPhase = ''
          mkdir "$out"
        '';
      };
    in {
      checks = {inherit fmt-check;};
    });
}

It’s probably not idiomatic Nix (for some definition of idiomatic) but the entire thing has been a trial and error anyway so 🤷

I’m very much a noob when it comes to Nix so any feedback is very welcome and appreciated!

Alternative solution (June 2024 update)

The above ‘hack’ can also be changed to use pkgs.runCommand which I recently learned about.

{
  description = "A very basic flake";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
    flake-utils.url = "github:numtide/flake-utils/main";
  };

  outputs = {
    self,
    nixpkgs,
    flake-utils,
  }:
    flake-utils.lib.eachDefaultSystem (system: let
      pkgs = import nixpkgs {inherit system;};
      files = pkgs.lib.concatStringsSep " " [
        # Individual shell scripts from the repository
      ];
      # Variant of runCommand intended for commands
      # that run quickly and will be slowed down by
      # the network round-trip.
      fmt-check = pkgs.runCommandLocal "fmt-check" {
        src = ./.;
        nativeBuildInputs = with pkgs; [alejandra shellcheck shfmt];
      } ''
          shfmt -d -s -i 2 -ci ${files}
          alejandra -c .
          shellcheck -x ${files}
          mkdir "$out"
        '';
    in {
      checks = {inherit fmt-check;};
    });
}

I recently came across a blog post from Maarten Balliauw that explained how they had managed to create an ActivityPub compatible identity for themselves, without hosting Mastodon or any other ActivityPub server.

I recommend going to their blog and reading the whole thing, but here’s a TL;DR

• ActivityPub has the notion of an “actor” that sends messages
• This “actor” must be discoverable via a protocol called WebFinger
• WebFinger is ridiculously easy to implement

For all practical purposes, WebFinger is essentially a JSON document that is served at /.well-known/webfinger from a domain and is used to identify “actors” across the Fediverse.

Maarten’s approach to implementing this was to simply place the JSON document at /.well-known/webfinger on their domain balliauw.be, which allowed @maarten@balliauw.be to become a WebFinger-compatible identity that can be searched for on Mastodon and will return their actual @maartenballiauw.be@mastodon.online profile.

Maarten did however note that since they’re relying on static hosting, they’re unable to restrict what identities they can enforce as valid, and thus a search for @anything@balliauw.be will also return their mastodon.online identity.

The implementation

I wanted to also set up something like this, but without the limitation Maarten had run into. Since my website runs on Netlify, I decided to try out using an Edge Function to build this up.

Similar to Maarten, I first obtained my current Fediverse identity from the Mastodon server I am on: androiddev.social (incredible props to Mikhail for making it a reality).

➜ curl -s https://androiddev.social/.well-known/webfinger?resource=acct:msfjarvis@androiddev.social | jq .
{
  "subject": "acct:msfjarvis@androiddev.social",
  "aliases": [
    "https://androiddev.social/@msfjarvis",
    "https://androiddev.social/users/msfjarvis"
  ],
  "links": [
    {
      "rel": "http://webfinger.net/rel/profile-page",
      "type": "text/html",
      "href": "https://androiddev.social/@msfjarvis"
    },
    {
      "rel": "self",
      "type": "application/activity+json",
      "href": "https://androiddev.social/users/msfjarvis"
    },
    {
      "rel": "http://ostatus.org/schema/1.0/subscribe",
      "template": "https://androiddev.social/authorize_interaction?uri={uri}"
    }
  ]
}

With this in hand, now we can get started on wiring this up into our website.

First, create an Edge Function using the Netlify CLI. Here’s the options I chose.

➜ yarn exec ntl functions:create --name webfinger
? Select the type of function you'd like to create: Edge function (Deno)
? Select the language of your function: TypeScript
? Pick a template: typescript-json
? Name your function: webfinger
◈ Creating function webfinger
◈ Created netlify/edge-functions/webfinger/webfinger.ts
? What route do you want your edge function to be invoked on?: /.well-known/webfinger
◈ Function 'webfinger' registered for route `/.well-known/webfinger`. To change, edit your `netlify.toml` file.

Next, add the following code to the TypeScript file just created for you. I’ve added comments inline to explain what each part of the code does so you can customize it according to your needs.

// Netlify Edge Functions run on Deno (https://deno.land), so imports use URLs rather than package names.
import { Status } from "https://deno.land/std@0.136.0/http/http_status.ts";
import type { Context } from "https://edge.netlify.com";

export default async (request: Request, context: Context) => {
  // We obtain the value of the 'resource' query parameter so that we
  // can ensure a response is only sent for the identity we want.
  const url = new URL(request.url);
  const resourceParam = url.searchParams.get("resource");
  if (resourceParam === null) {
    return context.json(
      {
        error: "No 'resource' query parameter was provided",
      },
      {
        status: Status.BadRequest,
      },
    );
    // I want to be searchable as `@harsh@msfjarvis.dev`, so I only
    // allow requests that set the resource query param to this value.
  } else if (resourceParam !== "acct:harsh@msfjarvis.dev") {
    return context.json(
      {
        error: "An invalid identity was requested",
      },
      {
        status: Status.BadRequest,
      },
    );
  } else {
    // Here's the JSON object we got earlier
    return context.json({
      subject: "acct:msfjarvis@androiddev.social",
      aliases: [
        "https://androiddev.social/@msfjarvis",
        "https://androiddev.social/users/msfjarvis",
      ],
      links: [
        {
          rel: "http://webfinger.net/rel/profile-page",
          type: "text/html",
          href: "https://androiddev.social/@msfjarvis",
        },
        {
          rel: "self",
          type: "application/activity+json",
          href: "https://androiddev.social/users/msfjarvis",
        },
        {
          rel: "http://ostatus.org/schema/1.0/subscribe",
          template: "https://androiddev.social/authorize_interaction?uri={uri}",
        },
      ],
    });
  }
};

And that’s it! You can test it out as below to verify things work as expected.

➜ curl -s https://msfjarvis.dev/.well-known/webfinger | jq .
{
  "error": "No 'resource' query parameter was provided"
}

➜ curl -s https://msfjarvis.dev/.well-known/webfinger?resource=acct:anything@msfjarvis.dev | jq .
{
  "error": "An invalid identity was requested"
}

➜ curl -s https://msfjarvis.dev/.well-known/webfinger?resource=acct:harsh@msfjarvis.dev | jq .
{
  "subject": "acct:msfjarvis@androiddev.social",
  "aliases": [
    "https://androiddev.social/@msfjarvis",
    "https://androiddev.social/users/msfjarvis"
  ],
  "links": [
    {
      "rel": "http://webfinger.net/rel/profile-page",
      "type": "text/html",
      "href": "https://androiddev.social/@msfjarvis"
    },
    {
      "rel": "self",
      "type": "application/activity+json",
      "href": "https://androiddev.social/users/msfjarvis"
    },
    {
      "rel": "http://ostatus.org/schema/1.0/subscribe",
      "template": "https://androiddev.social/authorize_interaction?uri={uri}"
    }
  ]
}

Thanks again to Maarten for doing the initial research for this and writing about it!

Paparazzi is a Gradle plugin and library that enables writing UI tests for Android screens that run entirely on the JVM, without needing a physical device or emulator. This is massive, since it significantly increases the speed of UI tests as well as allows them to run on any CI system, not just ones using macOS or Linux with KVM enabled.

Unfortunately, Paparazzi does not directly work with Kotlin Multiplatform projects so you cannot apply it to a KMP + Android module and start putting your tests in the androidTest source set (not to be confused with androidAndroidTest. Yes, I know). Why would you want to do this in the first place? Like everything cool and new in Android land, Compose! Specifically, compose-jb, JetBrains’ redistribution of Jetpack Compose optimised for Kotlin Multiplatform.

I’ve sent a PR to Paparazzi that will resolve this issue, and in the mean time we can workaround this limitation.

Setting things up

To begin, we’ll need a new Gradle module for our Paparazzi tests. Since Paparazzi doesn’t understand Kotlin Multiplatform yet, we’re gonna hide that aspect of our project and present it a pure Android library project. Set up the module like so:

// paparazzi-tests/build.gradle.kts
plugins {
  id("com.android.library")
  id("app.cash.paparazzi")
}

android {
  buildFeatures { compose = true }
}

Now, add dependencies in this module to the modules that contain the composables you’d like to test. As you might have guessed, this approach currently limits you to only being able to test public composables. However, if you’re trying to test the UI exposed by a “common” module like I am, that might not be such a big deal.

// paparazzi-tests/build.gradle.kts
dependencies {
  testImplementation(projects.common)
}

And that’s pretty much it! You can now be off to the races and start writing your tests:

// paparazzi-tests/src/test/kotlin/UserProfileTest.kt
class UserProfileTest {
  @get:Rule val paparazzi = Paparazzi()

  @Test
  fun light_mode() {
    paparazzi.snapshot {
      MaterialTheme(colorScheme = LightThemeColors) { UserProfile() }
    }
  }

  @Test
  fun dark_mode() {
    paparazzi.snapshot {
      MaterialTheme(colorScheme = DarkThemeColors) { UserProfile() }
    }
  }
}

Consult the Paparazzi documentation for the Gradle tasks reference and customization options.

Recipes

Disable release build type for test module

If you use ./gradlew check in your CI, our new module will be tested in both release and debug build types. This is fairly redundant, so you can disable the release build type altogether:

// paparazzi-tests/build.gradle.kts
androidComponents {
  beforeVariants { variant ->
    variant.enable = variant.buildType == "debug"
  }
}

Running with JDK 12+

You will run into this issue if you use JDK 12 or above to run Paparazzi-backed tests. I’ve started working on a fix for it upstream, in the mean time it can be worked around by forcing the test tasks to run with JDK 11.

// paparazzi-tests/build.gradle.kts
tasks.withType<Test>().configureEach {
  javaLauncher.set(javaToolchains.launcherFor {
    languageVersion.set(JavaLanguageVersion.of(11))
  })
}

Testing with multiple themes easily

Using an enum and Google’s TestParameterInjector you can write a single test and have it run against all your themes.

// paparazzi-tests/src/test/kotlin/Theme.kt
import androidx.compose.material3.ColorScheme

enum class Theme(val colors: ColorScheme) {
  Light(LightThemeColors),
  Dark(DarkThemeColors),
}

// paparazzi-tests/src/test/kotlin/UserProfileTest.kt
@RunWith(TestParameterInjector::class)
class UserProfileTest {
  @get:Rule val paparazzi = Paparazzi()

  @Test
  fun verify(@TestParameter theme: Theme) {
    paparazzi.snapshot(name = theme.name) {
      MaterialTheme(colorScheme = theme.colors) { UserProfile() }
    }
  }
}

Gradle’s convention plugins are a powerful feature that allow creating simple, reusable Gradle plugins that can be used across your multi-module projects to ensure all modules of a certain type are configured the same way. As an example, if you want to enforce that none of your Android library projects contain a BuildConfig class then the convention plugin for it could look something like this:

com.example.android-library.gradle.kts

plugins {
  id("com.android.library")
}

android {
  buildFeatures {
    buildConfig = false
  }
}

Then in your modules, you can use this plugin like so:

library-module/build.gradle.kts

plugins {
  id ("com.example.android-library")
}

Setting up convention plugins in your project

Gradle’s official sample linked above mentions buildSrc as the location for your convention plugins. I’m inclined to disagree, buildSrc has historically had issues with IDE support and it’s special status within Gradle’s project handling means any change within buildSrc invalidates caches for your entire project resulting in incredible amounts of time lost during incremental builds.

The solution to all of these problems is composite builds, and Josef Raska has a fantastic article that thoroughly explains the shortcomings of buildSrc and how composite builds solve them.

A full explainer on the topic is slightly out of scope for this post, but I can wholeheartedly endorse Jendrik Johannes’ idiomatic-gradle repository as an example of setting up the Gradle build of a real-world project while leveraging features introduced in recent versions of Gradle. I highly recommend also checking out their ‘Understanding Gradle’ video series.

Why would you want to make binary plugins out of convention plugins

First, let’s answer this: what is a binary plugin?

A Gradle plugin that is resolved as a dependency rather than compiled from source is a binary plugin. Binary plugins are cool because the next best thing after a cached compilation task is one that doesn’t exist in the first place.

For most use cases, convention plugins will need to be updated very infrequently. This means that having each developer execute the plugin build as part of their development process is needlessly wasteful, and we can instead just distribute them as maven dependencies.

This also makes it significantly easier to share convention plugins between projects without resorting to historically painful solutions like Git submodules or just straight up copy-pasting.

Publishing your convention plugins

To their credit, Gradle supports this ability very well and you can actually publish all plugins within a build/project with minimal configuration. The changes required to publish Android Password Store’s convention plugins for Android are:

build-logic/android-plugins/build.gradle.kts

-plugins { `kotlin-dsl` }
+plugins {
+  `kotlin-dsl`
+  id("maven-publish")
+}
+
+group = "com.github.android-password-store"
+
+version = "1.0.0"

After that you can run gradle -p build-logic publishToMavenLocal and it will Just Work™️. You can configure additional publishing repositories in similar fashion to how you’d do it for a library project.

If like me you need to publish these to Maven Central, you’ll need slightly more setup since it enforces multiple security and publishing related best practices. Here’s how I use gradle-maven-publish-plugin to configure the same (gradle.properties changes omitted for brevity, the GitHub repository explains what you need):

build-logic/settings.gradle.kts

+pluginManagement {
+  repositories {
+    mavenCentral()
+    gradlePluginPortal()
+  }
+  plugins {
+    id("com.vanniktech.maven.publish.base") version "0.19.0"
+  }
+}
+

build-logic/android-plugins/build.gradle.kts

+import com.vanniktech.maven.publish.JavadocJar
+import com.vanniktech.maven.publish.JavaLibrary
+import com.vanniktech.maven.publish.MavenPublishBaseExtension
+import com.vanniktech.maven.publish.SonatypeHost
+import org.gradle.kotlin.dsl.provideDelegate
+
-plugins { `kotlin-dsl` }
+plugins {
+  `kotlin-dsl`
+  id("com.vanniktech.maven.publish.base")
+  id("signing")
+}
+
+configure<MavenPublishBaseExtension> {
+  group = requireNotNull(project.findProperty("GROUP"))
+  version = requireNotNull(project.findProperty("VERSION_NAME"))
+  publishToMavenCentral(SonatypeHost.DEFAULT)
+  signAllPublications()
+  configure(JavaLibrary(JavadocJar.Empty()))
+  pomFromGradleProperties()
+}
+
+ afterEvaluate {
+  signing {
+    val signingKey: String? by project
+    val signingPassword: String? by project
+    useInMemoryPgpKeys(signingKey, signingPassword)
+  }
+ }

This will populate your POM files with the properties required by Maven Central and sign all artifacts with PGP.

Consuming your new binary plugins

With your convention plugins converted to shiny new binary plugins, you might be inclined to start using them like so:

autofill-parser/build.gradle.kts

 plugins {
-  id("com.github.android-password-store.published-android-library")
-  id("com.github.android-password-store.kotlin-android")
-  id("com.github.android-password-store.kotlin-library")
-  id("com.github.android-password-store.psl-plugin")
+  id("com.github.android-password-store.published-android-library") version "1.0.0"
+  id("com.github.android-password-store.kotlin-android") version "1.0.0"
+  id("com.github.android-password-store.kotlin-library") version "1.0.0"
+  id("com.github.android-password-store.psl-plugin") version "1.0.0"
 }

However, this fails because kotlin-android and kotlin-library plugins resolve to the same binary JAR that encompasses all plugins from the build-logic/kotlin-plugins module and results in a classpath conflict. To better understand how this resolution works, check out the docs on plugin markers.

The way to resolve this problem is to define the plugin versions in your settings.gradle.kts file, where these classpath conflicts will be resolved automatically by Gradle:

settings.gradle.kts

@@ -14,6 +14,25 @@ pluginManagement {
     mavenCentral()
     gradlePluginPortal()
   }
+  plugins {
+    id("com.github.android-password-store.kotlin-android") version "1.0.0"
+    id("com.github.android-password-store.kotlin-library") version "1.0.0"
+    id("com.github.android-password-store.psl-plugin") version "1.0.0"
+    id("com.github.android-password-store.published-android-library") version "1.0.0"
+  }
 }

And you’re off to the races!

Closing notes

This post was motivated by my goal of sharing a common set of Gradle configurations across my projects such as Android Password Store and Claw, which maintain a nearly identical set of convention plugins shared between the projects that I manually copy-paste back and forth. I’ve extracted the build-logic subproject of APS to a separate aps-build-logic repository, set it up for standalone development and configured publishing support. My goal is to supplement this with a continuous deployment workflow where an automatic version bump + release happens after each commit to the main, after which I can migrate my projects to it.

There are many good reasons why you’d want to archive your pictures outside of Google Photos, having an extra backup never hurts, maybe you want an offline copy for reasons, or you just want to get your stuff out so you can switch away from Google Photos entirely.

How to archive your images from Google Photos

1. You can use Takeout, except it always strips the EXIF metadata of your images, and often generates incomplete archives. Losing EXIF metadata is a deal-breaker, because you can no longer organize images automatically based on properties like date, location, and camera type.

2. You can download directly from photos.google.com which preserves metadata, but is embarassingly manual and basically impossible to use if you’re trying archive a few years of history.

So, what’s the solution?

gphotos-cdp

gphotos-cdp is a tool that uses the nearly-perfect method number 2 and makes it automated. It does so by using the Chrome DevTools Protocol to drive an instance of the Google Chrome browser, and emulates all the manual actions you’d take as a human to ensure you get copies of your pictures with all the EXIF metadata retained.

Setting up gphotos-cdp

Disclaimer: I’ve only tested this on Linux. This should be doable on other platforms, but it’s not relevant to my needs so I will not be investigating that.

Ideally you’d want to run this tool on a schedule on a NAS or a server to keep archiving images automatically as they get added to your Google Photos. I personally run this inside a hosted VM on a daily schedule.

For gphotos-cdp to run in a non-interactive manner, it requires your browser data directory with your Google login cookies. You can easily create this with the following command:

google-chrome \
  --user-data-dir=gphotos-cdp \
  --no-first-run  \
  --password-store=basic \
  --use-mock-keychain \
  https://photos.google.com/

This will launch Google Chrome with a brand new profile. Login to Photos, and then close the browser. Optionally, re-run the command to ensure that you do not need to login again.

The flags passed to google-chrome are extracted from the default set of parameters used by gphotos-cdp. I wish I could explain why each flag is necessary, but all I know is that it does the trick. I got them from this GitHub comment on the issue tracker for gphotos-cdp.

Once done, you’ll have a gphotos-cdp directory that you’ll need to move to the /tmp directory of whichever machine you wish to run gphotos-cdp on.

gphotos-cdp is written in Golang so you’ll need to install it first. Once done, run the following command to install the latest version of gphotos-cdp

go install github.com/perkeep/gphotos-cdp@latest

Then you can go ahead and start using gphotos-cdp, as given below

~/go/bin/gphotos-cdp \ # go install puts things in ~/go/bin by default
  -v \ # Enable verbose logging
  -dev \ # Enable dev mode which always uses /tmp/gphotos-cdp as the profile directory
  -headless \ # Run Chrome in headless mode so it works on servers and such
  -dldir ~/photos # Download everything to ~/photos

The automation techniques used are not completely reliable and can often fail. You’ll want to implement some kind of retry-on-failure logic to ensure this is run a few times every day.

Monitoring

With anything built on such a brittle foundation, it’s useful to be able to constantly monitor that things are working as they should.

Using healthchecks.io you can easily set up alerts that notify you of failures running the tool or unintentional gaps in the schedule you run the gphotos-cdp on. I use my healthchecks-monitor CLI in a cron job to run gphotos-cdp every day, and healthchecks.io notifies me via Telegram when it fails. The script running in cron looks like this

#!/usr/bin/env bash

HEALTHCHECKS_CHECK_ID=<UUID for check as given on healthchecks> \
HEALTHCHECKS_USERAGENT=crontab \
~/bin/healthchecks-monitor --retries 3 \ # Try running the command thrice before giving up
  --timer \ # Start off a server-side timer on healthchecks
  --logs \ # Record execution logs on healthchecks in case of failure, to help with debugging
  --exec "~/go/bin/gphotos-cdp -v -dev -headless -dldir ~/photos"

Conclusion

As evident, it’s not an easy task to automatically archive your pictures from Google Photos. The setup is tedious and prone to breakage when any authentication related change happens, such as you accidentally logging out the “device” being used by gphotos-cdp or changing your password, in which case you will need to create the gphotos-cdp directory with Chrome again.

Also, the technique in this post could easily stop working at any time if Google chooses to break it. That being said, gphotos-cdp was last updated in 2020 and still continues to function as-is so there is some degree of hope that it can be used for quite a bit more.

Hopefully this setup causes you minimal grief and allows you to back up your precious memories without relying only on Google :)

As expected, the library ecosystem (specifically the material-components-android library) took a while to support Material You but with the 1.5.0-alpha05 release of the MDC Android library, things are finally at a place where migration to Material You (henceforth referred to as M3) is viable for simpler apps like APS.

APS has had some design work done to it but for the most part remains the culmination of ad-hoc choices (often bad) over a period of several years. With this migration I sought to change that and make things a bit more cohesive as well as give the app some much needed oomph.

Getting the basics in

I began with scouring through the resources in the conveniently isolated M3 website, where the Material Design team has helpfully created a lot of great tools and content to help developers and designers through the process. There’s the “Migration to Material 3” blog post, and the Material Theme Builder to generate palettes/styles/themes for apps, for both Jetpack Compose and Android’s View-based system. These were extremely helpful in getting a headstart on the whole process. It’s all documented in the commit history of the migration PR but I figure some additional context can’t hurt.

Once I had the themes in, I decided to take the opportunity to also introduce a custom typeface. The app has been using Roboto since forever and it felt like it was time to spice things up. I decided to go with Manrope since it is a font I’ve previously used and found to be excellent for visual appeal and accessibility. I’m still not a 100% confident in my choice, so if people have better options in mind I’d love to know down in the comments.

Once the new font face was in, I opted to enable dynamic colors. Admittedly not the right choice, since I should’ve validated the “default” palette first but that’s what I did ¯\(ツ)/¯.

Bugfixes and improvements™️

Once the M3 themes were all prepped, it was time to actually start migrating.

I switched our activities to use the M3 themes, and immediately started noticing bugs from non-idiomatic and straight up incorrect theming we’ve been lugging around for the past couple years.

First step was to update our iconography, which was using inconsistent tints throughout. I updated all of them to use ?attr/colorControlNormal which made them blend in correctly with the rest of the updated UI.

Earlier this year we had migrated a selection UI in one of our screens to use Chips, but we never got the theming right so it always looked kind of wrong. With M3, we were able to revert back to MaterialButtonToggleGroup without regressing on the accessibility issue which made us do it in the first place.

There were a lot more smaller changes that were made to address the remaining visual bugs

• Our onboarding flow was using an incorrect interpretation of ?attr/colorPrimary for theming, and was migrated to use ?android:attr/colorBackground
• A lot of screens were using hard-coded colors, which were migrated to theme attributes
• Many screens used hard-coded styles for buttons and text fields, and were also migrated to theme attributes
• Multiple layouts also referenced typography styles directly and were migrated to the corresponding M3 attributes, based on the mapping table in the “Migration to Material 3” article.
• System bars and Toolbar had to be given explicit styles and colors to match the “flat” aesthetic from our M2 designs.

The final stretch

With the visual fixes out of the way, I went in and cleaned up the themes and styles. I commonized shared attributes such as fonts and widget styles, created M3 variants of other special-purpose themes we had, and got rid of all the now unused M2 theming. Overall, the PR touched 60+ separate files and generated a final diff of +603,-314 lines. The PR can be seen here.

We use a third-party library by Max Rumpf called ModernAndroidPreferences for our settings UI, and it hard-coded the use of AppCompat dialogs. Max was extremely helpful and made that customisable for us over the weekend which allowed us to use the appropriate Material You dialogs consistently. Huge thanks to Max, and check out his library! <3

Screenshots!

Before

After

Closing notes

APS is a very low-effort app when it comes to UI work. We do not have a custom design system, everything follows Material to a T, and we try to stay in that lane. Our migration took me around 9 hours of work over two days, most of which was really spent on menial work such as manually checking all layouts for hard-coded styles and replacing them with attributes. This isn’t representative of what this process would look like for any project which rolls its own design system on top of Material, since they have a lot more to do before they can even begin the migration of their screens.

I’d like to thank the Material Design team once more for the fabulous work they have done both in creating Material You as well as the technical documentation around it. Material Theme Builder was an extremely crucial tool for me that set the tone of the whole process, and I would have certainly repeated the same mistakes I did with Material 2 if it wasn’t for the tooling and guidance from the team.

Unfortunately, if you try to directly use it with RUSTFLAGS='-C target-feature=+crt-static' cargo build there is a good chance you’ll run into an error similar to this:

cannot produce proc-macro for `async-trait v0.1.51` as the target `x86_64-unknown-linux-gnu` does not support these crate types

This is a bit of a head scratcher, because the target (your host machine) definitely supports proc-macro crates. Turns out, even Rust contributors were confused by this. The “fix” for this is apparently to pass in the --target explicitly. The reason behind this seems to be a bug with cargo, where the RUSTFLAGS are applied to the target platform only when --target is explicitly provided. Without it, RUSTFLAGS values are set for the host only which results in the errors we see. More details are available Rust issue #78210

Therefore, the correct way to build a statically linked glibc executable for an x86_64 machine is this:

RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-unknown-linux-gnu

Other potential problems

You may be unable to statically link your binary even after all this, due to dependencies that mandate dynamic linking. In some cases this is avoidable, such as using rustls in place of OpenSSL for cryptography, and reqwest or ureq in place of bindings to cURL for HTTP, not so much in others. Thanks to the convention of native-linking crates using the -sys suffix in their name it is fairly simple to find if your build has dependencies that dynamically link to libraries. Using cargo’s native tree subcommand and greping (or ripgreping for me), you can locate native dependencies. Running cargo tree | rg -- -sys against androidx-release-watcher’s v4.1.0 release gives us this:

$ cargo tree | rg -- -sys
│   │   │   │   ├── curl-sys v0.4.45+curl-7.78.0
│   │   │   │   │   ├── libnghttp2-sys v0.1.6+1.43.0
│   │   │   │   │   ├── libz-sys v1.1.3
│   │   │   │   │   └── openssl-sys v0.9.66
│   │   │   │   ├── openssl-sys v0.9.66 (*)
│   │   │   ├── curl-sys v0.4.45+curl-7.78.0 (*)
│   └── web-sys v0.3.53
│       ├── js-sys v0.3.53

This indicates curl, zlib, openssl, and libnghttp2 as well as a bunch of WASM-related things are being dynamically linked into my executable. To resolve this, I looked at the build features exposed by surf and found that it selects the "curl_client" feature by default, which can be turned off and replaced with "h1-client-rustls" which uses an HTTP client backed by rustls and async-std and no dynamically linked libraries. Enabling this build feature removed all -sys dependencies from androidx-release-watcher, allowing me to build static executables of it.

Thoughts™️

Allocators

Rust’s standard library performs allocations as and when necessary, automatically, and provides a separate no_std mode that eliminates the standard library from the build and only retains the libcore which is suitable for use in bare-metal deployments (more on that in The Embedded Rust Book). Zig takes an alternative approach, where the convention is that all functions in the the standard library that require allocations take in an explicit Allocator which can either be implemented by the user, or you can pick one of the many options available in the standard library itself.

I’ve never given an extreme amount of thought to the allocations my programs perform, so I’m rather unopinionated on this. That being said, whenever I start working on the healthchecks library I’m definitely going to follow the ecosystem’s practices and make allocations explicit for consumers.

Filesystem

The FS APIs appear to be quite expansive, covering everything that I can think of, and I finally discovered a practical use-case for defer! I’m pretty sure I’ll end up passing the incorrect CreateFlags on more than one occasion, but at least they’re documented and discoverable, and I have some memory of doing the same with Python back when I still believed in my country’s education system :P

Formatting

Finally an answer to my “how to format an array” mystery! It’s fascinating that formatting also requires allocations, but then Rust does return an owned String rather than a &str when you format things so that should have been obvious in hindsight.

JSON

The native JSON support looks pretty great, though I am left to wonder how much of serde’s flexibility is available here. Guess we’ll find out soon enough :)

Random numbers and crypto

Great to see native crypto in Zig! I’m not the target audience, but lack of crypto primitives in Rust seems to come up often so it’s a nice plus for Zig.

Formatting specifiers and Advanced Formatting

Zig’s formatting system seems about as powerful as Rust’s, maybe more, so it’s definitely a plus for me since I’ve ended up debugging a lot of code with println!("{:?}", $field) in Rust 😅

Conclusion

I skipped through the parts about HashMaps, Stacks, sorting and iterators since those are fairly straightforward concepts that Zig does not appear to reinvent in any way.

Overall, I’m liking everything I’m seeing. Very excited to start building things in Zig!

Thoughts™️

Same as yesterday, this section will be a brain dump of what I think of the things I learn today.

Pointers

Rust made pointers a very friendly concept thanks to the borrow checker and amazing compiler diagnostics, and Zig seems to follow the same path with keeping them straightforward. I’m not the biggest fan of the variable.* syntax for dereferencing a pointer, since it breaks my existing muscle memory in a major way but I’m sure I’ll get used to it in no time.

Just like Rust, mutable and immutable pointers are explicitly distinct which is a ✅ in my book.

There wasn’t a lot of content on ZigLearn about many-item pointers so I’m still not sure I understand any of it. That’s probably just me though.

I’ve known and used Rust’s usize in my programs, but only after reading about pointer-sized integers on ZigLearn did I actually make the connection that the size of a usize is that of a pointer. 💡

Enums

On a syntactic level, Zig enums are closer to Kotlin than to Rust w.r.t. declaring functions in them, which is very nice.

Structs

The syntax note from enums applies here as well, with an additional nicety about pointers. Specifically, a struct function that accepts a pointer value will automatically dereference the value inside the function body. This only goes one level deep though, so keep that in mind.

const Rectangle = struct {
    length: i32,
    width: i32,

    pub fn swap(self: *Rectangle) void {
        // No explicit dereferencing needed!
        const tmp = self.length;
        self.length = self.width;
        self.width = tmp;
    }
};

Unions

I have never worked with unions, but tagged unions gave me awful ideas about matching Kotlin’s sealed classes functionality so I’m looking forward to writing some cursed code :D

Integer rules and Floats

Zig’s type coercion syntax is nicer than Rust’s, though the lack of a runtime error concerned me initially. Rust’s TryInto trait is explicit about the fact that the conversion is fallible, and thus returns a Result. Zig on the other hand attempts to validate these conversions at compile-time. Given this code:

fn returnsNum() u32 {
    return 1_00_000;
}

test "typecast" {
    testing.expect(@TypeOf(@as(u8, returnsNum())) == u8);
}

The build will fail with:

./src/main.zig:182:46: error: expected type 'u8', found 'u32'
    testing.expect(@TypeOf(@as(u8, returnsNum())) == u8);
                                             ^
./src/main.zig:182:46: note: unsigned 8-bit int cannot represent all possible unsigned 32-bit values
    testing.expect(@TypeOf(@as(u8, returnsNum())) == u8);

This is good, but I haven’t yet found a way to force the conversion to go through in instances where I can confirm that the incoming u32 definitely fits in a u8.

Optionals

Zig’s Optionals are a very good parallel for Rust’s Option, though Zig provides a lot more syntactic niceties.

The fact that you can use a while loop to capture values until they become null is pretty damn sweet.

var numbers_left: u32 = 4;
fn eventuallyNullSequence() ?u32 {
    if (numbers_left == 0) return null;
    numbers_left -= 1;
    return numbers_left;
}

test "while null capture" {
    var sum: u32 = 0;
    while (eventuallyNullSequence()) |value| {
        sum += value;
    }
    expect(sum == 6); // 3 + 2 + 1
}

Conclusion

Everything following optionals was either uncontroversial or too powerful/low level for my current interests, so I admittedly glossed over some of the gory details.

Chapter 2 introduces JSON, which we’ll need for our eventual healthchecks.io library, so I’m looking forward to it! I do have work tomorrow, so we’ll have to see if I can keep up the daily streak :)

My preferred method of learning new languages is rebuilding an existing project in them, like I did when going from Python to Kotlin to Rust. For Zig, I’ve elected to rebuild my healthchecks-rs library. It’s something I use on a day-to-day basis for keeping an eye on my backup jobs, and it would be a great addition to the healthchecks.io ecosystem.

Getting the basics down

Among the resources enlisted on the Zig getting started page, I opted to go with ziglearn.org for learning the ropes of the language. It is concise yet detailed, and the chapter-wise breakdown makes for great mental “checkpoints”, much like the Rust book.

For this post I’m going through chapter 1.

Thoughts™️

I’m going to use this section to jot down my thoughts about Zig, broken down by the sections on ZigLearn. I’ll skip the parts that I don’t have anything to say on.

Assignment

The presence of undefined is very interesting to me. It appears to be functionally identical to Rust’s Default trait, as shown in this snippet (had to skip to structs for this since I was so curious about it).

const std = @import("std");

const Vec3 = struct {
    x: f32,
    y: f32,
    z: f32,
};

pub fn main() void {
    const inferred_constant: Vec3 = undefined;
    std.debug.print("Hello, {d}!\n", .{inferred_constant});
}

which prints:

➜ zig build run
Hello, Vec3{ .x = 0, .y = 0, .z = 0 }!

I have clearly not gotten very far yet, but initial thoughts here: Rust’s trait-based implementation means I can customize the “default” values for my structs, which I’m not seeing in this implicit coercion yet. Guess we’ll find out soon whether or not this can be handled explicitly :D

Arrays

The syntax for array declarations is quite clear and explicit, which I like. Notably, while you can do this:

const implicitly_sized_array = [_]u8{}; // _ means "infer the size"

you cannot have an inferred size reference as the type:

const implicitly_sized_array[_]u8 = {};

Rust also disallows this, but the error is surprisingly worse than with Zig. Rust’s resident diagnostics magician Esteban assures me this is a regression and is being tracked.

The only problem I encountered here was that I can’t figure out how to print an array!

const implicitly_sized_array = [_]u8{0, 1, 2, 3};
std.debug.print("This is an array: {}\n", .{implicitly_sized_array});

// Outputs: "This is an array: "

I found a PR that overhauls formatting but nothing there gave me any pointers on why my code doesn’t work. Hopefully this’ll get cleared up later.

If

Nothing special here, aside from the early introduction to testing, which is slightly more pleasant than with Rust. I do however have qualms about the test output, which is unnecessarily noisy:

Test [2/2] test "while with continue expression"... expected 2080, found 10
/nix/store/nhd75c4sr3l9wlaspilkwawx5ixkn74w-zig-0.7.1/lib/zig/std/testing.zig:74:32: 0x206f9a in std.testing.expectEqual (test)
                std.debug.panic("expected {}, found {}", .{ expected, actual });
                               ^
/home/msfjarvis/git-repos/zig-playground/src/main.zig:29:24: 0x205abd in test "while with continue expression" (test)
    testing.expectEqual(sum, 10);
                       ^
/nix/store/nhd75c4sr3l9wlaspilkwawx5ixkn74w-zig-0.7.1/lib/zig/std/special/test_runner.zig:61:28: 0x22e161 in std.special.main (test)
        } else test_fn.func();
                           ^
/nix/store/nhd75c4sr3l9wlaspilkwawx5ixkn74w-zig-0.7.1/lib/zig/std/start.zig:334:37: 0x20749d in std.start.posixCallMainAndExit (test)
            const result = root.main() catch |err| {
                                    ^
/nix/store/nhd75c4sr3l9wlaspilkwawx5ixkn74w-zig-0.7.1/lib/zig/std/start.zig:162:5: 0x2071d2 in std.start._start (test)
    @call(.{ .modifier = .never_inline }, posixCallMainAndExit, .{});
    ^
error: the following test command crashed:
./src/zig-cache/o/5a647f3d3394214b30b3861a6b0ffbbb/test

Defer

The defer language feature is something I’ve always been curious about since seeing it in Go, so I’m excited to discover use-cases for it when I finally have it available. Through ZigLearn I discovered that defer calls can be stacked to be executed in LIFO order, and Golang does it in the exact same fashion.

Errors

I like how easy it is to define errors, but the syntax feels kinda icky. Having each error enum I declare ‘magically’ become a property on the error keyword doesn’t sit right with me :(

const NumericError = error{};

fn mayError(shouldError: bool) anyerror!u32 {
    return if (shouldError)
        // This is different from what I'm accustomed to as a user of
        // Either/Result type monads.
        error.NumericError
    else
        10;
}

Runtime Safety

Being able to turn off runtime safety features (like bounds checking) in specific blocks is pretty interesting! Not sure if I’ll ever have a valid use for it though…

Conclusion

I like most of what I’ve seen so far in Zig. Aside from the issues I mentioned above, the lack of string type is a very confusing thing for me. I’ve kinda come to expect it everywhere based on my previous experiences with Python, Java, Kotlin, and Rust; but maybe I’ll now learn to appreciate how every character on my screen is just numbers :D.

I was very easily distracted today, so I only made it a third of the way in 3 hours for a chapter that is supposed to take 1 hour for the whole thing. Hoping to finish it tomorrow!

Compiler and IDE setup

The installing Zig page recommends that while using the Zig stable releases is fine for evaluating it, their stable release cadence matches LLVM’s ~6 months which means they are often rendered outdated by the fast pace of Zig development.

Since I wanted to stick with using Nix to manage my (currently) temporary Zig environment, I went with the stable 0.7.1 release available on nixpkgs.

A quick nix-shell -p zig later, I now had access to the Zig compiler.

➜ nix-shell -p zig
➜ zig version
0.7.1

To be able to use VSCode for writing Zig, I also installed the official zls language server for Zig. This did get me go-to-declaration support for the standard library, but not syntax highlighting. I’m not sure if that’s intended, or a bug with my local setup. Syntax highlighting is also present, thanks to Lewis Gaul for his suggestion of using the tiehuis.zig extension.

Learning resources

The Zig team frankly admits that they do not yet have the resources to maintain extensive learning resources, but the Zig community has stepped forward to fill in those gaps. ziglearn.org is a great jumping off point for people who prefer to learn language basics directly, and there is a rustlings counterpart in ziglings for learning by looking at code.

On the official side of things, you get the standard library reference as one would expect, as well as a fairly detailed language reference.

This is in contrast with Rust, which has an officially maintained book and maintains rustlings as a first-party learning resource. They do however are a significantly larger and older team, so maybe with sufficient funding we’ll see Zig be able to devote effort towards this as well.

Your first program

The zig CLI contains commands to generate new projects easily, so let’s create a new binary project.

➜ zig init-exe
info: Created build.zig
info: Created src/main.zig
info: Next, try `zig build --help` or `zig build run`

The build.zig file appears to describe to the zig CLI how to build this program, and src/main.zig is our application code. Here’s what zig init-exe gives you for a “hello world” program:

// src/main.zig
const std = @import("std");

pub fn main() anyerror!void {
    std.log.info("All your codebase are belong to us.", .{});
}

Cheeky.

This post is just a brief overview of how I went about setting things up for learning Zig. I intend to post more detailed blogs as I progress :)

Due to it being a first-party addition to GitHub, Actions has nearly infinite potential to run jobs in reaction to changes on GitHub. You can automatically set labels to newly opened pull requests, greet first time contributors, and more.

Let’s go over some things that you can do with Actions, and we’ll end it with some safety related tips to ensure that your workflows are secure from both rogue action authors as well as rogue pull requests.

Running workflows based on a cron trigger

GitHub Actions can trigger the execution of a workflow in response to a large list of events as given here, one of them being a cron schedule. Let’s see how we can use the schedule feature to automate repetitive tasks.

For Android Password Store, we maintain a list of known public suffixes to be able efficiently detect the ‘base’ domain of the website we’re autofilling into. This list changes frequently, and we typically sync our repository with the latest copy on a weekly basis. Actions enables us to do this automatically:

name: Update Publix Suffix List data
on:
  schedule:
    - cron: "0 0 * * 6"

jobs:
  update-publicsuffix-data:
  # The actual workflow doing the update job

Putting the cron expression into crontab guru, you can see that it executes at 12AM on every Saturday. Going through the merged pull requests in APS, you will also notice that the publicsuffixlist pull requests indeed happen no sooner than 7 days apart.

Mine is a very naive example of how you can use cron triggers to automate parts of your workflow. The Rust project uses these same triggers to implement a significantly more important aspect of their daily workings. Rust maintains a repository called glacier which contains a list of internal compiler errors (ICEs) and code fragments to reproduce each of them. Using a similar cron trigger, this repository checks each new nightly release of Rust to see if any of these compiler crashes were resolved silently by a refactor. When it comes across a ICE that was fixed (compiles correctly or fails with errors rather than crashing the compiler), it files a pull request moving the reproduction file to the fixed pile.

Running jobs based on commit message

Continuous delivery is great, but sometimes you want slightly more control. Rather than run a deployment task on each push to your repository, what if you want it to only run when a specific keyword is in the commit message? Actions has support for this natively, and the deployment pipeline of this very site relies on this feature:

name: Deploy to Cloudflare Workers Sites

on:
  push:
    branches:
      - main

jobs:
  deploy-main:
    if: "contains(github.event.head_commit.message, '[deploy]')"
    # Set up wrangler and push to the production environment

  deploy-staging:
    if: "contains(github.event.head_commit.message, '[staging]')"
    # Set up wrangler and push to the staging environment

This snippet defines a job that is only executed when the top commit of the push contains the text [deploy] in its message, and another that only runs when the commit message contains [staging]. Together, these let me control if I want a change to not be immediately deployed, deployed to either the main or staging site, or to both at the same time. So now I can update a draft post without a full re-deployment of the main site, or make a quick edit to a published post that doesn’t need to be reflected in the staging environment.

The core logic of this operation is composed of three parts. The github context, the if conditional and the contains method. The linked documentation for each does a great job at explaining them, and has further references to allow you to fulfill even more advanced use cases.

Testing across multiple configurations in parallel

Jobs in a workflow run in parallel by default, and GitHub comes with an amazing matrix functionality that can automatically generate multiple jobs for you from a single definition. Take this specific example: