One item purchased, Ten emails

2026-04-08T00:00:00Z

Online shopping is fantastic. A few clicks and you've ordered almost anything from anywhere.

But I've noticed a huge uptick in the volume of emails relating to an online order which makes it frustrating to order anything.

I recently had a purchase which included the following chain

Thanks for your order
Create your account
We've got your order
We've shipped your order
We're expecting your parcel
We've got your parcel
Your order is scheduled for delivery
We've delivered your item (Courier) 8a. We've delivered your item (Vendor)
How was your delivery
Are you happy with your purchase

I'm sure there are a myriad of A/B tests that these businesses have run to optimise the living day lights out of their email campaigns to burn both the brand and the experience as a joyful one in the minds of any consumers. But as Goodhart's Law teaches us, that when a measure becomes a target, it ceases to be a good measure - these sorts of email chains are prime examples of that. Created to optimise whoknowswhat, but ultimately resulting in a frustrating experience.

My solution is ultimately to use a simplelogin alias that I turn off immidiately but this feels like a solution to a problem that shouldn't exist.

Anti-Book Book club

2026-02-28T00:00:00Z

The Anti-book book club started as an excuse to meet with friends every month or two. I wanted to document what the idea was and how it worked.

Regular book clubs all chose a single book to talk about. There are a few disadvantages to this though:

It doesn't cater to personal preferences on the book
You all need to buy the same book

By contrast, the anti-book book club is where we all chose a different book but around a central theme. This has a few advantages

Get to share insights from multiple books and different perspectives
Trade books after
Books cater towards your preferences

So far it’s been quite successful! We’ve had a few themes covered so far

Plays
Human brain
Books we disagree with
Books told from different perspectives
Break the fourth wall
Books you can read in a day

But we also have a huge host of theme ideas

Biographies
Airport books
Great books that are fatally flawed
Take place in a single day
Widely loved, you hated it
You loved it, most hate it
Guilty pleasures

...And many more.

Supply Chain Irony

2026-02-27T00:00:00Z

There is a certain irony that large organisations carry out a myriad of checks, due diligence, impact assessments, contract reviews and more for any and all business they themselves do business with. But, their team of developers npm, pip, or cargo install any and all dependencies built by a single person on the other side of the world.

This is not to say open source projects are untrustworthy. But if you suggested to an enterprise business lawyer that you want to run some random code you found on the internet in production, they would tell you no.

Todoist Setup 2026

2026-02-26T00:00:00Z

I’ve used Todoist for almost a decade and completed over 50,000 tasks on there. Over time my setup has changed quite a lot.

Previously I used a standard GTD setup - one that Todoist itself lends itself to.

If you haven’t read Getting things Done, one thing it defines that anything that requires 2 or more actions as a “Project”. So if you need to plan a holiday, you may have a holiday project and then task related - book flights, define budget etc.

Over time I found this project setup became a burden, tasks were everywhere and I was always juggling what was and wasn’t a “project”. It also took a long time to review my tasks and priorities them because they spanned so many different areas of life.

Instead I now have a simple low maintenance system

4 Projects
- Now - what I must do asap
- Next - things I need to do but aren’t as urgent as now
- Later - tasks with no deadline
- Routines - recurring tasks divided into section of daily, monthly, yearly. These account for a lot of my day to day tasks as I prefer having routine systems.
Labels corresponding to areas of my life. For example:
- @_work is day job tasks
- @_house is anything house related
- @_project/repowarden is anything related to repowarden one of my side projects. These types of projects get a prefix because I can then filter and get all side project related tasks
No more than 3 tasks scheduled per day - from most important to least. I reviewed my Todoist habits last year and found that less than 0.7% of my tasks were marked as important. The things I were doing was important but it makes me consider whether I was prioritising visually for myself to make it easier to know what to do next.

And that’s it.

Crucially, to keep things light I make the following UI tweaks

Make all Labels Grey - I’m deliberately not using them to colour code tasks
Remove Task count - more tasks makes me feel overwhelmed

Finally, it wouldn’t be a system in 2026 without using AI so I find it helpfully to use AI in the following ways

Identify tasks that haven’t been priorities - this is based on giving it context around my overarching yearly theme and priorities
Identify tasks that need to be split (normally they contain the word “and”)
Identify tasks that need more information to complete “speak to Jack” is meaningless, “speak to Jack about the quarterly sales report to identify opportunities” provides the context and offloads the memory.
Find tasks that can be delegated or removed.

Hopefully you find this system useful. As with anything, it could do with further refinement and naturally adapts as my requirements change.

Power Puttering

2026-02-25T00:00:00Z

You know those tiny jobs you never think to do until you’re sat on the loo with no paper? Yeah me too.

To solve those tiny jobs that don’t deserve dedicated time, I decided to start bundling them together into a sort of power hour - hence Power Puttering.

I schedule this power puttering time when I know I have some time free and I do the following (not every week)

Refill all the toilet rolls in bathrooms
Review all PR’s created by Repowarden
Empty household bins in offices and other rooms
Water the house plants
Clean and maintain the vacuum
Send invoices to clients
Set the dishwasher to a clean cycle
Update my wife’s computer
Update my computer
Update any other technology things
Filter my email
Stock coats and cars with tissues, antibacterial, doggie bags, chewing gum etc
Sharpen the kitchen knives
Refill our tea bag thing (we buy the bags in bulk)
Refill our rice, lentils, pasta, flour and other things (again we buy in bulk)
Fix any tech issues
Tighten handles
WD-40 Handles
Charge batteries - Apple TV, mouse, keyboard, doorbell, lights etc.
Clean the coffee machine
Floss
Update servers

Maybe others have certain things like this. But I quite like the regularity of this quiet system to keep things going.

Launch: Repowarden - AI-powered repository maintenance

2026-02-15T00:00:00Z

Today I am launching Repowarden, an AI-powered repository maintenance tool.

The pitch is simple:

Automated dependency updates
Test generation for changed code
Custom code tasks based on your rules
Everything delivered as clean, reviewable pull requests

Why I built it

Most of the burden of maintaining my projects falls into a few categories:

Death by dependabot PR's that stack up
Test coverage falls behind changes so there are gaps I'm not quite aware of
Repetitive clean up tasks never make it into sprints
General bug fixes, QoL improvements etc.

I wanted something that is like a "dev in a box". Sure I could boot up claude code in each of these projects, but that's a huge chore. This is completely automated, like another member of staff.

What Repowarden does

Repowarden connects to your repository, understands the codebase, and opens PRs that do specific maintenance work.

It can:

keep dependencies current without noisy upgrade spam
generate or improve tests where changes happen
run repo-specific tasks like refactors, lint migrations, docs updates, and other custom workflows

The key constraint is quality: changes should be readable, scoped, and easy to merge.

How it fits into a team workflow

I built Repowarden to work with existing engineering rituals. It's not designed for feature development. It just keeps things ticking along so you don't need to worry about how to upgrade Eslint config's, or patch an obscure security problem.

Repowarden just handles the repetitive work and leaves the final call to humans.

What is next

Next up is improving task customization, broadening language support (currently Node, Python and Rust), and giving teams better control over how aggressive maintenance should be.

If your repo has a backlog of "we should tidy this up later," Repowarden is built for that.

Check it out at repowarden.dev.

Thoughts on LLMs and AI

2025-08-29T00:00:00Z

Thou shalt not make a machine in the likeness of a human mind. - Dune

LLMs have been with us for a while now, and the rate that they evolve continues to accelerate. As a developer, I was the prime market for these tools. Not just for programming, but for searching the web, technical questions and the occasional cooking recipe.

These tools are incredible. Claude Code creates a website in minutes. The ever-patient ChatGPT answers questions. Gemini can summarise research papers in language even I can understand.

After using them for a while though, I’ve formed some thoughts on how I use these tools, and how they shape thinking in general.

Inputs over outputs

Like many people, I started by typing a vague question into ChatGPT and waiting for an answer. Then I’d poke at the result until it seemed right.

The problem is obvious: if you don’t define the problem clearly, you can’t judge whether the output is correct.

What I do now is load the model with context up front - a full problem statement or a 5 Whys breakdown. That forces me to clarify what I’m asking, and gives the model something useful to work with.

For programming, that means I don’t ask Claude to “build a backend.” Instead I’ll ask it to suggest edge test cases based on my code coverage, or to compare two different approaches I’m considering. For research, I’ll gather papers using Elicit and then ask Gemini to summarise and distil them.

Improve the input, and you improve the output. Garbage in, garbage out.

Overreliance

Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them. - Dune

When I first used Claude Code it was magical. I typed a few words and zip - off it went.

But over time, I noticed my own abilities atrophying. Without exercising the basics, I started forgetting them. I eventually removed AI from my editor and went back to coding manually, only asking models to review my approach.

Telephone

AI also extends the game of telephone that programming already is. Customers tell analysts, who tell product managers, who tell designers, who tell developers. By the time it reaches you, intent is already muffled.

Now add an AI into that chain: a brand-new “teammate” with no context beyond your codebase. No wonder the outputs can be so poor.

Bus factor

“Vibe coding” makes the AI tool the bus factor.

I often found myself spending more time reviewing AI-generated code than if I’d just written it myself. As a hands-on learner, the process was both laborious and unhelpful.

For me, autocomplete is enough - small nudges without handing over the wheel.

Yes men

Even with attempts to fix it, AI models remain sycophantic. Look through Claude Code’s GitHub issues and you’ll see people complaining at its endless "You’re absolutely right!" replies.

The problem isn’t comedy, it’s that models: A) lack critical thinking, and B) echo whatever you feed them.

That echo chamber has led some to form unhealthy attachments to models. The uproar over GPT-4o being replaced by GPT-5 was so loud that OpenAI restored it for Pro users.

But models aren’t good friends or coworkers. They won’t disagree, push back, or tell us when we’re wrong. Without that, we don’t learn.

Garbage

At work, I tried GitHub Copilot for reviewing pull requests. I hoped it might catch obvious mistakes.

Instead, it nitpicked irrelevant details, suggested edits that broke the build, and ignored the big picture. Copilot has a reputation for this, but Claude and ChatGPT have given me similar results.

People say “90% correct, 10% wrong.” My experience is closer to 70/30 - and you spend more time fixing that 10–30% than if you’d just done it yourself. Diminishing returns set in quickly.

Should you learn to code?

With reports of job cuts and articles declaring “coding is dead,” panic is natural. But the truth is nobody knows what happens in ten years.

We’re living through an AI bubble. Even leading players admit as much. The whole house of cards could collapse - especially if issues like model collapse materialise.

So should you learn to code? I don’t know. Neither does anyone else. That uncertainty itself is the point.

Going forward

LLMs are magical. But they need to be appreciated for what they are: magical parrots¹.

Avoid enslaving yourself to them. Let them sharpen your inputs, but keep the outputs - and the thinking - yours.

[1] As an aside I asked ChatGPT what it thought of the term "magical parrot" for itself. It replied (clipped): "If I were self-reflecting, I might say: being called a 'magical parrot' captures the spectacle of what I do, but not the structure. A parrot repeats without context; I generate within probabilistic context windows. The magic isn’t that I parrot - it’s that I can combine, remix, and weigh information in ways that often feel like original thought, even though the mechanism is utterly different from yours." ↩

Just launched, PlanPacer - The easiest way to add payment plans to your app using Stripe

2025-06-25T00:00:00Z

I launched a new microsaas - PlanPacer! PlanPacer offers payment plans for Stripe. Allowing you to create flexible instalment payments and boost your conversion rates.

Now the elevator pitch is out the way, here is some more information about why and how I built it - as well as some musings on the advantages LLM’s can give solo founders.

Why I built it

Whilst in a meeting with some coworker's, we discussed Stripe as a potential candidate to serve as a payment gateway. The one blocker to using them however was a lack of dynamic payment plans based on products and other variables.

Another coworker lamented that they had a similar issue and ended up rolling their own solution.

I was surprised that stripe didn’t have this functionality since it had such a rich ecosystem. So I dove into their documentation to see what I could find. To my surprise, there wasn’t anything.

I concluded that there must surely then be product offering for this. Again, much to my surprise my search turned up short. There was one product but the documentation was complex and this feature wasn’t their core offering.

Now sprung the idea. If two people had the same problem, that’s product validation. In order to validate it further I would need to put the feelers out. A couple social media posts later and I found that others did have this problem but there was no good solution.

How I built it

On the train home, I decided to get to work. Using a hono template on cloudflare workers and Claude sonnet, I managed to get a bare bones MVP up within a couple hours.

I took the rest of the evening to then polish it up manually as the AI had spuriously created a lot of things that weren’t needed. I also ran through some manual testing of the API.

It broke me from a developer point of view, but I concluded that to launch an MVP there would be no automatic sign up (I run a DB insert and email api keys), or dashboard at all.

Next stage was to launch, I posted to socials and Reddit and am now continuing to post to share the product. Whilst it’s not been a stampede to gain access to the product, I was able to launch in just a few hours. This is crucial because it meant I hadn’t wasted time building a product that no one might have wanted.

If nothing else, it was a great exercise in learning how to launch a proper MVP. And I hope it can serve as a good micro-saas that gains a user base.

Book Notes - The Mezzanine

2025-06-06T00:00:00Z

Nicholson Baker’s The Mezzanine is ostensibly about nothing. An account of a man who purchases shoelaces and milk during his lunch break. That’s it.

But within that hour, Baker stretches the mundane into something strangely majestic. A 120-page meditation on escalators, office supplies, bathroom etiquette, and the fragile genius of product packaging. It’s one of the funniest books I’ve read in years.

What really struck me, though, wasn’t just the humour — it was the tactile world that is evoked. The protagonist lives in a clearly analogue age, where staplers clunk, paper is sacred, and the objects of daily life are designed with care and purpose. Reading it felt like being in a beautifully curated stationery shop where every item has a story.

The footnotes—sprawling, frequent, and delightful—mimic the way thoughts naturally branch and loop back. At times they’re longer than the main text. But rather than feel cluttered, they create a layered, rhythmic inner monologue. If your brain tends to zigzag like mine, it feels like home.

This isn’t a review. I’ve already forgotten some of the book (it was for our anti-book book club some months ago). But it left a strange comfort behind. A sense that someone else, even a fictional someone, thinks the way I do—treating the trivial with the seriousness it sometimes deserves.

Event Driven Architecture Rules of Thumb

2025-01-09T00:00:00Z

Event driven architectures are a fantastic mechanism for powering decoupled services. But they depend on the contract - the actual data points within each event. As with these sorts of things, there is always an "it depends" of what data should go within events. Therefore, this guide is not prescriptive and should be followed as a "rules of thumb" - guiding principles, not rules.

1. Think about testing and versioning early.

Each service should be considered in its own little silo and be looked upon by consumers as a third party. Often overlooked is when you need to change your event to modify the payload. How do you version your events? How do you tell consumers to upgrade? It's certainly not an easy problem to solve. The first step is to define some guidance. In general this what I try to do

Add a "deprecation" message to the old event - inform consumers of when the event will be retired.
Publish both the old and new event for a time
Publish documentation of how to upgrade to the new event
Do a codebase search across the GitHub organisation to find instances of consuming that event - inform those teams who manage those services.
Utilise an event catalogue such as eventcatalog.dev or event bridges schema discovery features to find other areas it might be consumed.

An informal way is to just make sure you inform all the teams who manage the consumers verbally. A more robust way could be to include a "deprecated" property and make sure that consumers check it. Another approach I have seen is to publish explicit "deprecation" events and have them published somewhere publicly (like to slack or something). That said, once you have an established pattern for releasing the old and new events, there is basically zero cost for keeping the old one around (unless there is some kind of security issue).

As regards testing, there are tools such as Pact that can test contracts. But all testing libraries can perform mocks of the event emitter you are using and therefore mandate a certain event structure.

2. Avoid inter-service events

An anti pattern I see in "pure" event driven architectures, is when the service begins to emit events that are consumed only by itself. Whilst events are good for decoupling outside of your applications scope, there is no need to do the same within the service - it's already coupled. It often defeats the object of decoupling by creating more internal complexity for your application developers. If you need to do some an action performing asynchronously, I'd suggest using queues. Otherwise, if you need data on the fly within a system then just perform a direct request to the function internally.

3. Structure events

In general you should try to have the minimum amount of data required for the consuming service(s) to perform the desired action. Across your organisation, it's useful to have a set structure that all events should follow for consistent parsing and error handling. In my view, all events should contain a handful of properties:

The source application name.
The version of the event/source application.
A trace identifier.
The data - a generic JSON object containing all the info required
A time stamp of when it was emitted.
If you want to have a zero-trust architecture, then a sha256 hex digest "signature" property based on the above values.

4. Be strict with accidental coupling

One common mistake I see in event driven architectures is "physiologically coupled" services. This often occurs when team 1 comes along and says "Hey Team 2, we need to build this feature, can you start notifying us when the name of a user changes". And team 2 begrudgingly agrees and starts emitting events when the name changes. No harm done right? Whilst an isolated example, once you start crafting events bespoke to one consumer you create an immediate coupling. And it means in the future, you have to maintain that event for a single consumer. Another example is when another service needs an extra piece of information not entirely related to your event, but adjacent. For example, you might emit an event called "OrderCreated". Another system consumes this but has a requirement to display the expected payment information. Now this should theoretically be handled in a "InvoicePaid" event. But alas, due to the phase of the moon and/or time constraints the payment information associated with the order is added to this event. In this case, we've created a three way coupling - between orders, invoices and the end consumer. Whilst challenging, it's prudent to push back at requests such as these and treat any consumers as a black box. Emit events with the data that tells them about the event. Nothing more. Nothing less.

5. Avoid event chain

Sometimes your system might need to do a sequence like so Order placed > invoice created > invoice paid > order approved

Whilst the instance above is a valid one, it’s extremely easy to fall into a trap where your event driven architecture becomes an “eventually consistent sync architecture”. You eventually get nested chains of events each sequenced one after another.

As an antidote consider:

Do those intermediate steps need to exist?
Could I fan out an event to multiple consumers instead of sequencing?
Could I process the event in parallel? Do I need the result of the preceding event to action the next?
Is this an event in the true sense? Or more of a pub/sub architecture?

Let’s think of some examples based on this:

In the payment example above, it might seem impossible to architect around this. At the end of the day, the invoice should be paid before the order can be approved and shipped. In this case, an EDA pattern is not what you’re after. Rather it’s a saga pattern, an event orchestrator to ensure an exact sequence execution.

Let’s consider another example: a social media system. You may have an event chain like this PostLiked > NotifyPostOwner > GenerateFeedEntry In this case, the events can be processed in parallel. The post owner does not need to be notified of the like before the feed entry is created.

One final example - a notification system. Your event chain might look like this UserRegistered > WelcomeEmailSent > AccountActiviatinReminder > OfferSent We’ll gloss over that you shouldn’t roll your own drip marketing funnel. Anyway… In this case you can use a fan out pattern to react to the original user registered event. If there is some logic around not sending one email before another then create the event with an appearance delay.

In all the examples, dependencies are created unnecessarily making it difficult to debug and each step assuming the previous succeeded.

I'm sure you can think of many more than 5 rules of thumb for event driven architectures but these are just some I think are often overlooked.

ClickOps and IaC - which are Pets and Cattle?

2025-01-08T00:00:00Z

Ancient history tells us that a peoples known as the "sysadmins" or "web masters" used to manually configure servers via the archaic command line. These people literally SSH'ed into machines and ran commands on them. Utter animals.

And these peoples then needed to baby the servers. Nursing them as a young child to make sure they never broke, never went down or got tired.

Then the cloud renaissance came, ushering in a revolutionary way of configuring servers - Infrastructure-as-Code. Gone are the days of running CLI commands - no no. Instead you're writing Terraform or CDK, and configuring servers via version controlled, repeatable methods. Rather than pets that we had to baby, we had cattle - a cow is just a cow. We can get rid of as many as we like and still rebuild the herd.

To many people this appeared like a good thing. We lost a lot of the complexity that was many legacy systems configuring by relics long gone.

The problem is, that Infrastructure-as-code is not synonymous with having cattle.

IaC != Cattle

The promise of infrastructure-as-code is that we should have infinitely repeatable infrastructure. And buy-and-large it brought us that. It brought us 95% of the way there. The problem is that last 5%. That 5% is all the things not repeatable like

Integrations with other systems
Custom scripts
SSL certificates - there are ACM certs but these don't always fit in complex environments.
Literally any EC2 instance - yes there is cloud-init, but it's not a silver bullet

Many may chalk this down as "bad engineering" but this is the reality of the world. If your business honestly could recreate their entire infrastructure from scratch using only code, then great for you - but this is not the norm. This is not even including the fact that no matter what IaC you use, it still involves some kind of initial manual work to provision the cloud accounts. At , we have a "manual-account-setup" terraform module that is run to provision new accounts, but this has to be done manually using local AWS credentials - not in a pipeline.

But let's talk about that 95% that is automated. Now sure, you have a singing and dancing pipeline that runs your Terraform. But then how do you upgrade your Terraform, add governance across multiple teams and ensure that the system is secure? I'll admit that this process is more observable than sshing into servers, but it's still babying of a kind.

And whilst your walled garden of Terraform is performing great. You then get a requirement to connect to a DB outside of your IaC and before you know it you end up manually provisioning security groups based on the IP address of your EC2 instance. Quickly, it becomes only marginally better than the old days.

Manual Setup of a Server does not mean Pets

Let's remember the original goal of IaC - not simply automation, but rather repeatability and immutability. A manually provisioned server could still be "cattle" as long as the exact process to stand it up again is documented. This process is much more difficult than with code, but it's not impossible.

Before the "IaC" tools that we have now, most of the time there was at least some sort of "setup.sh" script used to configure the server. It meant that provisioning a new one was as simple as adding it to the internal network and running the script through a KVM.

So did it matter if your server burned to the ground? No, it was just a server. And as long as you didn't literally only have one server, you can just roll to the backup and provision a new one.

How to be truly resilient

The question is, how do we make our infrastructure truly resilient? Truly repeatable?

1. Shoot your cattle - regularly

Though this might be unpopular with the rest of your team, I'd highly recommend performing a terraform destroy on your non-production environments. Then run a terraform apply to recreate it. If something breaks, you can quickly address your code to make sure that it completely recreates the environment.

2. Adopt the ephemeral mindset

As a team, it's key that you drive the ephemeral mindset into all that you do. This mindset should encompass development, architecture and operations. This means moving beyond the idea of "environments" and instead think in terms of sandboxes that can be destroyed and recreated. In connection with the first point, a good hallmark of this is that you should be able to create an entire sandbox environment for all new pull requests or locally on a developers machine.

There are other tactics to drive resiliency in infrastructure but these are the two that I believe give outsized impacts. Building resilient infrastructure is not a trivial task but by adopting these two tactics you'll set yourself up with a system that requires much less manual intervention, makes upgrades painless and ensures that the infrastructure is always in a deployable state.

Learning to Type Fast

2024-12-12T00:00:00Z

Typing is the conduit by which thoughts flow via the keyboard to the computer. Doing this quickly can massively improve not only your productivity but your flow. Remember when you had a phone that looked like this?

Remember how slow it was? It took ages to write something - 'wic iz y we rote lyk this'.

If you work as a software engineer, you'll be typing all day - whether programming or more predominantly, emails and slack messages.

I found that I was typing reasonably fast (70WPM) but often found myself being unable to keep up with my thoughts as I typed. And I made a LOT of mistakes. The reason? I typed with my index fingers only. I was super quick with it but I knew that I would never be able to write faster unless I retaught myself to write with all 10 digits.

My first scope was to decide what my baseline was - a test that I could use to track my typing speed over the course of this learning experience. For me this was the standard 1 minute typing test on 10fastfingers.

Next, I researched a tool that could re-teach me how to type. I found typingclub to be an incredible tool for this.

The build up

Typingclub starts by building you up what fingers you use. Initially it just focuses on the index finger and builds the muscle memory for what keys it should type. It keeps things simple by having everything lower case and focusing just on the position of your fingers first before typing anything. Once I had reached a stage where I could use 3 fingers I started to apply this in my job. My typing was biblically slow but I was building the muscle memory.

A new keyboard

This has nothing to do with typing speed per se (more on that after), but a new keyboard would affect my typing experience. Previously, I was using a Logitech K380. A fantastic keyboard with multi-bluetooth and plenty of action keys. For £30, it's a steal and I've owned two of them over the years (after the first one met it's unfortunate end with a "e" key being totally broken). But I found the typing experience quite flat. A mechanical keyboard was the obvious answer. But this posed a problem. When I was a teenager, I used to play copious amounts of starcraft 2 - with a mechanical keyboard. Soon, my left forearm swelled with fluid and I eventually had to get carpal tunnel surgery to correct the problem. Wanting to avoid all that pain again (literally), I was a bit hesitant about picking up a new one. Fortunately, mechanical keyboards have come a long way. And one of those new developments is a "low profile" mechanical keyboard. They are a slimmed down version of mechanical keyboards that are less raised so don't strain your hands as much.

After a great deal of searching culminating in an excel spreadsheet comparison, I landed on the Nuphy Air75v2 with Wisteria switches. These switches are most akin to cherry brown switches and have a nice tactile bump on each keystroke. Crucially, they have multi-bluetooth to allow switching between my work and personal machine.

The new keyboard actually reduced my typing speed by a lot. Because I kept accidentally hitting keys or not pressing the correct one entirely. It was worth the persistence though because the keyboard massively improved my enjoyment of typing. The satisfying "thonk" with each keypress echoing like a sturdy industrial-era engine.

More keys

Last up was my pinky finger. I found this difficult to practise because "non-pinky" way of typing was so heavily ingrained. With practise and being conscious about it, I managed it. Although, I'll confess that I'm far from perfect in this regard. I kept grinding on these lessons daily - usually doing 5 a day. They soon moved onto more complex topics such as punctuation, numbers and macros. I'm continuing these lessons now but by around lesson 300 I had a fairly good typing speed.

Once I had grasped using all 10 fingers, I moved on to measuring my progress and practising more common phrases. For this I used, 10fastfingers.com and the top 250 words test. I applied the 80/20 rule in this regard. My thinking was that if I could type the most used 250 words then this would account for 80% of my typing.

Progress

My highest words per minute (as measured on 10fastfingers) is 81WPM and my average is staying well above where I started so I'm count that as a success!

If you work on a computer daily, I'd strongly recommend improving your typing speed as your fluency of transmitting information will increase dramatically.

Allgood - Instant healthcheck webpage and API for JS/TS projects

2024-11-25T00:00:00Z

Over the weekend, I shipped a new open source project - allgood. It's an npm module designed to instantly add a /healthcheck page to your app. Out the box it supports Express, Fastify and Hono. It could be adapted to use Next as well (although I haven't tested this).

After you set it up, you get a page like this:

You can find the code on GitHub and the library published on NPM

It was a fun project to build and satisfying to ship something in a weekend. I haven't built an npm module in over 5 years(!) so it was nice from that perspective also.

Why I built allgood

It mainly boiled down to 3 key reasons:

Shipping is good. It's often easy to get caught up in the trap of endless delivery timelines, interspaced with laborious planning meetings, discussions and the like. Creating something and putting it out into the world is amazing. And this project was something small I could deliver and provide value. It was satisfying to actually see the project live.
Open source is good. A lot of my recent programming work outside of the jobby job has been on commercially focused products (a side hustle if you will). To avoid the complexity that billing and marketing bring, I wanted to create something entirely free and for no tangible benefit to myself other than self-satisfaction. Doing so enabled me to focus on the code and to craft a nice API (internal) - thinking about extensibility and simplicity.
It's a tool I'll use. Like most, I scratch my own itches. It's one of the great things about knowing how to code. You stumble across a problem you have, and you can solve it!

How I built it

By far, the most difficult part was setting up the build system so that it worked with both require and import syntax. It was a total pain, and tells me something not that nice about the state of the JS ecosystem - I digress. I knew this was going to be a pain though so I figured I'd use a boilerplate. The one I landed on needed a little bit of fudging but I got there in the end.

Afterwards, I got started on writing the check code. I started with memory usage as I thought this would be the easiest - simply using process.memoryUsage().

Now though, there was a challenge. The user could configure N checks to run on their healthcheck page. I needed to run all of the checks (in parallel, for speed) and then display the results in a consistent order. I also needed to call the correct check function. The solution I landed on was for all checks to use a consistent interface. This meant I could guarantee the output of each check was consistent. They are sort of decoupled from the app itself meaning the display message and other properties are entirely within their control.

Here is how the interface looked

export interface HealthCheck {
  status: Status;
  value: string;
  componentName: string;
  message: string;
  time: number; // Time in milliseconds that elapsed running the check
}

// The interface that each check implements
export interface CheckFn {
  (config: Config): Promise<HealthCheck>
}

export interface CheckRegistry {
  [key: string]: CheckFn;
}

I then created an object mapping of "checks" to their functions. And then tethered it together with .maps and Promise.all.

Knowing my own monkey mind, I wanted to make this as maintenance free as possible. So when new dependency PR's are created, I can just merge them straight away. Therefore, I took some time to create a test suite. Initially I wanted to use the new native node test suite. But after trying to wrangle with mocks I gave up and used Jest instead - which just worked.

One surprising outcome of this project was the package manager. Using pnpm was a really nice interface and extremely quick. There wasn't a lot of dependencies to install but I never got into the usual hassle of dependency wrangling I usually do.

The future

I'm not sure what this project will yield but after sending it round to some colleagues I've got a good response and had some good feature requests. It's fun to code and create things purely for the sake of the act of creation. If you know how to code, do the same - you'll feel amazing.

Vendor Lock-in is an Imaginary Problem

2024-10-23T00:00:00Z

Vendor lock-in is often cited as a reason to chose on-premise hosting, or to be mindful of "avoiding" it when using the cloud (whatever that means). Whilst there are some compelling arguments to mitigate risks associated with vendor lock-in, I'd argue that this is a completely imaginary problem.

1. You're already locked-in - at every level

Vendor lock-in is always discussed through the lens of the cloud. But, in reality the cloud is not a special class of tooling, it's like everything else. If you use a language like Javascript, you're already locked in to using the myriad of NPM packages you have. Worst still is if you're using NextJS, React, Rails or Laravel. These frameworks all lock you in to some degree. What about your continuous integration? The same organisations that worry about vendor lock-in to the cloud providers never bat an eye lid writing thousands of bespoke lines of github actions workflows. The point is, the cloud is not the only place you're locked in. You're locked-in everywhere. But, it's about making appropriate trade-off.

2. Businesses rarely migrate from a cloud provider

Asides from a few extremely rare cases, it's likely that once AWS or whatever other cloud provider puts their claws into you, you won't escape it's clutches. Rather you'll likely embrace it. Abandoning the lift-and-shift you originally oversaw, and build "cloud native" - but why? Because it's cheaper (on the surface) and the recommended way to do things (according to the cloud providers). You also end up hiring staff with the skills to wrestle these tools to do your bidding. It's unlikely then that after doing all that work you would want to migrate to another provider. No only is the cost so astronomically high to make it unfeasible but there is also no reason, commercial or otherwise, for doing so. Even if they heavily raise prices, it's still a footnote for the costs that most businesses pay. In some cases however, at enormous scale, it does become economically advisable to migrate elsewhere. If you built the product in a vendor agnostic way, I'm sure the migration team will thank you. But if not, you're likely at a scale that the cost of doing this migration is just viewed as "one of those things".

3. You know what you're signing up for

When you go to a shopping centre (or mall), you might need to buy some clothes, gifts or yet another Apple device. But then you suddenly realise that you need shampoo. Now, will you go to the pharmacy shop that sells the shampoo despite the cost being a little higher? Or will you get it from your regular supermarket that is 20 minutes away? Of course, you'll pay the premium and buy it at the shopping centre. There are no tangible benefits to the product - they are the same. What you are paying for is convenience. The cloud is no different, you might go in first of all to simply get some compute. But then quickly you start using it for everything - it's convenient. The business has already signed off on it, there is budget and your tooling is setup for it. In other words, you know what you signed up for when you started using the compute. And if you need to pay a bit extra to use other things then big deal, you would have paid that cost elsewhere in the first place. Public cloud pricing is publicly available (although notoriously difficult to forecast). So, due diligence should be taken up front to understand what kind of cost implications the cloud could have. While convenience often outweighs cost at smaller scales (like buying shampoo one time), in larger enterprises, even slight cost increases can lead to substantial expenses. This is why some companies consider multi-cloud or cloud-agnostic strategies to balance convenience with financial prudence.

The only sound argument against vendor lock-in

Local development. That's it. Once you use bespoke cloud software, it becomes extremely challenging to create working development environments that mimic your live deployment. Therefore, I do favour against using "cloud-native" tools because it makes your life a lot more difficult for doing development work (which makes the money).

Summary

As discussed vendor lock-in is largely an imaginary problem. There may well be 1 (or 2) extremely valid reasons for avoiding vendor lock-in but generally it's something that you need to not worry about and get on with shipping.

DynamoDB Considered Harmful

2024-10-21T00:00:00Z

I think DynamoDB is quite a useful database. But, I'm here to tell you should pretty much never use it whether in a greenfield project or a mature product. Let me explain why.

1. It's inflexibility will slow you down.

Initially, DynamoDB's speed and schema-less nature can make development fast. Although DynamoDB isn't modelled around schema's (like a traditional SQL database), it is modelled around queries. This means you need to know the query model up front. In a mature product, you might have a good idea of what possible queries you would want but in a greenfield product it's downright impossible. But, regardless of how mature your product is we all suffer from a fog of war. It's impossible to know what feature requests will be fired at us by a product manager. This is where DynamoDB will start to become as cumbersome as jeans in a rainstorm. Because you originally modelled the database around the queries you knew about, it becomes inflexible to change for the new queries you need to perform. Often times, teams just add new global secondary indexes (which behind the scenes is a complete copy of your database). But these are limited to 20 per table. And it creates another problem, deciding what index to use when. This becomes a headache to maintain and build upon. Some may reason that they can create other tables around the new query model, or change the existing database. But these solutions are also rife with complexity. For example, DynamoDB has a 1MB limit on results from scans. Therefore, to write a migration script you'd need to loop through all this data, update it in memory, remove the old data and then write it back to the database. Try doing that in a no-downtime fashion. SQL on the other hand can be mashed, broken, cracked and mangled in order to create whatever queries you want. You can join tables where there are separations of relationships. And if you ever get into place where the table design is bad you can do a SQL migration - it might take a while but there are no query limits to contend with.

2. You don't have the scale nor will ever need it

DynamoDB was designed with hyperscaler levels of traffic. Sites receiving this level of traffic clock in at less than 50 across the entire internet. Ergo, you do not need DynamoDB's scale. Even Facebook who started on MySQL (but eventually use other supporting databases), didn't need DynamoDB level scale (initially) and when they did they refactored. Choosing a database technology for scaling reasons is akin to getting "CEO" business cards before writing a line of code - it's premature optimisation.

"But what about scale?" I hear you cry

This is a non-argument made by people who haven't launched serious products. Any SQL database with enough money thrown at it can scale plenty fine. SQL databases power some of the most used websites in the world - including all Wordpress sites (43.3% of all websites), Spotify (626M MAU) and Twitter/X (516M MAU). If you reach a scale where SQL becomes the blockage then you likely have the money to solve the problem either by refactoring parts of your app or mitigating the issues (with caches, increased horizontal capacity, sharding etc). In any case, scale is not a reason to discount SQL.

3. You need other technologies to handle basic database functions

Let's say you are using DynamoDB for your little CRUD app. Percy the product manager swaggers up to your desk and asks "Can we add search and sorting of tables to our product pretty please?". After a few umms and ahhs, you probably realise that DynamoDB ain't gonna cut it. You're going to use another tool, like Opensearch or Algolia. Before any DynamoDB rage nerds ask, yes you can do search using "contains" and you can do sorting. But try doing something even remotely complex and it becomes impossible to perform. You know what could add search and sorting to your app, SQL! DynamoDB needs all this other cruft to provide basic functions to your app. And it's not just a case of spinning up some new system and hey presto it works. Nope! You've got to keep those systems in sync (DynamoDB streams or Kinesis), then you've got to configure indexes and so on. Your technology choice is slowing you down from delivery of features to your customers. Guess who also doesn't care about your database - your customers!

4. It's challenging to work on your system locally

Working with DynamoDB locally isn't as simple as running a docker container (like ehem, MySQL or Postgres). So, then you're focused to have a "remote" development environment. Where you have resources deployed to the cloud that are used by each developer. These can work, but provide horrendous experiences. Changes to system configuration have to be deployed and you can't work offline. There are a whole host of problems that arise as a result of systems that cannot just be run on a computer.

In summary

DynamoDB isn't a bad database. It's simply a tool that came from a certain context - in Amazons case high throughput writes and reads. In 99% of cases, you are not going to be in that same context. Focus on getting there first with the power of a SQL database. Most apps are just CRUD, SQL is super good at that - use it, and get back to building the thing.

PS: A good litmus test to see if your tech stack is working is asking "in the last 6 months to what extent has our technology impeded or prevented the development of new features or fixes?". If the answer is > 3 then you probably chose wrong.

An engineers guide to the Solutions Architecture role

2024-08-21T00:00:00Z

For the past couple of years, I've worked as a solutions architect. Originally I stumbled into the role after being offered a contract, but found I'd been doing something akin to it for years. Coming from an engineering background (rather than business), I found I had to make lots of changes with how I approached work. This article is centred around those who are coming from an engineering background but have minimal exposure to business.

Defining the role

A solution architect can be difficult to define as they often have a broad remit. It can often include (ordered by technical involvement from low to high)

Enterprise Architect - working at an organization level to align IT strategy with business goals.
Solutions Architect - Focuses on specific project designing end-to-end solutions based on business requirements and technical implementation.
Technical Architect - Focuses on low level code based design decisions influencing the implementation heavily.

But, presuming we fit nicely in the second category, you will be responsible for:

Creating end-to-end designs for systems based on technical and business requirements.
Innovating and introducing new technologies to the platform.
Working with delivery teams to make sure they understand the designs created.
Working with business users (and business analysts) to translate requirements and pain points into technical improvements.
Creating PoCs for new systems.
Selecting vendors to solve business problems.

This isn't an exhaustive list but gives you a flavour.

Understanding the business needs

This is the key role that transforms you from a developer into an architect. I'll admit, it's something I'm still working on. Unlike technical skills you can "train" by smashing leetcode, these softer skills cannot be approached in the same way. But there are some good patterns to follow:

Spend lots of time gathering requirements. I've always ended up making mistakes when I've rushed this part. This "discovery" should end up making up at least 60% of the overall project work - depending on the complexity. Principally you're trying to uncover the known unknowns, and the unknown unknowns. It's challenging to know when you're "done" here but a litmus test I do is "could I explain how this project works down to the minute details to someone outside of this team?". This stage involves carefully diagramming the current architecture. It also involves creating sequence, bpmn, or state machine diagrams (amongst others) that map out business processes, and other artifacts of the system. Use different diagramming systems based on the problem you're trying to solve. Consider, "what am I trying to communicate?". Before moving onto the solution phase make sure to have all the business and technical requirements in one place. Then you will be able to validate all your possible solutions meet those requirements.
Consider the solutions. Notice the plural, solutions. Although you may have a good idea of what the "best" idea is, it's vital to consider more than one solution. Why? Because you risk narrowing your focus and going for a solution that appeals to you rather than the business. Openly share the possible solutions with other architects, developers and engineering leaders. These ones will be able to share insights to improve your designs. They may also shed light on other systems that have an impact on your system - I find this especially true when being new to an organization.
Handover - Once a solution has been selected, it's time to make sure it's documented so that the delivery teams can implement it. This phase involves things such as story mapping (what tickets do you need to build the thing), RFCs (for cross cutting technological decisions), ADRs (for project specific concerns), and workshops with developers. I've often found it useful to record a talk track which commentates the architecture diagrams and talks through how it all works.

Tools of the trade

In most organizations, the technology choices should be fairly well trodden - AWS, serverless, NoSQL databases, you get the idea. So what are the "tools" of an architect. Some have already been mentioned but here is a complete list of tools that I use regularly as part of my work.

Diagramming. There are a myriad of diagramming techniques. Try to learn as many as you can. If in doubt, ask others in your team. And when you feel confident enough, run a knowledge share session where you teach others about this technique. In general, for architectures use the C4 model. It's by far the most effective way of communicating technical designs. Ultimately, all diagramming is about effectively communicating to others about how a system or process works.
Frameworks - Think TOGAF, AWS Well Architected and Zachman. These are frameworks that can be used to inform how you create system designs. In particular, the Well Architected framework provides a huge amount of value through its best practices. Make sure you can competently complete the well architected review and understand the terms within it.
Documentation - In addition to diagrams, written documentation is a vital resource that you can provide to stakeholders and delivery teams. Becoming well adept at written communication is a great skill to have that will stand you in good stead. Documents you can produce (but not limited to), are RFCs, ADRs, Vendor comparison matrices and runbooks.
Technical thinking - Coming from an engineering background you should have a good sense of if something you're designing will be secure, performant and cost effective (in that order). Make sure to validate these understandings against the real world by perhaps doing a proof of concept for your selected design. On this point, consider making yourself knowledgeable in things like the OWASP Top 10.
Misc - I couldn't think of an umbrella term for this one. But, I've often found myself reaching for calculator.aws in order to provide a cost analysis/comparison for the systems I design. There are many other tools that can enrich the designs you do.

Measuring success

Engineers can easily measure success by looking at what is delivered. Solutions architects are an "enabler" - they don't "deliver" anything customer facing in the true sense of the word. But, as we have discussed, the success measure is that the project was discovered correctly (i.e., there were no further iterations required after new requirements came to light), good documentation was produced, and the designs were handed off smoothly to the team. Further, you can make sure that your systems were aligned with business goals and the costs matched what was expected. It's a challenge to be disconnected from delivery but still measure success - but it's still possible. And crucially this makes sure that you are delivering value.

Closing thoughts

I've enjoyed working as a solutions architect. Business was a weak area of mine and so it has been useful to understand and expand my knowledge. Further, the opportunity to teach and communicate (through diagrams, code and words) is what I love to do. I hope this small collection of thoughts helps you to become an architect!

Should I learn to code if AI will replace me?

2024-05-07T00:00:00Z

In the past few years, AI has gone from a nerdy pipe dream to reality. Its use has become commonplace amongst professionals and enthusiasts alike. Some are using it to create art, write tweets or improve their appearance.

Although it was predicted that self-driving cars and menial work would be automated, instead it’s targeted a lot of creative pursuits. Already some sectors have seen layoffs as a result of automation.

As a software engineer, and I know I’m not the only one, I’ve been questioning whether I’ll have a job in 5 to 10 years. Will my job even exist? And for those new to software, what should they learn?

I don’t have a crystal ball but these are some personal musings on this topic. Hopefully, I can say “I told you so!” In 5 years. Or I’ll be writing a reflective piece on showing scepticism toward prediction pieces. Time will tell.

Will software engineering disappear in 5 years?

I highly doubt this one. One analogy of how I see software engineers in the future is to that of the draughtsman. Historically, a draughtsman was employed to create various technical drawings of various things - buildings, bridges etc.

In the 60’s CAD came along and changed the whole scene. Now draughtsman had a large portion of their work removed as it had been automated away. Nowadays the profession doesn’t exist per se but has in large part become “CAD Technicians”.

Now that sounds like I think software engineers are going to die out but I don’t! I instead think that, like draughtsmen, our role will change. We will work collaboratively with AI models to be more productive.

Again, the sceptic would say that more productivity means less need for people. But again, just as CAD opened up a world of careers, so too will AI. I can’t even imagine what these might be.

It’s important to note that AI (not AGI) will always need some kind of “prompt” - the instruction on what to do.

Software development is not just about writing the code. It’s about taking customer requirements, having them filter through a chain of product managers, and then translating that into functional and non-functional requirements that you can code.

For example, let’s say your manager comes to you and says he needs a new API point created to handle students' exam results. Now you know from experience that around 100,000 students are taking their exams - so you need scale. And you know the company is an educational institution so can't spend too much money. Based on these parameters (and doubtless countless others), you'd recommend using a serverless solution. This is just a small example, ask this same thing of Chatgpt and it quickly falls down.

Of course, this all changes if it becomes trivially cheap to train specific models for each business context. But there is still that translation layer. Often software needs multiple iterations. AI can write stuff but it can’t read people’s minds. It can’t think in abstract creative terms (yet).

This is all to say that the profession of software engineering will not disappear. Our role may however change to work collaboratively with AI. This will mean less coding and more prompt engineering.

Should I train to be a software developer?

If you’re new on the scene and learning traditional web development, you might wonder if you should continue. I’d emphatically say yes! Software is changing all the time. I’m quite new in my career but have still seen multiple technology changes - from frontend frameworks and languages. If you are learning, expect to keep learning and adapt as the market changes.

Many recommend becoming the "robot operators" rather than the people who are going to be replaced by robots. In other words, becoming an AI engineer. This is certainly a new growing career path - if it interests you, go for it. If not, then there will be plenty of career paths that remain in web development, backend development and more.

While learning, I've always recommended learning principles over frameworks. For example, all languages have a way to create loops, declare variables and call functions - these are the "principles" or building blocks of technology. This will help you to stay adaptable and not stuck to one way of thinking.

We're living through a very interesting time in technology, AI has become a reality and it presents a huge opportunity for working in tandem and increasing our productivity with it.

Components of a Great Architecture Diagram

2024-05-02T00:00:00Z

Info - key, author, legend, version history
Flow diagrams
VPC and markings
Services wrapped up
Understand the different scopes - overarching be more general "image service" but a more indepth diagram would include the components or even the classes of that service etc.
Organise it from right to left or top to bottom
Try to avoid the mess of arrows
Use the C4 Model
Mark clear domain and service boundaries
Add interaction comments
Record questions asked about the diagram, along with the diagram
Record an explanation of the diagram
Have both short term and long term vision for the architecture

Set your time zone manually when in the Canary Islands

2024-05-01T00:00:00Z

On a recent vacation to the Canary Islands, I decided to book a romantic evening at a sea side restaurant for my wife and I.

Dutifully, with plenty of time to spare, we leisurely strolled along the beach and arrived at this restaurant.

But, on first glance it was quite empty. Too empty for a restaurant on a summers evening. When I asked to be seated, they told me they were closed and the table wasn’t for another hour.

Then it hit me, my phone and computer had switched to Spanish time - not Canarian time.

You see, the time in Spain is 1 hour ahead of the UK. But, the Canary Islands operates on the same time zone as the UK.

The iPhone, trying to be the clever stick it is, helpfully changes your time zone to a Spanish time zone. I imagine because the canaries are spanish, and so when pinging time.apple.com, it says “you’re in Spain!”.

So this is a small reminder, to both my future self and any other travellers, to manually change your timezone. Bon appétit

AWS Summit 2024

2024-04-26T00:00:00Z

I was fortunate this year to attend the AWS Summit at London's ExCeL. In this post I wanted to outline what the event was like (for a first timer), what I learned and other tips I found.

What it was like

The conference itself is enormous. Over 25,000 people all flocked to the ExCeL in London ~~to worship at the alter of Werner Vogals~~ - I mean talk about all things AWS. It was good being surrounded by people who worked on similar things to you.

There are two key parts of the summit - the talks, and the vendors.

Overall, the talks were great. I think your experience depends largely on what talks you attend. From speaking to my colleagues who attended other sessions that the "difficulty" ratings for the sessions do not represent the actual technical detail they provide. There are a few categories of talks

Talks from a company representative describing how they use a particular AWS technology. Usually these talks have 2-3 speakers - with 1 coming from the company in question and the others from AWS themselves. For example, I attended a talk by Flo Health who had their CTO speak about how they were using DynamoDB to store data in their system. They then had 2 AWS representatives give talks about how DynamoDB worked under the hood and cost optimisation strategies. These were the best talks to attend in my opinion.
Talks sponsored by a company. These are more like sales pitches to demonstrate how you can use a companies technology.
Talks by AWS themselves. These are usually focused on a particular area or AWS service. I found these to be very high level and to "drink the coolaid". For example, when speaking about analytics for serverless systems they of course recommended Cloudwatch (which is terrible). And for increasing resiliency recommended Blue/Green deployments with Codebuild (which is trash).
Community talks. These were likely not recorded so are valuable to attend in person. I attended one of these in the morning and found it to be like a talk you might find at a good tech meet-up.

The vendors are a sprawl of booths that litter the conference floors. They are different software companies of all flavours advertising and pitching their products. They usually give away swag of some description (I got a bucket hat from PagerDuty) and some have challenges where the winner gets a prize. These challenges usually have huge queues so I avoided them and focused on what I could get out of the experience as a whole.

What I learned

I was able to attend 4 talks in total here are my learnings from each.

"remocal"

"COM201: Patterns for Efficient Software Architecture"

Focused on local development for serverless.
Use Lambda to transform data NOT transport data
Try to leverage no code serverless solutions where possible for transport - event bridge pipes, dynamodb streams etc.
Suggested strategy for local development and testing is "remocal" - a portmanteau of "remote" and "local"
- This entails testing against mocks and true resources (deployed into AWS).
- So you deploy your code to an ephemeral environment. And then write tests that use mocks for DynamoDB responses but then call the actual API gateway (for example). This is targeted at testing and debugging the code in the Lambda's themselves.

"ARC302: Building observability to increase resiliency"

One pain point that many businesses face is there is no granularity to the errors they get - for example "the site is down".
The key to resolving this is multi-dimensional errors - for example "latency per trace id", or "requests per AZ" etc. This can all be provided by Cloudwatch.
Increased observability can then feed into automated rollback systems. The multi-dimensional alarm would be "errors per code revision" for example.
JustEat engineer spoke about how:
- They regularly have engineers review logs for noise. I think this is a good practise but difficult to understand what is noise.
- They standardise labels across their services - environment, service, team and version/code revision
- These tags feed into cloudwatch alarms which help them to detect bad deployments etc.
- Teams have regular "graph club" meetings where the goal is to understand and review telemetry data and action any monitoring and alerts.
Game days are recommended to be done on a regular cadence. A good game day is described as
- Realistic - like production if possible
- Reasoned - having a why and desired outcomes
- Regular - with a regular cadence.
- Controlled - targeting a certain system or area. For example, how do our API's react without database access.
- Tools such as AWS Fault injection service can be used to facilitate these game days.
They brought out a good illustration (which I foolishly didn't note down) where the idea was that you practise to build your own luck. In this context, they were talking about how it could be viewed as "lucky" that justeat rarely goes down. But the reality is that they practise the system breaking so much that they never face it in production and if it does, they know what to do to fix it quickly.
The final note was to have an observability strategy. This should align with the business goals. It's not just a case of saying "more metrics, more alarms" but knowing what to measure and why.

"DAT202: DynamoDB Deep dive with Flo Health: Powering critical data for 300M users"

Data for their customers is very sensitive as it's around reproductive health.
They also had the challenge of scaling to large user numbers without increasing costs. Uniquely they give their app away for free in 66 countries where education and help is needed but income levels mean they couldn't afford a subscription app.
One of the main challenges for them is that all the data is unique, specific and needs to feed into a personalised experience. They describe storing over 1 trillion data points - all in DynamoDB.
Their PII data is triple encrypted - using AWS KMS, the default server encryption from DynamoDB and encrypting the data itself at an application level.
Under the hood of DynamoDB, and how it scales so well, is that each partition is provisioned with 1000 write units, 3000 read units and is up to 1GB in size. They can scale these partitions limitlessly.
To partition data, they hash the partition key.
Each GSI is like a materialised view and a different version of the database under the hood.
Sometimes it's ok to throttle! For example, if something is inserting data to DynamoDB from the back of a queued job it doesn't matter if the insert gets throttled because it's not customer facing. So you can more freely use provisioned access because the impact is low. On the other hand, favour using on demand for customer facing tables.
1KB = 1 WCU. Therefore the max consumption is 409,600 Bytes = 400WCU - because the max item size is 400KB.
This means increased item size = bigger costs. If you want to update a single attribute in a 400KB item, you used 400WCU - because there is no such thing as an update with DynamoDB - it's a delete and insert.
The recommendation therefore is to "vertically partition" your data. This is the push of the single table design model. Where the sort key is used to differentiate different properties of data related to a single partition key entity.
This reduces cost by only having to access or update smaller records. Lots of data changes infrequently so doesn't need to be updated much and can be collected together.

"ANT306: How the BBC built a real time media analytics platform to process over 5B events a day"

The BBC talked about their unique challenges with data
- Multiple platforms - weather, iplayer etc.
- Needing to provide real time feedback to customers.
- Data has a "half life" of usefulness - meaning the longer it takes to provide insights to that data the less useful it is to a customer. A recommendation to watch a program 40 minutes after they have finished the last program is not useful.
Primarily they use Kafka, Kinesis and Flink.
They developed an SDK which abstracts the analytics data gathering from developers. This allows for consistent data gathering across all their products.
They initially used timestamps sent from the clients to dictate when an event occurred. This led to issues with data processing (due to processing data by time chunks), so they switched to the timestamp when Kafka received the event.
There other issue was with bad or malicious data. Using JSON schema definitions, which power their event system, they were able to tighten these definitions and dead-letter any events that were bad or malicious.
They chose JSON schema's (over protobuf etc) because of compatibility with existing API's as well as compatibility with tools like Kafka and Kinesis. In particular it allows them to make no-code changes when a new event gets added to their platform. The definitions get stored in S3 and then registered by Kafka.
The final problem they described was how they distribute Flink tasks (machine learning)
- If they do random then it makes querying for time specific data very memory intensive (for example "unique readers in the past 5 minutes").
- If they do it by articleId they often overloaded one Flink instance. This was because people would view a single page on the website and refresh lots - for example when the champions league was being drawn.
- In the end they chose to accept a certain level of inaccuracy and went with a system called HyperLogLog. This is an algorithm for calculating high cardinality with minimal memory usage.

Other tips

Plan your day in advance, ideally to things that are close together. If you have to walk around a lot of the centre to get to your next talk then you might not get in as there are always queues to the great talks.
Allow space in the day to talk to vendors. Target both those that you (or your business) are using and see if they have any new features/releases coming up. For vendors that you don't or haven't used, approach with curiosity and have an idea of where you might want to use them.
Most of the activities and challenges have huge queues, I personally avoided these because it cut into time doing other things but YMMV.
Registration is listed from 8am-10am. My train was late so arrived late and was surprised to see hoards of people still queuing for the registration.
Lunch is provided but there are plenty of (overpriced) options in the excel centre.
Unless you want to do a workshop or take notes, then you can leave your laptop at home.
Usually there are drinks at the booths between 4:30 and 5:30.
Although I arrived late to the keynote, I don't think I missed much. It really just outlined the day. It also is recorded and available online. I'd recommend using the time to explore the venue and talk to vendors.
Download the Map, your agenda and anything else you might need to your phone in advance. The wifi is quite spotty and there is barely any phone signal.

Overall, if you have the opportunity, I'd highly recommend attending. It's well worth it, even if just for the stickers.

P.S., Thanks to this dog for keeping us safe!

Markdown for Slides

2024-03-12T00:00:00Z

Working in tech, if you wanted to "share your knowledge" chances are you used Powerpoint to create a presentation. It's been around forever so can run on anything and, on the surface, is simple to use.

But, Powerpoint sucks. It really does. The interface is incredibly cluttered, the snap to grid system is completely unhelpful in real world scenarios, and there are just too many options. All I need is to have some images and text on a page and be able to scroll between them. I can forgo my beloved "star-wipe" transitions for simplicities sake.

Images, and text. That sounds a lot like something I could do in Markdown. Turns out someone already had the same idea, and created a tool called Marp.

It's incredible simple to use and makes the process of creating a presentation like writing a blog post. It's actually pleasurable to use. A simple CLI, good defaults and the flexibility of markdown.

I've only created a few presentations with it and my usage is quite basic. But I have settled on the below as being a good configuration for all my presentations.

---
theme: gaia
_class: lead
paginate: true
backgroundColor: #fff
backgroundImage: url('https://marp.app/assets/hero-background.svg')
style: |
  table {
    width: 100%;
    margin: 0 auto;
    margin-top: 1em;
    font-size: 0.75em;
  }
  h1 {
    font-size: 1.5em;
  }
  h2 {
    font-size: 1em;
  }
  p {
    font-size: 0.75em;
  }
---

The styles were added as a result of tables overflowing the slides. Smaller fonts fixed this.

Starting a new presentation is super easy:

# create the slides
$ touch slides.md

# Start the slides server in Watch mode
$ npx @marp-team/marp-cli@latest -w slides.md

You can then open the HTML file that is created in your browser. It will live reload whenever you make changes.

Then when I want to export them to Powerpoint (or PDF) for others to use:

# for pdf
$ npx @marp-team/marp-cli@latest slides.md -o slides.pdf

# or for pptx
$ npx @marp-team/marp-cli@latest slides.md -o slides.pptx

And hey presto that's all there is to it. Marp is a great tool and I highly recommend using it if you like working directly in markdown.

Cool Links from Around the Web

2024-03-11T00:00:00Z

Despite web search providing an interface to a wealth of human knowledge, one thing that it can't crack is finding cool stuff. And that's mainly because cool is relative and hard to quantify.

I've stumbled across a bunch of cool sites across the web that you might not have heard of.

If you have suggestions, submit a pull request on this blog!

Where possible, I've attempted to categorize them. Enjoy!

Web

https://solar.lowtechmagazine.com
https://grumpy.website
https://www.abandonedamerica.us/abandoned-theaters
https://designmanifestos.org

Psychology

https://untools.co/

Utilities

https://coolbackgrounds.io/

Common phrases that probably aren't true

2024-03-10T00:00:00Z

There are a litany of common phrases that are found on lots of packaging and in marketing materials. The problem is, that most of them aren't true. This is a collection of phrases I've come across that just aren't true and why you should be skeptical of them.

"Sustainably sourced" - sustainable doesn't mean it's not harmful to the environment
"Web scale" - ambiguous and lacks clear definition
"We care" - when said by a cooperation of any kind
"Clean" prefixing any fossil fuel (e.g., "clean coal") - science doesn't support that any fossil fuels can somehow be carbon neutral
"We practise agile" - most businesses practise fast waterfall
"No hidden fees" - usually has caveats around when the business will honour this
"Lifetime guarantee" - this guarantee is not a binding legal statement that can be easily reversed
"Detox" - there is no scientific evidence to support getting rid of "toxins" in the body.
"World-class" - subjective term
"100% natural" - hydrochloric acid is natural but not something I'd want to consume.

Tips for Battling Alert Fatigue

2024-03-09T00:00:00Z

When your first outage happens, alerting and monitoring becomes top priority. You don't want to be woken up at 3am again.

So you add alerting. Lots and lots of alerting.

But soon enough, the alerts start returning false positives and everyone gets used to the alerts. They become background noise.

Alert fatigue is a real problem. And one I've seen at almost every company I've worked with.

But the process of reducing alert fatigue is laborious. You have to commit hours of time and there isn't always a best way to approach it.

Nonetheless, here are some tips I've found useful:

1. One problem causing multiple alerts

If your database irrecoverably crashes, you probably get a myriad of alerts. The database is down, the website is down, the API is down etc. In isolation we want alerting for all these components. But when the root cause is the same, we want to consolidate the alerting if possible.

The implementation will vary depending on the tool you use. But, let's say the database goes down. You could only have the API alert trigger if the response does not indicate that the database is down. This would mean that the API monitor would only alert if the database was up but the API was actually down.

2. Alerts for self-healing problems

If your primary server goes down, but the second takes over. Do you need alerting about this? Maybe, but likely not.

In reality, if you have a "cattle not pets" approach to infrastructure, that primary server should be terminated and another started up. In such a case, the problem has "self-healed" - it has resolved itself without any human intervention.

This is a case where alerting is not needed as there is no direct action you or your team need to take.

3. Non-actionable alerts

Similar to the above, if an alert doesn't require an action, then it should be ditched or filtered out into a separate "notifications" bucket.

As an exercise, do the following:

Go through your alerting channel (Slack, Email, etc.)
Analyse the language of the alert to see if it has a clear action (e.g., "Reboot the server", "Increase auto-scaling capacity", "Contact 3rd-party vendor as the API is broken").
If it doesn't have a clear action then either: A) Delete it - the preferred choice or B) Reword it - include the action as part of the alert message

By spending just 15 minutes on this, I bet you'll handle some of your noisiest alerts and restore sanity to your team.

4. Alerts that don't have a clear owner

Another symptom of impending alert fatigue is alerts that don't have a clear owner. Now you have actions in your alerts, you need to consider who will complete those actions. Some alerting tools now have this built in, that can automatically @ the code owners/maintainers who will then swarm on the problem. Even so, if there is multiple individuals, it's likely worth making sure expectations are clear. It shouldn't always be person A that picks up the alerts. It should be shared around the team. But equally, if a person is in back to back meetings, they can't be expected to pick up any issues. Keeping clear expectations within your team will make sure one person doesn't end up being the "go to" when a siren sounds.

Conclusion

Hopefully these tips are actionable and help you reduce the noise of alerts on your platform. Alerts are a useful tool, but there are seldom thought of as being a key part of the developer experience. But, to run systems reliably at scale great observability is crucial - with alerts being a cornerstone of that.

What people mean when they say - we do TDD

2024-01-16T00:00:00Z

Test-driven development is that - test-driven. Not test passenger.

This means that tests come first.

What people usually mean when they say “We do TDD” is that they believe that all the code shipped has tests against it. Rarely in my experience is this actually true. [^1]

When the common understanding of a methodology is not true to its original meaning, it ceases all meaning. You then have people talking across each other who are speaking slightly different languages.

Worse than this is a business that claims to be both BDD and TDD! This leads to some lead developers and product managers pushing for BDD and therefore implementing systems to support that. Whereas another team may be doing the same but with TDD.

Like a car, you can only have one driver.

So what’s the solution? As with a car having a single driver, you need a single methodology to lead the charge. This doesn’t preclude you from having tests if you’re BDD or vice Versa. But, having clearly communicated expectations about the software development life cycle will be crucial to make sure that teams can work productively.

Having chosen a single driver, support it! For example, in the case of TDD, invest in fast CI runners, put developers through training courses, and communicate the impact across the product team.

[^1]: I would also argue that in TDD you do not need tests for every single function you write. This has been written about a lot in other places, so I won't rehash it. Generally however, my rule of thumb is to primarily test integrations and only unit tests where it makes sense.

P.S. I’m not saying that BDD is better than TDD or vice versa. I’m just saying that you can’t be led by both.

11ty or Bust

2024-01-03T00:00:00Z

It’s been four years since I last did any major work to my site. But over those years, things have got a little chaotic.

Personal websites are usually fairly low down on peoples priority lists. Test coverage, clean code and the like all get thrown out the window in favour of tinkering and writing new posts. At least that’s what happened to me.

When I first ported to Gatsby it was fantastic. Having a React-ish framework felt like I was just doing an ordinary days work. But over time, like many react projects, it lumbered to a grinding halt. The site itself was still quick. But the development process was painful. Dependency upgrades had meant that I could no longer run the site on my own laptop. It was a mess of “peer dependency unmet” and “X type is not recognised”. This site is barely 5 years old (in its Gatsby form), but the fast moving Nodejs changes had meant that it hadn’t broken significantly.

It was time for a change.

First, I looked at my requirements:

Minimal dependencies - I don’t want a repeat of the current Gatsby spaghetti.
Minimal code (so I don’t need to mess with it in the future to upgrade)
Dynamic Opengraph images - for every blog post that look nicer for social media
Web mentions - make it easy to integrate the indie web features I currently have.
Fast build times - Gatsby is extremely slow if you have lots of markdown files.
Easy to add new features - I don’t ideally want to be maintaining a React component library.
Extremely small payload size - my gatsby site used 1g of C02 per request! I wanted to reduce that.
(Nice to have) Create my resume dynamically based on a “work” page

Next I took a look at the options, ultimately there were two it boiled down to:

Hugo. A relative new comer to the static site scene but a strong contender already. It’s written in Go, which is nice. Ultimately, although there were no major issues with Hugo from my requirements, I realised my Golang knowledge was not strong enough. I want my site to be exceptionally simple to maintain. This is not a learning side project, I have those.
Eleventy. Again, a newcomer on the static site scene. Written in JS but uses Nunjucks or Liquid for templating rather than something heavy like React.

Ultimately, I settled on eleventy.

The nice thing about it is that it ships no javascript by default to the client.

It was a fairly simple migration path:

Clone down 11ty template project. I used this one https://github.com/tomreinert/minimal-11ty-tailwind-starter/tree/master
Move posts from /content/blog in gatsby to /src/blog/posts in 11ty
Updated all the front matter to include the layout tag.
Encountered an error because of a code block. Solved this by wrapping them in and .
Found a github issue that said it was fixed in a later version. Realised that this template uses 11ty v1 not v2. So I upgraded that. Thankfully the upgrade was fairly simple but still not got the site working.
Removed webpack because it’s the spawn of the devil and anything else I didn’t need.
Configured Tailwind according to this guide - https://ben.page/eleventy-tailwind
Added RSS via the 11ty rss plugin
Converted the homepage, blog post page and now page to 11ty. This process was simple enough, just copy pasting content. In the process, I restyled the look of the blog posts page to be simpler, and (for the time being) removed photos and notes from the homepage.
Add a new projects page!
Updated cloudflare pages to build the 11ty site.
After doing an accessibility scan, I updated a number of colors and alt tags to ensure that the site was accessible.

And that was it!

Overall, I’m really happy with the migration. The site now uses a mere 0.01g of C02 per request and ships no JS (although Cloudflare injects some that I’m trying to remove). It’s much easier to maintain and I am enjoying the templating engine. I have learned my lesson to keep things as simple as possible and prefer stability over feature development. To keep on top of inevitable upgrades, I’m going to configure some basic snapshot testing that means I have some basic reassurance that the site builds, and displays content.

Lambda Warming is an Antipattern

2023-12-18T00:00:00Z

At a certain stage during most monolith to microservice migrations, teams that choose to migrate to Lambda often encounter a classic problem: cold starts.

Searching this issue on Google (or DuckDuckGo for those conscious of privacy) yields thousands of results. Strategies abound, from reducing cold start times to using Lambda warmers and other seemingly haphazard tactics.

On the surface, optimizing your Lambdas seems beneficial. Lambda bills per millisecond of compute time, so optimizing at scale can lead to marginal cost savings, not to mention the customer impact of a faster app.

I won't rehash the numerous articles suggesting sound methods to improve Lambda performance, such as increasing memory, avoiding placement in VPCs, and choosing quick languages like Python, Go, or Node.js (in that order).

Instead, I'll focus on another strategy often suggested to mitigate cold start problems: Lambda warming.

Lambda warming is a method where Lambdas are pinged to maintain an active, spun-up instance. Theoretically, this eliminates the time AWS takes to boot the container. However, as Yan Cui points out on The Burning Monk, this assumption is flawed. By implementing a Lambda warming mechanism, you're only keeping a single instance of your app alive (until it recycles about every 45 minutes, a built-in AWS quirk) rather than having many concurrent instances ready for traffic.

Beyond this apparent flaw, Lambda warming disrupts the pattern of Lambda's transactional, functional nature—stateless input and output.

If you've concluded that you need Lambda warming, it suggests a larger issue: the need for low latency in your backend systems, either for customer experience or technical reasons. And by using Lambda warming, you're likely willing to invest money to combat 300ms of latency.

If these statements ring true, then the reality is you might be better served by a different service like ECS Fargate, Kubernetes (EKS), or traditional EC2—essentially, anything long-lived.

Lambda is a fantastic tool, but it's not a one-size-fits-all solution. Often, a HTTP API with some cold start latency is perfectly acceptable. However, for real-time services like chat, it might not be the best fit. Yet, if your traffic for such a service is consistent, then perhaps it is suitable!

The key is to gather data, understand your use case and pain points, and then evaluate your options. Don't rush to a workaround without thorough consideration.

How to buy a car

2023-08-30T00:00:00Z

Recently, I purchased a new-to-me (used) car. Unfortunately, after I purchased it there was something wrong with it. It wasn't a costly repair but annoying nonetheless. And there are a lot of horror stories of buying used cars. So, I decided to put together a checklist (for myself mostly) that I can run through whenever buying a car (or helping someone else to).

The list

Before you view the car

[] Check the MOT History - ask the seller if the minor defects have been addressed
[] Review common faults with the car for the mileage - ask the seller if they have been addressed
[] Check if it has a service history and who with (dealer service history is better) - I wouldn't recommend buying a car without a service history.
[] Get an insurance quote on the car - can you afford it?
[] Check the vehicle tax
[] Do a HPI check on the car

When viewing the car

[] Check the tyre tread depth - if it's low, you can factor in the cost for new tyres as a negotiation point
[] Check the service history paper work and the electronic service history (if available).
[] Check all the interior functions work - AC, Radio, Bluetooth, Steering wheel controls, wipers, windscreen washer, all lights.
[] Check the boot to make sure the spare tyre/repair kit is there.
[] Check the engine bay for any leaks or damage.
[] Check fluid levels in the engine bay - coolant, oil, brake fluid, power steering fluid etc.
[] Make sure the oil cap isn't milky - this could indicate a head gasket failure.
[] Check the bodywork for any damage.
[] Turn the wheel all the way to the left and right to check for any clicking noises.
[] Turn the steering wheel each way and check the tie rods and ball joints
[] Make sure both keys work
[] Give the car a test drive - make sure it can rev high, no warning lights appear and the breaks are responsive. If possible, make sure that the parking break holds the car on a hill.
[] After test driving leave the car idle, make sure the idle sounds smooth and underneath the car there are no leaks.
[] Check the VIN of the car matches the V5C
[] Examine how clean the car is inside and the condition of the interior, usually this is a good sign of the cars care.
[] Examine the brand of tyres the car has, if they are budget tyres and/or don't all match then this may be a sign the car has been maintained on a tight budget.

Business Version 2

2023-08-30T00:00:00Z

I have been working as a software engineer for 8 years. Over the past 2 years, I've been freelancing for startups, enterprises and everything in between.

The next stage of my business is pivoting away from pure freelancing work to a more complete offering of products and services.

TL;DR

I launched a business 24 months ago chiefly to work as a freelancer. I'm now looking to diversify and offer a broader suite of products and services to businesses. These include:

🧙‍♂️ Cloud architecture and optimisation
⚙️ Hands-software development
🌅 Engineering leadership, management and training

If you want to work with me, please drop me an email at me at joshghent.com

What I (now) offer

The key to any successful business is bringing the maximum amount of value to customers. This one is, thankfully, no different. I've chosen 3 key areas where I believe my skills and experience can make a difference.

🧙‍♂️ Cloud architecture and optimisation

Having worked as a software engineer for over 8 years, I've come across every type of cloud architecture imaginable. I've successfully transformed monolithic applications into microservices, migrated applications from on-premise to the cloud and optimised cloud infrastructure to save money.

When I work with any organisation, I focus on the following four pillars (in priority order):

Observability. Helping organisations to get a handle on the numbers behind their products and developers to gain insight into their code. Having worked with a number of different observability tools, I can help organisations to choose the right one for them.
Reliability. The only thing worse than an app that is slow is an app that is down. I help organisations to design, build and ship robust products that can scale as the business grows.
Performance. Application speed is a feature and competitive advantage. Taking a holistic view, I can provide indepth insights into your applications performance and work with your team to improve it.
Cost. No one wants an AWS bill that leaves you sobbing uncontrollably in a corner. I can help bring confidence and restore consistency to your cloud spend. I approach this by looking at the other three pillars and optimising them to reduce costs.

Generally, my approach is simple, I work with you to understand your business and goals. And then I build you a plan to achieve them by means of the technology you're building. I don't recommend a one-size-fits-all approach or that you need to hire a giant devops team to manage your infrastructure. I believe that the best solutions are the simplest ones and I work with you to find the right solution for your business.

⚙️ Hands-software development

I've spent the past 8 years working directly with code. My areas of expertise are back-end development (particularly API's and event-driven microservices) and infrastructure (particularly serverless). And I've extensively worked with both greenfield and legacy projects.

I've worked with lots of different languages and frameworks, but generally favour these stacks:

Stack 1

Frontend: React/Next.js, TailwindCSS, TypeScript
Backend: Node.JS, Typescipt
Datastore: DynamoDB, S3, SQS
Infrastructure: AWS, SST, Cloudflare Pages

Stack 2

Frontend: Laravel
Backend: PHP/Laravel
Datastore: PostgreSQL, Redis
Infrastructure: Render.com, Cloudflare

The skills listed above are not exhaustive. I'm always happy picking up new technologies and languages as required.

Software development goes beyond just coding. I take pride in producing robust code, complemented by concise documentation, and prioritize clear communication with both technical and non-technical individuals.

🌅 Engineering leadership, management and training

Encompassed within many of my roles is leadership and training. I believe I can help your organisation to train, and retain your engineers - and build a thriving engineering culture in the process. To do this, my approach focuses on

Creating systems that empower the team to drive their own growth.
Introducing fair and accurate metrics to motivate the team and measure progress.
Building a culture of sharing and learning by introducing knowledge shares, pair programming and code reviews.
Establishing psychological safety by means of blameless postmortems, retrospectives and one-to-ones.
Making sure that communication between stakeholders and the team is transparent and effective, with shared goals and values.

I have primarily worked with engineering teams who have a headcount of 1-20 people and successfully introduced these practices to them.

📟 Fixed cost services

These fixed price services are designed to give your team a boost in a particular area. They are designed to be delivered in a short time frame and provide you with a clear plan of action to move forward.

If you would like more information about the services below please get in touch.

Cloud Architecture Plan or Review - £1,300
Website/Application Performance Audit - £1,500
Observability Audit - £750
Team Strategy Documentation - £2,500
Team Training - £1,250

All prices are exclusive of VAT.

🎁 Products

In addition to the services above, I'm also working on a number of products that I have built as a result of problems observed in the organisations I have worked with. These include:

LoginLlama - A suspicious login detection service for your applications.

These products can be provided to your organisation for a discounted rate and implemented by myself.

Work with me

If you're interested in the above, and want to work together, please drop me an email at me at joshghent.com. Or book a call with me here.

I look forward to hearing from you!

TIL - How to send an SQS message from a Lambda inside a VPC

2023-08-15T00:00:00Z

Sending a message to SQS from a Lambda inside a VPC should be trivial. Unfortunately, this is AWS so they like to make it as complex as possible.

Here is the process to follow if you're stuck:

Setup

The lambda function was in a subnet on the VPC
Create a new VPC endpoint with a Full Access policy in that same VPC and subnet as the lambda. You will need to create the VPC endpoint first, then click on it, then select the "Policy" tab, then edit the policy.
Create a security group with HTTPS -> 0.0.0.0/0 Inbound and All traffic outbound.
Attach this security group to the VPC endpoint.

Sample Lambda Function code

import { SendMessageBatchCommand, SQSClient } from "@aws-sdk/client-sqs";
import chunk from "lodash/chunk";
import { logger } from "./logger";

const BATCH_SIZE = 10;

const sqsClient = new SQSClient({
  region: AWS_REGION,
  endpoint: SQS_VPC_ENDPOINT || null,
});

export const addToQueue = async (
  messages: Array<{ type: string; data: Record<string, string> }>
): Promise<void> => {
  logger.debug(
    `Sending ${messages.length} emails. Data: ${JSON.stringify(messages)}`
  );
  const batches = chunk(messages, BATCH_SIZE);

  logger.debug(`Sending email to ${QUEUE_URL}`);

  await Promise.all(
    batches.map(async (batch) => {
      const command = new SendMessageBatchCommand({
        QueueUrl: QUEUE_URL,
        Entries: batch.map((message, index) => ({
          Id: String(index),
          MessageBody: JSON.stringify(message),
        })),
      });

      try {
        await sqsClient.send(command);
        logger.debug(`Sent SQS email(s) to queue`);
      } catch (err) {
        logger.error(`Error when queueing email: ${JSON.stringify(err)}`);
      }
    })
  );
};

Adapted from the answer here: https://github.com/aws/aws-sdk-js/issues/3203#issuecomment-786372586

Pull Request Environments in GitHub Actions (with SST, AWS and Cloudflare pages)

2023-07-24T00:00:00Z

Pull request environments are a useful tool to have in your CI/CD pipeline. They allow you to preview your changes in a production-like environment before merging them into the main branch. You can send these environments to stakeholders, QA teams and even customers to request early feedback.

Recently, I was tasked with adding this into a project. The project used:

SST
AWS (for deploying the backend)
Cloudflare pages (for deploying the frontend)
NextJS frontend
NodeJS backend

I couldn't find a complete guide on how to do this, so I thought I'd write one.

The setup

Principally we have 4 things to do

Deploy the backend
Deploy the frontend (pointing to that backend)
Post a comment to the pull request with the URL's.
Destroy the environment when the pull request is closed or merged.

Let's break this down.

Starting out

First we need to setup the workflow. We'll trigger for new pull_requests and give permissions to the workflow to allow it to post comments, access secrets etc.

Additionally, we'll add a PR_PREFIX variable. This is so that we can deploy multiple pull request environments at the same time.

Throughout the jobs, you will likely see code to filter out dependabot triggers. This is because these jobs will not work when triggered by dependabot. You can remove these if you don't use dependabot.

name: Pull Request Ephemeral Environment

on:
  pull_request:

permissions:
  contents: write
  pull-requests: write
  id-token: write
  deployments: write

env:
  PR_PREFIX: pr-$

Deploy the backend

As we're using SST, this is pretty easy.

One caveat I did discover is that if you have multiple stacks that you want to deploy individually, you'll need to set the output from the .sst/outputs.json file and then deploy the next stack. This is because when deploying with SST, it will wipe the existing .sst/outputs.json file.

Here is the job for deploying the backend:

backend:
  name: Deploy Backend for PR
  if: github.actor!= 'dependabot[bot]'
  runs-on: ubuntu-latest
  outputs:
    api-endpoint: ${{ steps.sst-api-outputs.outputs.apiUrl }}
  steps:
    - name: Checkout
      uses: actions/checkout@v3
    - name: Configure Non Prod AWS Credentials
      uses: aws-actions/configure-aws-credentials@v2
      with:
        role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
        aws-region: ${{ secrets.AWS_REGION }}
    - uses: actions/setup-node@v3
      with:
        node-version: 18
    - name: Cache node modules
      uses: actions/cache@v3
      with:
        path: node_modules
        key: node_modules-${{hashFiles('package-lock.json')}}
        restore-keys: node_modules- # Take any latest cache if failed to find it for current yarn.lock
    - run: npm install
    - run: npm run build
    - name: Deploy Global
      run: npx sst deploy global --stage $PR_PREFIX
    - name: Deploy API
      run: npx sst deploy api --stage $PR_PREFIX
    - name: Extract Api URL and set output
      id: sst-api-outputs
      run: |
        cat .sst/outputs.json
        API_URL=$(jq -r '.[].ApiEndpoint | select(. != null)' .sst/outputs.json)
        echo "apiUrl=$API_URL" >> $GITHUB_OUTPUT

Deploy the frontend

Next we need to deploy the frontend. As we're using Cloudflare pages, this is also pretty easy!

How this works is using the cloudflare/pages-action. This will deploy it to a unique URL as a preview deployment for that pages project.

frontend:
  name: Deploy Frontend for PR
  if: github.actor!= 'dependabot[bot]'
  runs-on: ubuntu-latest
  outputs:
    url: ${{ steps.cloudflare-publish.outputs.url }}
  needs:
    - backend
  steps:
    - uses: actions/checkout@v3
    - uses: actions/setup-node@v3
      with:
        node-version: 18
    - name: Cache node modules
      uses: actions/cache@v3
      with:
        path: node_modules
        key: node_modules-${{hashFiles('package-lock.json')}}
        restore-keys: node_modules- # Take any latest cache if failed to find it for current yarn.lock
    - name: Cache NextJS Build
      uses: actions/cache@v3
      with:
        path: |
          ~/.npm
          ${{ github.workspace }}/packages/web/.next/cache
        # Generate a new cache whenever packages or source files change.
        key: ${{ runner.os }}-nextjs-${{ hashFiles('**/package-lock.json') }}-${{ hashFiles('packages/web/**/*.[jt]s', 'packages/web/**/*.[jt]sx') }}
        # If source files changed but packages didn't, rebuild from a prior cache.
        restore-keys: |
          ${{ runner.os }}-nextjs-${{ hashFiles('**/package-lock.json') }}-
    - run: npm install
    - name: Build
      run: npm run build -w packages/web
      env:
        NEXT_PUBLIC_API_URL: ${{ needs.backend.outputs.api-endpoint }}
    - name: Publish
      uses: cloudflare/pages-action@1
      id: cloudflare-publish
      with:
        apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
        accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
        projectName: ${{ secrets.CLOUDFLARE_PAGES_PROJECT_NAME }}
        directory: packages/web/dist
        gitHubToken: ${{ secrets.GITHUB_TOKEN }}

Clean Up

Finally, we need to clean up the resources that we created. This is done by using the sst remove command. Cloudflare pages resources are automatically cleaned up for us.

This process requires a completely new GitHub action workflow that triggers when pull requests are closed (this means merged or manually closed).

name: Destroy PR Environment

# only trigger on pull request closed events
on:
  pull_request:
    types: [closed]

env:
  PR_PREFIX: pr-${{ github.event.pull_request.number }}

permissions:
  id-token: write
  contents: read

jobs:
  remove:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Configure Non Prod AWS Credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
          aws-region: ${{ secrets.AWS_REGION }}}
      - uses: actions/setup-node@v3
        with:
          node-version: 18
      - name: Cache node modules
        uses: actions/cache@v3
        with:
          path: node_modules
          key: node_modules-${{hashFiles('package-lock.json')}}
          restore-keys: node_modules- # Take any latest cache if failed to find it for current yarn.lock
      - run: npm install
      - run: npx sst remove --stage $PR_PREFIX

In addition to this we need to make sure that our default destroy policy is set to remove all resources. By default SST will never remove DynamoDB tables, S3 buckets and other data sensitive resources. In this case though, we want to completely remove everything.

In your sst.config.ts file, add the following:

if (app.stage !== "production") {
  app.setDefaultRemovalPolicy("destroy");
}

This will remove all resources when the stage is not production.

Conclusion

And that's all! You now have a custom preview environment for your pull requests.

How Part Time has helped me in life

2023-07-11T00:00:00Z

Part time working has been the best career move I've ever made. This change was natural and made a lot of sense. But, I understand it's hard to fathom for many people. Do I just sit around? Do I actually get anything done?

I wanted to expand on how I'm working now and why I believe it will be the next era of working.

I believe that in the coming decade, just as remote work has arisen this decade, part time work will become increasingly commonplace.

How I work now

Currently, I work 3 days per week. These days vary from client to client but are generally fixed.

The other 2 days a week I do voluntary work for charity.

My work is solely remote and centres around a few key clients. I also create SaaS products on the side.

Why I find part time work so great

Here's the secret, you don't need 35 hours a week to accomplish your work. Often it takes half that time. Parkinson's law is real. By having 35 hours, "stuff" expands to fill that time. After all, if you're a software engineer, how much time do you actually spend time coding verses planning meetings, catch ups, speaking with stakeholders and stand ups? In addition to this, the complexity of software nowadays means that it takes much longer to create new features and fix bugs.

Parkinson's law still applies when it comes to part time working.

The difference is, less hours forces me to cut through the noise and focus on outputs. Let me break these down.

Cutting through the noise

Seen a calendar that looks like a losing Tetris game? I bet you have. Because everyone has this structure, people think nothing of adding another meeting.

By not working on some days, you can simply reject the meeting that you don't work that day.

You can also ask questions - does this need to be a meeting? Do I personally need to attend?

That critical thinking greatly reduces the amount of distracting non-work that you're part of.

Beyond meetings, it also prevents you from becoming a single point of failure. People get used to relying on you for answers and help if you're around all the time. But, working part time pushes your team to work from documentation and share knowledge. Critically, it cuts down on you being constantly queried.

Focusing on outputs, not inputs

If you are contracted to work 35 hours a week, what is the incentive to work hard all of that time? Very little. Of course, you don't want to miss deadlines. But people are used to scope creep and tickets taking longer than their allotted points. So what's the harm if you slack off a bit? After all, Vouchercloud found that their office workers were "productive" for a mere 2 hours and 23 minutes per day.

Fewer hours mean a shift in mindset. My contracts are about deliverables within a set weekly timeframe. The spotlight is on what I've done, not time spent.

Overall, I've found working part time incredibly beneficial for both myself and my clients. It's helped me reduce the amount of busy work, balance work and life, and importantly, makes things happen.

Why are software companies so obsessed with doing anything but work?

2023-06-21T00:00:00Z

Recently, my favourite time building software I've ever had has been developing Loginllama.

It's so abundantly simple.

Just NextJS pushed to Vercel.

I don't write fancy commit messages - just "x". And then deploy straight to production.

Creating a new product

❌ No development environment
❌ No local development
❌ No commit messages (just "x")

It's the fastest I've ever shipped.
— Josh Ghent (@joshghent) February 14, 2023

Contrasting this with most of my "jobby job" software development, and the joy is quickly sapped.

To work on anything the ticket needs scoping and then writing. Followed by an hour long "refinement" to point out that Kevin the designer still hasn't attached their the Figma link to the ticket (come on Kevin!), and Janine is raising concerns that it's not possible to do (it's technology, it's always possible!).

Then you come to evaluate the amount of time it will take, which if you do not work on software involves one simple step - stick your head out the window and just gather a general "feeling" then call it a 5 or 3, or 8, none of it means anything anyway. Software estimates are one step away from the ridiculed pseudoscience of homeopathy. It's a total farce, but some people believe in it. And that's the important part.

Now that you've got your estimation, your project manager and your team sit down and prioritise it and divide the work into a sprint. Basically a sprint is the amount of work you can conceivably regurgitate to meet an imaginary deadline.

Of course, bug fixes are priority number 1 you think. Think again! If it's not affect that many customers then it's fine! Just add more features and the shareholders are happy.

Before you get started on the actual work, you need to meet with your manager to discuss OKR's - the things you hope to achieve to make the company more money.

When you get back, you check your inbox and see Sarah has requested you give 360 feedback on her. So you spend the next hour trying to understand the questions and then just resigning and giving them "excellent" on everything.

I could go on, but I'm hoping some of this resonates with you.

Of course, my scrappy "x" commits and pushing to production does not really scale for a large team. And arguably, rushing to code before looking at the metrics and requirements is foolish. But, I would hope, there is a happy medium.

Unfortunately, many organisations I have worked for solve this problem by aggressively hiring. Even during a down period for the economy, most software companies (which is to say, companies) are hiring aggressively. They need QA engineers, frontend engineers, full stack engineers, devops engineers - you name it, they need it.

But, the question never gets ask of why we have all this complexity in the first place.

No body wonders why we need more documentation than the great library of Alexandria to write an app that helps find you someone to walk your dog.

It's a classic case of Parkinsons Law. Technology should enable us to do more with less. In reality, the opposite is true.

What's the solution?

Unfortunately, there isn't a one size fits all solution. And in many cases, lots of people don't want a solution, because it would mean admitting that their job is unnecessary and their work is unvalued.

In any case, you can try to do the following

Ask good questions. Questions such as "help me to understand why X" or "what is preventing us from Y approach?" helps everyone clarify their thoughts and justify the position. This should be an interrogation but rather a friendly discussion with the aim of getting things done, not out of a personal vendetta against busywork (however tempting).

Be careful what you measure. Goodharts law states that whenever you begin to measure something, it ceases to become a good measure. For example, if you want to measure the productivity of car builders, you could measure how many cars are built per day. But if all these cars break down due to quality issues, of what value is the measurement. Instead, try to measure things that lead to better outcomes. For example, in the car factory you could measure the ratio of vehicles produced to the number that have defects.

Aggressively cull meetings. It's already been written about at length how meetings can be corrosive for deep work. But still, most organisations have lots of regular and "quick catch up" meetings. As teams scale, complexity of communication also scales. It's not uncommon to have teams dedicate 1 day per "work cycle" to meetings - sprint reviews, refinements and planning. Be a leader and ask if these meetings need to be in place, can they be reduced by doing more work async? Are they achieving the outcomes they set out to solve? Can we focus the discussion more?

Busy work will always exist but it doesn't have to just be accepted. But be kind, some peoples entire life is busy work.

Lifehacks

2023-06-09T00:00:00Z

Inspired by https://guzey.com/lifehacks/

Last updated: 2023-06-09

you'll get far more benefit from reading books than anything on the web
Be cautious of people who overcomplicate things
be requirements led
ask good questions
time spend configuring a system (productivity, software, computer) is time wasted.
the goal of a productivity system is to be simple and reduce cognitive overhead. evaluate if yours is accomplishing those goals.

Setup a Repo in Github

2023-05-03T00:00:00Z

I end up creating quite a few repos in Github for customer projects. And I always end up having to remember how to best set them up. In line with the whole "blog-umentation" thing, I thought it would be best to write it down for myself.

This setup lends itself to a "modular monolith" setup but can be used for any kind of setups.

Features

This setup brings you the following

✅ Conventional commits check (making sure commits adhere to guidelines)

✅ Ensure checklists in PR's (as defined in a template) are complete

✅ main branch is protected

✅ Dependabot alerts are setup for security

✅ Dependabot is setup for github actions and npm packages

✅ Jest coverage reports get added to PR's

✅ PR's are scanned for secrets

Steps

Go to Settings and change the following
1. Protect the main branch - require 1 or 2 approvers, prevent force push
2. Change merge types to Squash and Merge only
3. Enable dependabot
4. Enable automatically delete head branch
Add the following Github actions
1. Secrets Scan (https://github.com/marketplace/actions/trufflehog-oss)
2. Google release please (https://github.com/google-github-actions/release-please-action)
3. Require PR checklist complete (https://github.com/mheap/require-checklist-action)
4. Conventional Commits (https://github.com/amannn/action-semantic-pull-request)
5. Coverage report (https://github.com/ArtiomTr/jest-coverage-report-action)

Add the dependabot config (.github/dependabot.yml)

version: 2

updates:
  - package-ecosystem: "github-actions"
    directory: "/"
    schedule:
      interval: "daily"

  - package-ecosystem: "npm"
    directory: "/"
    schedule:
      interval: "weekly"
    rebase-strategy: "auto"
    open-pull-requests-limit: 2
    ignore:
      - dependency-name: "*"
        update-types: ["version-update:semver-major"]

Hopefully this helps you setup your Github repos faster!

SQS, SNS, Eventbridge, DynamoDB - Chosing the right queue system in AWS

2023-05-02T00:00:00Z

AWS has so many different queuing services.

SNS
Eventbridge
SQS
Amazon MQ
DynamoDB - a database but also can trigger lambda's so it's kind of a queue!

For newcomers to AWS, having so many solutions for a seemingly simple problem can be overwhelming.

Here is a breakdown of each queuing service and when you might use it.

SQS

Sending: 1 to 1

What it is: Literally a "simple queue system". Does what it says on the tin, send a message to a queue, have it consumed, easy.

When to use it:

Buffer for API requests or 3rd party system. E.g., if you needed to send some data to a 3rd party system but don't want to hit their rate limit and add resiliency, then SQS can be used for this. You could also use it to buffer writing data to a database.
Queuing jobs. e.g., If you have a pipeline to optimise images and upload them to a CDN, you can use SQS to queue up those images. This means you have resiliency if an optimisation fails (it will auto retry) and will not overwhelm your down stream systems. A queue message is also long lived (unlike an API request) so you don't need to worry about time outs.

SNS

Sending: 1 to Many

What it is: High throughput "fan-out" message distribution.

When to use it:

You want to send SMS or Mobile notifications.
You want a dumb pipe to send to lots of downstream targets.

Eventbridge

Sending: 1 to Many

What it is: Event bus's. Think of them like queues but messages can go to multiple places. Similar to SNS but you can define rules for which messages go to which consumer.

When to use it:

Need to distribute to multiple targets
Designing an event driven architecture
You want to integrate with third parties (like Datadog, Shopify, Zendesk etc.)

Amazon MQ

Sending: 1 to Many (depending on configuration)

What it is: AWS's managed RabbitMQ solution

When to use it:

You have a pre-existing RabbitMQ cluster and you don't want to migrate to SQS (yet).

DynamoDB

Sending: 1 to 1

What it is: NoSQL database

When to use it (for queues):

You want to keep logs of data your system has processed. e.g., you could use a table as a store for a CMS. When you upload a new post, it triggers a pipeline that sends it to your social media accounts.

Hopefully you feel a lot more confident in making an architectural decision about which queuing technology to use in AWS. If you have an application and you're confused as what to use, send me a message on twitter @joshghent

Raising £1000 for MSF

2023-04-28T00:00:00Z

TL;DR I'm raising money for Medicins Sans Frontiers by running my first half marathon. You can donate here

I love running. It's such a simple exercise. All you need is some good shoes, good music and, ideally, good weather.

I love that I can run literally anywhere I am in the world and see new things and explore the terrain.

When I first started, 5 kilometres was like climbing Everest. I wouldn't say I was unfit, but being in a sedentary job for many years, and not much attention paid to fitness led to a weak stamina.

Unfortunately it took some health problems from family members to prompt me to pay proper attention to my health.

As I worked up from 5 kilometres, to 10, then to 15. I was looking for another challenge. A full marathon has always been a goal of mine, but seemed like too much of a stretch.

When I saw the Birmingham Great Run Half Marathon advertised, I knew that was just the challenge I needed.

Alongside this increased attention to fitness, I was also reading through Peter Singers book, "The Life you can Save". I won't rehash it here because it's been discussed at length elsewhere. But, crucially, it made me realised that although my wife and I gave to charity already, we could do more.

Kill two birds with one stone.

So, the thought occurred to me - "why not run and raise money for charity at the same time?".

I launched a fundraiser through JustGiving in the coming days. I chose the charity MSF chiefly because of the great work they do and the high amount of money they spend on their activities versus marketing, salaries etc.

I'm still quite worried about doing 21.2 kilometres. But, the important thing is I am running and raising money for a great cause.

I need your help to reach the goal of £1000. So, please, if you have a few quid donate to the fundraiser here: https://justgiving.com/fundraising/josh-ghent

The business my wife and I run, Turbo Technologies, will be doing a donation match of up to £1000 ourselves. Hopefully in totally we can raise £2000.

Whimsical Software

2023-03-24T00:00:00Z

Software takes itself too seriously. It's time to have some fun. For far too long, software has been designed and created around one single metric - revenue. And often, increasing revenue creates negative behaviour for us - the humans. Designs are created to maximize clicks and to be more addictive than nicotine. We are constantly bombarded with notifications and alerts, and we are forced to make decisions in a split second. On top of this, building software has never been more complex. Although we have a wonderful array of technology at our disposal to scale an application to millions of people, it's also incredibly complex to build and maintain.

We cannot abandon these metrics. Ultimately, it keeps the world turning. But we should introduce some "whimsey".

| whimsical wim-zi-kal | definition: playfully quaint or fanciful behaviour or humour.

We should be able to build software that is fun to use, that makes us smile, and that makes us feel good about ourselves.

Enter Whimsical Software.

The Whimsical Software Manifesto

Be playful. Discard the boring blues and greys. Use bright colours, delightful interactions, playful sounds and animations. Software should be like a playground, not a prison. In some cases, people perhaps spend more time using your software than they do with their family. So, it should be an enjoyable place that allows them to do their job.
Be kind. Most revenue generation from software arises from hoovering up masses of data. Whimsical software should be kind, capturing only what data is needed to deliver a better experience. It should be kind to the environment, running on efficient architectures (e.g. serverless) and using minimal resources. And it should encourage us get the task done, and then log off.
Be helpful. How many times have you wondered what an error message was, how a particular edge case function could be done or how to use a particular feature? Whimsical software should be helpful, providing guidance when needed. And automating as much as possible.

Whimsical software is not a particular aesthetic or technology. It's a way of building tools to empower people, not to enslave them. You could have an old school terminal interface, or a modern web application. It doesn't matter. What matters is that it's fun to use, and that it makes you smile.

Devblog - LoginLlama 001

2023-03-23T00:00:00Z

I'm starting work on a new SaaS!

I want to document my process because I used to love devlogs on tumblr and tigsource.

I am creating an API-as-a-Service that monitors suspicious login attempts.

At the moment there is a couple of solutions in this space. But they fall down in a few areas

Bad pricing
Bad marketing
Rigid integration path. In the case of the main competitor, they make it so they have to send the emails out, rather than send them out yourself. It also doesn't have the concept of teams so not setup for enterprise.

My MVP will be:

A single API endpoint to check if a login is suspicious or not
- Check against known VPN's, TOR nodes, time of day and other factors.
Allow customers to control the factors that affect whether a login is considered suspicious or not.

Here is my bullet pointed notes as I did this first stage of development!

You can checkout the in-progress site here: https://loginllama.app

started with the subscription starter pack from vercel
Had to upgrade from 12->13
Main fix was done by the codemod dependency. Mainly had to update Link components to not include a elements
Then had to manually migrate the database
And get the types form the repo
But then all setup!
Added the first edge function.
Decided I’m going to for an mvp of
- Basic admin interface to control the sensitivity
- Api to check if the login is suspicious and return the reason why
- Good looking doc pages
- NodeJS SDK
- Allow customers to get the data in a computer readable way so they can send their own emails.
- Homepage listing features
- Pay as you go pricing model
- - $1 per seat
  - $0.00015 per request
- Teams but off only one person
Added basic /api/v1/login/check endpoint
Added basic accounts screen with API Key

Rules of Thumb for creating API's

2023-03-14T00:00:00Z

Building good software often starts by having good principles upon which to design it upon.

But, principles are high level and so often don't allow for much practical application. On the other hand, rules, although more specific and practical, are used by many as something to bash others over the head with.

In both cases, these are hardly desirable to build good software.

What we need, is something inbetween. The practicality of a rule, but the heuristics of a principle.

Enter the rule of thumb! We've all used rules of thumb ("righty tighty, lefty loosey" anyone?). But, recently I was asked by someone to create a list of rules for creating API's. Instead, I created some rules of thumb that I have documented here:

Pluralise routes. For example use /users not /user.
Make use of HTTP methods. It sounds basic but many API's break this convention. If you have an update use a PUT or PATCH. If you're getting data, you use GET - get it? Why use this convention? Mostly because it's logical. It stops people guessing your routes. For example if you tell someone you have a route of POST /users to create a user, then the person may logically assume that doing GET /users will fetch users as well.
Don't include method in the name. For example, don't create routes such as a /createUser or /changeName. The action "create", "change", or whatever, should be clear from the HTTP method (POST, PATCH, DELETE etc.).
Return HTTP response codes appropriately. Only return a 200 when successful. And equally, only return a 4xx/5xx code when there is an error. HTTP response codes are a simple way for API consumers to respond to issues when calling your API.
Return JSON to the client. Makes handling your data handling far easier on your frontend.
Use query parameters for filtering data. Don't use POST bodies for filtering data for a fetch request. Instead add filters via query parameters. For example ?age=88&surname=Ghent&company=Turbo Technologies.
Avoid backwards incompatible changes. Generally, API's are tightly coupled to the consumers that are built against them. To avoid having to tie releases together and having to issue lots of updates to your frontend, try to avoid backwards compatible changes. For example, instead of removing a field, just add a new one. Or alter the response based on a "Version" header.
Keep versioning basic. It's not a lunar mission. Favour using V1, V2 and so on rather than anything complex like dates, or semver versions. You likely won't need it.
Keep data, business logic and request life cycle handling separate. Clear separation of these different areas will make maintenance of your application far easier. Additionally, by segmenting the request life cycle (for example, error handling, authentication etc.) it standardizes responses, making it easier to use for consuming applications.
Put changes into OpenAPI first. Specifications seem like a huge pain to write. If you're like me, then you likely want to jump straight into the code. But, OpenAPI specifications are a great resource so that your frontend team can develop their code against a contract. Further, if you're API is used by 3rd parties then this OpenAPI spec can serve as documentation for them.
Mock DB responses when testing your API. When testing, you want to test your code, not the database. Mocking the database response will accomplish this and mean you can run your test suite in your CI suite without having to spin up a local database.
Emit events. Event-based architectures are not 100% practical. REST API's are far easier for handling basic CRUD operations. But, beyond this, it's prudent to release events (to Eventbridge or another event bus) from your CRUD API, which can then be consumed by other systems asynchronously. For example, let's say you have an endpoint to update a customers details. When the customer updates their email you need to send an email to verify that new email and notify the old email of the change. You could place all this logic in your CRUD API system. But, to improve resiliency and segment logic, it would be wiser to release an event of "CustomerEmailChanged", and then have another consumer send the email, and another notify the old email address of the change.

This list is by no means complete, or exhaustive. But it does serve as a good set of guidelines to build your API around.

Simple sites

2023-03-09T00:00:00Z

I saw a tweet the other day from Dan Rowden that hit on a class of software I really love.

Simple, small, minimalist tools are in ascension. And I like it.

Check also:https://t.co/objPWGsriu https://t.co/U2i8tW9Qq3 https://t.co/GYt5FPPEbb https://t.co/snSoIs6UPY https://t.co/CLXFQGqMOW

— Dan Rowden ⚡️ (@dr) February 14, 2023

In creating my "digital garden", I wanted to publish a list of these sorts of sites.

If you have further suggestions, please tweet them at me!

https://tasks.wtf/
https://imgs.so/
https://glass.photo
https://read.cv/explore
https://bmrks.com/
https://lfe.org/
https://savee.it/

You can't fix engineering culture with communication

2023-03-08T00:00:00Z

It always bothers me that people say that "communication" is the problem in engineering organisations.

We have near-constant access to each other via Email, Slack, JIRA and even GitHub PR comments.

Saying "communication" blindly is like telling a potentially Olympic runner to "run". That might be part of the overall strategy, but it's an oversimplification.

For example, if the problem is that a new feature broke another area of the system, then that's not a communication failure. Why?

The idea was conceived and translated into a ticket (communication)
That ticket was then iterated upon, and a specification was formed (communication)
That ticket was then delegated to a developer (communication)
That ticket was spoken about at stand-up (communication)
That ticket was then submitted for code review (communication)

More communication is not the answer here.

So what is?

The answer, of course, varies on a case-by-case basis.

But, I have found people blaming communication as a scapegoat for the real problems in the team.

Some teams attempt to get around this with a "5-why's" system. But, in practice, the same groups that blame communication are also blind to the problems they are trying to solve. Many see a problem and jump to a solution they read that a FAANG company do.

First, list the problems and the impact on the team (from low to high). Then look at the high-impact issues and understand why they are problematic. For example, if the issue is that shipping features is slow, then its high impact is because a business needs to ship certain features by deadlines.

Next, being requirements led, look at your needs for the "ideal" system. Following the above example, the conditions might be that you can ship once per day, not require manual work to check new features and deploy new code in 15 minutes or less.

From there, you can design solutions to solve those problems.

This way, you solve your problems and are mindful of the issues. It's easy to read about solutions that FAANG's use. But the reality for many organisations is that those solutions would not fit and even do more harm.

Finally, you can measure the impact of these changes and iterate on the approach you have created.

Communication might be the problem. But I'd suggest looking at other failings in your engineering process, as it might surface some more positive changes you can make.

The Cobra Effect and Software

2023-02-28T00:00:00Z

We like to focus on inputs, not outputs. Inputs are simple. Sugar for baking, metal for factories and petrol for cars. Software is no different.

But by focusing on inputs, we often encourage the dreaded cobra effect.

Basically, we incentivize bad behaviour.

For example, let's say we have metrics for a team to write 30 tests a sprint and maintain 90% code coverage.

On the surface, it sounds good! 90% coverage will mean our code is tested thoroughly and 30 tests a sprint means we definitely will be making the system more robust.

But, consider the other side of the coin.

The team is now disincentivized from writing new code to not reduce the code coverage.
Tests are written for the sake of it, not because they are required and therefore the CI/CD pipeline takes longer to run.
Any new feature that is added has to be tested to absolute oblivion, thereby increasing the time to ship new features.

What's the solution?

Focus on what you want the desired outcome to be and then design metrics that would change based on that outcome.

In the above example, rather than enforcing code coverage, we could measure the number of defects raised, the number of releases, and the number of failing test cases.

Metrics are important. But, consider the kinds of behaviour that those metrics will encourage. Otherwise, you'll need to watch out for the cobras.

Five Things I Wish I learnt sooner

2023-02-13T00:00:00Z

I have been a software engineer for the best part of a decade (yikes, I'm old now?). And in that time, I have learnt a great deal about software development - languages, frameworks, architecture and much more.

But, since becoming a freelancer, I have learned how to make my work more successful and less chaotic.

I'm sharing those lessons here because I wish I had learned them sooner.

Attach metrics (KPIs) to work. If your work doesn't have metrics that business people can see go up or down, you will be in trouble. Metrics (or KPIs) are the genesis for doing successful good work. Developers often think that things such as technical debt cannot have business metrics. This was how I reasoned for many years. But turns out they do! Technical debt, for example, can be used against the metric of user retention or revenue. How? By reducing technical debt, we speed up the application, which increases customer retention.

Additionally, by attaching metrics to your work, you can gain insight into how that work is progressing. Teams are motivated by seeing progress being made, so make sure to be transparent and share these metrics with them.

Always attach metrics to your work. If there are no metrics, it's a sign that the project will likely not get done.
Be requirements led. This means to start with the problem first and the solution second. Developers often need help creating a solution to something before fully exploring the problem. It's a natural tendency - after all, we are builders. But, by approaching things from a solutions-first mindset, we fail to appreciate all the requirements customers (both internal and external) have.

For example, many people start their new year's resolutions with "I need to start going to the gym". This is a solutions-first mindset. The requirements are more complex than that. Solutions are difficult. After all, why do you need to go to the gym? To look a certain way, feel a certain way, or stop eating certain foods? It raises many questions. But by asking and exploring the answers to them, we get down to the root of what you actually want. Armed with a list of requirements, you can now build a solution to satisfy them and maximise the value. Statistically speaking, when it comes to the resolution "go to the gym", by the 3rd Thursday of January, 80% of hopefuls will have stopped. In the same way, being solutions-first will often lead to a project failing and getting ditched.
Adhere to engineering principles. Principles are overarching guidelines that can motivate us to action. I don't mean "SOLID" or "DRY" when I speak about principles here. I'm talking about a way to approach problems and solutions with engineering rigour - collecting metrics, testing, challenging assumptions and being proactive. Principled thinking.

For example, if someone asked you to migrate a REST API in a VM to AWS. You might start by asking what the motivations behind this move are. When they say it's because it's slow, you challenge that assumption by first running a load test. Then, review the APM metrics to see the average throughput from API consumers. If it turns out the API was slow, then great, the assumption has been verified. But without hard metrics to back things up, it was a stab in the dark. Engineering rigour helps developers to be more successful at the work they do. This particular lesson has made my work smoother.
Secure, simple, fast, in that order. When developing a product, we should adhere to this order closely. If you're like me, you likely want to make something as performant as possible - pre-optimising. But this is a waste of time at this stage. This order of making a product first secure, then simple, then fast allows us to prioritise the correct thing and work down the chain. It also means we can make something secure even if it's complicated. But we shouldn't make something fast but complex.

Why this order, though? Security always takes top priority. In my experience, a security breach is far more damaging to customer trust than a slow application. Simplicity comes second. We develop software with a team of people, so we want to keep things simple (even if it's not as fast compute-wise) so they can understand and build upon our code. Finally is speed. We want to make all our efforts to make our application fast. Slow apps are the bain of our existence. And often, people don't wait around. Google's target for site loading is now 300ms and has seen colossal bounce rate increases when sites exceed that number. Speed drives revenue, so it should always be something important to us.
Take the lead in making notes. How many meetings have you been to where nothing gets accomplished. And afterwards, you are asked what the meeting was about and need help remembering. For me, it's a lot. I've just shared my screen with an Apple Note and then taken notes to remedy this. You can still make notes even if you're not "leading" the meeting. It's important to document what was discussed, what the actions are, who is responsible for them, and what the deadline is. Sharing your screen with everyone keeps people focused on the problem at hand and not getting sidetracked. It also means that everyone clearly knows why they are a part of that meeting and their responsibilities. Doing this has changed my "wasted" meeting percentage (which should be an SI unit at this point) from around 80% to 60%. A massive reduction, even though I have only started doing this recently.

All of the above deserve articles, so I may write about them more one day. But for now, keep learning!

Favourite Subreddits

2023-01-20T00:00:00Z

Most of Reddit is a trash fire, but there are some brilliant pockets of communities. Here is my list of subreddits I occasionally browse.

Albums I listened to in 2022

2023-01-19T00:00:00Z

In the past couple of years, I started tracking albums I listened to and what I rated them. The motivation was to be able to re-discover music in coming years and to satisfy my tendency to record data about little things.

Here is my list of albums I listened to in 2022. Note that a lot of the albums didn't come out in 2022, just that I listened to them in 2022.

Feel free to drop me recommendations.

Name	Artist	Rating
Gemini Rights	Steve Lacy	9
Mellow moon	Alfie Templeman	9
Harry’s House	Harry Styles	9
This is really going to hurt	Flyte	9
Topical Dancer	Charlotte Adigery	9
Today we’re the greatest	Middle Kids	9
Sometimes I might be an introvert	Little Simz	9
Let me do one more	illuminati hotties	9
SOS	Sza	8
Blue Revs	Alvvays	8
Beatopia	Beabadoobee	8
Pocketknife	Mr Little Jeans	8
Hold the girl	Rina Sawayama	8
Midnights	Taylor Swift	8
Dropout boogie	The black keys	8
Love is yours	Flasher	8
Space Island	BROODs	8
Fake it Flowers	beabadoobee	8
Pieces	IU	8
Flowerland	Pearl & The Oysters	8
1/6	SUNMI	8
Dawn FM	Weeknd	8
Negro Swan	Blood orange	7
Yaeji	Yaeji	7
Shake Shook Shaken	The Do	7
A Mouthful	The Do	7
The Myth of the Happily Ever after	Biffy Clyro	7
MUNA	MUNA	6
The Cat Empire	The Cat Empire	6
Laural Hell	Mitski	6
Delta kream	The black keys	5
WE	Arcade Fire	5
Pompeii	Cate Le Bon	5
Blue Banisters	Lana Del Rey	5
The Car	Arctic Monkeys	4
Loner	Alison Wonderland	4
Palaces	Flume	4
Reward	Cate Le Bon	4
The nearer the fountain, more pure the stream flows	Damon Albarn	4
Inside Voice / Outside voices	KFlay	3
Mr Morale & The Big Steppers	Kendrick Lamar	2

Onebag

2023-01-18T00:00:00Z

In 2020, my wife and I decluttered 10 carloads of stuff from our home. Since then, we've vowed not to devote our lives to things. We want to focus on travelling more, spending less and generally have more freedom.

This ethos of "less" carried over into our travel setup as well. And we've now done a few trips with this setup, learning a lot on the way.

It's hard to describe a "eureka" moment of "ultralight" travel because it happens in such small ways, from being able to breeze through checkout to not getting caught on cobbled European streets with a giant carry-on case or worrying that you're over the weight limit for a flight.

This post is not a guide or filled with affiliate links; it's merely a collection of the stuff I use and like. Because there are only a few things here, extensive research was conducted before buying them. Also, everything was purchased gradually over time due to the cost of some items.

But you don't need fancy gear to pack ultralight. Most of the items you probably have in your wardrobe already.

I decided to include and exclude certain items from my bag based on my requirements; yours might be different.

Enjoy :)

Packing

North face surge 2. My go to pack for daily tasks and a workhorse when travelling. It has 3 main compartments for separation of items and 3 mini pockets for little bits like pens, passports and keys. It has chest and hip straps to make it easier to hike with and has handy pockets and cables to attach things to. It's easily the best backpack I've tried and shows no signs of falling apart.

Tech

MacBook Pro 14” 2021. Brought after I spilled an entire litre of water on my old laptop (yes really - and no, rice didn't help). It allows me to do my work and that's it. I could have used some more RAM occasionally for running VM's and such but it's great 99% of the time. I generally limit software or OS upgrades because I find stuff just break after upgrading.
iPhone 14 Pro Max. Upgraded a few months ago and it's made a massive difference to the speed that I can accomplish tasks with. When travelling, it's my constant companion, for everything from maps, music, ride sharing, and travel guide.
- Apps installed
  - Reading: Kindle, Feedly
  - Travel: Uber, Alltrails, Flighty
  - Journaling: Day One, Notes
  - Utilities: 1Password, Notion
  - Entertainment: Spotify, Pocket casts
Airpods Pro. Used for music and calls. Due to the noise cancelling, I use them to get silence when on a flight.
Anker powerbrick. Mine is so old and battered you can't buy it anymore. These things are bomb proof and are great for providing power on the go. Because it weighs about 500g, I generally only bring it when I really think I'll need it.
Universal plug. The amazon page has disappeared but search for any generic worldwide travel adapter and you'll get something decent. I chose something light and that can deliver fast charging. This plug means I don't need to carry USB plugs as it has the ports integrated itself.
Apple Watch. Used passively for tracking walks, sleep and, occasionally, telling the time.
2x USB-C to Lightning Cable
USB-C cable
Kindle
Nikon D5600

Clothing

Patagonia torrent 3L. Only had this for a little bit. It was a replacement for my previous rainshell that had lost its waterproofing after 10 years. This jacket seems pretty robust and is minimal in design.
Rab microlight alpine. Another new comer to my gear list! This jacket is a proven warm midlayer that will keep me toasty.
Adidas ultraboosts black. The only shoes I ever bring abroad. I walked 200km in Italy over 2 weeks and my feet were as fresh as the first day. They are incredibly light and dry quickly. Because they are black it means double as shoes for getting into a club or bar.
Merino wool socks. I must admit I was a little skeptical of the hype around merino. But after giving them a try I'm a total convert. Because they keep you dry, they also keep you warm. It's an odd mental model to think of, but moisture (read, sweat) can either cool (and therefore cool your body) or warm up (and make you uncomfortable). Merino solves this problem by being highly wicking. I got a pair as a gift and now plan to replace all my current socks with them.
Jeans. A pair of Levi's. Usually I wear these on flights. Jeans go with everything so are a good item to have.
5x Uniqlo T-shirts. 2 white, 3 black. I don't always bring 5, but 5 is my maximum. If I'm away for more than that, then I just wash the clothes. Black and white mean that dressing to match is super easy.
Shorts. A simple navy pair. Sometimes add another pair of shorts here depending on the type of trip we are taking.
Underwear. 5 pairs. All black. From Amazon so they are easy to replace.
Swim trunks. For swimming...
North face fleece. As this is largely worn to and from the airport in rainy and cold England, the goal of this item was to be light and packable. This fleece fits both those criteria.
Patagonia beanie. Not the warmest hat in the world so may swap this out, but keeps me pretty warm.

Other

Card holder.
- 2 credit cards
- 1 debit card
- Driving license
- Some cash
Tote bag. Used as a beach bag and for groceries. Because it's so compressible and weighs nearly nothing, it's a must have for me now.
SIM card tool. Kept in a little pocket in my bag. Useful for international travel when I need to swap SIMs.
Washing powder (optional). Although I've heard some horror stories of it being confused for drugs, we've not been pulled asides for this in an airport so far! Usually washing powder is sold in large boxes so we carry a small ziplock bag of it. We take powder because we found tablets exploded constantly. So for the extra weight, it's worth it. It's marked as optional as we don't always take trips that require washing clothes.
Toiletries.
- Toothbrush
- Toothpaste
- Plaster
- Paracetamol
Sharpie or pen. The pen is mightier than the sword as they say! You never know you need this, until you need it. And when you need it, it seems impossible to find a pen unless you have one yourself. For the microscopic weight that a pen has, it's worth packing even if I don't use it all the time.
Sunglasses. I always break/lose sunglasses so I just got the cheapest UV400 pair I could find. UV400 means way more light is filtered and protects your eyes. If you're squinting on a beach whilst wearing sunglasses then you probably don't have UV400.

Hiking

This is the stuff I add for hiking trips. So far, I haven't taken my hiking stuff through an airport so just watch out for that! Currently, I'm in the middle of picking a tent and quilt - I will update this list when I get them.

Soto Amicus. A powerful and lightweight stove. Resilient against wind and has saved me a few times. Able to boil a litre of water in around 5 minutes (depending on altitude).
Befree 1L. Probably my best hiking purchase. This bladder bottle is collapsible and can filter any water you find on a trail. Although I generally look for clean water to gather, this removes any nasty stuff.
Stanley Pots. Normally used to boil water for instant meals. Although a little bulky, it covers stuff for 2 people.

FAQ

But what about X?

In all likelihood, I don’t take it. But if you feel you need it, pack it. After you have travelled, see if you used that thing or not and go from there. Continue to cut the fat.

For activity related gear like snorkels and goggles, you can usually rent this or borrow it. Just make sure to clean it first - I got a nasty cold in Turkey from a snorkel 🤮

No toiletries?

Again, to save time, I try to carry no liquid toiletries. Most airbnbs have shower stuff and small toothpaste is generally brought wherever I go.

How much does this all weigh?

Generally speaking the pack weights 8kg or less. 6kg without a laptop.

Thanks for reading!

Ten Software Architecture Rules of Thumb

2023-01-04T00:00:00Z

I love a good rule of thumb. They are instantly understandable and based on the practice rather than the theory of a particular topic.

Through my learnings as a software architect, I have often created these rules of thumb to apply the patterns to other systems.

There is always a bottleneck. Even in a serverless system or one you think will "infinitely" scale, pressure will always be created elsewhere. For example, if your API scales, does your database also scale? If your database scales, does your email system? In modern cloud systems, there are so many components that scalability is not always the goal. Throttling systems are sometimes the best choice.
Your data model is linked to the scalability of your application. If your table design is garbage, your queries will be cumbersome, so accessing data will be slow. When designing a database (NoSQL or SQL), carefully consider your access pattern and what data you will have to filter. For example, with DynamoDB, you need to consider what "Key" you will have to retrieve data. If that field is not set as the primary or sort key, it will force you to use a scan rather than a faster query.
Scalability is mainly linked with cost. When you get to a large scale, consider systems where this relationship does not track linearly. If, like many, you have systems on RDS and ECS; these will scale nicely. But the downside is that as you scale, you will pay directly for that increased capacity. It's common for these workloads to cost $50,000 per month at scale. The solution is to migrate these workloads to serverless systems proactively.
Favour systems that require little tuning to make fast. The days of configuring your own servers are over. AWS, GCP and Azure all provide fantastic systems that don't need expert knowledge to achieve outstanding performance.
Use infrastructure as code. Terraform makes it easy to build repeatable and version-controlled infrastructure. It creates an ethos of collaboration and reduces errors by defining them in code rather than "missing" a critical checkbox.
Use a PaaS if you're at less than 100k MAUs. With Heroku, Fly and Render, there is no need to spend hours configuring AWS and messing around with your application build process. Platform-as-a-service should be leveraged to deploy quickly and focus on the product.
Outsource systems outside of the market you are in. Don't roll your own CMS or Auth, even if it costs you tonnes. If you go to the pricing page of many third-party systems, for enterprise-scale, the cost is insane - think $10,000 a month for an authentication system! "I could make that in a week," you think. That may be true, but it doesn't consider the long-term maintenance and the time you cannot spend on your core product. Where possible, buy off the shelf.
You have three levers, quality, cost and time. You have to balance them accordingly. You have, at best, 100 "points" to distribute between the three. Of course, you always want to maintain quality, so the other levers to pull are time and cost.
Design your APIs as open-source contracts. Leveraging tools such as OpenAPI/Swagger (not a sponsor, just a fan!) allows you to create "contracts" between your front-end and back-end teams. This reduces bugs by having the shape of the request and responses agreed upon ahead of time.
Start with a simple system first (Gall's law). Galls' law states, "all complex systems that work evolved from simpler systems that worked. If you want to build a complex system that works, build a simpler system first, and then improve it over time.". You should avoid going after shiny technology when creating a new software architecture. Focus on simple, proven systems. S3 for your static website, ECS for your API, RDS for your database, etc. After that, you can chop and change your workload to add these fancy technologies into the mix.

Hopefully these rules of thumb can help you when designing new systems. Remember though, they are just rules of thumb, not rules!

Book notes - The Seven Deaths of Evelyn Hardcastle

2023-01-03T00:00:00Z

🧠 Thoughts

The Seven Deaths of Evelyn Hardcastle first grabbed my attention when my friend quoted the opening paragraph.

I forget everything between footsteps.

"Anna!" I finish shouting, snapping my mouth shut in surprise.

My mind has gone blank. I don't know who Anna is or why I'm calling her name. I don't even know how I got here. I'm standing in a forest, shielding my eyes from the spitting rain. My heart's thumping, I reek of sweat, and my legs are shaking. I must have been running, but I can't remember why.

It instantly made me want to read it. Although amnesia can be a gimmick, in this opening paragraph, we immediately get a sense of the problem that will be tackled in the book.

The premise is that you are Aiden, who has seven days to solve a murder. But as each day passes, he inhabits seven different people's bodies. Oh, and you live the same day over and over. It's like a combination of the bourne identity, groundhog day, and an episode of CSI all rolled into one.

After my friend generously loaned it to me, I got right in. But as I trudged to page 30, I found it quite a slog. My main gripe was that the story needed to explain itself earlier. For example, it only explains what Aiden's motivations are in this story much later. And the pacing of how the complexity evolves is like a hockey stick graph - not much in the beginning, but then it ramps up hugely.

The entire first half was a trial. Despite this, it's clear that the author had a wall of post-its and a ball of yarn to organise the plot. It's cleverly woven together to create a rich tapestry of a story. Full of twists and turns right to the end. The second half was much more fast-paced and went from a book I didn't want to pick up to a book I was glued to.

The writing throughout is very well done. The challenge of inhabiting different people is not one to be sniffed at. There is an entirely different personality and way of looking at the world. And the writing perfectly conveys that feeling of being an alien in someone else's body. Almost as if the book takes place just behind the eyes of the body you are in. Standing at 525 pages, the author does a great job of balancing moving the story forward whilst capturing small details.

In my eyes, it was an excellent fiction novel. I could get lost in an engaging story and have a changed worldview after reading it.

🪄 Actionable Takeaways

Our lived experience shapes our personality.
We have the autonomy to change the future despite our past.
"Another set of eyes" can provide new perspectives. It is important to listen to those different voices and seek them out.
A good mystery is a lot like an onion with many layers and often conflicting motives
Many people mask in public.
Secrets often lead to more secrets or even lies.
Judging a situation at face value can provide an inaccurate picture of events.
Multiple competing personalities is a great plot device.
Consider the people you are with now, and don't judge them based on their past.
Strive to be your true self in public and private.

💬 Favourite Quotes

If this isn't hell, the devil is surely taking.

We are never more ourselves than when we think people aren't watching

Nothing like a mask to reveal one's true nature

Every man is in a cage of his own making

Watchful like a deer in the woods that's just heard a branch snap

Their memories crowd the edges of my mind, the weight of them almost too much to bear.

So many memories and secrets, so many burdens. Every life has such weight. I don't know how anybody Carrie's even one.

Five books

2022-12-14T00:00:00Z

I love books.

Books are portals to worlds that others have created. Books are expert knowledge distilled into the everyday. Books can make you cry having been written thousands of years ago. They are true magic.

It was natural therefore that, like with Pokemon, I wanted to "catch 'em all'".

So I purchase lots of used books, picked them up from charity shops, local sellers and more. But eventually I got so many, the small space I had on my window sill for storage didn't cut it. They overflowed onto my desk, in the living room, on my bedside table, even the bathroom. I'd clearly made an error.

Some may have purchased a book shelf at this point. And whilst I contemplated this for likely far more time than anyone else I know, I eventually decided against it.

The driving force was that I didn't want to continue purchasing items that encouraged larger consumptions.

Ultimately I can only physically read one book at a time. And at most I can carry around 5-8 books worth of information in my working memory.

Also, wanting to reduce my personal footprint has motivated me to want to pare down my possessions as a whole. Books are a small part of that effort.

So I've decided I will only allow myself 5 books at a time. They can be swapped or changed, purchased and sold. But there can only be 5.

Why 5? As I tend to read multiple books at once, each will have a purpose.

A relaxing fictional read.
A non-fiction information book.
Poetry or something arty.
A biography.
A big book. 700 pages+ that I can't finish quickly.

I can't be bothered to count my currently collection but it stands at around 50-60. Not that many but enough that I have had to shove them in a random container.

I'm going to read through these and then trade, sell or donate them. Then I'll be down to 5!

Book notes - Project Hail Mary

2022-12-07T00:00:00Z

Project Hail Mary is the 3rd major novel to be released by Andy Weir, most well known for The Martian.

It chronicles an astronaut afflicted by amnesia attempting to save the earth. You know, the normal every day activities.

Project Hail Mary has been my favourite book of 2022 and I've been recommending it to all my friends. To me, it perfectly weaves semi-realistic science, human struggle and entertainment into a rich narrative with characters you can put yourself into. Simultaneously balancing conflict with calm scientific reasoning. There is tension but not in the "the bomb is about to go off" sense. More so in the we've discovered a problem and now use a sound scientific approach to resolve it.

It's almost like reading a doctors report on a patient. Breaking down their behaviours and questioning what it means to be human, why do we communicate with words and why do we sleep the way we do? Weir has an ability to take a microscopic focus on a small unassuming basic truth and rip it to shreds.

I read this at a time where I needed to escape to another world - to step outside of the everyday and enter into a situation where there was everything to play for.

In an effort to remember more from what I read, here is my brief review of Project Hail Mary and my key takeaways.

Actionable takeaways

Humans have the inbuilt ability to thrive in subpar environments.
Food is a seldom thought of limiting factor in the growth of civilisation.
Engineering principles trump knowledge in many areas.
By contrast, people who are good engineers and knowledgeable are the true geniuses.
Scientific literacy is so vitally important.
The characters battling various real world physical limits (like the distance between stars and the speed of light) is a hugely engaging and motivating for the storytelling.
Our names are tied heavily to our identity and memories.
The threat of going crazy with loneliness is not something to be sniffed at.
The book used science to solve real world problems with real world consequences if they were wrong. The protagonist couldn’t rely on computers. They needed the information in their brain. It made me think that the sciences (including maths) are so poorly taught. Completely abstracted away from any real problem, simplified to the equivalent of brain baby food and then spoon fed to children. Science is interesting, useful and in my view, pretty cool.
Simple ”truths” like waving, morse code, saying “hi”, sleeping and other things are so beautiful and could have been completely different. The first interactions with the alien is fascinating because you have to build up this base of language, which in their case isn’t even words!

Favourite Quotes

“Hurry”, “ok I’ll wait faster”

I’m smart enough now to know I’m stupid.

Human brains are amazing things. We can get used to just about anything. I’m making the adjustment.

Oh thank God. I can’t imagine explaining “sleep” to someone who had never heard of it. Hey, I’m going to fall unconscious and hallucinate for a while. By the way, I spend a third of my time doing this. And if I can’t do it for a while, I go insane and eventually die. No need for concern.

What it takes to become a lead developer

2022-11-16T00:00:00Z

"Lead Developer" - now that would look good on a business card.

Most developers I have worked with have all been striving after this title. But, many see the word "Lead" and stop reading there. Based on this, the idea is conjured up that a lead developer will command their team, revolutionize the technology stack and code 75% of the tickets. Meanwhile, others think it's about being a bridge between development and other groups - representing their vision for the company, and communicating their concerns.

Simply put, there is a lot of confusion about who a lead developer is and what exactly he or she does.

Recently some developers have approached me asking what it takes to be a lead developer. In their mind, they are already there. In their mind, they ship quality code, push the team forward and introduce new technology. What more could you want?

Rather than focusing on specific actions that you'll be performing, it's far better to focus on attributes you should develop to be a lead developer. The reason is that actions are difficult to measure their effectiveness. Sure, you may feel your ignored suggestions would transform the company but are you in touch with broader business contexts? When reaching out for a promotion, be mindful about coming from a place where you're already there. You may well not be (and that's ok!).

Build the attributes below, and you'll be promoted in no time.

Communicate effectively. In the era of remote work, communication is everything. Aside from the obvious, not complicating a topic, sticking to the agenda and choosing clear words, there is another crucial element of good communication. Translation. Not from Hebrew to Maltese but from non-technical teams to technical teams and vice versa. As a lead developer, you play a vital role in helping both teams work together cohesively. A good measure of this is how much one team hates the other - the less, the better! Doing this translation work also comes with being able to listen (yes, communication is both speaking and listening) to stakeholders about wider business concerns and use them to inform technical decisions.
Take responsibility. As Uncle Ben (spiderman's uncle, not the bloke who sells microwave rice) would say, "with great power comes great responsibility". As the team lead, you are responsible for how well the technical staff under your care perform. It is your responsibility when something goes wrong, even if it's not your fault. Your job is chiefly to make your team look good, not yourself. Building responsibility helps prove that you're ready to shoulder people problems.
Be principle and process driven. There is no exact playbook for each situation that will arise as a lead developer. But by being principle-driven, you will always know what to do. And processes will form the basis of consistent actions across various decisions. For example, by taking the lead in incident postmortems, you will help develop the principle of transparency and discovery.
Trust others. You don't know everything, and you can't know everything. And that's good! It makes humans interesting and gives different perspectives. When you don't know something, don't be afraid to empower someone on your team who does. Trust them, and ask questions to learn more about what they know.
Trust yourself. As a lead developer, there should be a lot of knowledge you do have that your team needs. Don't be afraid to mentor them and teach them everything you know. Rather than reducing your job security, you'll get a better relationship with your team and improve the quality of the team's work.

Lead Dev or Bust?

Junior dev, mid-level, lead developer. Job done. Right?

I used to think so.

But it's really not as simple as that path. Rather than taking "upwards steps" into management, you can take horizontal steps. If you're a web developer, look at DevOps, Security engineering, site reliability engineering and solutions architecture. Because of the complexity of modern-day systems, there are so many technical roles now.

Management might not be for you. It's not for me. I got bored of managing people directly, doing 1-to-1's, and sitting in planning meeting after planning meeting. I wanted to code, write, read about cool tech and create automated tools. It might not be for you, either.

Having said that, we can all use the principles outlined above. Developing them will lead to being better engineers and improve teamwork.

No Calendar

2022-11-10T00:00:00Z

I used to plan everything - individual tasks, lunch breaks, exercise. My favourite phrase in the house was, "have you put it on the calendar?"

Now, I don't care.

Why?

Planning took time. Every week I would religiously put in my time blocks and appointments; this was good for gauging how much (or little) I could accomplish. But it also took a considerable chunk of time, with no tangible benefit when doing the work itself.
I was anxious about being behind. Time blocking my calendar led me to feel like I was constantly behind. One task would take longer than I had budgeted, knocking out my entire day. Time-blocking evangelists recommend adjusting your schedule when this happens. But it feels like a constant game of cat and mouse. I was always "chasing" the time rather than getting things done.
I was feeling less accomplished. I have a relatively good idea of how much I can do in a day. But, when I time-blocked my calendar, I found it got less done.
Meetings are usually pointless. Although meetings can be helpful, it's rare to find a meeting where everyone is relevant to the discussion, it's for making a collaborative decision, and action points are recorded. Most of the time, meetings are bloated and accomplish nothing in the name of "teamwork". I now reject 99% of meetings I'm invited to and favour asynchronous working practices.

But now I don't use my calendar much, what do I do?

How?

I don't accept meetings. As mentioned before, I prefer asynchronous working - discussions over documents, Slack or email.
Only plan things for the weekend ahead. This means my wife and I can remember our plans and be mindful of other weekend tasks we may need to do.
Reduce my commitments with other people so I can usually remember all our plans.
I removed regular appointments. Every week, I have about four or so regular appointments of various sorts. I realised I don't need my calendar for this. So, I have replaced them with a phone alarm or just remember.
Don't plan too far in advance. Plans too far ahead often overwhelmed me because if circumstances changed, I had already committed to something. This simple approach means I can be more flexible with my time. The exception to this is rough travel plans and dentist appointments.

When I first ditched my calendar, it made me feel uneasy.

Coming from a time-blocking world, I felt like I was wasting time. Not doing what I was "supposed" to do.

But, I realised I needed to pay better attention to myself. If I've done my daily tasks, I can exercise, take a walk, be with my family, or call someone.

I can relax because I have a single source of "truth" for what I need to do - my to-do list. Life is far simpler, and I get more done.

Common objections to CI/CD and why they are wrong

2022-11-02T00:00:00Z

Continuous integration and delivery is something many of us take for granted. In reality, the vast majority of businesses are still using manual testing and deployments.

If you’re inside one of those businesses then doubtless, CI/CD has been suggested - but never implemented.

The reasons vary but here are a few common objections to CI/CD and why they are wrong. Use it as a template to speak with your team about this.

We don’t have time

Often organisations are caught chasing their tails. They don’t deploy automatically or write tests, which leads to bugs, which leads to more time spent. To take a data driven approach, measure how much time you spend doing manual deployments and fixing bugs that could have been caught by tests. Based on this you can say, if we implement a CI/CD system we can eliminate at least 70% of this time (bugs will always happen no matter how many tests).

People won’t fix the build if it breaks

This statements suffers from a negativity bias - looking to the worst case. And the person who made the statement also doesn’t trust their staff.

A good CI/CD pipeline empowers the developers to resolve problems themselves. And whilst it’s impossible to eliminate 100% of issues, it is possible to add passing test suites as part of the requirements to accept a merge request.

Additionally, appoint a “build master” that rotates week to week who is responsible for fixing the build. Make sure that person has that time factored in for any planning.

Releasing all the time will break things

This statement suffers from zero risk bias. If you’re wanting to introduce an automated system, my guess is that the manual system has gone wrong - a lot.

Taking a data driven approach, you can trial a CI/CD process, and then measure the amount of errors that occur, and the mean time to fix them.

After the trial period, compare it to the data taken when doing manual releases. If the results are favourable, then you can proceed the discussion further.

We don’t have tests

This is, in some way, the only valid reason. And while it’s not possible to get a full CI/CD pipeline running, you can start. But how?

Start small. Require tests for new features and bugs. Invest heavily (via contractors or dev time) in building an E2E test suite.

If your business cares about shipping new features quickly and doesn’t want bugs then this investment should be an easy sell.

—

Ultimately, CI/CD is an investment. And many businesses are unwilling and afraid to take the plunge. Recognise that humans have biases that govern their thoughts and actions. You have your own biases too! But, by taking a data driven approach it becomes a much more collaborative effort rather than one person championing a large change.

Get rid of your retrospective meetings

2022-10-27T00:00:00Z

The end of the sprint rolls around - you know the drill. Pile into a room with the rest of the tech team for 2 hours and discuss how the sprint.

The goal of this is to action any improvements that could be made.

But that's rarely what happens.

What happens is:

People rant and don't offer solutions.
Teams assign actions/blame to people outside of the meeting
Action items aren't followed up correctly.
Action items don't always reflect where business priorities lie.
A small percentage of the team dominates the conversation. Making it impossible for others to speak.

And that is repeated week in, week out.

I have seen this pattern emerge in many organisations I have worked with. And the solution was quite simple - get rid of retrospectives.

What would this look like practically?

No regular meeting
Create a group retrospective board that is the same week to week. This surfaces recurring problems.
Action points "open" at any one given time are capped at 3. This pushes more ownership of those changes rather than just piling them up.
The items raised on the retrospective board are done asynchronously via chat. This encourages less extroverted members of the team to get involved. The idea is that this process doesn't always need to be repeated. Just as and when there are new items to discuss.
Once action items are decided as a priority, they are created as a ticket or task in a shared area (like JIRA).
Reminders are set weekly to remind the team to follow up on these action items.
All action items should have a few states - Done, In Progress, Outside of team scope, Not a business priority. In the case of the latter two, this means they are dropped or a workaround needs to be implemented.

Why this approach?

No meetings. Meetings cost a *lot* of money. It's total madness to block important work in favour of ranting about all the problems. In the words of Elvis Presley - a little less conversation, a little more action, please. If you're curious about the cost, you can estimate it using this tool.
All persons get an equal say. Whereas in a meeting a small minority dominate the conversation, this approach encourages all to participate.
Asynchronous. If you're a large organisation, no doubt you're desperate to expand to other countries. But without asynchronous working practises, it will be chaos. Encouraging these practices early sets you up for success in the future.
Focus on solutions, not rants. As the conversation is entirely oriented around capturing action items that line up with business goals and can be accomplished by the team; it stops conversations about how Joe in marketing is blocking work or Elizabeth in admin keeps sending requests directly to one developer. There is a place for those discussions but this is something to capture as and when they happen and escalate this through managers. A retrospective sprint discussion focuses on the action items and nothing more.
Documented. Visibility to stakeholders and the tech team is important. It means we can easily go back and see recurring issues, actions taken, and if those solutions worked. This all builds towards an organisation that documents its work. All too often are retro meetings held for 2 hours, and produce nothing more than a couple of messages on Slack.

I have used this approach to great success in many organisations and I hope you can too!

Give it a try for a while. After all, no one is going to complain about missing a 2-hour meeting every other week.

Creating my own personal Instagram

2022-10-06T00:00:00Z

This website is my own little corner of the internet. Historically though, this site, has been a bit "distant", a more professional reflection of my own self.

Its purpose has been to share documentation on things I have done, and allow people to reach me on other platforms. This site is only a small slice of my personality with Twitter and Instagram filling in the more "personal" gaps.

But recently, I have logged out of Twitter and it was over 3 years ago that I deleted Instagram entirely. My reason for getting rid of both was

Creepy advertising. Particularly from Instagram. My wife, who still has instagram has had a number of extremely personal adverts that she never searched the web for.
They had no purpose. Twitters main purpose for me was to "build an audience". Building an audience seems to be the catch all advice these days for influencers in the indie maker space. But, I'm not making a product at the moment - so it serves no purpose.
Ownership of data. Slowly, I'm wanting my data to be more contained on my own website. I don't want to have tonnes of logins and platforms to check, scroll and update. I want to have my little corner of the internet that I can prune and shape.

But enough of the tin foil hat, tie dye tshirt and discount James Bond villain sayings. The question is, what am I doing about it?

Well the first step is in a new page I have added.

Introducing /photos!

As you could guess from the URL, it's a page for my photos! Specifically ones that I am proud of an label as "photography". How it works is a more exciting process that I will explain in another post. But, to cut a long story short, I can upload photos directly from my phone to my site. And all the data is kept in the repository (I may move them to a CDN in the future). On my homepage, I have also added a photos slider that displays the 5 most recent images. Soon, it will be available as an RSS feed, so that people can subscribe to it like you may follow an instagram account you like (but I'm working on that!)

The downsides

My website is obviously not a social platform. The photos I post here will not get as much exposure as on Instagram. And without using Webmentions, there is no simple way to comment on these posts.

This is just the first version of the photos page, but marks the start of me moving my data onto my own website. Next up is my reading list, tweets (notes), and music!

How to fix 'Public key authentication failed' for Azure DevOps

2022-10-06T00:00:00Z

If you're using DevOps and tried to clone down a repository with a Mac you might have stumbled across this error.

Cloning into 'example-repo'...
remote: Public key authentication failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

You double check your SSH Key is correctly added but it's all correct. What gives?

This error usually appears when you are using multiple ssh keys.

The solution, thankfully, is simple.

Solution: Add 'IdentitiesOnly yes' to your SSH config

Here's how to do it

Open up a terminal window and type `nano ~/.ssh/config
Add the following to your SSH config

Host ssh.dev.azure.com
    UseKeychain yes
    IdentitiesOnly yes
    AddKeysToAgent yes
    IdentityFile ~/.ssh/id_rsa
    PubkeyAcceptedKeyTypes=ssh-rsa

And hey presto you're done!

You can now git clone projects with SSH using Azure.

Microsoft vaguely hide this fact in some documentation here.

Blog Roll

2022-09-28T00:00:00Z

This page will be updated periodically. It is a list of websites that I enjoy reading. The list is in no particular order.

Making the case for CI/CD

2022-08-31T00:00:00Z

If you browse developer forums, continuous integration and continuous delivery will not be new concepts to you. We all love the utopian ideas of being able to open a pull request and commanding an army of robots to do your bidding and get the application tested and shipped.

Unfortunately, the reality of many businesses is different. Developer experience is an area seldom invested in; despite being such a huge opportunity for large productivity gains.

Perhaps already you have been pleading to tinker with github actions, switch to a PaaS or use a IaaC but to no avail.

So, how can you champion the move to a continuous integration and delivery?

First let's look at the reasons people may object.

They are scared of change. This is a powerful force not to be underestimated, even in the fast-paced technology industry. This excuse is particularly potent with stakeholders, who don't (understandable) know the ins and outs of technology.
They don't trust the team. If things are already going wrong, why would they invest time in a risky new deployment strategy? Often upstream trust issues rear their heads in these discussions and it's important to navigate as carefully as a sailor does with rocks by a shoreline.
They lack metrics to determine if change is successful. As Peter Drucker famously said "that which cannot be measured, cannot be improved". Without numbers going up, down or sideways, a stakeholder cannot correctly understand the team and systems overall health.
The vision seems too good to be true. If we dream of deploying 50 times a day to production like Instagram does, then it can seem like we'll never catch up to that level.

Start with approval

It's utterly pointless starting any of this without some kind of green light. Although we love stories of people doing things under ground and against the odds, the fact is that you're not creating a cure for cancer. You should speak to your immediate team and get their feedback. Then create a formal proposal and, again, get more feedback. Then get the green light from the relevant stakeholder.

The proposal should outline the overall vision for the system, the advantages, the risks and the downsides. If you can't think of risks and disadvantages, then you haven't thought about this problem enough.

Throughout all this, make sure to not just push your vision on the team. Be collaborative. Keep an open mind. And don't pretend that it will be a breeze and nothing will go wrong (hint - it will!). The worst thing to do is to try and do it all yourself or with only a couple true believers. This is a sure fire way to make sure the initiative is abandoned.

Start with the data

Often times, stakeholders say they want a fancy dashboard with burndown rates, bug reports and the position of Saturn. In reality, an excel spreadsheet will work.

Create a spreadsheet with the following attributes:

Release frequency. For tracking the number of releases on a given day/week.
Rollbacks or defects detected. A count of the number of times you have had problems you have had to fix or rollback on a production system.
Hours to deploy. This one requires a bit more coordination. But, developers could start tracking their deployment time with a specific time tracking measure in software like Toggl or Harvest.

Start measuring these metrics for at least 4 weeks before making any changes. And make sure the spreadsheet is shared, collaborative and documented well - both inside the team and to any stakeholders.

Start small

Now you have your numbers, you can start to look at improving them!

Using your original proposal as a guide, build up a step by step plan that can be implemented in a sprint (a 2 week period) or less. Be mindful of any steps that will create blockers for others and plan around this accordingly to mitigate their impact.

Unlike with code, we often can't rollback changes made to deployment and testing systems. So, think carefully about how those changes can be reversed if needs be. I've been burnt many times by not considering this carefully enough.

After each step, take a 1-2 week break before making other changes, continuing to record your data points. Verify that the numbers are heading in the right direction. And if they aren't, you can address this in a retrospective meeting.

A sample plan may look like this.

Goal: Be able to merge into the `staging` branch from a feature branch and have the code automatically tested and deployed to our staging environment.
Currently: FTP files onto a server manually once a week on Tuesday. No test suite.

Step 1: Create a testing suite for critical components of the application
Step 2: Create a github action (or other CI service) to run the test suite for all new pull requests
Step 3: Add a github action that FTP's files from the `staging` branch to the staging server

Obviously I have glossed over a number of details, but hopefully you get the idea. It's about making small changes and not just going for a "big bang" type change (which are always doomed to fail).

It can be tempting to prioritize big change first, but as a word of caution, try to focus on being iterative rather than catching the big fish.

As you go forward, and the metrics improve, the buy in from stakeholders and others will improve. Nothing makes a stakeholder happier than knowing that it's cheaper to do some work!

This can drive further change which follows a similar process as outlined above.

By the end of it, you may not be deploying to production 50 times a day, but doing something much more humble. Saving you and your colleagues, hours of mundane, repetitive and error prone work. And isn't that just as valuable?

Question your Rate limits

2022-08-02T00:00:00Z

If you are building a system with an API, there is a good chance it has a rate limit or you stay awake at night afraid of a DDoS attack.

Rate limits are a method to reduce network traffic by putting a cap on the number of times an action (like calling an API endpoint) can be performed in a certain time frame.

Rate limits for APIs are spoken of as a security measure. And, principally, they are a good idea. They prevent heavy usage of your system in a short space of time. The kind of behaviour that a malicious actor would have. Or would they?

What is spam?

The problem with rate limits is that they are blunt instruments. More of a mallet, than a katana.

You can define a limit on the number of requests, let's say 100 per second, per API key. But then you get a call from your golden goose customer, ACME Corp. They say they are getting a 429 Rate limited response when calling your API because they are such big customers of it!

Granted, in this situation, you can raise the rate limit for them individually. But you don't want to involve manual work as you gain larger customers. You're trying to prevent malicious usage, not mass usage.

Firewalls on top of your application aren't a silver bullet either. They can enforce semi-complex rules for rate limiting but the problem of differentiating spam from legitimate traffic remains.

Masking over scaling problems

In serverless environments, there is an associated cost with lots of requests. But for a long-running service hosted on EC2, ECS or EKS (or the Azure/GCP alternative), do more requests really matter? Not really.

In my experience, rate limits are often introduced to protect against scalability problems. And even if they are not, they often mask over-scaling issues. Although we don't want to prematurely optimize the product, it is prudent to have at least a 50% buffer from your peak load. In other words, if your system can handle 10,000 users at peak time, you should be aiming to be able to cope with 15,000 users.

Using rate limits to avoid scaling problems can be wise and you certainly want to protect against malicious use of your system. But, it's only one side of the coin.

Be mindful of the use case

When someone mentions adding a rate limit, ask why. What attack vector are you trying to mitigate? What specific type of traffic are we trying to avoid?

In any case, there should be some data behind this to say we are getting X traffic which is pushing up 95th-percentile response times by Y.

As we have discussed, rate limits are blunt instruments. They are only one weapon in our fight against malicious actors. The solution is a multi-pronged approach:

Implement bot detection (like Cloudflare Bot Management) to detect bot attacks
Add a firewall to deny incoming connections that aren't legitimate.
Add cautious rate limits to key areas of the system. You like will need a higher rate limit on your GET /customer endpoint than a /login endpoint. Work based on the 50% buffer limit.
Add systems to detect if a person is trying to login with known-to-be-breached passwords for a single user. This is likely a credential stuffing attack or a brute force against a key user.
Implement a "blocked" user concept. If there are N number of failed attempts, then they need to unlock their account using a code from their email.

There are of course many more defences you can use to fight spam. In a majority of cases though, your site won't be the target of malicious use. So, don't think you need the same security as MI5 - because you probably don't. Rate limits are an easy method to implement, but question the why. It might open up a larger conversation about more robust security measures. But, always keep the customer and their data in mind, not the protection of response time statistics.

Site Upgrades - Gatsby V4 and Webmentions

2022-07-28T00:00:00Z

I was fed up. I was writing a post for my site the other day and was reminded of the fact that I couldn't run the site locally. It was time to fix that, once and for all. I was going to take on the task of lifting my site from the stringed together mess and add some new features along the way.

Here's how I did it:

Gatsby V4

First thing on the agenda was upgrading Gatsby. The existing site was on Gatsby V2 and now didn't even run on my computer. As I was upgrading 2 major versions, and dozens of other plugins, I thought I was in for a difficult time. But, thankfully the upgrade process was relatively painless.

I simply run ncu -u && yarn and I was away!

I did have to update the gatsby-plugin-feed as it now required configuration. I popped in the following, based on the example in the documentation and hey presto!

{
      resolve: `gatsby-plugin-feed`,
      options: {
        query: `
          {
            site {
              siteMetadata {
                title
                description
                siteUrl
                site_url: siteUrl
              }
            }
          }
        `,
        feeds: [
          {
            serialize: ({ query: { site, allMarkdownRemark } }) => {
              return allMarkdownRemark.nodes.map((node) => {
                return Object.assign({}, node.frontmatter, {
                  description: node.excerpt,
                  date: node.frontmatter.date,
                  url: site.siteMetadata.siteUrl + node.fields.slug,
                  guid: site.siteMetadata.siteUrl + node.fields.slug,
                  custom_elements: [{ "content:encoded": node.html }],
                });
              });
            },
            query: `
              {
                allMarkdownRemark(
                  sort: { order: DESC, fields: [frontmatter___date] },
                ) {
                  nodes {
                    excerpt
                    html
                    fields {
                      slug
                    }
                    frontmatter {
                      title
                      date
                    }
                  }
                }
              }
            `,
            output: "/rss.xml",
            title: "Developer Musings - Josh Ghent RSS Feed",
          },
        ],
      },
    }

Fixing the archive page

This is something that has bugged me for ages. For the eagle-eyed among you, you may have noticed all the way at the bottom of the page that February 2018 was listed before December and November 2018.

Initially I thought this was to do with my custom graphql grouping that groups based on the year-month. But after some debugging I found that it was grouping correctly, but not sorting the groups.

I added a sort to the allMarkdownRemark graphql statement but that had no effect.

Then I noticed that all of the months were just single digits. So to graphql, "2018-10" was greater than "2018-2".

I updated my code that added this node to this:

// gatsby-config.js

function pad(n) {
  return n < 10 ? "0" + n : n;
}

exports.onCreateNode = ({ node, actions, getNode }) => {
  const { createNodeField } = actions;

  if (node.internal.type === "MarkdownRemark") {
    const value = createFilePath({ node, getNode });
    createNodeField({
      name: "slug",
      node,
      value,
    });

    const date = new Date(node.frontmatter.date);

    const year = date.getFullYear();
    const month = pad(date.getMonth() + 1);
    const yearMonth = `${year}-${month}`;
    const day = date.getDate();

    createNodeField({ node, name: "year", value: year });
    createNodeField({ node, name: "month", value: month });
    createNodeField({ node, name: "year-month", value: yearMonth });
    createNodeField({ node, name: "day", value: day });
  }
};

After restarting the site build, the dates were all prefixed with zero's and the sorting worked!

Unfortunately, when I created a pull request for a site, and Netlify attempted to create a deployment preview, it failed. After reviewing the issue, it was due to a plugin Gatsby-plugin-preact. This plugin massively reduced the bundlesize of my site. But, I couldn't find a way of working around the problem so I had to remove it. It appears to be a problem associated with the fact that Preact and React are now no longer interchangeable as of V18+. Unfortunately, this means my sites payload is up to 400Kb. I'll be working to reduce this as that's far too bloated.

Design

Design has never been something that came naturally to me. But, I employed a trick from some great artists - I copied.

For example, the little block page breaks, they are from Mu-An Chiou. I loved the clean, minimal design of her site. Mine looked a little rough around the edges and needed something to make it a bit different. I'll be continuing to iterate upon the design.

I also fixed a long standing issue with the mobile navigation. The buttons were so close to each other you couldn't really click on the link you wanted. Or at least I couldn't with my fat fingers.

So I updated the links to give them more space.

The last thing was fonts. Websites should be interesting to look at. And, in a small part, I tried to accomplish that by using a new title font - Space Grotesk.

I feel this font face immediately tells you its a blog about software development. I'm going to work on other small design tweaks to make the site more charming.

Webmentions

Something that's been on my list for ages is adding indieweb features, like webmentions. A number of my friends, Carol, Jamie and others, have added webmentions to their sites.

The basic premise is that you can collect instances where you site has been referenced around the entire web. It sort of goes back to what the web was originally intended to be, a network of pages.

In the past, I had a comments section on Discus. It was removed shortly after due to lack of usage. No one leaves comments on sites anymore. But they do tweet about it or add it to their newsletter. And webmentions allows us to collect those instances. You can give it a try by tweeting a link to this post and seeing your tweet appear at the bottom of this page!

The webmentions are fed from https://webmention.io. A fantastic free API that collects all of these mentions for your site.

H-Feed

A small Indieweb update I made was to include a H-Feed for my blog posts. Think of H-Feeds as RSS for the Indieweb.

I added the following code to my /archive page.

<ul style={{ display: "none" }} className="h-feed">
  <h1 className="p-name site-title">{siteTitle}</h1>
  <p className="p-summary">Archive of all posts from joshghent.com</p>
  {data.posts.edges.map(({ node }) => (
    <li>
      <article className="h-entry">
        <Link className="u-url" href={node.fields.slug}>
          <h2 className="p-name">{node.frontmatter.title}</h2>
        </Link>
        <address className="p-author author h-card vcard">
          <a
            href="https://joshghent.com"
            className="u-url url p-name fn"
            rel="author"
          >
            Josh Ghent
          </a>
        </address>
        <span>
          <time className="dt-published" dateTime={node.frontmatter.date}>
            {node.frontmatter.date}
          </time>
        </span>
        <p className="p-summary">{node.frontmatter.description}</p>
      </article>
    </li>
  ))}
</ul>

Tests

Ok so this bit might be a bit overkill. I added tests to my website.

My motivation was simple:

Wanted to reliably merge dependabot (and my own) updates without issue
I wanted to learn more about site testing (where there isn't code per-se to be unit tested)

I added a few tests using Jest that make sure certain components render correctly when given certain inputs. I utilized snapshots which have worked great so far and even caught some bugs!

In the future, I'll be adding screenshot testing to make sure the webmentions component renders correctly and more.

What's Next?

My next phase of development for my site is adding new data sources and capturing "notes". Many Indieweb people prefer to use apps that publish to a micropub endpoint. But I don't want to do anything "special" outside of my normal workflow (like tweeting, listening to music etc). So instead, I've decided to attempt to do it all via Github Actions.

I already have the first action setup that will record data associated with a page that I bookmark. It was created by Katy DeCorah, with the idea that each time you want to bookmark a page, you can do so by creating a github issue. The action then reads the URL and records it to a yaml file.

The last part, which I haven't done, is using that data to dynamically create pages.

But, onwards and upwards!

The Culture of Estimations

2022-07-04T00:00:00Z

In business, moving fast is everything.

Sales depend on them, marketing wants to advertise for them and the CEO want's to award themselves a large paycheck without any quarrels.

But, it's the software engineers who have to design, build and test the features. And because software is an inherently creative pursuit, figuring out when something will be done is hard.

Thus, we are led to the creation of software estimations. You may know them as story points, or ticket points as well.

On the surface, the goal of trying to "estimate" when work will be done is a good one. Software is not something done in isolation, and we have to recognise that other teams depend on the work we are doing.

But the reality is that estimations are just that - estimations. But they are treated as absolute values. Blame is assigned when your 3-point ticket took longer than a few days. Your job and any promotions are weighed in the balance of a single figure, that was perhaps not even decided by you.

But where did we go wrong here? Why are estimations and the culture around them harmful?

Managers ask for estimations but want deadlines

One of the biggest issues with estimates is that they are used as a proxy word for a deadline. In reality, deadlines are important. It allows other teams to prepare their department. For example, a deadline for a feature allows marketing to prepare a campaign, sales teams to be trained on the new feature and support teams to learn the ins and outs.

But estimates are not deadlines. So why not remove the middleman?

In some cases (not all), it would be best to communicate with your manager and ask them what deadline there is. And then, practically, discuss the work item to meet that deadline somehow.

Developers rarely have all the information required

At their core, estimates are only as good as the information provided to them. And, unfortunately, there is no extra "confidence" number to put against an estimate. You have the estimate, and that's final.

In my experience, tickets rarely have all the information you could need to make an informed estimate. You don't need everything and the kitchen sink. But, I have often seen cases where critical business logic was not listed in the requirements. Other times, designs are not listed because the work item is "basic". Only for the feature to be redone when it turns out the stakeholders had very precise ideas of what the feature would look like.

I'm sure you have stumbled across cases like this before, maybe even being on the receiving end of them. One solution I have found practical is a checklist along with a ticket. It goes something like this:

Do we have designs for this item?
Do we have all the critical business logic listed in the acceptance criteria?
Do we have a clear idea of what systems we need to change to deliver this work?
Are there any potential blockers we can foresee in doing this work item?

In practice, you can never have 100% of the information. It's a ticket, not a crystal ball. But, having a high confidence level when making your estimates will improve a developer's happiness and decrease the chance that the feature needs to be changed after being built.

Estimates are formed based on a small set of factors

The premise of estimates is that the ticket being evaluated is a neat loop of functionality that can be rounded off and shipped by anyone in the team. Although in a software utopia that might be the case, the reality is much different.

Tickets often overlap each other. Or only contain one part of the work involved in a larger feature. A common pattern is to divide backend and frontend tasks. In large companies, one team may need to create a ticket for another team to add a new value to an API. In all these cases, there is a question mark as to where the "effort" is assigned. To this ticket or that ticket.

Furthermore, in my experience, estimates are based on previous work experience at a company in that project. The more experience a developer has, the less effort score they will give. I know I have been guilty of this. So, the estimate has an inherent bias towards people who have knowledge of the system. That then ties the ticket to an individual - which creates a bus factor.

Additionally, there is the psychological factor. Estimates assume we are worker robots churning out code ad infinitum.

But that's not the case. Here's an example.

Your team has just finished a huge rewrite of a spaghetti ball of ASP.NET into a nice modern framework. Your team is pretty happy with their work. But, after that large set of work where progress seemed a foreign concept, they are given an enormous feature to do. Although their estimates may reflect their expertise on the project. The actual time it will take to complete the feature will be longer. Why? Because they are mentally exhausted from the first feature they built.

Another example, you have a family member who has a serious illness. Again, your estimates may reflect the theoretical time it would take you. But the reality is different. You struggle to complete the task because of the mental tax on you from other sources.

Although it's not possible to control these inherently human issues. We need to keep them in mind when making estimates, drawing up quarterly timelines and promising features to customers.

Estimate numbers are meaningless

What does 3 points even mean exactly? Is 3 a day, a week? In all teams I've worked with, when you go in it seems like an advanced race who have understood time in a new dimension. A lingo evolves that categorises a 3 as a day, a 5 as 3 days and a 13 as two weeks.

But there is no consistency. The numbers are simply a proxy for the days.

The original purpose was to be an estimate of both risk and time.

And this, in theory, is good, but in practice is hard to pin down a single number that represents two values and communicate that across a set of people. Especially given the fact that the number is arbitrary. When a doctor provides an estimate of the risk to a patient's life during an operation, it's given as a percentage. And this gives us some grounding. Because we know that it represents the number of people per 100 that will die during that operation, in theory.

But estimate points lack that grounding into something real and practical.

When I've joined new teams in the past, they've circulated an image listing the number of points and how that correlates to days. What results is that you are working like Alan Turing to decode the estimates during sprint planning meetings.

My preference is to just drop the proxy. In the vast majority of cases, you can simply use days. Or, if possible drop estimates altogether and prioritize rigorously. Often work takes as long as it takes. Enough trust should be built in your team to do the work diligently but not take too much time.

Conclusion

If you use estimates, be mindful of what that number represents.
Consider switching to another estimation system.
Don't invent lingo that new team members need to learn to take part in the planning meetings.
Estimates are not one size fits all. Sometimes you may need a deadline. Other times you don't need an estimate at all.

Like many things, estimations started out with good intentions. But, along the road, they took a bad turn. Question the pre-existing routines and you quickly realise they've been passed on as gospel. But, working as a team, you can build a system that works for all and continues to deliver a quality product.

My journey with carpal tunnel

2022-06-21T00:00:00Z

About 5 years ago, I went through a time when I doubted I could continue programming.

I had carpel tunnel syndrome. A repetitive strain injury where the tendon going over the wrist at the base of your hand compresses the median nerve.

This compression causes your hands to feel numb, aching in the forearm and the inability to move your fingers without stabs of pain reminding you of your injury.

Carpal tunnel is the injury of the internet age. With many of us hunched over our screens and using them for everything (work, banking, socializing and entertaining etc), RSI is a natural consequence.

Many programmers have written about their experiences. I wanted to share mine. Cutting through the noise and presenting, what I believe to be, reality.

First, to give you a context of my overall health, I was 17-18 at the time, reasonably thin but not someone who frequented a gym.

Treating myself

When I first noticed pain in my forearm and tingling in my fingers, I did what all good internet children do - Google it. I stumbled across many different articles discussing tennis elbow, repetitive strain injury and carpal tunnel syndrome. Based on my own symptoms, I believed I had CTS.

Foolishly, I had ignored the pain for the longest time. Choosing instead to block it out, favouring writing endless code to maximise my learning and playing Starcraft until the early hours.

By the time I decided to do anything about it, it was already too late.

My wrists ached with a pain I couldn't get rid of. And the underside of my forearm had filled with fluid to protect the area (aren't bodies clever?).

I quickly searched YouTube for stretches I could do - including my favourite by "Dr Levi". Although these exercises gave me some relief, it was temporary. I soon turned to painkillers desperate to rid myself of the pain.

Lesson: Don't ignore your body. I could have managed the pain a lot better had I done things earlier.

Diagnosis

Getting diagnosed was easily the most annoying part of the experience. When you go to a doctor (on the NHS), the first thing they have to do is refer you to physiotherapy. One of the many failings of the British healthcare system is it's based on flow charts. It standardizes care but doesn't fairly give a patient autonomy to skip stages that will likely be a waste of time and money.

Treatment

Physio

The first lot of treatment was 5 sessions of physiotherapy. I tried to be as open-minded as possible but knew deep down that it wasn't going to give any lasting benefits.

I was given a large rubber band to pull on for resistance training. Tennis balls for rolling out the affected area. And the same stretches I had been doing from YouTube.

I got the impression that many of the doctors I met weren't well equipped to deal with someone of my age and fitness. Carpal tunnel usually affects pregnant women and the elderly.

After all of this, my initial impression had been correct. It gave me some benefit, but nothing lasting.

It wasn't all in vain, however. One particular doctor did figure out that I had the median nerve trapped in both my wrist and back (near my shoulder blade). This gave me a formal diagnosis of carpal tunnel syndrome.

Lesson: Try to keep an open mind to people even if you think they can't solve the problem at hand.

Osteotherapy

During the time I was getting physio, my mum had been going to an osteopath to help with her back. She recommended that I go and visit him and see what he could do. I'm skeptical of alternative medicines but had seen the impact he had on my mum so I decided to give it a go.

A single session was £50 for 30 minutes. That's a lot of money for me even today. But back then it felt like a huge chunk of my wage was being spent on these sessions. Nonetheless, I was willing to pay for just some relief and the glimmer of hope that it would work long-term.

Despite numerous different techniques, nothing worked. However, after massage (not just the affected area), I did come out feeling incredible and was able to put away my painkillers for at least 2 days.

Surgery

After deciding osteopath treatments were a waste of time and physio had yielded no results, I decided to push my doctor to get me seen by someone else.

Thankfully my GP referred me right away to a "hand specialist".

She also gave me a generous supply of co-codamol (30mg codeine + 500mg paracetamol). A drug I'd been taking out of necessity after my body began to resist the effects of paracetamol and then Paramol (7mg codeine + 500mg paracetamol).

The result of this cocktail of pills was that my work became extremely difficult to do. I became aware of a lack of focus and tiredness. Almost as if I was slightly tipsy and hadn't slept the night before. Initially, I had to time the drug carefully. If I took it too soon before or after eating, I felt like vomiting. And if I took 2 pills (the recommended dose) at the same time, I felt like my brain wanted to float out of my skull. Eventually, I figured out that I could take 1 tablet, wash it down with a coffee and then take another 20 minutes later. This would have minimal effects on my focus and not make me high, whilst also giving me pain relief for the day.

Around this time, I came home one day to my mum pointing out that my skin had started to discolour. My livers protest at having to deal with the painkillers.

She prescribed a mountain of spinach and other iron-rich foods, which helped alleviate this.

But the message was clear, I needed surgery.

When I saw the hand specialist, they immediately put me on the list for surgery. That was the good news. The bad news was that I'd need to wait 4 months. But, he did offer that he had availability next Tuesday at his private clinic. I forget how much the operation would have cost, my family and I were unwilling to pay. I could wait it out.

Whilst I waiting for D-Day, I tried a laundry list of treatments that many with RSI recommend. With mixed results.

What did work

Using a low-key-travel keyboard (not mechanical). This helped immensely as the amount of force I needed to put in from my forearms to my fingers was minimised. As much as I loved my cherry red switches, it wasn't worth it.
Rest. As much as this helped, it wasn't practical to never use my wrists. I'd recommend resting as much as possible, as soon as possible. I foolishly continued with hobbies based on a computer.
Stretching. My forearms had tendons so tight, I could have played Beethovans 5th using it as a violin. Stretching helped me loosen up.
Exercise. In small, light doses exercise did help. Flexibility workouts and lifting weights to strengthen my back aided in maintaining good posture.
Hand warmers. One side effect of carpel tunnel is it affects blood flow into your fingers. Cold fingers being forced to type quickly feels like coaxing an unoiled engine to life after years of idle standstill. A USB-C hand warmer is a life saver for the price.

What didn't work

Believing it wasn't there. I don't know what voodoo scientist came up with this idea, or how it became so popular in the tech community but believing the pain wasn't there (quelle surprise) didn't work. The book entitled "Healing Back Pain: The Mind-Body Connection" came highly recommended by both hacker news and other programmers who had faced RSI. I read a portion online and decided quickly that it stank of bad science. If it works for you... great. You clearly have access to a realm far beyond my reach.
Exercise. As previously stated, doing small, light exercises did help. But the wisdom of lifting lots of heavy weights did not. I think this would have worked had I started sooner. But by the time I attempted it, my wrists felt like they would snap.
Wrist splints. I used wrist supports every single night. I even took them to social events sometimes if my pain was particularly bad. Although in theory, they keep the pressure off your wrists by tilting them upward. The reality is I felt no material difference in pain when I did and didn't use them.

The Surgery

When the day came for the surgery, I felt relief but also a little anxiety. I had never had surgery before. I was unsure of the "setup" you know. In movies where people are having bullets removed and organs transplanted, we are familiar with the look of an oval operating theatre with dozens of doctors milling about.

But my operation was different. I even, sort of, knew how to do it.

Dr. Youtube had provided me with a real-life glimpse into what my procedure would look like. It involved slicing the tendon that goes over the carpal tunnel. Simple as that. When it heals, you pray that it doesn't compress the nerve again.

The surgeon himself was very friendly. There was soft classical music humming in the background and tubed lighting flickering above me. It was a small rectangle room with the operating table in the middle, two swivel stools at opposite corners and an ancient windows XP tower whirring in one corner.

A sort of screen that resembled a thick paper towel was erected to obscure my view from the operation that was taking place.

After the surgeon ran through his checklist, confirming the operation I was having and my details, it started. First, I was injected with a local anaesthetic near the incision site. And then the surgery was underway. Due to the numbness, it felt like knives scrapping at a brick. It felt odd. The pressure was familiar, but the lack of pain was something my brain couldn't quite fathom.

In about 15 minutes, the surgery was done.

I was helped to sit up and the doctor started asking me questions. About 30 seconds after, I passed out. It was the first time I'd passed out (and I hope last). I saw what resembled a wormhole. With walls of a metallic jelly herding me towards the infinite centre. I tumbled down. And then suddenly woke up, in a different room with my, then, girlfriend at my side. She asked how I was doing as I came round, bleary-eyed as if after a long night's sleep.

The nurse assured me this was quite common but I felt quite weak having fainted at such a minor operation. I hadn't seen the blood or the wound. I just passed out as the blood rushed back to my head.

Day by day, my hand recovered. I gained mobility back in around a few days but my hand was stiff from the stitches for a couple of weeks until they were removed.

After 2 weeks, I returned to work. My hand was quite stiff and the stitches itched but my pain was gone!

The fluid that had built up around my forearm still remained so I was a bit concerned it hadn't worked completely. But about a month afterwards, it was clear that the procedure had worked. But the fluid was not going to go away.

Conclusion

After all these treatments, this is how my hand now looks.

You can see the small scar just above the ridge of the wrist where the incision was.

Overall, surgery was the best thing I could have done. My only regret is not acting sooner. Here are my main takeaways:

Start the chain of care early. It can take time to get referred to different clinics.
Listen to your body. Don't ignore it. Get things resolved now. Like right now. Health is the most important thing you can have.
Surgery is not scary and does work. Seriously. Look at the stats. You can't believe the pain away. If you feel you need surgery, and doctors agree, then go for it. It means you don't need to worry about having pain again.

If you have any questions about surgery or anything else then hit me up on twitter.

Deeply Remove a Key from an Object

2022-06-14T00:00:00Z

Recently I had the problem of removing a specific key from an object. Normally I would use omit from the Lodash or Ramda library. But, there was a catch - I also needed to remove the key from nested structures within the object.

Here is a code snippet of how I solved it in NodeJS with Lodash.

I'm sure you can make this VanillaJS but transform is a handy function to use.

import { transform } from 'lodash';

const isObject(value) {
    const type = typeof value
    return type === 'function' || type === 'object' && !Array.isArray(value) && !!value
}

// Deeply remove keys from an object
// @param - obj: Object - the object to remove the key from
// @params - keysToOmit: Array/String - string or array of strings of keys to remove
const deepOmit = (obj, keysToOmit) => {
  const keysToOmitIndex = Array.isArray(keysToOmit) ? keysToOmit : [keysToOmit]);

  function omitFromObject(o: any) { // the inner function which will be called recursivley
    return transform(o, (result, value, key) => { // transform to a new object
      if (keysToOmitIndex.indexOf(key) !== -1) { // if the key is in the index skip it
        return;
      }

      result[key] = isObject(value) ? omitFromObject(value) : value; // if the key is an object run it through the inner function - omitFromObject
    })
  }

  return omitFromObject(obj); // return the inner function result
}

Usage

You can use the function like this

const obj = {
  _id: 1,
  name: "Josh Ghent",
  title: "Software Engineer",
  metadata: { _id: 2, company: "Turbo Technologies" },
};

const result = deepOmit(obj, ["_id", "name"]);
// =>
// {
//   title: 'Software Engineer',
//   metadata: { company: 'Turbo Technologies' }
// }

Creating legacy code is ok

2022-06-08T00:00:00Z

"Legacy code". Words that will strike fear into the hearts of most developers. We hate to work on legacy code, it's complicated, untested and often larger than the observable universe.

Worse still, we shudder at the thought that any of our code would be considered "legacy". And so, as developers, we try hard to create "clean" code, choose the "right" framework, and implement solid principles.

But I'm going to argue that you shouldn't worry about creating legacy code.

Now, this isn't a post to blag on about how you should forget programming principles and write spaghetti code. Rather it's to say that you shouldn't worry about creating legacy systems[^1].

Why? Because most[^2] systems become legacy on a long enough time.

Think about it. How many systems have you built, or contributed to, professionally that is still running more than 5 years later? I'd wager the figure is below 3[^3].

And this is ok.

But, why does this happen?

Requirements change. Probably the biggest reason for system change is because requirements do. If a product is made a certain way and it's cheaper to rebuild it with new requirements, then it will be. Requirements are one of the few things that I have never seen done flexibly. So, there is a tendency to commit the "not invented here" bias.
People change. As teams change, so does the knowledge that is inside the team. In my experience, knowledge transfers between developers can take place twice before the link to the original ideas is broken. If a system is complex enough, and the team has long gone, then it might be easier to rebuild the application. Often teams justify this with the "better the devil you know" argument.
Technologies change. New frameworks, libraries and patterns come up all the time. What was once considered cutting edge soon becomes cumbersome. The length of time varies depending on the language. But, in Javascript-land, large changes happen yearly. As teams evolve, so will the preferences and ideas that they bring to it. So, it's natural to assume that the coding style will change as individuals change and the ecosystem matures.

Where does this leave us?

Don't sweat choosing the "right" thing. I remember sitting through lots of meetings where we would debate the merits of various frameworks. Although this felt like important work at the time, it was largely pointless. Developers often forget that technology will improve. We tend to focus on the inputs rather than the outputs. Because that's where we live - the code, the framework, the deployment system. Rather than the customer who lives with the outputs - the website, app or game. Ultimately, technology dies and gets replaced over time. And this is a good thing.
Write tests, but not too many. There are two polar situations I have seen with tests. Either they have tonnes that break when you indent a line, or there are absolutely none. Aim for enough to cover your main happy paths, the complicated parts that no one likes to change and customer critical functions. But don't sweat creating a large test suite, it won't reduce the likelihood that the system will be replaced.
Write documentation, but not too much. Documentation is difficult to keep up to date. So, aim to write documentation that answers logical questions and covers the most common use cases in a sensible order. This will assist a new team when they come along. This is a piece of work that may, principally, stand the test of time. Because it could be used by a new team to develop a compatible system in a new framework or language.

Overall, remember that technology moves fast. Be accepting of change. And don't try to make stuff that will outlive you. Solve the problem the customer has, and keep the system tidy whilst doing it. Write tests that save you time and documentation that saves your customers time. Legacy code is not something to be feared. It's to be embraced.

[^1]: By "system" I mean a module or discreet unit of code. For example, an email system, an authentication system, a customer API etc. [^2]: I say most because there are obviously exceptions to the rule. The Curl code base for example is 24 years old at the time of writing. [^3]: This is based on my own personal experience. Having had some contact with developers at businesses I previously worked at.

Questions for Developers to ask at interviews

2022-05-24T00:00:00Z

Interviews are for both the interviewer and interviewee. Largely though, they are designed to ask questions to the interviewee.

Recently, I've had a few interviews and was searching for questions to ask the company I was applying to. I realised that there are lots of lists of questions for interviewers but not the other way around.

The goal of your questions should be to clarify:

What's it like to work there?
Can I perform the role and grow?

With that in mind, my general approach is to prepare 2-4 questions to ask my interviewers. Usually a mix of a couple of general questions (that can be asked of any software business), and some specific ones (bespoke to that company).

Here are my general questions:

What is your software development life cycle like?
How big are teams?
What are the opportunities to grow in this role?
What metrics are being used to measure success in this role?

Besides general questions, here are three pieces of advice to craft bespoke questions. Doing so will mean you stand out and show you've done your homework about the business.

Look at their online presence. Reading the company's Twitter, LinkedIn and blog can allow you to glean what their culture is like. Notice the "voice" they use. What do they share? If there are photos of the office does that look like the place you want to work? A developer may have a blog where they share stories of software they've built or bugs overcome. Understand the process they used, and if the tech they used is of interest to you. I usually spend 20 minutes on this at most. You don't need to read everything in depth, just skim the details.
Research their latest projects. Using the research you gathered in step 1, you should be able to find their latest project. Build questions around this. If they used some new technologies, ask them why they chose their technologies. Perhaps it's in an unusual industry, ask about the unique challenges of that industry. Maybe it's a regular CRUD app or Shopify store, you can ask about how they approach adding specific business logic into those applications.
Learn about who started the company. It's essential when interviewing to establish who you're working for. You can ask questions about the history of the company and its future growth plans. This will paint a complete picture of how the business runs. You can also ask about leadership. The CEO won't have a huge impact on your day-to-day, but they will govern the overarching strategy of the company. So understand where their focus is - sales or technology.

Your questions should be things you want the answer to. Don't ask for asking sake. Remember you are trying to establish if the company is a good fit for you.

Interviewing can be a daunting task because you feel like you're under a microscope. But it's a two-way street. Judge them too! Use interesting questions to gather information and stand out as a candidate.

Bookshelf

2022-05-22T00:00:00Z

I believe reading and travel are the best ways to expose yourself to new ideas. Travelling can be expensive. So, largely I "travel" (through time and space) through the books I read.

Here is a semi-complete list of the books I own. I have lent many to friends and family and donated others. I prefer to have at least 80% of my books unread. I only keep a book I've read if I am going to re-read it or use it for reference later.

I've highlighted good books in blue, and great books in orange. These recommendations are contextual though. At one time, a certain book might be life changing, but at another time be meaningless.

This page will be updated as I get new books. Send me recommendations!

Four hour work week
Dealers of Lightning
Courage to be disliked
Skunk works
Soul of the new machine
Jobs
Stalingrad
Creativity Inc
Infinite Jest
Crying in H Mart
The Sun also rises
The old man and the sea
The Game
Kafka
Klara and the sun
Guernica
Thursday Murder Club
Thursday Murder Club 2
Prey
Girl A
Groupthink - A study in self-delusion
Surface Detail
The Great Indoors
Walkable City
Do androids dream of electric sheep
Meditations
The Island
The picture of dorian grey
Talking to strangers
Walden
Surely you're joking Mr Feynman
Cats Cradle
Zen and the art of motorcycle maintainence
All quiet on the western front
The handmaidens tale
The rum diary
Mans search for meaning
Stuff matters
Checklist manifesto
Bad blood
The man who mistook his wife for a hat
Perfume
Bridge over the river kwai
Principles
Spy the lie
Bad science
Why we sleep
Chekov Plays
Shoe dog
Business for punks
Flash boys
Zero to one
Trust me, I'm lying
Masters of doom
Animal farm
Undercover economist
Blitzed

There is always more

2022-04-26T00:00:00Z

Life can often seem like we don't have enough hours in the day. The media glorifies many who work 80+ hours a week. And there is an attitude that if you're not launching a startup and earning 5k MRR, then you're not doing it right.

I wrestle with the thought that I'm not doing "enough". I should be building another business, growing my client base, writing, or creating videos.

Like many, I have read countless articles and watched numerous videos on "productivity". With the goal of getting more tasks done.

But, I'm trying to talk myself around to the idea, that there will always be more to do.

No matter how much you prioritize, refine or cut, there will always be another feature to add, bug to squash or email to reply to. And that's ok.

That isn't to say prioritization and refinement are worthless. But it's a fallacy to think this will leave you feeling like your work can ever be "complete".

Instead, try to live slowly. Accept that there will be stuff that doesn't get done.

But in the moment, be happy about what you're doing, and live so that at the end of each day you're pleased with what you've done (not ticked off a list).

Advancing from a Mid-Level to Senior Developer

2022-04-12T00:00:00Z

"Senior" developer is a coveted title amongst software engineers. Many you work with will be promoted seemingly overnight, despite you thinking you're a better engineer. While you stay firmly as a mid-level developer.

This is the situation I found myself in. I was curious about what would make me a senior developer. I figured that the title was when I would have finally "made it". I could tackle interesting problems, make decisions and work on more areas of the product.

But I soon realised I had an incorrect view of what a senior developer was.

I thought a senior developer was someone who

Had 5+ years of experience.
Wrote great code (which I understood to mean never receiving a PR comment).

But, the reality is much different.

I'm going to discuss what it means to be a senior developer and what you should do to become one. Hint: It's nothing to do with your pull requests!

What it takes to become a senior developer

1. Don't chase the title

The best things come in life when you don't search for them.

In the same way, one of the best ways to become a senior developer is not to strive after the title. Because the title is meaningless without a solid foundation. I could call myself an architect. But do I know how to design a house? Not at all. In the same way, you have to have the knowledge and attitude of a senior developer to be called one.

But titles are not the be-all and end-all. Reflecting on it, I now consider titles meaningless. I've met plenty of "senior" developers who have that title by being at a job the longest.

Be driven by the knowledge you will gain at the end of the process and not some words on your company's "about" page. You don't need a title to be a good engineer. Stay humble and be a junior.

2. Develop T shaped understanding

"T" is a great shape. It's got the bottom stick and the broad top. We can liken this to our knowledge.

Let's first look at the bottom of the "T" - depth of understanding.

By now, you will likely have at least 1 programming language you use more than others. Develop further understanding of this language. How? Learn how the compiler for the language works, its flaws and what situations is it good in.

Developing a deep understanding (the bottom of the T) of a particular topic will enable you to debug the difficult problems and tell others about it. You can speak from a position of knowledge, rather than repeating blog posts you read verbatim.

There is a joke that touches on this topic:

A priest is giving a nun a ride home.

As they're in the car, each time the Priest goes to switch gears, he rests his hand on the nuns knee.

The nun looks up at the priest and says "Father, remember Luke 14:10."

The priest moves his hand away, embarrassed. The next time they stop at a light, he places his hand a little higher on her leg.

Once again, the nun says "Remember Luke 14:10, father."

The priest apologizes, "The flesh is weak" he says.

The priest drops the nun off, and when he gets home, he reaches for his bible and flips to Luke 14:10, which says.

"Friend, come up higher. Then shalt thou have glory."

The lesson? Know your subject.

The other part of the "T" is the top - breadth of knowledge.

It's all well and good being an expert in one topic. But, without a wide breadth of knowledge, you will struggle to apply that knowledge. You need to have a surface understanding of a bunch of topics.

For example, if you're a genius at NodeJS, then you can learn React, HTML and CSS. Continue to skill yourself on new topics, even if they are only slightly related to your field of expertise. You might choose to learn Algebra, Game theory or Design psychology. Learn what interests you and supports you.

3. Apply principles and previous experience

A key feature of a senior developer is that they've been around the block. They've seen the destructive effects of forgetting tests and have seen the successes of teams that foster psychological safety.

Using these experiences, their principles, and their "T" based knowledge, they can apply these to future scenarios. When a decision is being made, reflect on your past and use these to make comments and suggestions.

Additionally, when doing your day-to-day programming, apply programming principles consistently such as KISS, documenting code, and handling edge cases and performance issues proactively. To illustrate this, consider a new carpenter and an experienced one. Likely, they can both construct a chair. But the experienced carpenter will be able to apply his experience to discern that the chair needs extra lumbar support and so requires a particular technique of joining the wood. In the end, the experienced carpenters' chair will stand the test of time.

In the same way, consistently applying your principles will enable your code to be robust and stand the test of time.

Of course, there are many other ways to develop from a mid-level to a senior developer but these three stand out in particular. And, in my case, were the deciding factor. Enjoy the process of continuous learning and growth. Although cliché, it's about the journey, not the destination.

Quarterly planning for life

2022-03-29T00:00:00Z

Most People Overestimate What They Can Do in One Year - Bill Gates

By now, as we enter the fourth month of the year, many of us will have abandoned our New Years' resolutions or lost sight of our goals. I did this for many, many years.

I would start January, hopeful that this year was going to be different. I was confident that I could get a 6 pack, run a marathon and wake up at 5am. Quelle surprise, I only ended up with a 6 pack... of beer.

What happened to all that time?

Like most, I'd bitten off more than I could chew. And I found my motivation waned after a month or two. I would constantly reason that I had X many months left so I had plenty of time to get that 6 pack or run that marathon.

What helped me was breaking down the year in Quarters.

Commonly, businesses break down their financial year into 3-month blocks named "Q1", "Q2" etcetera. How their key metrics change between these quarters affects their share price.

But you can use this same system in your life.

Why use quarters?

Benefits can be seen quickly.
Helps to prune the scope of your goals.
Able to continuously re-evaluate your objectives as circumstances change.

How to do a quarterly plan

Using a notebook, or your project planning tool of choice, write down what your future self would be proud of accomplishing.
Next, reduce that down to a maximum of four goals. Ideally, these should centre around different areas of your life (financial, health, relationships, work, spiritual).
Now, for the key part, divide the outcome of that goal into 4 segments. For example, if your goal is to read 40 books. Then each segment would be to read 10 books.
Now spread those segments for each goal across the four quarters. The result will be that every 3 months, you are completing a quarter for each of your goals.

If after looking at the quarters it seems like that's too much to tackle, reduce the scope. It's as simple as that. A goal that you overachieve is much better than the one you underachieve. The latter will lead to disappointment, whilst the former will lead to joy.

What if I can't separate my goal down?

Sometimes this can be a challenge. It appears that the goal is just to "do the thing". But, in nearly all cases there is some kind of planning or preparation that goes into things. For example, if your goal is to do one month of no alcohol. A segment of that goal might be to stock up on alcoholic replacements. Or, it could be to make an effort to turn down social settings where you may drink alcohol for that month.

Even though we are partway through the year, we still have three quarters of the year left! Use it!

Should I split my monolith into microservices?

2022-03-22T00:00:00Z

Likely you have clicked on this article to find the answer to the aforementioned question. The quick answer is - it depends.

Rephrase your question

Before I explain why try to rephrase your question. What problem are you trying to solve by splitting up your monolith?

Answering this will help you to clarify if the approach you take is going to solve those problems.

If you simply want to split it up because you feel microservices are "better". Then you should think again. Because it will create more problems than it solves.

But, if you have a burgeoning team who are spending hours resolving merge conflicts, then microservices are something to consider.

Write down your problems

Using the list of "problems" your team has, write down next to each of them how microservices will solve or alleviate that issue.

But do the same with other solutions. Consider refactoring parts of the system, removing them, or restructuring the folders.

This can sometimes be a deceptive process, however. Often, people consider microservices because of both/either of these factors:

The monolith is buggy and/or slow
Developing new features is a nightmare

Unfortunately, on a long enough timeline, you are going to face those problems with microservices. Except for this time, instead of having one buggy and slow place to fix, you have fifty. It's worth doing your research and thinking about whether these things will actually solve your problems or just mask over them.

Consider the downsides

Microservice advocates often gloss over the downsides of microservice architectures. But they are important to consider.

As a guiding principle, if you don't have a dedicated DevOps team or your engineering headcount is below 25 then I'd strongly recommend keeping things as simple as humanly possible. Remember that value comes in the form of features and fixes, not in restructuring the application.

Everything has a "cost", both upfront and ongoing (in technology as well as life). Make sure you know what these two figures are. For example, it might require a 4-week project upfront by 3 engineers and 1 day of engineering time per week ongoing. Weigh this up against the alternatives to tell you if it's truly valuable.

Some other downsides include

The learning curve for the new project structure
Bug fixes are harder to track
Monitoring is more challenging
End-to-end testing is difficult
Lots of surface area to secure, configure infrastructure and release pipelines for
The learning curve for infrastructure deployment
New problems such as latency and load balancing

In light of these downsides, maybe microservices are less appealing. What's the alternative?

The alternatives

Depending on the problems you're trying to solve, I'd suggest some alternatives. You can use many of these approaches.

It's worth noting that just because moving to microservices isn't the best right now, it might be in the future. These suggestions will help to make that migration easier if/when you make it.

Write tests. Focus primarily on integration. Attempt to cover your entire application.
Remove the cruft. Over time, components of an app are retired but the code lives on. Remove these parts to pare your codebase down to only what is used. If they can't be removed, then make sure it's well documented.
Configure monitoring. Make the components of the system extensively monitored. For example, the email notification system, user creation and back-office reports should all notify you if they go wrong. This should be separate from the code itself using a monitoring solution like Sentry or Datadog.
Upgrade your tools. Make development fast by investing time in your tools. If your app takes 15 seconds to recompile, make it so it compiles in less than a second. The payoff for this time is ten-fold and will make your team rejoice.

Sometimes microservices are the right approach. Sometimes they aren't. Mindfully consider the situation and make the migration easy.

I've talked only about the decision of "monolith to microservices". But, lots of this advice applies to any technical decision your team is making. It depends.

How to make changes as a Junior Developer

2022-03-15T00:00:00Z

When you start your software development career, you come into a new job with excitement filling your eyes. Think of the cutting edge technologies you will use, collaborating with other amazing engineers and building a fantastic product. Sadly, it doesn't always go like this. It rarely does. And that's ok! Ultimately products have budgets and deadlines.

Perhaps many of the expectations of a job that you had, have been unfulfilled. Testing is a secondary concern. And principles have been abandoned in favour of "throwing something together because it needs to be done by Friday".

At this stage, it is easy to become disenfranchised. After all, you do not have the experience or position as a manager to make widespread changes. But you feel like you're stomping over the software engineer's credo every time you ship a pull request. What can you do in this position?

First of all, especially if you're new and/or it's your first job, don't make testing, security updates, plain text passwords or anything else a hill you're willing to die on. I've been there, done that, and got the t-shirt (many times). And I can tell you it never works.

The decisions have already been made, often by your manager's manager - who is equally saddened by the product's form. It takes time to build credibility to make a change in a company.

And sometimes, you might be wrong. A decision may have been made that it is the wrong one by the "golden standard", but the correct one for that company.

Take time to digest a company, ask questions and be receptive to the answer. If you find yourself saying: "so why do you just...", stop.

So, what can you do? Make a personal change. After all, you're at the start of your career, there is still plenty to learn.

1. Write tests for yourself

Even if you're not "supposed" to write tests due to time constraints, do it anyway, but don't take ages with it. Block out 10% of the total time it took you to create a feature to write a test or two. It doesn't need to be a suite with 100% coverage, just to cover your little corner of work.

Use Puppeteer to create your own End-to-end (E2E) tests in a separate folder to the codebase. And use the defacto testing framework of choice (Pytest for Python, Jest for Javascript etc.) for writing integration tests. Again, put these in a folder that can be git ignore'd so it's not in source control for the rest of the team.

2. Review your code

Reviewing your code is a vital step to correct your work. Like proof reading an essay.

Before you create a pull request, look through the green part (the code you've added) and ask the following questions

Can I explain what this code does to another (imaginary) person with little knowledge of this system?
Can I explain why I've implemented this solution?
Is it clear to others reading the code base the answer to the above questions? (i.e., is it in code comments)

Following this process encourages you to keep pull requests small and helps you to justify your technical decisions.

3. Do a personal post-mortem

After an incident occurs, many companies do a "post-mortem". A review into the problem, its root cause and ways to prevent it in the future.

When you make a mistake like taking down the app, shipping some broken code, or deleting the production database (check, check and check), write your post-mortem. There are many templates online that can help you formalize this document. Review these once a quarter and reinforce the learnings from each incident.

It can be frustrating if you want to make decisions but can't. Everyone goes through this process. Don't dwell on it. Take the time to learn, understand and ask good questions. Be an agent for change in your work and then use your expertise to help others. Doing so will build a solid foundation of knowledge and connections throughout your career.

Be friendly and don't ignore Recruiters

2022-03-10T00:00:00Z

Increasingly, I've noticed an increased level of resistance to recruiters among engineers. There has always been a love/hate relationship between the two parties. From the beginning of my career, I always heard "recruiters are bad". And, to be honest, I accepted that as truth for the longest time. But, reflecting on it now, I don't understand it at all.

I'm writing this article for fellow software engineers to not accept the rhetoric of what others say - see for yourself. In a nutshell, you and I are not "better" because we are engineers. Recruiters are another role that is necessary for the sector to function.

Here's my advice for software engineers dealing with recruiters.

1. Use canned responses

One of the main aversion to recruiters is sending countless emails to irrelevant jobs. This is understandable annoying. Especially when you are specific about not looking for a job, or the types of job you want. But consider for a second why this happens. Recruiters are paid for successful placements. Because there is a shortage of talented engineers, it figures that most engineers are already employed. If recruiters took the fact that someone was already employed as a reason to not approach them, it would be impossible to hire. For myself, I've always been employed when approached about another job.

So, we can't avoid the inMail and emails. But, what's the solution? Canned Response

Canned responses save you time by being clear about your expectations for a job and what kind of work you might be open to. You can find many templates for this online, but they are mostly full of snark.

Here is one I use:

Hi X,

Thanks for your message! This looks like a fantastic opportunity.

Unfortunately, I have recently accepted a position at a new company and am not currently seeking new opportunities.

I’ll be sure to bear you in mind for future roles.

Kind regards,
Josh
https://joshghent.com

Short, friendly, clear. Simple as that.

Or, if you are open to work. Here's another template I use for that:

Hey X,

Thanks for your message! This looks like a fantastic opportunity.

Although this job doesn't fit me, I am currently looking for a Remote Senior Developer position working with NodeJS, React, and AWS. I have worked with these technologies for over 5 years across an array of projects. Most recently, I have been architecting and building a greenfield project for a large eCommerce company.

I have attached my CV in case you have anything that fits the bill.

Kind regards,
Josh
https://joshghent.com

Again, it's short and sweet but gets across the message.

2. Recruiters are great at building communities

Due to being involved with a large network of developers, recruiters are incredible at creating communities. Many events that I have run, have been so well supported in large part due to the work of recruiters leveraging their networks. Meeting these people at events can then further your own career.

Additionally, they have more access to commercial ends of businesses. If you have a technical event you're running, recruitment agencies are usually among the first to sponsor the event. They get the exposure and you get the finances to run a kick-ass event. Win, win. Leverage these resources that they have access to.

3. Don't let a bad egg put you off the whole batch

Now, I know what you're thinking. You've read this article so far and said to yourself "that's all well and good. But this recruiter was truly awful".

I agree.

There are bad recruiters out there. Terrible ones. But there are lots of bad engineers too. There are bad healthcare workers, builders, architects, designers, painters. With anything and everything, there is a "bad" version of. And that's ok.

At worst, a "bad" recruiter might spam you with some emails or calls. You can easily block these. A bad engineer might give you a haunting nightmare of yarn that you have to untangle over the next year. The effects of these are vastly different in size.

But just as when we recognise a bad engineer, we don't assume all engineers are bad. We should think the same about recruiters. Don't let one bad egg spoil the whole batch. There are good eggs out there. This brings me nicely onto my next point...

4. Work with individuals, not companies

A recruitment agency, like any other company, is a faceless emotionless entity. Inside each company, there will be some great people and some not so great people. Find the individuals in those businesses that you get on with and place you into jobs you enjoy. Then work with them throughout your career. Personally, I've held 5 jobs given to me by 2 recruiters that I work with and have built up trust over time.

You can build trust with them by running events with them and referring people in your own network to them.

5. Accept them as part of the process

Many believe we should live without recruitment agencies entirely. This is an understandable viewpoint. But this belief underscores a fundamental misunderstanding about business - stuff costs money. And if that "stuff" is hiring people, then it costs a lot. Why? Reviewing resumé's/CV's, interviewing and technical skill tests all takes time. Time from someone who is paid by the company. A recruitment agency's main value offering lies in getting high-quality candidates from a large network and handling all the marketing associated with advertising for a job.

Just as many developers "outsource" their code by using third party libraries to save time, businesses do the same with recruiters. It saves time, and there is no point in reinventing the wheel. It allows them to unlock access to resources that would have taken a considerable amount of time to develop otherwise.

For better or worse, recruiters are part of the process of getting a job and are here to stay.

Hopefully, this post has helped soften your attitude toward recruiters. I'm not trying to win favours with recruiters by writing this. It's a response to several snarky posts about the recruitment industry and tech. At the end of the day, these are people trying to do their jobs - like you and I. Sure there are some bad apples, but where isn't there? Default to truth and follow the advice outlined above, it will work out in your best interests to do so.

Mistakes I made as a self-taught developer

2022-02-15T00:00:00Z

Learning to become a software developer is not a trivial task. There is a plethora of guides, tutorials and courses to take. And of course, there is the question of self-teaching or going to university.

But no matter, which path you chose you will always make mistakes.

I made a tonne of mistakes. And I want to share them with people who are learning software development. Here are 5 mistakes I made whilst learning to code.

1. Analysis paralysis of resources

I wasted a lot of time analysing the "best" place to learn to code. Early on, I often found myself reading articles about how "good" Codecademy was vs. a collection of "Head-first" books. This time would have been better used on doing the work. It's easy to enter this state of panic thinking you're committed to a certain learning path. But the truth is, you aren't. Mix and match, try different things and go with what works for you.

I also did this same analysis when choosing a language to learn.

Is Python the best? What about Javascript? My favourite sites use Rails, maybe I should learn that?

That was the list of questions that plagued me. It was exhausting and I shouldn't have spent so much time overthinking this. My advice for new programmers is to learn Javascript and web technologies. Yes, you might want to learn games, but Javascript is so widely used (now even in space!) that it will allow you to go down any path you want later on.

Lesson: Just learn Javascript. Mix and match resources that work for you to build a consistent practice of learning.

2. Not building early and often

What do I mean by "building"? I mean creating libraries, API's, demo sites and more. But to begin with, I thought the idea of "building" something was too complex to handle. Translating the code into something practical would have made me familiar solving problems. When I did start building projects, I found it challenging to shift from syntax to finished products. To liken it to real languages, it's the difference between knowing the word for "Apple" and knowing how to say "Would you like an Apple?".

This fixation on learning the syntax also had the downside that I didn't have anything to show for it at the end. To a potential employer, I was just someone who said they had learnt to code and could do some simple problems. By having a portfolio of projects that I could show off, I would have proven to myself and others that I had the skills to work.

Lesson: Apply syntax to real-world style projects after learning it.

3. Being tied to tutorials rather than problems

In line with the above, I spent too much time on specific tutorials. I should have learnt to break a project down into small problems and seek solutions. This practice of breaking larger projects into small problems would have been valuable when I started working. It would have also helped me train my "google-fu" to search out error codes, and problems I needed to solve.

Lesson: Learn to break projects down into small solvable problems.

4. Sweating "interview" preparation

Before I began to interview I spent hours doing code "katas" - small challenges using no libraries in your language of choice. I did this because they are supposedly common interview questions. But, I have never been asked to do specific coding challenges like this on a whiteboard or otherwise (having interviewed for 7 jobs in total). What I have had is take-home projects, and asked general technical questions.

Lesson: If you are interviewing for the FAANG companies you are likely to get these questions. If you are not, then skip this. Focus on projects instead.

5. No SQL Exposure

In self-taught land, starting a new project is quite simple. Install NodeJS, install React, launch Chrome - job done. But, installing and using SQL? That was far too scary for me to tackle. There were ports to configure, connecting it to NodeJS and then the table structure. In part, MongoDB gained popularity because it's so simple to set up. By not having the exposure to SQL, I struggled when I got a job.

It also meant that my coding style tended to lean on parsing data within the language. Over time, I've trained myself (for software performance reasons) to use the database to crunch and mould the data for me.

Lesson: My advice on this point is to set up a free account with Render.com or Heroku and add a MySQL or PostgreSQL instance.

If you're learning to become a software engineer, don't give up! It is difficult no matter which path you chose. I hope by listing my failures that you can avoid them. You will make others in your journey and I implore you to write about them and learn from them.

Building Collaboration with Remote Teams

2022-02-08T00:00:00Z

TL;DR: Provide the tools, empower people to use them and embrace remote work for what it is - remote.

Steve Jobs designed Apple Headquarters to maximise the length of time it would take people to get to the bathroom. He did this to increase collaboration stemming from running into others in the corridor.

Since the pandemic, remote working skyrocketed. Owl Labs recorded in 2021 that almost 70% of full-time workers in the US were working from home. And based on the stats from job boards such as RemoteOk.com, you can see a massive uptick in remote job opportunities that are not dying down.

But the question is, how can you "bump into" people in the corridor in the remote-first working world? Many say that weekly meetings to keep people aligned are the answer. But I disagree, meetings are viewed with disdain. The same study by Owl Labs found that 80% of remote workers wanted at least a day per week without any meetings at all.

Instead, I'd suggest a different approach.

Make communication public by default. The main rebuff of remote work is that the communication is too "formalized", recorded and preserved. People use that excuse to communicate only in direct messages. But, putting messages in public channels, allows everyone to be informed and to take part. Additionally, making individuals comfortable with public communication will open the doorway for asynchronous work. It will discourage people from calling meetings simply to gather a consensus or input.
Don't recreate the watercooler. Commonly, I've seen teams create a #watercooler chat in Slack, an informal space to post memes and have a chit-chat. Although in ultra-large businesses I've seen these spaces improve individual relationships, I have not seen them successfully increase collaboration. These spaces are attempting to recreate an in-person space. Whilst well-intentioned, these spaces do not translate to the online realm. Embrace remote work for what it is, remote. It will naturally take time for an in-person organisation to change into the "remote" frame of mind.
Avoid hybrid working. Where possible, avoid having some of the team in an office and the rest remote. I've been on both sides of the office divider for this one. And it doesn't do well on either. On the office side, you can often be blocked by a remote worker not working the same hours and not having the tools to perform a task asynchronously. But on the remote side, you have a constant FOMO for all the undocumented decisions that have been made. Pushing all this communication asynchronous and online alleviates these issues and keeps everyone up to date.
Use the tools. We are all using Zoom and Slack. But that's not the end. There is a myriad of tools to help you unite a remote team. Use Google Drive for documents and presentations (share them without a meeting!), Tuple for pair programming and GitHub for tickets and code. Directing people toward these channels (asynchronous) and away from meetings (synchronous) will accelerate collaboration for your team by accomplishing meaningful objectives.
Build a documentation culture. In an office, decisions can be made and passed around in small conversations. In remote teams, that vanishes. So, make sure that every decision is documented in a consistent and precise way. Encourage people to write about problems they've solved and how they solved them. Help people to write up accurate guides on getting up and running with a system. And document various approaches to a particular ticket. Make sure you lead by example here. Don't rely on Slack to store a decision. If you ever hear the phrase "I forget exactly what was said", it is a chance to write it down. Further, address this at its source by making sure to hire individuals with good writing skills.

Not all these techniques will work with your team. I encourage you to experiment and see what works.

Overall, embrace the work situation you are in and capitalize on its advantages. If your business can be in-person, capitalize on the fact that it will likely be more collaborative. If you have a remote business, embrace a diverse global workforce, lower costs and asynchronous work for increased productivity.

Facing the Legacy Code Monster

2022-01-25T00:00:00Z

I start new jobs like a spelunking caver, exploring all the systems, code and pipelines. I time myself to see how long I can ask questions about a system before I get a dreaded response:

"Oh, that's a critical system that handles our core business. A developer wrote it years ago, who has since left. Now we reboot it when it breaks. You don't want to mess with that."

I'm sure we've all heard words like that.

It seems almost a law at this point that every mature software company has at least one system that "works". But, has a few horrendous bugs meaning it needs rebooting and there are workarounds to its inflexibility that have built over time.

It's software that has no tests. Runs on an extremely specifically configured server (likely in a closet in the office). Where documentation has been passed down in folklore. And the person who originally wrote it now resides on the moon where they are unreachable.

Oftentimes, this software is left to rot because it's so critical that it's safer to have the workarounds and reboots than to fix it for good.

But, you shouldn't be deterred by this rhetoric of what current employees tell you. Why not?

No one deliberately writes broken or bad software. I say no one, there are of course exceptions. But buy-and-large, software is written with good intentions in mind. And as it is running in production, it means it solved the original problem. Adopting this ethos will help dissolve the image of this evil unknown developer spiting you.
Current employees might be misinformed. Depending on how long ago the software was implemented, current employees might have been misinformed. Swaths of developers may have come and gone and simply repeated what they were told on their first day. What started as a healthy respect for a critical system, soon becomes the subject of fear and anxiety.

Fighting these tendencies to buy into the anti-legacy cult will mean you can deal with legacy code. It will be dirty work, but gaining a deep level understanding of these systems will make you a hero among your team and an invaluable employee.

How can you gain insight into these systems?

Understand why it became legacy in the first place. Finding the why on these systems is vital because it can help clarify your own journey's priorities. In some cases, it might be that the system was impossible to run in a sandbox. If so, that should be your first port of call. Focus on first principles and get to the core of the problem you're trying to solve.
Get it running in a sandboxed environment. Provide safety for yourself and others, by setting this up to run in a sandbox. This might need the buy-in from someone on the DevOps team. This is a critical dependent step to working with a legacy system as without it you do not have the psychological safety to make changes. This process might take a long time and should be considered just as important as writing tests.
Follow the code, and document it. Target your reading of the codebase by following the various paths it takes. Review the entry points and build a map of the various "journeys". Then, pretending you are one of those calls, follow the path that the code takes. It's sometimes helpful to draw these paths on paper for further reference. After reading it through once, read it again, but document the functions as you go. In some places, you might not know "why" something has been written a certain way; mark these areas with a quick TODO comment for later review.
Write integration tests. Not unit, functional or anything else. Focus on integration at the moment. This maximises the test coverage whilst minimizing the level of understanding you need. Early on in this investigation, you likely don't know all the in's, out's and gotchas of the system. So, to avoid getting overwhelmed, it's best to write a suite of integration tests that mirror the "journeys" you discovered in the previous step.
Share the knowledge. Don't let yourself become another bus factor. Be quick to share the knowledge, even if there are gaps. This also encourages other developers to get involved, helping write tests and documentation.

This process is by no means perfect. But, be sure not to skip any of the steps, as they are all as critical as each other. It will likely not mean you are a complete expert in this system. Nor will it mean you can rewrite it in a more modern tech stack with the confidence of a solid test suite behind you. But, it does mean you have shed some light on an otherwise dark scary legacy code monster.

How to Ship Software Faster

2022-01-18T00:00:00Z

Remember when software came on a physical medium like discs, USB sticks or punch cards? Me either. Software release lifecycles used to be lengthy - years-long in most cases.

As software flourished on the web, we grew accustomed to "moving fast and breaking things". This approach has a lot of drawbacks. Not least because some customer bases are more sensitive to problems than others.

They still wanted to "move fast", but not "break things". The speed of the web, with the safety of physical releases.

The solution that many teams came up with was to mass up lots of work and then release it every so often. Taking the bad of both release systems. The lack of safety from a "move fast" release and the slow speed of physical releases.

It's likely that you work in a place like this, or have worked there in the past. Organisations where DevOps are a secondary concern to the application itself. Places where "continuous delivery" is considered voodoo reserved for the FAANG's.

I have found myself at these organisations all my working life. I quickly noticed the following patterns:

Developers have little to no confidence that a new release will not break something.
That low confidence means there is anxiety when it comes time to release.
The time between releases meant upstream work causes conflicts.
Manual testing cycles had to be done to establish any confidence.
Bugs upon release caused finger pointing; with psychological safety diminishing as a result.

What can you do if you notice these patterns?

The solution is to ship faster. How?

Set expectations of delivery time. Start by opening a discussion, with stakeholders, about what the expected time to ship new versions will be. Establishing these rough boundaries govern the setup of processes used to ship software. Generally speaking, shareholders will want features as soon as possible. But, if you are currently releasing once a month, you should aim to start releasing bi-weekly. Get a bit further along before promising to deploy 30-50 times a day like Instagram.

Make systems observable. Low confidence in releases often originates from systems with little observability. This means that if something does go wrong it's a nightmare to figure out why. Before starting to increase deployment frequency, you need a system you trust. Focus on the fundamentals - searchable logging, automatic monitoring of key website pages and API endpoints (using UptimeRobot) and automatic tests (integration and unit at least).

Set small concise deliverables. Doing manual releases requires an immense amount of cognitive overhead. Having a small number of tickets and clear deliverables in each release reduces this cognitive load. There is less to remember to test and check. And other areas of the system are less likely to be affected. If releases are simple to do, it's more likely they'll get done.

Invest in your DevOps. This is the crucial technical step. There are many other articles about having top quality development tools to aid deployment, so I won't add to them. But principally, look at the areas that take the most time or have the least confidence, and automate them. For example, a bash script written 2 years ago for bundling the app is unreliable, take time to address this.

Use Feature flags. Often releases get delayed because stakeholders don't want to reveal new features to customers. Using feature flags allows you to ship unfinished features without breaking things for everyone. A further selling point for stakeholders is that feedback can be gathered from select customers before a full rollout is done.

Make mistakes a non-issue. If the risk of a new release causing a bug is on par with starting a nuclear war, people will shy away from it. By making it easy for developers (or better yet, a blue-green system) to rollback to the last known stable release, it will break down the fear around releasing.

Use checklists. Anything that cannot be automated (or you don't have time to do so), should be made as programmatic as possible. Using checklists reduces the guesswork out of manual tasks. It means reliable releases are simple to do.

Shipping software faster is a mix of both cultural and technical aspects of an organisation. Both are equally as difficult. Work towards the "release nirvana" that awaits once these systems are set up. Your team will be rewarded with lower blood pressure and your business will be rewarded by getting and retaining more customers.

Cache Auth0 M2M Tokens

2022-01-11T00:00:00Z

Auth0 is an easy to integrate service that handles all your applications authentication needs. But, if you've worked with it before, you'll know it's downfalls.

One of them Machine-to-Machine (M2M) tokens; used to authenticate between your services.

But the limits are restrictive for serverless infrastructures. In the free plan you only get 1000 per month. And even on a paid plan, it would be expensive to get the number of tokens you might need in a given month.

The solution is to cache Machine-to-Machine tokens so we don't need to request new ones until they expire.

In traditional infrastructure, this would be trivial. Save the token globally somewhere and done.

Serverless architectures are a tricky because there is no persistence between instances.

Here's how to handle caching Auth0 Tokens for AWS Lambda Microservices. But, the same principles apply for other cloud providers.

Create the DynamoDB Table

(or equivalent serverless DB table in other cloud providers)

Set your own name for the table, and set the partition key to token as a String

Add the name of the table as an environment variable CACHE_TOKEN_DB

Retrieve and store tokens

First let's add a method to store new M2M

// ===
// cacheToken.ts
// ===
import AWS from "aws-sdk";

const storeNewToken = async (token: string) => {
  const docClient = new AWS.DynamoDB.DocumentClient();
  const response = await docClient
    .put({ TableName: `${process.env.TOKEN_CACHE_DB}`, Item: { token } })
    .promise();
  return response;
};

The code is simple enough and fairly self explanatory.

So, let's move on and add method that we can use in our Lambda Handler to retrieve a new M2M token.

There are two paths for this method

There is a existing unexpired token in DynamoDB, so we use that.
There is no token or only expired ones, so we generate a new one, store it in DynamoDB and use that.

We will design this system to only store one token at a time. This means we do not have to worry about old tokens and filtering them out on each initialization.

So let's write our method!

// ===
// cacheToken.ts
// ===
import request from "request-promise";

export const getAuthToken = async (): Promise<string> => {
  const token = await getExistingToken();
  if (token !== "" && hasTokenExpired(token) === false) {
    return token;
  }

  const params = {
    method: "POST",
    url: `https://${process.env.AUTH0_NAME}.auth0.com/oauth/token`,
    headers: { "content-type": "application/json" },
    body: `{"client_id":"${process.env.AUTH0_CLIENT_ID}","client_secret":"${process.env.AUTH0_CLIENT_SECRET}","audience":"${process.env.AUTH0_AUDIENCE}","grant_type":"client_credentials"}`,
  };

  const result = JSON.parse(await request(params));
  if (!result["access_token"]) {
    throw new Error("No Access Token returned");
  }

  await deletePreviousTokens(token);
  await storeNewToken(result["access_token"]);

  return result["access_token"];
};

Let's break this down a little

We first get the existing token in DynamoDB. It returns the token or an empty string.
If it returns a token, we check it's not expired and then return that token.
If it is expired, or there is no token, we go ahead an generate one from Auth0.
We then delete the old token in DynamoDB, and store the new one.

Potentially, with this flow (and the fact that DynamoDB is non-locking), could mean that multiple instances of your service save a token at the same time. But, this will be minor compared to how much you're able to save by caching in the first place.

Let's now create the methods we referenced in the getAuthToken function that help us interact with the tokens storage and validation

// ===
// cacheToken.ts
// ===
import jwt_decode from "jwt-decode";

const deletePreviousTokens = async (token: string) => {
  const docClient = new AWS.DynamoDB.DocumentClient();
  const tokenRecords = await getAllTokens();

  // Clear down the table
  if (tokenRecords.Items) {
    tokenRecords.Items.forEach(async (row) => {
      const token = row.token;
      await docClient
        .delete({
          TableName: `${process.env.TOKEN_CACHE_DB}`,
          Key: { token: token },
        })
        .promise();
    });
  }
};

const hasTokenExpired = (token: string) => {
  const decoded = jwt_decode(token) as { exp: number; iat: number };
  if (decoded) {
    return decoded.exp < new Date().getTime() / 1000;
  }

  return false;
};

const getAllTokens = async () => {
  const docClient = new AWS.DynamoDB.DocumentClient();
  const response = await docClient
    .scan({
      TableName: `${process.env.TOKEN_CACHE_DB}`,
    })
    .promise();

  return response;
};

const getExistingToken = async () => {
  const response = await getAllTokens();

  if (response.Items && response.Items.length > 0) {
    return response.Items[0]["token"];
  }

  return "";
};

Again, let's break this down

In deletePreviousTokens we grab all existing tokens and delete them one by one. This is to avoid concurrency issues where another instance has written a new token which we do not want to delete.
In hasTokenExpired we do a basic JWT validation to make sure that it is not expired. This could be improved by not using the token if it's only got 1ms left but has worked so far for me.
In getExistingToken we get all rows in the table and return the first token or an empty string if none is found.

Usage in the handler

Now all that's left to do is to add it to your Lambda functions handler method.

export const handler = async (event: any, context: any) => {
  const token = await getAuthToken();

  // Do something with the token
  await sendResultsToService(token, event.Results);
};

Hopefully you found this interesting and saved some money on your Auth0 Bill!

How You Work

2022-01-07T00:00:00Z

Learning how you work best is a superpower. Imagine, creating and seeking environments where you succeed best. Likely you remember times where you got into a flow state and produced magic, but you can't pinpoint why.

I was in this position and so took some time to figure out how I work best.

Product user manuals have a section dedicated to "ideal working conditions" - to get the best out of the machine. This includes the maintenance, the temperature and location and how it should be operated. In the same way, you can develop a "personal user manual" documenting how you work best and where.

In my user manual, it's broken down into five sections.

The kind of work I do best
The environment for doing that work
How I enjoy doing that work
How I receive feedback
How I get motivated

For example, my user manual is included below.

The kind of work I do best

Creating things that I know will benefit people.
Automation and saving time.
Turning rough specs into tangible products.
Thinking of edge cases and being able to dream of scenarios where bugs will occur at scale.
Building clear API's that are secure and scalable.
Writing technical documentation.
Architecting and building systems that can handle millions of customers.

The environment for doing that work

I'm informed of the bigger picture and the impact of my work.
I like a small to-do list.
Requirements stay reasonably consistent. But I have the autonomy to figure out how to meet those requirements.
The benefit of my work is somewhat measurable.
There is a culture of gathering and reviewing data to make decisions.

How I enjoy doing that work

I prefer to work asynchronously and reserve synchronous work for solving specific problems.
In line with the above, I like to keep meetings to an absolute minimum. Including many of the sprint reviews, standups and retros that are commonplace in most software businesses.
I like to have the flexibility to work inconsistent hours. Often, I find I solve problems better away from the computer rather than bashing my head against the wall.
I believe in transparency and equality, I prefer to work with organisations that foster that same ethos.

How I receive feedback

I love to improve my work and constantly question personally on how to do this. But, I need to understand the reason why something is better or is important.

The Benefit

By preparing a user manual, you can quickly establish a rapport with your co-workers and effectively communicate your "ideal working conditions" to your manager. Having led teams, I ended up putting together versions of a "user manual" for everyone in my team in my head. A written document from that person would have proved vital.

The advantages are that both yourself and the people around you can understand each other and make everyone happy by aiming to keep work within those ideal parameters. Of course, this is not always possible. Tough, stupid things sometimes need doing. But a keen-eyed manager will always be aiming to balance these tasks with work you love.

I've found my manual allows me to reaffirm my ideal work. Whenever I am thinking "I hate this work", I review this manual and figure out why I dislike it.

Maybe don't hire

2022-01-06T00:00:00Z

Increasingly, I've become sceptical of businesses looking to "rapidly scale" their technical teams. All businesses seem to get a certain amount in their backlog, or an urgent request from a large would-be customer, or just too much money and decided that the only solution is to hire at a tremendous scale.

I'm going to argue that perhaps you don't need to hire at all. And doing so would be counter productive.

Why startups need to hire

Hiring is driven by demand for product features. Shareholders within the business get requests for certain things. "Can we add this quickly?", "I need this urgently for X", "Our competitor has Y so we need Y". These are all common phrases that I'm sure we've all heard. These requests are understandable. These shareholders are trying to do their job by making the company (and by-proxy the product) more profitable. I don't blame them for these requests. But, they stem from an environment of their own making.

These businesses start, like many, with few resources. They build an MVP, sell it to a few customers and start to make money.

But they have a problem - they want to grow, and don't have the money to hire a full time team. So, they get funding. That funding puts them into 6th gear.

Because they are now at the mercy of investors, they need to get more customers, which means more features, which means more developers.

And more developers means they need more money. So, they raise more money and agree to add more features to get customers. And on and on.

Unfortunately, this cycle never stops.

So what's so bad about this?

The cycle I've described above is what some would call "growing a business". Whilst that might be largely true, it's not the only solution. One problem with this approach is hiring.

Hiring is viewed as a task that can simply be checked off. Recruitment agencies can help you feel like this. But the fact is that hiring is extremely difficult and time consuming. From initial resume review, to interview, to code challenge, whatever your recruitment process, it takes time on your part.

And after the contracts are signed, the process isn't over. You've got to onboard, educate and give a lead time before those developers get productive (in my experience, even the most senior developers aren't truly productive until at least 2 months after starting).

The silent killer

The silent killer of many of these business is they start creating work for the sake of it.

When a carpenter creates a chair, they skilfully sculpt the wood, attach the parts together and sand, oil and paint it before declaring it finally finished. Software is similar, but we miss the finish. How many pieces of software have you worked on where it was declared "finished"?

In hiring a huge army of developers, eventually these features from customer peter out. What then fills the void is an endless game of optimizations, minor pivots of existing features and "reckons" about things people think are good ideas (without validating them). Evernote and Dropbox are classic examples of this at play. They created a great piece of software, but continue to annoy its customers and create meaningless changes that get killed. Without the pressure of continuous growth, it would be more acceptable to put software into "maintenance mode".

The Alternative(s)

Increasingly, "indie makers" are building out products in their spare time and waiting for them to become profitable before making the leap full time (or hiring staff). This means there is not the huge impetus to continue growing at all costs.

Others, in the wake of the pandemic, are building out remote, part-time teams. Gumroad has done this to great success after suffering from many of the failings described above.

In another case, I had a non-technical founder approach me to build out an MVP for him. He was a reasonable chap, had a savings pot to pay me, a specific scope and a clear market. But, rather than investing all that money, I advised him to build out a prototype using Nocode tools like Bubble and Webflow. Even if you are not technical, the barriers to entry for building products is lower than ever.

These approaches are not foolproof, they have their own issues. For example, for many products, it isn't possible to "build it in your spare time". Still others may not have spare time to build a product in. But, don't be afraid to break the mould and do things the non-traditional way. Think of creative solutions to the problems you have. Resist the rush to build lots of new features, seek to please customers at all costs, and compromise your product. Your product and customers will thank you.

In summary:

Relax your requirements about the pace of work. It will lead to better quality product as a result.
Don't get sucked into the investment strategy grind. You'll eventually have an army of developers with nothing to do.
Seek alternative solutions. The barriers to create are lower than ever, use Nocode tools or build the project on the side.

Software Beauty

2022-01-05T00:00:00Z

Design is a funny word. Some people think design means how it looks. But of course, if you dig deeper, it's really how it works.

Those famous words spoken by Steve Jobs, was the cornerstone of Apples great success in building beautiful products. But these words can be applied, not just to hardware, but to software too. Code at the end of the day is simply words - albeit in a wonky form. Asimov can write beautiful works in English. Why can’t we do the same in code? The answer is, we can. But why should we as developers be concerned with the “beauty” of our code? In this post, I’m going to answer this question. But also drill down into specifics on how (often posts like this can be a bit too romantic and theoretical).

Why Beauty is Important

So, why is software beauty important? After all, we are engineers, not artists! By way of example, let’s say you buy a new car. It’s the fastest one on the market. When you get it, you’re all excited - until you open the drivers door. Before your eyes you see the pedals on the dashboard, the steering wheel on the roof and a literal bucket for a seat. Safe to say, you wouldn’t be impressed. What’s the connection to software? Although something is usable, it may not be nice for someone to use.

If you work on a system with an API of any kind, whether internal-only or integrated with third-parties, beauty should be a concern. Bad design breeds bugs, extra cognitive load, increased time to develop new features and more.

Gliding past the obvious examples of having a nice REST API like Stripe (which we will cover later). Let’s say you have a large React frontend and express backend. If your code bases are not “beautiful”, on boarding new developers becomes an increasing hassle. You may try to fight this by having lots of documentation. But there comes a point in software, as with jokes, where if you have to explain it, it’s bad. Further, when someone comes to add something to the code base, because it wasn’t designed in an extensible manner, it becomes brittle and wonky. If you’ve ever played Tetris you’ll know this feeling. You keep having to deal with these fast falling blocks and before you know it, you’ve lost the game.

Where has beauty gone?

If beauty in software is important, why is it seldom considered? In your organization, there is some care and thought given to the general architecture. But, the specifics of how the software plugs together is handled ad-hoc - without any guide rails to support developers. This leads to individuals writing software for them, not for others. It's natural - we are all a little bit selfish.

A further reason is that rarely are companies "dogfooding" their software. Sometimes, it's not possible because the developer is not the customer. That's fine. But failing to have a deep understanding of the software and how it will be used leads to false assumptions and clumsy design.

Principles

What's the solution? Design has no hard and fast rules. And generally, people have a good sense of something working "well" or not simply by using the thing. Software design is no different. Therefore, principles make sense in order to guide developers to design great software. I’ve attempted to distil the principles of beautiful software into three key attributes.

It should be a joy to work with. No headaches or screaming at the computer should be seen. It should be joyous to work with or on your system to be able to accomplish their tasks.
Simple. “If you can’t explain it simply, you don’t understand it” - Einstein. Beautiful systems need to be simple by definition. That doesn’t mean they cannot be complex. Rather, the complexity should be presented simply. If quantum mechanics can be explained then so can your system.
Extensible. Doubtless, requirements change and you often need to modify or add functionality to meet a use case. In the case of beautiful software, this should be easy to implement, document and test.

How to Use These Principles

On a practical level, I've found it best to codify these principles into a short checklist.

Are all successful use cases of methods of the API documented?
Are error codes and edge cases documented for each method of the API? - i.e., if you pass X with a value of Y, you also need to provide Z.
Does the API have side effects?
Do all the methods do “what they say on the tin”? - in other words, does the API method in question do what is described by the method itself. For example, if we have a REST API method GET /packages - does this return a list of packages for a customer? It’s always good at this stage, if you’re experienced with this system to ask someone who has never seen it - even if they are non-technical. Just ask “if you asked me to get packages”, what would you expect me to answer you with?
Is it possible to run the API with 2 commands or less? - if the answer is no, then we can look into creating a setup script.
Are testing patterns already established to test the API, including mocking data or dependant systems etc.
Can I quickly tell what version of the API I’m using
Can I quickly resolve any errors myself? - does the API return error messages that are actionable and concise.
If it is not possible to rectify an issue myself, can I provide a means of recreating an issue to the API author? - requestId’s, trace logs and the like are all helpful here and need to be accessible to the consumer.
Are there significant efforts to mitigate issues? - Does it handle retries and other complexity that should not be a concern for the end user.

There is a lot of overlap here between sound software development practises and beauty. As the famous design adage goes "form follows function". By creating good development practises, you end up creating beautiful software.

Takeaways

Keep an eye on beauty, no one else will. Even in your little corner, strive for beautiful design. If you’re stuck making something beautiful, ask coworkers, friends, customers or join the #softwarebeauty irc channel on freenode.

Continuous Delivery to ECS with Terraform

2021-08-11T00:00:00Z

Continuous delivery is something that we're all striving for. I was doing the same, but there was a snag:

My terraform code and API code were in separate projects
I wanted to make updates to the API code and have it build and update the ECS service
I didn't want to manage the container definition separately as it had too many dependant resources (Datadog sidecar etc.)
You have multiple environments and don't want to have to use a separate git branch for the API code.
You have a branch for each environment for your infrastructure that is independently deployed.

Maybe you have this problem too. Because Terraform uses a specific ECR image path to build the container definition, how do we ever update it automatically?

There are several ways to solve this problem, many of which are discussed in this thread. But, today, I'm going to show you how I resolved this issue.

Setup your Docker build/deploy pipeline

First things first, we need the API Docker image inside ECR. This will vary according to what CI system you use - in our case we use Azure DevOps.

When the image is being built, we want to tag it uniquely. Ideally, you want something numerable. In our case, we chose to use Azure's built in "BuildId" parameter to tag the images.

Below you can see the build steps we take in the CI pipeline. After the image is built, it creates a text file with the BuildId in it and ships that as an "Artifact". This will become important later. But the main thing is you need to trigger a further pipeline for your environments based on that parameter changing.

- task: Docker@2
  inputs:
    command: build
    DockerFile: "$(Build.SourcesDirectory)/Dockerfile"
    repository: $
    tags: |
      $(Build.BuildId)

- task: ECRPushImage@1
  inputs:
    imageSource: "imagename"
    sourceImageName: $
    sourceImageTag: "$(Build.BuildId)"
    repositoryName: $
    pushTag: "$(Build.BuildId)"

- task: Bash@3
  displayName: "Upload Build Artifact of the Docker image Id"
  inputs:
    targetType: "inline"
    script: |
      # Add the build Id to a new file that will then be published as an artifact
      echo $(Build.BuildId) > .buildId
      cat .buildId

- task: CopyFiles@2
  displayName: "Copy BuildId file"
  inputs:
    Contents: ".buildId"
    TargetFolder: "$(Build.ArtifactStagingDirectory)"

- task: PublishBuildArtifacts@1
  displayName: "Publish Artifact"
  inputs:
    pathToPublish: $(Build.ArtifactStagingDirectory)

Make sure to run this pipeline now you've created it.

Setup an SSM (Systems Manager) parameter

SSM is an AWS service I had previously never really used. Its parameter store feature will allow us to store a variable that we can update later - in this case, the docker image tag.

Create a new parameter by going to AWS Systems Manager > Application Management > Parameter Store. Name the parameter something like /my-api/${env}/docker-image-tag (where env is the environment, you'll need to duplicate this parameter for all the environments you have). It should be a "String" variable and be the unique tag that is generated by your CI build pipeline - in my case, the BuildId.

Create your deployment pipeline

Now, we need to define a way to update the image just in a certain environment (e.g., just development). How can we do that? Because of our setup, the duplication effort is fairly minimal. We already have our image build (which is consistent across all environments). We just need to update that SSM parameter to use the unique tag (BuildId) that the build pipeline generated.

In Azure, you can trigger a pipeline based on an artifact as we generated in step 1. I then configured 3 tasks based on this:

Get the BuildId from the file and add it to the runners environment
Update the SSM parameter for that environment to the new BuildID
Trigger the Infrastructure/Terraform pipeline for that environment - this is where the new SSM parameter value will get picked up and used in a container definition.

Update Terraform to use the SSM parameter

Now you've got the SSM parameter being updated each time there is a new build, we need to set up Terraform to use the SSM parameter as part of the image name.


// Import the SSM parameter
// This can be done on a module level because it depends on the environment
data "aws_ssm_parameter" "docker_image_id" {
  name = "/my-api/${var.environment}/docker-image-tag"
}

// Use it later on...
container_image = "<AWS_ACCOUNT_ID>.dkr.ecr.<REGION>.amazonaws.com/my-api:${data.aws_ssm_parameter.docker_image_id.value}"

All done!

Now you've set up a complete continuous deployment pipeline with Terraform and ECS. To review, here's how the system works

Creates and builds a new ECR image tagged in a unique way
The build pipeline notifies the release pipeline of this new image tag in some way (in Azure's case, a build artifact)
The release pipeline updates the SSM parameter based on the image tag and triggers the Terraform deployment pipeline for that environment.
Terraform picks up the new SSM value and implements it

There are two obvious downsides to this approach:

Multiple updates to multiple API's could cause locking issues where you have to manually run the Terraform pipeline once.
It's a bit slower than other approaches, but the best one I could find. Plus if your Terraform repo deploys in less than 2 minutes like ours, then it's not a big problem.

Web Performance for Developers on a Deadline

2021-04-29T00:00:00Z

Web performance is a vital part of your business to ensure customers keep returning. One retailer found that by reducing their page load by 65%, they saw a 63% increase in organic traffic. Google is also beginning to use the "core web vitals" to rank your page in search results. Despite this, it can be difficult to get the time for it. So often, teams will de-prioritize performance for the sake of more features.

Although there are specific ways on how to improve site performance based on your technology stack, server, customer requirements and more, here's a guide on how to improve the performance of your website if you're on a deadline - regardless of those factors. Got no time to read the article? Simply read the headings for each section.

1. Gzip and Brotli

Gzip and Brotli are two compression algorithms, they are used to compress resources on your server before sending them to the customer's browser. This makes the file smaller, thereby making it quicker to transfer. The browser then unzips these files and then uses them. All the major browsers have support for at least 1 of these compression types. This is such an easy change that gives such a large impact. A report by Akamai in 2016, reported a file size reduction of 63% by Gzip for JS files and 68% by Brotli.

Here are links to a few guides on how to do it for your setup

2. Cache-control on all resources / Use Cloudflare

Caching can hugely speed up subsequent page loads by storing resources on the customer's computer. This means you only need to deliver any dynamic content such as API responses and so on. Add a long cache control so that resources are cached for a month or more. Will this cause bugs? No! If you use a build tool like Webpack, it will create new file names that will force the browser to re-download the page when you do a new release. This very blog is cached heavily, so try this to see the impact of caching on page speed.

Open up chrome dev tools (F12)
Go to the Network Tab
Notice the red "Load" time.
Now click and hold on the Refresh button - a dropdown menu will appear. Click, "Empty Cache and Hard Reload"
Check the "Load" time - when I did this test it was 482ms.
Click the Refresh button normally
Check the time again - now it's only 154ms! A 68% percent improvement!

Here are some links on how to do it.

AWS S3 + Cloudfront - Scroll to the bottom
IIS
Nginx
Apache

3. Implement a build process for front-end resources (JS and CSS)

If you haven't already, try to implement a build process for your Javascript and CSS files. Why? Because using a build process can make the files smaller and compatible with the browsers you need to support. If you have tested your site on Google's Page Speed Insights and have seen the dreaded "Remove unused Javascript" then this step is for you. A build process will significantly reduce your initial bundle size thereby reducing load times.

Here are a few links on how to do this. Webpack is a little complex to setup but is worth the effort in the long run. You can also expand its usage to optimize images and other funky things.

4. Defer and Asynchronously load 3rd party resources

"You've got a massive head" - that's what I say when I visit most websites. But seriously, they have a huge amount of resources in their <head> tag! Why is this a problem? Because resources, such as JS and CSS, are loaded at the top of the page, this is the first thing the browser loads. Thereby blocking it from loading any of the content further down! You might say you need to load Google Analytics, Intercom and a myriad of other trackers right away, but the answer is simply that you don't. By deferring and loading resources asynchronously, you will drastically improve load times for your customers and the resources will still be loaded quickly anyway.

How can you do this? I'll share this directly because it's a simple change.

<!-- This -->

<link
  href="https://fonts.googleapis.com/css2?family=Ubuntu:wght@400;700&display=swap"
  rel="stylesheet"
/>
<script src="https://mysite.com/script.js"></script>

<!-- Becomes This -->

<link
  href="https://fonts.googleapis.com/css2?family=Ubuntu:wght@400;700&display=swap"
  rel="stylesheet"
  media="print"
  onload="this.onload=null;this.removeAttribute('media');"
/>

<script defer async src="https://mysite.com/script.js"></script>

Now you can move these tags to before the closing </body> tag at the bottom of the page. Job done! You should see a huge improvement in your First Contentful Paint Times.

5. Load polyfills only when needed

Polyfills are small snippets of code that allow developers to use modern Javascript features on older browsers. But the problem is, how do you load polyfills only for customers who need it? Otherwise, you will compromise the modern browser experience which likely makes up the majority of your customer base. Here is a great write up by Phillip Walton on this very topic - https://philipwalton.com/articles/loading-polyfills-only-when-needed/

Bonus 6. Set Performance budgets

Bonus round now. Hopefully, you've made some performance improvements and you can see a difference in page load times and your Lighthouse score. But so often, without monitoring something, it's likely to drift. For example, if you are putting up shelves in your house, you need to constantly keep an eye on the level using a spirit level. You cannot just put it there once and then eyeball it the rest of the way. In the same way, you need to monitor your key performance metrics to make sure they don't return to how they were.

You can do this with tools like bundlesize, webpack dashboard, speedcurve and more.

So, now you can monitor the performance you need to set a budget. In other words, what is the limit of performance the business can take (e.g., what is the slowest average load time we can take without affecting revenue)? I will write an article on this in the future, but I'd recommend taking the worst result of each metric in the past week and set that as the budget. This can then be in-forced by tools such as LighthouseCI.

Takeaways

Overall, I hope you've learnt some quick ways to improve your web performance. As mentioned, there is a myriad of factors to consider when looking to improve web performance so the methods above will not be a one-size fits all. If you'd like to know why your site is slow, I can help you. I'm a software performance consultant who works with organizations of all sizes to analyse and fix site speed issues.

How to Run Sequelize Migrations in Azure Pipelines

2021-04-07T00:00:00Z

Database migrations are the concept of managing your database schema via reversible, version controlled files. A program is then used to run these "migrations" and keep track of which ones have been run on your database. Migrations are immutable, meaning if you want to change a column name, type or anything else then you have to create a new "migration". Handling your database programmatically gives you many benefits. Namely, providing a consistent schema across all your environments and portability if something happens to your DB. Further, with these files being committed to source control, migrations can be reviewed by others on your team. If you're not already using a tool to do this, I'd encourage you to do so.

Throughout this article, I'll refer to two terms

Migrations - meaning files that change the schema of the database but not the underlying data
Seeders - files that insert anonymous data into our staging environments for testing purposes

Normally, these migration and seed files would have to be run manually from a developers computer against the different databases. But, with my obsession with automation, this wouldn't fly. I decided to create an Azure Pipeline runner to handle this for us. It will run automatically whenever new commits on our development or master branch are found. It also reduced stress for me as I know that I will make mistakes, whereas a computer, configured correctly, won't! 😅

Although this article is designed around building azure pipelines for Sequelize migrations. This process can be adapted to other ORM's such as Knex and TypeORM.

Create Your Artifacts

If you're not familiar with Azure, it has a concept of "Artifacts". These are a collection of files that can then be used by other pipelines. We need to create two artifacts, one for our migrations pipeline and the other for our seeding pipeline. In your source control create the following two files - you can copy-paste the code below!

# azure-migrate.yml
pool:
  name: azure-pipeline-runner
pr: none

steps:
  - task: CopyFiles@2
    displayName: "Copy migration scripts"
    inputs:
      contents: "$(Build.SourcesDirectory)/migrations/**"
      targetFolder: $(Build.ArtifactStagingDirectory)

  - task: PublishBuildArtifacts@1
    displayName: "Publish Artifact"
    inputs:
      pathToPublish: $(Build.ArtifactStagingDirectory)
      artifactName: migrate

# azure-seed.yml
pool:
  name: azure-pipeline-runner # the name of your azure pipeline runner
pr: none # Don't run this pipeline for pull requests

steps:
  - task: CopyFiles@2
    displayName: "Publish SequelizeRC"
    inputs:
      Contents: .sequelizerc
      FlattenFolders: true
      TargetFolder: "$(Build.ArtifactStagingDirectory)"

  - task: PublishBuildArtifacts@1
    displayName: "Publish Seed"
    inputs:
      PathtoPublish: seeders
      TargetPath: "$(Build.ArtifactStagingDirectory)"
      ArtifactName: seeders

  - task: PublishBuildArtifacts@1
    displayName: "Publish Sequelize Config Folder"
    inputs:
      PathtoPublish: config
      TargetPath: "$(Build.ArtifactStagingDirectory)"
      ArtifactName: config

In the code above, it first copies the .sequelizerc file that is used to denote various configuration parameters to Sequelize. Next, it creates "build artifacts" for the seeding and migration folders where all the files related to setting up the database are stored. Finally, it publishes them to the azure artifacts library.

Configure Your Runner (Optional)

This step is optional because it depends on your existing setup. All you need to make sure is that your Azure runner (whether self-hosted or not) can access the Database.

We use MySQL Aurora to host our database which sits in a VPC. Our azure-pipeline-runner (defined in the "pool" parameter) is hosted inside the same VPC but a different security group. So, we needed to allow access from the runners' security group to the RDS' security group. This is called "ingress" in AWS. The port you need to allow access to may vary - in our case, it's 3306 which is the default for MySQL.

Getting this setup is a simple process. Check this guide for more info.

Create Your Pipeline

Now we've got our runner configured and our build artifacts published, we can move onto creating the actual pipeline. Go to Pipelines and Releases and click "+ New" and select "Create Release Pipeline" from the dropdown.

You'll be prompted to select a template but we can click "Empty Job"

Next, click the Artifacts box on the left and then find your artifact by searching for it.

You'll be able to find the name of the artifact under "Pipelines > Pipelines". You should see your migration or seeding pipeline that you created earlier. Clicking into one of the runs of this job will reveal the artifact name that the job created.

Now, configure a stage. Click on the "Tasks" tab at the top of the page. This will take you to the list of "tasks" that will run for each stage.

Click to add a new task and search for "npm". We want to first install Sequelize globally on the command line so that it can be used to run the migrations or seeding process. Because we are using MySQL, we also need to install the mysql2 package.

The job should end up looking something like this:

Now we need to add the stage that runs the migrations or seeding process. Click the plus button again and select the "Command Line" job. This will allow us to run the Sequelize commands.

The command we want to run for migrations is:

sequelize-cli db:migrate --url ${DB_URL}

For the seeding process it is:

sequelize-cli db:seed:all --url ${DB_URL}

The documentation for these commands can be found here.

We aren't doing anything fancy asides from passing our Database URL. Since this won't be stored in our Git repo, we need to provide it here as an environment variable.

If something goes wrong. You may need to check that the runner is running files in the correct directory. Ensure that the directory is the root. It should contain a folder called "seeders" or "migrations". These folders should contain the migration and seed files.

Below is how our job ended up

Now you've configured one stage, you can clone it for the others! Go back to the Pipelines view and click the clone button beneath the stage card.

Wrapping Up

I hope this helped you configure your database migrations! Here is a quick summary of what we have learnt.

Database migrations and seeding processes are important to codify for consistency, portability and review purposes.
How to configure Azure runners to allow ingress to RDS
How to create build artifacts in Azure Pipelines
How to write basic azure pipeline jobs
How to migrate and seed your database using Azure pipelines.

I'm glad to have this work sorted as it was a bit of a hassle to configure. But, we got there in the end and this is now a durable process that will scale with the team.

How to Improve Your Typing Speed

2021-03-30T00:00:00Z

Recently, I've started practising my typing each day - for around 5-10 minutes. So far, according to 10fastfingers, I've increased my typing speed from 70WPM to 80WPM. Not much, but I can feel myself getting more "familiar" with the keyboard. In this post, I wanted to dig into why I've started this daily habit and how I'm practising.

Why?

This all started with trying to adopt a "keyboard-first" mentality. If you're anything like me, you mix and match between operations performed by your mouse and keyboard. But, when increasing my usage of Vim, using the mouse isn't possible. All operations have to be performed via the keyboard. Not only does this make actions faster but also keeps your hands on your keyboard, enabling you to get back to typing as soon as possible. I decided I could translate this mentality to other programs by using their keyboard shortcuts. Learning simple shortcuts like selecting the URL bar of your browser (CMD+L) and cycling through tabs (CMD+SHIFT+] or CMD+SHIFT+[) has saved me a bunch of time.

The keyboard is the primary interface between my brain and the computer. So increasing the "bandwidth" of this connection is super valuable. As shortcuts improve the speed of actions, typing faster will help me get my thoughts out quicker and code written faster and with fewer errors. I always seem to get in a flow of programming, only to be interrupted by making a typing mistake - reducing these errors was a priority.

How?

Principally, I'm doing this through a daily typing test on 10fastfingers.com and practise on keybr.com. Both of these tools differ in key ways (no pun intended).

10FastFingers

10FF, gives you a 200-word typing test and a time limit of a minute. The challenge is to type as many words as you can within that period. The words chosen are the most used in English. I like taking this test regularly because it builds my muscle memory for common words and phrases. Given that you can learn 800 words and know how to speak 75% of a spoken language, it pays to be able to type these common words quickly. I'm ok with taking a couple of seconds to type verisimilitude - however much I love that word. There are a bunch of social features with 10FF, but I don't care for these. My only opponent is myself. Often, I catch myself in this test in a sort of "flow" where I magically type all the correct keys without even thinking about it. This momentary realisation makes me smile, but soon turns into scowls when I mistype "soccer" as a result.

Keybr.com

Keybr is a beautifully designed tool that first takes you through the entire keyboard to see where your weaknesses lie. Most of the words are "riffs" on familiar words - "influencecapa", "comprom" and "discuse". In doing so, it breaks your muscle memory and forces you to think carefully about where you place your fingers. This tool makes up the bulk of my practice as it "moves" you around the keyboard in a way that typing common words won't.

Wrapping up

Maybe you're sold on improving your typing speed, maybe not. In any case, I encourage you to sharpen the saw and look at the fundamentals rather than chasing productivity through "tools". Now the sun is out, and I've typed enough today so cya!

Super Fast React/Node App Testing with GitHub Actions

2021-03-25T00:00:00Z

A seldom thought of component of performance is that of continuous integration performance. Here at York Press, we are big users of both Azure Pipelines and GitHub Actions. Due to us hosting our Azure pipeline runners, "job minute" restrictions were never a concern from a billing perspective. Although, the long running jobs did frustrate the team. Having moved some key processes over to GitHub Actions, I decided it was time that we looked at improving the performance of one of our core repositories. Not only would this mean developers had quicker feedback, but it would also mean we burned through our actions minutes a lot slower. Here's how I did it.

Initial Investigation

GitHub actions (and I promise this isn't an ad) has a handy feature whereby you can see how many seconds each stage of the job took.

The first thing I spotted was how long installing npm modules took - nearly 3 minutes! Because of this, I chose to combine the Test and Lint pipeline so that we would not need to duplicate the module installation.

Secondly, I swapped out my normal npm ci command for another action bahmutov/npm-install@v1. This action handles all the cache invalidation and storage of node modules across builds so you can save time with installing them. After those changes, here is what the timings looked like...

Half the time gone! That's a good start but still not far enough. I found the modules were taking ages to install due to a Webpack plugin responsible for optimizing images, something we didn't need in the CI process. I moved this out into an optionalDependencies and then set the command to npm ci --no-optional

ESLint

The other big fish to fry was ESLint, it took nearly 2:30 minutes to run. I tried to debug this locally using an environment variable TIMING=1. This gives you a table view of how long each ESLint rule took to check.

Interestingly, it was the import/ rules that were taking the longest. After some google-fu, I discovered that it was due to having to build a dependency graph across the codebase. Our codebase is fairly large so it was understandable why it would take this long. I didn't want to remove the rule entirely as it was useful, but surely there was a way around it...

Actions to the rescue! Fortunately, a kind internet person has created a github action that will run ESLint only on the files that have changed. I swapped this out like so

- uses: tinovyatkin/action-eslint@v1
  with:
    repo-token: $
    check-name: eslint

This completely eliminated that time taken if no files had been changed matching the scanning glob.

From there, I spent more time than I care to admit trying to trim the time down. The main blockers were the dependency install (1:20s average) and the Jest test suite (50s average). Although there are ways to run the Jest suite in parallel it sort of seems redundant at this stage. The install is the big job, but the unfortunate battle is that we have Webpack image-loader as a devDependency. This then installs a whole host of binary packages that then get built from source - every. single. time. Anywho, I'm pleased with reducing it by 76%.

Here are my main takeaways:

Spend time speeding up your CI - fast developer feedback is important and saves you money (if you're restricted on pipeline minutes)
Use the pre-built actions - there is a huge marketplace of actions that solve a bunch of problems and have smart defaults. GitHub Actions is great (and I promise this isn't an ad), in part, because it's like code lego.

I hope this helps you with your journey in speeding up your CI.

Solve Your Problems. Not Others.

2021-03-24T00:00:00Z

What is your business? What is your product? What is your core mission? These are questions I ask myself and my clients, continuously. They help delve into what the customers are paying for, and in turn, what your staff are paid for. In the world of flowing VC money, it's often never questioned what practical value that £X is giving you. But businesses of all sizes are prone to this. Losing sight of these questions often evolves complex service criteria that leads teams to create a bespoke solution rather than something off the shelf. I'm willing to wager that, if you're a software developer, you've worked on a service or feature that could have been handled by a third party in the past month. Or, worse, you've resolved a bug that wouldn't exist if you had used a third-party solution. I know I have - on both counts.

Through my career and starting my own business, I've developed my understanding of what I'll dub "business thinking". Business thinking is putting your numbers hat on and practically evaluating options. Not just on "can I do this or not" but what it will cost the business in both money and time. As an example, at York-E we needed a contact form with some complex routing to different support teams based on the locale a customer-specified (an Egyptian customers request would go to the Egyptian support team etc). I began to scope this out before realising we could likely use something off the shelf. After some DuckDuckGo'ing, I found Formspree could handle this. The catch? It's $40/mo. That's quite steep for a contact form. Any developer worth their salt could bash one out in a few days. Right? At this stage, many teams would dismiss the idea of handling such a trivial task to a third party, but we chose to keep it as an option. Afterwards, we calculated that it would take 2 years before building our own contact form system would be worthwhile - not factoring in maintenance.

The lesson? Don't ignore third-party solutions even for trivial tasks. Contact forms are not York-E's business. Providing an online education experience is.

Boilerplate

When creating new products, I've reviewed these questions as well. I poured a great deal of time into creating TurboAPI from scratch. The only "boilerplate" I used was create-react-app. The express API, lambdas, serverless configs, GitHub actions CI/CD pipelines, all written from the ground up. In retrospect, this was a mistake, I should have taken more time to find a boilerplate and use that. The parts that took me the longest with TurboAPI, were fixing bugs with TypeORM queries, Stripe billing and user authentication - not the actual "app" itself. The core technology of TurboAPI was written between 1am to 3am on our hallway floor. This pales in comparison to the overall time I spent on TurboAPI for the MVP. Which, according to Toggl, was about 45 hours. To have used a boilerplate that saved me 20 hours would have been a lifesaver. Now, when developing new products, I use a boilerplate and am familiar with chopping and changing it according to my needs. Sure, I still need to build some bespoke but basic elements but this is small by comparison.

There are lots for free that offer a basic CRUD API with Mongo and React. And recently, there has been an explosion of paid boilerplates, either offering a one-time payment or monthly subscription. Although it seems like a great deal of money to part with, getting up and running quickly is invaluable. I'm in the process of creating a niche boilerplate that I'll share soon.

There is another lesson here. Solve the problems that are not solved. Displaying tables of data, handling authentication, and billing are problems that have already been solved and many boilerplates are stitching these together.

Some argue that boilerplates mean that you won't have system understanding and will make it more difficult to debug. This might be the case. But, the focus here is to get your MVP out and start delivering value. Even if this is the MVP of a feature inside a company, the same principle applies.

Here are the key takeaways for all software developers - regardless of experience:

Know how much your time is worth - this is important in evaluating if it's cheaper and faster to use a third-party service
Know what problems your solving - is it part of your core business? If not then try to outsource it in some way

SpellcheckCI

2021-03-04T00:00:00Z

Making sure you have correct spelling on your blog posts is vital to keep readers attention. Unfortunately, it's a laborious process and sometimes things fall through the cracks. Being the nerd I am, I decided I needed a shell script to solve this problem.

Thankfully, someone has created an open source markdown based spellcheck module that is Node based - mdspell.

Since I'm using Gatsby, my posts can be found under content/blog/*/index.md - where * is the name of the blog post. The command to run the spell check was then

$ npm i -g node-markdown-spellcheck && mdspell -a -n "content/blog/**/*.md"

This would go through each of my posts and then validate the spelling is correct. When it comes across an incorrect spelling, it notifies me and asks me if I want to correct it, or add it to a local dictionary.

But, because I often blog from my iPad, where I don't have a terminal, I wanted this feedback to be visible on the CI for the new blog posts. My workflow for creating new posts is create a new git branch, create the file and write the post, push to github and create a new pull request. You can find this exact blog post's pull request here.

Time to Automate

I'm a big user of GitHub Actions so I went with that to setup this process.

Initially, I went down the road of installing all the node dependencies, then installing mdspell and then running the spellcheck. However, I found that it took over a minute to download all the node modules! It turns out, I could have used npx to use mdspell without having to install the project.

Here is the complete GitHubCI - which, across over 50 blog posts, takes around 10 seconds to run!

# ./.github/workflows/spellcheck.yml
name: Spellcheck

on: [pull_request]

jobs:
  spellcheck:
    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [14.x]

    steps:
      - uses: actions/checkout@v2
      - name: Use Node.js $
        uses: actions/setup-node@v1
        with:
          node-version: $
      - run: npm i markdown-spellcheck -g
      - run: mdspell -a -n -r "content/blog/**/*.md"
        name: Spellcheck

I hope this proves useful to you for your own blog. If you don't have one already, I'd highly recommend creating one!

Setting up LightHouse CI for React in GitHub Actions

2021-02-16T00:00:00Z

At York Press, we noticed that our pages were gaining weight. In some cases, pages were loading over 1MB of resources before showing for the customer. This was unacceptable considering the modal broadband speed is around 1MB/s. So, we decided we needed stricter checks. This would ensure that pages are lighter than an ants leg made of clouds. And, faster load times would mean customers could get to studying faster - which I trust they yearn for.

Lighthouse to the Rescue!

Lighthouse is a tool developed by Google. It analyses a page and gives it a score, out of 100, on SEO, Performance, Accessibility, PWA and Best Practises. Although these are arbitrary numbers, they give a rough guide to how your website is doing. These scores are also used to rank your page in Google search rankings. So they are vital to maintain for business reasons, not technical prowess.

The challenge is how to get this tool setup as there are lots of outdated articles and guides. Furthermore, none of these seem to cover a regular use case - setting up Lighthouse for your React app.

Here's a definitive guide on how to setup LighthouseCI for your React app - and have it tracked in Github Actions.

Setup Lighthouse CI

First, you will want to install LighthouseCI and http-server locally for testing purposes.

$ npm i -g @lhci/cli http-server

The former is the LighthouseCI tool. The latter is a small module to run the React app after it has been built.

Next you can create a file called lighthouserc.json. This should have the following contents

{
  "ci": {
    "collect": {
      "url": ["http://127.0.0.1:4000"],
      "startServerCommand": "http-server ./build/client -p 4000 -g",
      "startServerReadyPattern": "Available on",
      "numberOfRuns": 1
    },
    "upload": {
      "target": "temporary-public-storage"
    },
    "assert": {
      "preset": "lighthouse:recommended"
    }
  }
}

The section under "collect" is where the server that runs the React app is defined. The interesting properties are the startServerCommand and startServerReadyPattern. The first tells Lighthouse how to start your application. And the second, tells Lighthouse what text to look for to see that the server is running and the test can begin. In this case, it starts the server via http-server and then it listens for the text Available On. Run the command shown above for yourself and see what text it displays in your terminal. You may need to change /build/client to the directory where your application gets built

Now you can give your LighthouseCI a whirl! Build your application (if you used create-react-app then run npm run build), then run:

$ npm run build
$ lhci autorun

You should then see an output like this:

✅  .lighthouseci/ directory writable
✅  Configuration file found
✅  Chrome installation found
Healthcheck passed!

Started a web server with "http-server ./build/client -p 4000 -g"...
Running Lighthouse 1 time(s) on http://127.0.0.1:4000
Run #1...done.
Done running Lighthouse!

Checking assertions against 1 URL(s), 1 total run(s)

33 result(s) for http://127.0.0.1:4000/ :

Setting up GitHub Actions CI

Now, let's automate that. The best way to enforce these sorts of checks is to make them part of your pull request workflow. This means preventing merge on requests that fail to meet these standards.

All we need to do with GitHub Actions is imitate the commands we did in the setup process. Paste the following into a new file called /.github/workflows/lighthouse.yml

# ./.github/workflows/lighthouse.yml
name: LighthouseCI

 on:
   pull_request:

 jobs:
   lighthouse:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v2

       - name: Setup node
         uses: actions/setup-node@v1
         with:
           node-version: "14.x"

       - name: Install
         run: npm ci && npm i -g http-server @lhci/cli

       - name: Build
         run: npm run build

       - name: LighthouseCI
         run: lhci autorun

Next, push up your changes and create a new pull request. You should see your Action running at the bottom of the pull request.

And that's that! I hope that has saved you a lot of time if you were struggling to get your React app to play nice with GitHub Actions.

So, You've Messed Up

2021-02-15T00:00:00Z

So you've messed up... big time. You've dropped the production database, pushed a broken update to production, throttled the API and literally set the internet on fire. Cold sweat gathers on your brow, you stare longingly at your alcoholic beverage of choice and you perhaps start to update your resume. You are convinced you're going to be fired.

I'm sure if you've been a software developer for more than 6 months, you've been in this situation. It can be an manic situation and you might be unsure about what you are supposed to do.

Here is my guide for when you inevitably make a catastrophic mistake.

Things to Remember

You probably won't get fired - I say probably as I'm sure there are some instances that a person may get let go, but in 99.99% of situations you won't. Employers know you will make mistakes, they themselves make mistakes. What matters if how you respond to your mistakes.
Don't Rush - The natural instinct when things go wrong is to try to resolve them - quickly. Try to slow yourself down and don't act irrationally.
Be Radically Transparent - Even if you feel you work in a company that thrives on concealing mistakes, it's best to be radically transparent with your mistake. If you literally did DELETE * FROM *, then admit that. Even GitLab admitted they did an rm -rf on a production server that caused an outage. It's best to be upfront about your mistakes and learn from them. You could even write your own personal post mortem.
Keep a Cool Head - It's easy when you've made a mistake to lash out and blame others. Instead, keep a cool head. Remind yourself that A) nothing will be accomplished by getting angry. And, B) the situation has most likely not resulted in direct physical harm to anyone, so it's not that bad in the grand scheme of things.

What's Next

Access the Impact - who is affected? Is it actually as bad as you think? Can you resolve the situation quickly? For example, if you have just done a deployment that has gone wrong, then can you roll back? In any case, continue to the next step.
Start a Log - You'll thank yourself later for this one. Open up a new text file on your computer and record the current time and date. Write down exactly what the impact is, and what has happened. It can seem like a waste of time to do this in the middle of an incident, but is important for reflection later on.
Report the Problem - Next, you need to report the problem to someone. You should know who this is - likely it's your line manager, the technical project manager or lead developer. Speak to them about the problem, your findings and ask if they have any suggestions. In some cases, you may need to send an email to your support team to inform them that there is an ongoing problem and updates will be given every 30 minutes - this gives them confidence that they can relay to angry customers. Make sure to follow through with this, even if you haven't found the "solution".
Debug the Problem - If you have made a direct mistake, then hopefully this step will not take too long. But, work to debug the problem and find the root cause. Use logs, graphs in your APM and anything else you can get your hands on. The focus here is the root cause. Often, it's the second order problems that are the surfaced issues. As an example, you may find a particular API endpoint that stops working and returns a 500 - problem 1. Upon further investigation, it appears that the request is timing out at the gateway - problem 2. After careful examination of the top logs, you see that the CPU usage spikes when that endpoint is called and crashes the server - problem 3. But why does it spike the CPU? You spot that Jerry has added a random for...loop calculating the digits of Pi to 10,000 places - the root cause.
Resolve the Problem - Now you know the cause, you can resolve it. Work with people on your team and be extra communicative during this entire process - you'll build trust and prevent work being done twice.
Review Your Log - Look into each stage of the process and analyse whether you used your time well here and what mistakes were made along the way. Root out any efficiencies you implement personally as well as changes to make at the organisational and team level.

But what if you're in a position where others come to you with their catastrophic problems? Here are some things to remember.

What to do when someone comes to you with an incident

Ask them to Keep a log - This will be a saviour when it comes to implementing preventative change and help you learn the mindset of those on your team.
Be empathetic - It's not the time to point fingers, or say I told you so (to anyone, not just the person who reported the incident). Focus on resolving the issue and learning.
Be a force for change to prevent future issues - As someone with some authority, you have the power to implement changes to prevent future issues. Otherwise, these things will happen over and over again and customers will get angry. You'd think that things fix themselves on their own and get sorted but you'd be amazed at how many teams spend large swathes of time, simply putting out fires. Implement changes such as mandatory testing, code reviews by experts and restricted access to prevent these incidents in the future.

I'd like to say this won't happen, but it will. Overall, learn from your mistakes and communicate abundantly. You'll learn valuable skills in the process and be an asset whenever more incidents come down the line.

Shutdown Routine

2021-02-04T00:00:00Z

In the current pandemic, it's even more challenging to switch off from work. With many working from home, it can seem impossible to "leave" work in a physical and mental sense.

I introduced a shutdown routine to help combat this. It's a practise touted by many "productivity guru's", which made me sceptical at first. But, the basic idea is to create a short checklist of things to do before you "finish" your day. The goals are as follows:

Realise how much you've accomplished that day
Be clear about what you need to do tomorrow
Tie any loose ends

I usually allot anywhere from 10-20 minutes to do this process. Yours may vary but here is my list and the reasons why:

Close any open tabs
Write down tomorrows time block plan
Reply to any messages (Signal, Text, Slack) - I've uninstalled slack from my phone as well as leaving it on my desk in DnD after 5pm. This task helps me to get rid of those niggles to check my phone after this.
Zero my inbox - Even if it means snoozing mail to the next day, I don't want it in my inbox. It all needs to be dealt with.
Clear my todo list - After I've finished work, I've finished and I want to spend time with my family. This helps me to stop feeling productivity guilt about things I haven't done. It also saves all the tasks rolling over to the next day.
Clean my desk - Sounds obvious but it's nice to not return to a stale coffee cup in the morning.

These steps may seem reductive. But, I have found the process mindfully winds me down from work and helps me get into the "relax" flow for the evening. I hope you too can leave your stress at work for the next day. There is always more work.

Resumé Red Flags

2021-02-03T00:00:00Z

After looking through tonnes of resumés, they all begin to blend into one. But, I wanted to share some red flags I see that are unique to software developers. It's a mine field to prepare an eye catching, yet informative CV. According to the undercover recruiter, hiring managers spend 5-7 seconds looking at a CV. I can attest to the fact that this is unfortunately true. It's another discussion about whether CV's are an effective way of conveying a persons ability (spoiler: it's not). But, that's by the by. It's a reality we all have to face. So, here's some things to avoid to help you gain that edge.

1. Don't include a picture of yourself

This is not so prevalent in the UK and US but is often the case in other countries. I would recommend airing on the side of caution and get rid of the photo. It is (or at least should be) irrelevant to someones decision to hire you.

2. Put Your Work Experience At the Top

This is what myself and others are looking for when hiring someone. Why put this above your education? You have to think about the purpose of a CV. It's for someone to see if you can do a job. If you have relevant work experience to the job then you want to bring peoples eyes there.

If you don't have any work experience, include your personal projects, freelance work or open source contributions.

3. Don't use Skill Meters

More and more, I have seen CV's that have bars indicating their skill at various technologies. Although many argue that it shows a "high-level" overview of someones ability, I find it loses all meaning. After all, what is 80% Javascript and 50% CSS? Illusory superiority exists because many think they are above average. In 1981, a survey took place to rate a persons own driving ability. 93% of participants in the US ranked themselves in the top 50%. Mathematically, this cannot be correct. So, evaluations of ones own skill level is flawed. And, should not be included on a CV. Speak with your experience, not fancy formatting.

4. List the Tech You Used in Various Roles

When recruiters look at your CV, they are looking to see if the technology you've used most recently is relevant to the job they have.

For the sake of easy parsing, I include a sentence at the end of each experience section that says "In this role, I used the following technologies...".

5. Use Short Concise Sentences

As we have already covered, people aren't reading your CV. And, if they do, you don't want to bombard them with information. Try to keep things information dense but concise. I recommend using hemingwayapp.com to help do this.

6. Don't Bother with Interpersonal Skills

No one is going to say they don't work well in a team. Or, that they're irritable without an intravenous supply of caffeine. So, remove it and speak about how you worked amongst teams in your work experience.

7. Try to Avoid Using "We"

This is a challenging balance. Rarely, as software developers, do we work alone. But, it's prudent to speak about your individual contributions rather than ones made as a group. Using the phrase "we" makes people think you didn't actually contribute much. You can embellish but don't lie.

8. Don't Name Individual Libraries You've Used

Unless the library or framework is something that someone would hire for, don't include it. It doesn't matter if you've used RxJS or Lodash. It does matter if you've used Typescript or React though.

9. Write what role you want in your personal profile

Your personal profile is a great chance to speak about what you want from a role. Remember your audience. If you're applying for a role as a full stack engineer, and you've before worked as a frontend engineer - then write about why you want to move into such a role. It's wise to add why you would be a good fit. For example, a friend was moving into cyber security engineering after being an electrician. They spoke about how their problem solving skills applied across industries.

10. Your GitHub and Website say far more than a CV

If you get passed the first filter, then it's likely that the person interviewing you will want more information. Including your website or GitHub will give them great insight into what drives you, your experience and determination.

I hope these 10 tips help you as you create your resumé. If you need any help with your CV then there are many friendly communities (such as MidlandsJS) that will help you.

My Advice on Become a Software Developer

2021-01-31T00:00:00Z

Whilst running the MidlandsJS meetup, I've been asked a number of times how to "get into" software development and the industry at large. So that I can clarify my own thoughts (and update them), here is my advice.

But, here are two points of advice that I preface my practical advice with:

Don't take any one's advice as gospel - All advice is given from that persons perspective. They are not to be blamed for this, but rather use their advice and experience to plot a course for your own path.
Don't pay too much attention to how people got into the industry - There are a million and one ways to learn software and get a job in the industry. I personally fell into a little "analysis-paralysis" looking at what the "best" way to get into the industry was. The truth is, it doesn't matter. Do great work and work hard and you'll succeed.

With that said, here is the steps I would take if I were getting into software development.

1. FreeCodeCamp

It shouldn't be required that you need to pay for loads of courses to get into software development. FreeCodeCamp is an amazing free resource where I learnt to code myself. They've changed the course since I took it, but the principles are the same. As with all learning, it's easy to power through all the lessons by googling the answers and asking on the forum. I'd strongly advise against this. You'll ultimately learn nothing and be only fooling yourself later on in the process. If you do need to learn quickly then I'd advise skipping over lessons and then coming back at a later date. Try to learn slowly, take notes, build the projects and have fun doing it.

2. Basic SQL and NoSQL querying

A large portion of learning to code focuses on the code part, but in reality the critical part is the data. Once you've got through FreeCodeCamp and got a couple of apps under your belt. Start to incorporate more SQL and NoSQL querying and integrate them into your apps. For example, if you've created a Todo app that stores the data in state, try to port it so it can store data in both MySQL and MongoDB.

If you get great at SQL, you can become an indispensable asset to any company. I recommend setting it up on your own PC and reading sqlzoo.

3. Get a Job

If you've learnt to drive, swim or ride a bike, you know that it's only through hours of practise that you fully master something. Looking back, I'm sure you remember many times where you fell off your bike, got a bit out your depth in the sea or crashed your car (I did, twice 🙈). The initial "learning" stage is where you form a basic understanding. In the same way, once you've gone through FreeCodeCamp and SQLZoo, you won't really know how to code, but you'll know the basics and that's all you need to get a junior programming job. So, don't be afraid to just dive in. Don't wait to "know" a technology or software at large - it's an endless journey.

It's a whole other article to discuss how to pick out a junior job, and how to apply for them. But principally, try to get your foot in the door somewhere, anywhere. It will 10 times more challenging to find your first job than your second. Wherever you land, be humble, learn something from everyone and keep up with building new projects and contributing to open source.

That was the advice I wish I had been able to tell my 17 year old self. I hope you are able to take something away from it. Feel free to contact me via twitter - @joshghent and I'd be happy to help with any questions you have.

Cut to the Chase

2021-01-25T00:00:00Z

Scheduling time with friends and family is even more critical nowadays. But I’ve noticed a trend that goes something like this:

“We should get together some time”
“Yeah when is good for you”
“I’m easy, when works for you”
“I’m easy too, could we do a weekday afternoon”
“Urrmm, I can’t do Tuesday and Thursday afternoons”

You get the idea... I'm sure you've all had conversations like that. We're all guilty of doing this. And what results? Nothing. Arrangements don't get made. In 6 months time, you might send or receive a message that says "oh we never did this? Are you free soon?" and thus, the cycle continues.

But how could we make it better?

“Hey we should get together to play some games, how does Friday at 7 sound?”
“Sounds great but can’t do then, shall we say Saturday at 7?”
“Done deal”

So much better. Why?

📆 You aren’t left guessing about the other's calendar
🙊 Less messages mean you’re more likely to actually book something in.
🤝 You are both assured of the others commitment to schedule something

If you’ve been sent to this website by someone, don’t take it personally! They love you and want to spend time with you! ❤️

Redesigning my Site - Accessibility, Privacy and 100 PSI Scores

2021-01-11T00:00:00Z

So, things look a little different around here. I took the time to overhaul the previous design of my site into something more simple. But why? This post is meta but bear with me.

The three reasons are as follows:

Accessibility
Performance
Privacy

Accessibility

After hearing a talk by Jadene Aderonmu about the WCAG guidelines, I began to include accessibility as a critical part of a website. Often, I had pushed accessibility to the side. Because it was difficult to test and seldom noticed. But accessibility is vital. Recently, an "Ask Hacker News" post asked, "How do I prepare as a developer going blind?". Although a small part, I realised how little notice of a large community I took. My website is only a drop in the digital sea. But, as with the boy and the starfish "I made a difference for that one". I don't have the power to change the web, but I do for my tiny corner of it.

I found my accessibility had many glaring errors - one being the colours I used. The orange (#ff6c2f), used for links and other highlights, didn't meet the contrast guidelines of the WCAG. And, the dark theme didn't meet the mark either. I ripped these out in favour of a simple blue and white theme.

The markup of the site was also lacking. Under the hood, my navigation bar wasn't getting identified as a navigation bar by the browser. I cleaned this up and now have made the site navigational via a keyboard.

I plan to improve my sites accessibility further by scanning all articles for basic metadata like alt tags.

The results so far! No violations - up from a score of 80%!

Performance

My blog had become bloated - over 300KB. For most modern websites being well over 1MB, 300KB seems like a small fry. Who cares right? I took a step back and analysed what this site was - a collection of static HTML pages. On that basis alone, the homepage should not be 300KB - it's text on a page.

I started trimming the fat.

First, I pruned Google analytics and Disqus (which I'll talk about later). Next, Google Fonts and Twitter embed went. Finally, I redesigned the site to remove the dark mode toggle, unused CSS and images. This produced a simple design that focused on the reading experience only.

After all this pruning, I was down to 140KB. There is still room for improvement I am sure.

Screenshot of Solarwinds tools, showing the page weight reduced from 300KB to 140KB

Privacy

The genesis of this redesign was simple. I had decided I wasn't getting benefit out of Google Analytics and Disqus. And, the readers of this blog had their privacy invaded. I decided to remove this, so now my blog is 100% my content.

For comments, readers can now contact me via email. And, analytics I've let go of all together. Ultimately, I'm writing this blog for me. I'd love if someone else got benefit out of this, but if they don't then that's ok too. I've let go of the idea of being some famous super blogger - it wasn't a conscience hope but something in the back of my mind. I'm not interested in optimising my content for clicks or ads. I'm interested in improving my writing craft, whilst building a personal knowledge bank for myself.

Upcoming

This simple design comes along with a principle that I'm now practising in my life - simplicity. I've realised I want to embrace the things I value and discard those that don't. I'm speaking not just of possessions here, but words, apps, health and more. I wouldn't describe myself as a minimalist - but it's along a similar vein. Soon, I'll be speaking more about this practise of "simplicity" and how it relates to me as a software engineer.

Downloading your Favorite YouTube Playlist Automatically

2020-12-10T00:00:00Z

In light of Youtube-DL being taken down from GitHub, I decided to give it a go with a use case I happened to have.

Lately, I've been listening to lots of concerts/festival sets that are not available for traditional purchase. Although I have listened to them on youtube, I didn't want to have the web page open and the auto-play/queuing features are not as fledged out as a proper music player.

I decided to write a quick script that I could run in a cron to pull down the latest version of the playlist I maintain. To keep an eye on it, I included a Slack webhook notification that let's me know when the playlist has been downloaded.

My first version worked but I quickly realised that the script did not handle updates, only a complete re-download. To resolve this, youtube-dl has a --download-archive flag that keeps track of downloaded videos.

Check out the code below and hopefully you find it useful.

#!/bin/bash

playlist_url="https://www.youtube.com/playlist?list=ZZZ"

cd ~/Projects/music

youtube-dl --embed-thumbnail --download-archive downloaded.txt --no-post-overwrites --extract-audio --audio-quality 0 --format bestaudio --audio-format mp3 --yes-playlist --output "%(title)s-%(id)s.%(ext)s" $playlist_url

curl -X POST --data-urlencode "payload={\"username\": \"youtubeb0t\", \"text\": \"Completed download of all Music playlist.\", \"icon_emoji\": \":tv:\"}" https://hooks.slack.com/services/ZZZ/YYY

Using RDS Snapshots

2020-09-21T00:00:00Z

Recently, I had a case where I needed to gain access to an RDS Instance that I had long since deleted. To add insult to injury, the Bastion host to gain access to the Database server had also been deleted and it's VPC, Security Groups and all the other architecture components! Yikes.

Fortunately, I had taken a final snapshot before it was deleted. Phew!

I thought it was as simple as downloading the snapshot, which I assumed was just an SQL Dump, import it into a local Database and then bobs your uncle. Not so fast... RDS Snapshots aren't just database dumps, after all that would just be too easy! Instead they are bespoke AWS format that is not able to be parsed and imported with a tool.

Despite there being documentation, it's fairly lacking so I thought I'd write a guide on how I gained access to the data within an RDS snapshot.

1. Create a new VPC

The first step is to create a new VPC. Name it what you like and give it any old CIDR Block. Leave everything else standard

2. Create a Security Group for the VPC

Next, you need to create a security group for the VPC you just created. Simple add a name, then select your previously created VPC you created in step 1.

Next in the Inbound Rules pane, create an new inbound rule to allow traffic on Port 5432 (the PostgreSQL default port). The source should listed as "Your IP", which will prefill it on the current IP address you are accessing the AWS Console on.

3. Restore the RDS Instance

Now you've done that, go to RDS and then Snapshots. Find the snapshot you're looking to get data from and click "Restore snapshot"

This will send you to a configuration page for the instance you are about to boot. First, change the instance type to something cheaper because the default is something more powerful than Deep blue and costs about as much as a mission to the moon.

Afterwards, change the VPC that the instance is in, to the one created earlier and change the security group to the one created earlier. In my case "snapshot-access-vpc" and "rds-snapshot-sg". Additionally, select "Public Access".

Now hit Create and your instance will launch with the snapshot data.

Hey Presto!

You should not be able now connect to your RDS instance using the details on the Connectivity Pane when clicking on your RDS instance in the list. Hopefully you found this helpful!

Make sure to tear down this access after you finish using it as it is by no means secure. It should purely be used for grabbing data retrospectively if a client requests it.

Improving Koru's API Performance

2020-09-21T00:00:00Z

If you don't know already - I love performance. It solves a genuine frustration for users and provides meaty problems to sink your teeth into. Koru's API performance was a little lacking since we had added a lot of features, rewritten large swaths of code and generally not thought about it, since our system is not dependant on returning results quickly. Never the less, we chose to do some performance improvement primarily aimed at reducing database load. This work was in preparation for moving to a multi-tenant architecture. When the move to a central database cluster is complete, reducing database load for each endpoint will be critical to ensure that we continue to have acceptable response times.

This work is still largely on going but here I'll discuss some of the foundation work we have done.

Gathering Data

Firstly, we started by collecting data on query performance in our system. We don't use an application performance monitoring, but this is simple enough by attaching onto the receive event on the PG Promise constructor. We then want to report queries that take over 500ms into Slack so that we can then focus our attention on them.

const pg = pgPromise({
  // {... other options }
  receive: async (data, result, e) => {
    winston.debug(
      `${e.query} received ${result.rowCount} row(s) in ${result.duration}ms`
    );

    if (result.duration && result.duration >= 500) {
      await Slack.reportSlowQuery(e.query, result.rowCount, result.duration);
    }
  },
});

This sends the query, the amount of rows it returned and the duration of the query to a Slack webhook which then produces a message like this:

Ok, so now we've got a steady stream of slow queries coming into our system! 🎉

Optimization

Quickly, we began to see a pattern. Most of the queries that were reported as slow contained two things:

A WHERE clause on the clients ID - added for prevention of accessing other clients resources
An ORDER BY of a property within a JSONB object

The first was a fairly easy change, simply add an index on the client ID column.

CREATE UNIQUE INDEX IF NOT EXISTS client_id_key ON clients USING BTREE (client_id);

I'm still unsure why we hadn't got an index here before, but I put it down to the fact that there was very little in the way of cross-client security. It was as if the system said "well you're authenticated so have what you want!". Thankfully, we have long since added these protections.

The second was a little more of a challenge. Initially, I began by looking up if it was possible to create an index on a certain property of a JSON structure or if you instead had to create just an index on the entire column.

Soon I discovered it was possible! But there was a lot of conflicting advise as to whether it actually improved query performance or not and if the building of the index netted a performance negative.

Anyway, I decided to create the index like this

CREATE INDEX candidate_response_score_indx ON responses((data->>'score'));

This took down the average query time from around 885ms to 7ms 🚀

Conclusion

There is still many more queries to improve the performance of but this was a great start. It taught me a lot about the internals of Postgres and experimentation.

Takeaways

Learn but don't be afraid to experiment - there becomes a point with learning that the best way to see if something will be a positive change or not, is to just try it out
"Reckons" don't always reflect reality - I believed there was lots of performance improvements to be made in a number of common operations but after trying the queries out and collecting data, they were already very performant. Therefore, be more data driven rather than "reckon" driven, even if those assumptions point you in the general direction.

Writing Useful Error Messages

2020-09-04T00:00:00Z

Server-side developers often overlook the fact that they are not immune to considering user experience in the design of their systems. Part of this is consideration around error messages. As I spoke about in my article about crafting quality health checks, people don't get angry when something doesn't work, but rather when something doesn't work for some inexplicable reason. In server sided systems, we explain those reasons through the form of error messages returned via an API, or some other mechanism to get a piece of text explaining the issue back to the originator of the request. But how can we write useful error messages? Here are my 5 tips.

Make them actionable

Actionable advice will always be helpful for your users. But don't just tell them, provide a means for them to take that action that you are suggesting. As an example, let's say your application requires the email address to be verified via a unique link sent to said email address. In the case where the user logs in without verifying the email address, you could return an error message like this. "Please verify your email address joe.bloggs@example.net to continue". This might seem perfectly sensible and even actionable advice. But to go a step further to make it 'great', we could also include a button to "Resend the verification email" because chances are, the verification email you sent originally is long gone into the rabbit hole of that persons inbox. By providing a button to, in part, perform the action you require, it will be more likely for the user to follow through with the action.

In other cases, it might not be possible to partially perform an action that your system requires. In these cases, be as clear as possible about what the system would need to work - even if it requires multiple steps. For example, you might have an API that requires parameters "A", "B" and "C". But the caller only passes in parameter "A". What should the error message be? In most cases, it would probably be: Please specify parameter "B"

The user would then dutifully add parameter "B" and rerun the request. Only to be given another error: Please specify parameter "C"

At this point, keyboards will be broken, fists will be clenched and Irish coffees will be set to brew - not the response you want from a user of your service.

Let's rewind and see if we can make the original error more actionable - the user has not passed in two required parameters for our API

{
  "errors": {
    "B": {"type": "number", "required": true},
    "C": {"type": "string", "required": true}
  },
  "documentation": "https://kryptoking.com/v2/bitcoin#parameters"
  "message": "Please specify parameter "B" of type Number and parameter "C" of type String. Please see the documentation at https://kryptoking.com/v2/bitcoin#parameters for more details"
}

Ok so this is a little more than a message, but still is included in the overall error response. What we have here is a concise breakdown of exactly what the caller of the API needs to do to perform a valid request, as well as being provided a link to further documentation if required. Additionally, the errors object contains a payload that is easily parsable by a computer to perhaps automatically correct errors on the fly without the developers intervention.

In both examples, it was clear what the user needed to do next and that's what you should aim to achieve.

Remove internal/tech lingo

This is one particularly for services consumed by non-technical audiences, who may be unfamiliar with terms we as developers use everyday. We need to use language that, as Lego would put it, can be used by ages 4+. For example, if a users authentication token is invalid, rather than displaying an error "Error: JWT has expired". We should sign the user out automatically and display a friendly messages such as, "We signed you out because we weren't sure you were still there! Please log in again using your account details below". Same error, different angle.

I am always on a crusade with whatever I work on, to remove terms from systems that would require a dictionary to understand. You should aim to do this in any messages that get displayed to users as well. Otherwise, further to point number 1, it would not be as actionable as it could have been.

Get rid of them!

The best error messages are the ones that don't exist at all. Although you don't want to be burdening your system with translating a million different data input types to the ones you want, I'm almost sure that whatever system you work on, there is a way you can automatically correct an error without the user knowing. Maybe it's a default value you can assign, maybe it's a common data type that users mix-up, whatever the case, take a look at calls to your system that return errors and find patterns in them to seek out ways you can improve the system or the documentation.

Don't blame the user

One of the worst things you can do in error messages, is pin the blame on the user. Avoid using terms such as "Your" or "You". In an example we used earlier we spoke of a system that auto logged you out if you were idle for too long. This could be blamed on the user - e.g., "You were idle for too long so you were logged out. Please log in again to continue". It contains actionable steps, it's not robotic and there was no tech lingo. But it makes the user feel like an idiot because of a constraint that the system built. Don't blame your users, blame yourself if anyone. I've seen no end of error messages, especially unknown ones that say words to the effect of: "Something went wrong in our system. Our engineers are working hard to fix it. Please site reference number: 12345 to our support channels if the issue persists".

Be empathetic to your users and use it as an opportunity to improve your service, maybe your documentation needs clarifying or maybe this is an easy mistake to make (misspelling of a parameter perhaps). Attack these problems, not your users.

Be direct but not robotic

"Invalid Password Provided", "This field has an error" and "Bad Request: Unexpected end of input" are all error messages I've had in the past, maybe you can remember some equally ridiculous yourselves. What's wrong with these messages? Well asides, from the fact that they are in-actionable, they also use a tone that might be preferred by a Terminator T1000 but probably not by the "humans". Robotic language is rife in all kinds of messaging, but especially errors - the idea being that by being robotic, it's clear and direct for the person to understand. But in reality, it comes off as distant, cold and frankly useless. Inject some humanity and a bit of fun into your error messages. Psychologist have studied and found that a great way to instantly lift your mood is to simply smile. Smiling is contagious - if we add fun and humanness (if that's a word) into our tone of writing, then even though a user has an issue and maybe slightly annoyed, it will be impossible to maintain that position by being so friendly. Kill them with kindness so to speak (but maybe avoid killing your users because that would make you a terminator).

Conclusion

Hopefully you've learnt some new tricks to help you craft great error messages. As an additional hint, try to include this as part of your code review process where you scan any external (or heck, even internal) facing messages and analyse them for quality. If possible, get someone unfamiliar with the system to review the messages so that they can be understood by all.

Building Awesome Application Health Checks

2020-08-28T00:00:00Z

For many, having a health check in your application may be somewhat of an afterthought. Maybe your application does have a health check, but you've got no idea if it actually works. I didn't give it much of a thought until I truly understood that one of the key pillars of good development is monitoring. And to be able to monitor a service, you need to know if it's up, down or fallen down the stairs.

Application health checks can be useful not only for internal usage but also external and can aid with supporting customers. Take a look at many large companies and they all have public status pages (especially developer focused ones). Aren't they embarrassed whenever they go down? Perhaps, but it's a natural part of all systems, and the benefit to them is that customers can easily see that they are aware of the problem and are actively working to fix it. Otherwise, they'd be crowding support channels and generally getting frustrated.

So health checks are important, but how can we create one?

If you have a health check in your application, chances are it might look something like this

import express from "express";

const router = express.Router();

router.all("/health", (req, res) => {
  res.status(200);
});

Ok! Good start. In this basic example, this express API is telling the caller of the /health endpoint that it is up. That can be important information to know, but fairly redundant in some cases. For example, if you host your application using AWS Lambda, Azure Functions, or whatever the GCP one is, then, in reality, all this endpoint is doing is telling you that the cloud provider has not gone down - which is far less likely that your application being unhealthy.

Define "unhealthy"

A good place to start is to define what unhealthy means for your service. In other words, at what stage will your core application become unusable to the point where your customers will react something like this.

Nowadays, systems are never usually completely "down" but rather have failure points across a range of services. For example, if you run a microservice architected infrastructure then perhaps one piece of functionality will not work but the core of the application will. Your health checks and your messaging to customers needs to reflect this.

At Cappfinity, we depend on a number of services to run a pipeline for customers and their candidates alike. Our "unhealthy" state would be when our scoring provider is down, our API and Database are unreachable or when our authentication provider is down. If any of these three components go down then we would not be able to deliver a core service to our customers. Depending on your business, you may have more or less components that you critically depend on.

As an aside, try to look into each one of these components to see if you can deliver a reduced experience if they are unavailable. Netflix do this to great success and you can read more about that here: https://netflixtechblog.com/making-the-netflix-api-more-resilient-a8ec62159c2d

Improving the health check

Ok so we've defined what "unhealthy" is, so now we need to build the health check itself.

First, let's dive into each of our critical components and add an endpoint or monitoring around if they can connect to all of their dependant third parties. For example, if our API depends on a Database and an authentication provider then we might have the following

import express from "express";
import Database from "./lib/database";
import Auth from "./lib/auth";

const router = express.Router();

router.all("/health", async (req, res) => {
  const db = await Database.connect(); // check we can connect to the database
  const auth = await Auth.getUserById(1); // check if we can get a random user from our auth provider

  return res.json({
    db: db ? "healthy" : "unhealthy",
    auth: auth ? "healthy" : "unhealthy",
  });
});

Great, so now we're able to tell any caller of this endpoint that:

The API is up
The Database is up
The authentication provider is up
The API can connect to the database
The API can connect to the authentication provider

The last two points are key, because although you might be able to create a new service that goes and pings these services to see if they are up, you are not telling the caller if those services can be reached. Often, firewalls, static IP's, bad credentials, and other application issues can prevent a service from connecting to another. So it's important to test the connectivity from which ever service relies on it.

Internal vs External

That health check endpoint is exactly what developers need to assist with debugging a service. However, saying that your database is down to a customer is not really relevant and may even confuse them. In all reality, if you are building a customer facing experience for your service status then you need to think about how to communicate what the impact for them is.

For example, take a look at Slack status page

You can see that it doesn't contained detailed break downs of information, but rather individual components of a system that a customer will be familiar with - messaging, login, notifications etc. You should follow a similar pattern, if your system is an API that delivers Woody Harrelson's face as placeholder images for frontend developers, then perhaps you'd have the following services you can report on:

The API
Harrelson import system - for importing a continuous feed of Woody Harrelson's and cropping out everything but the face
Harrelson's calculator service - the service for getting the correct Woody Harrelson picture based on the size requested by the caller

You can then report those data points to a frontend status page whilst keeping your in-depth health checks that are only accessible by your developers internally.

There we go! Ready for production!

Resistance

In the past, I've seen some companies be hesitant about revealing in depth information about the health of their systems. If you face this kind of opposition then you can reason that the company will benefit for the following reasons:

The company is transparent about it's successes and shortcomings which appeals to people and businesses
The company is proud of great engineers debugging systems and fulfilling any contractual SLA's
Developers who consume your application don't have to ring your support line to get help if the system is down
You're following in the footsteps of some of the largest companies in the world - Netflix, Slack, Stripe, Apple etc.

Overview

I hope you've learnt something and can go out and improve your systems health checks! Please feel free to share with me any of your tips for creating great health checks in the comments below. That's right! My pocky little blog has comments. It's right posh round 'ere now.

ATS Resiliency

2020-08-24T00:00:00Z

As with all modern enterprise SaaS platforms, Koru has a mechanism to send data about a candidates results back to a clients ATS (Application Tracking system) or CRM (Customer Relationship Management) system. Since Koru was event driven and the traffic is not consistent, the decision (which pre-dates myself) was made to make these "ATS Integrations" lambda's triggered from DynamoDB.

So the set up is this, when a candidate completes an assessment on our platform, it goes through all the pipes and tubes, passed Gerald the field mouse that generates the scores and then finally back into our API when the completion of the scoring is recorded. Now we want to get this completed candidate data to our clients system, so our API creates a new record in the DynamoDB table for that client's ATS with all the information it needs. This then triggers the lambda that sends the data to the ATS of choice. Sounds easy right?

I thought as much, but found there were a number of holes that the original developers fell into when developing this software that caused huge resiliency issues. I thought I would share my findings to all those trying to do the very common task of "submit data via HTTP to a 3rd party". In our case, the third party API is the Cappfinity external scoring mechanism which is built by another team at the company which allowed us to work closely with them. However, most fixes were addressed on the Koru side.

1. Lambda was suffering from "socket exhaustion"

This was the biggest problem we had "socket exhaustion". I hadn't stumbled across the term apart from a joke I'd seen on IRC, so it took me a while to discover what the problem actually was and why it was happening in our system. What we observed was that the Lambda function would fire a request for each record that we hadn't marked as "processed" in our system (It is considered "processed" when we get a 200 response from the 3rd Party API). Often this could just be one request, which would succeed. But sometimes, if records had failed over time, then we would build up this backlog of pending records to be sent to the third party. In these cases, the first few would succeed but quickly we'd get the error "ETIMEDOUT" or "ESOCKETTIMEDOUT".

A quick google of these errors lead me to discover that Node goes to look up the DNS the URL for each request it makes. After this, it makes the request to the URL you've specified. Often if you have multiple requests trying to process in parallel, and the DNS lookup for a few of them takes a bit too long then the program can't keep up with trying to issue new sockets to make requests on. Uber talks more about why it occurs over here.

So we knew the the problem was that it was trying to process too many requests in parallel. To the code! Immediately, I spotted the problem.

return new Promise((resolve) => {
  Promise.all(
    records.map(async (record) => {
      const payload = buildPayload(record);
      try {
        const response = await processPayload(payload);
      } catch (e) {
        await markRecordAsFailed(record);
      }
      return Promise.resolve(payload);
    })
  ).then((result) => resolve(result));
});

See the problem? Well there are a couple...

processPayload is an async function but an Array.map with an async callback will appear as valid javascript but will in fact not truly await functions contained within it
Because of not being awaited successfully, the array will quickly loop through and call the processPayload but not listen for the result and therefore max out the sockets

It's worth noting that this code was used in production for many months before we got these kinds of issues, in response to increased load.

What was the solution here? It turns out to be fairly simple, rewrite it as a regular for loop.

for (let i = 0; i < records.length; i += 1) {
  const payload = buildPayload(records[i]);
  try {
    const response = await processPayload(payload);
  } catch (e) {
    await markRecordAsFailed(records[i]);
  }
}

The code above means that we submit a request to send the data to the 3rd party one at a time.

2. The 3rd party was getting overwhelmed with requests

After we deployed the code above, we now found that the 3rd Party API was getting hammered with the amount of requests we had to process (even though they were one at a time). Although this sort of seems like "not my problem", I chose to issue a fix for this. The solution was to simply limit the number of requests that we sent too 100 per "run" of the lambda (i.e., we would only process a maximum of 100 records each time the lambda got triggered which was for all DynamoDB inserts into the table)

3. Use Https agent with a pool

We added a further fix to the problem above by creating our own Https Agent and passing that into the request-promise libraries options.

const agent = new https.Agent({ maxSockets: 25, keepAlive: true });
await request({
  options: {},
  agent: agent, // use the global agent
});

This meant that we would share the agent across all requests that were made from the program rather than creating a new one per request. This again prevented socket timeout issues as well as slowing the rate that we sent requests to the API.

4. Cache JWT tokens (or any other authentication tokens you are using for the life time of the request)

A few months after we deployed the service, the Cappfinity team called us up to say that we had maxed out their M2M Auth0 Tokens. Oops. When digging into the code, I found that the token was not only generated each time we processed records, but for each individual record itself. Double Oops. I was hesitant to cache JWT token's because it just feels a bit wrong and has the potential to become invalid in the case of rotated secrets and so forth, but we went ahead with it anyway. We cached the token by storing it in another DynamoDB table - the mix of cheap storage combined with quick and easy access was hard to compete with. In the program, each time it boots, we check the table for a token. We then check if that token is valid. If it is then we can proceed as normal. Alternatively, if it's not valid then we can regenerate it, and update it in the table.

5. Don't submit data to the 3rd party when it is down

Systems go down, that's a fact. Or maybe a deployment breaks everything. Either way, we should have some mechanism so that if we mark a candidate records as failed to be processed, then we can be sure this is a data issue rather than an issue with any part of the underlying system. There are many ways to do this, but without working too heavily with the third-party API, we were able to implement a basic sanity check. I asked if they had a health check mechanism, which they did. I asked to look at the code and this is what I saw (roughly)

const router = express.Router();
router.all("/health", (req, res, next) => {
  return res.send(200);
});

Yup, that's it! I'll dive into how to build great health check mechanisms in another post, but suffice to say, this doesn't give an accurate representation of a systems health. Nonetheless, we implemented a system that would first ping the health check system to see if we got a good response back. We created tickets to get this health check improved so that we can have more confidence in this check going forward but this is a good first start. Additionally, this basic check would have saved us in the case of an outage the Cappfinity API had for 2 days. In that time, Koru was firing requests at the Cappfinity API, getting no response and failing them accordingly. This is correct behaviour but was such a headache to get them to submit again when the API was backup so this health check will prevent those sorts of situations.

6. Filtered out DynamoDB update, delete and modify triggers to reduce retries

I said earlier that the Lambda was triggered for DynamoDB inserts, but as of August 2020, there is no way within AWS itself to be this specific. So how did we do it and why? We implemented this by looking at the eventName property on the event that is passed as the first argument to the Lambda function. This event name property, for DynamoDB events, signifies what type of event it was. When a record is updated, it's MODIFY, when a record is deleted it's DELETE and so on. The one we are interested in is INSERT, which occurs, logically, when a record is inserted. It was then a matter of checking the first records eventName like this

const triggerType = event.Records[0].eventName;
const acceptedTriggers = ["INSERT"];
if (!acceptTypes.includes(triggerType)) {
  return {
    message: "Invalid trigger for this lambda",
  };
}

But why did we do this?

What was happening was that as part of our system, we update the dynamo table to indicate how many "retries" a record has attempted or if it's been processed or not. This updating meant that each time we processed a record it also triggered the lambda, so it just spun in a circle continuing to retry, over and over again. Pretty hilarious really. By cutting down the lambda to trigger only on inserts, we not only saved money, but also made it more efficient. Additionally, we only wanted the lambda to trigger for new records and retry old ones and this bit of code achieved that purpose.

7. Special error cases

We worked with the Cappfinity team closely on this one. We found that in some cases, somehow, records were in the database ready to submit back to the ATS, that shouldn't have been there. These were the cases we were trying to handle:

Straight up invalid records
The candidate hadn't actually completed the assessment
We had already sent data but had not marked it as processed in our system.

To solve this problem, we needed to introduce some new status codes so that we could categorize the errors we got back. Previously, if we sent an invalid request of any kind, it would return a 500 error to us. This categorization of errors meant we could handle each one uniquely. For complete invalid records, this meant just deleting them. Whilst for duplicate submissions, we marked them as "processed".

After we deployed this change, we let it chew through a huge backlog of records that were marked as unprocessed and soon we were down to 0 records marked as "unprocessed". Great success!

Overview

I'm really happy with how solid and robust the system is now. There are still improvements that could be made but that's the way with all things. I hope to have this system taken off my list of "most modified" into a pile of systems that "just work"(tm). None of these techniques are new, but the key is to take the time to identify first why things happen. This system resiliency work too me to places that felt out my comfort zone and put under the microscope the gaps in my knowledge - namely, how NodeJS actually works under the hood so to speak and networking principles. I've since taken a couple of courses and watched a number of conference talks on these very topics.

Takeaways:

Use your work to identify gaps in your knowledge
Don't assume - look for the why's and measure to test the hypothesis

Rebuilding a Monolith

2020-08-20T00:00:00Z

It's no secret that microservices are the hotness right now (I won't say new at this point because they're fairly well established). But you only have to Google the phrase "Why not to use Microservices" to see a treasure trove of articles about why to be sceptical of this pattern. As with all things, in some instances it's the "correct" approach and in other's, it's not. I'm not entering into a holy war of which is better, nor am I a Basecamp hipster who despises frameworks. This is a story about why and how we collapsed a few "microservices" into our central API.

The Setup

Koru is a pre-interview assessment filter. That sounds wordy but basically it's designed for large corporations who get thousands of applications to jobs, and this tool aids companies in seeing what different strengths a candidate has based on an assessment. The product is fairly simple infrastructure wise. An event-driven microservices architecture with a central API, a React frontend for customers and a admin interface for employees. In addition to that we have systems to generate PDF's and send emails that are part of the candidate lifecycle through our system.

What we wanted to combine was the following:

Our admin interface and it's API
Our internal tooling API
Our internal API documentation ...into our main API that serves customers and their candidates.

It's worth noting that these were all separate repositories.

Why?

The reasons for doing this were simple

Our admin portal was incredibly buggy and since we had moved over to a new assessment provider, our API no longer provided us with all the functionality that we needed. Additionally, it had no tests suite, or documentation. The biggest reason however, was that Koru runs (currently) as a single tenant model. In other words, each client gets their own API and DB instance. The problem with this is that the admin API needs to then connect to each customers database. And because we're not insane that means we have to create a VPC peer with each individual customer database from the admin API. Not only is this complex to maintain and debug but also a pain to update. Furthermore, it was hosted on it's own EC2 instance which meant that it was costing us a reasonable amount of money. This would be fine if it was high usage but it was barely ever used.
Our internal tooling API had the same issue as the admin API, we wanted to keep things simple from a "what has access to the DB" point of view so we chose to collapse this one as well.
The API documentation was something that we wanted to be constantly updated along with new changes to the API. To make developers go to a whole new repo, make a whole new PR and such meant that it seldom got updated. We chose to combine it so it can be referenced as part of a Pull request change to any code.

How we did it

As all things should be, we tackled these one at a time and released each one of them so we didn't have any confused boundaries about where large swaths of code got introduced.

The documentation The documentation site was written using Slate, which is a Ruby based project. To add this to our NodeJS/Express API, sounds painful but all we did was put it in new folder 'devdocs' and treat it as it's own separate project. Next we added a GitHub Action (Our CI/CD tool of choice) to build and deploy the 'devdocs' to the S3 bucket where they are served from.

Well that was easy

Right, next... 2. Internal Tooling API Since this was a new service written only a few months prior, it was already similar to our main API in terms of the patterns used. Additionally, it was written in Typescript so the code could quite literally be copied across. We took the time to fix up some tools that were broken and created new ones. Additionally, we added a few simple sanity tests for each one so that we could verify they worked.

Admin Portal This is where things became a bit tricky. As stated previously, one of the main reasons we wanted to merge these API's was because the current API didn't really serve the needs of the system, we were sort of bending it to our needs and in a number of cases, it snapped under the pressure. We embarked first on a mission of discovery, what were all the things that it did, why did it do them, and how did we want to improve them.

The API was written in Python using Flask as this is what the previous team had mostly used. But since there was little expertise within the team for Python, we decided to rewrite it in Typescript. One of the main improvements we wanted to make was to merge the creation of different records in various tables into one easy to use endpoint. Previously, each individual section when creating anything (like a study, survey etc.) had to be saved individually. This lead to confusion from users and confusion when debugging issues where some data was relied on by another part of the system. By combining them, we made the save process one simple API call, and more importantly, one set of errors to deal with. Along with this merge, we added new tests, both unit and integration so that we could verify the integrity of this process. We also made plans to improve the service further by adding tools for Koru employees to be able to "Copy" existing studies to make them easier to setup.

The difficult part with this merge was authentication. In our main API we had no mechanism to authenticate requests only from Koru Employees. Thankfully this was a breeze, because we use Auth0 to handle authentication. We simply wrote a new middleware handler that reads from Auth0 based on the JWT req.user.sub field and checks certain properties of that user in Auth0's system.

The interface was built in React but although not inherently bad, it was clearly coded in a bit of a rush. Things over time had been hacked in as new features came along so we took the time to clean this up (as the boy scouts that we are) before merging it into our central tooling interface. Again we added tests to make sure that the interface loaded the data correctly and behaved the way we expected.

Conclusion

Overall, I'm really happy with "the big merge", because it's meant that we've created a useful resource for Koru employees and they can self-serve for whatever customers ask of them. Additionally, it will be easier to add documentation for the API and consolidate that back to a single work ticket. On top of this, the main Koru API is now a complete CRUD API rather than having bits of "R" and "U" with "C" and "D" being all the way over in a different service! Hooray!

Re-architecting our PDF Generation

2020-07-31T14:06:03Z

When I joined the Koru team, one of the biggest issues with our pipeline was the PDF generation system. We have PDF's in the first place as a nice report that customers get about each of their candidates that takes an assessment on our system. These PDF's are saved to S3 and then we send the customer a signed link to download the PDF.

The major problem with it however, was that the PDF system ran on two big expensive m4.large box (~$74 a month each!) and then there was another one for our QA environment. You wouldn't mind paying a premium for the machine if the service worked but it constantly failed to generate PDF's for unknown reasons. It was one of those services that was just a black hole to us. I don't like having any black holes in systems that I'm charged with running and maintaining. It pays to be extremely knowledgable about the system (within reason of course).

So we've got the classic, too expensive, doesn't work and a "known unknown". What would you do?

How the PDF system worked historically was to open the web page in an instance of Chrome and then "print" it to a PDF. This was extremely memory intensive and the box could only handle a couple of PDF's at a time. However, there was no mechanism to rate limit itself. Therefore, if we fired five requests at it, then the box crashed and it had to be manually rebooted. Although we tried a number of ways to fix it, what we really wanted was something that was event driven so it could fail and manage it's own processing rate with ease. Since the application was hosted on EC2, we couldn't configure it to trigger from SQS without additional overhead, so we decided to rewrite.

The first MVP was a Node Lambda that called a service called Api2Pdf - a simple API-as-a-service platform that, given a URL and some parameters, creates you a PDF. We then saved this PDF Buffer to our own S3 bucket and then deleted the one that they had just generated. Unfortunately, we had a number of bugs with this system. Because the web page that we took a PDF of often took up to 700ms to load, the PDF was being generated before any of the content had loaded. It was seemingly only an edge case though. The obvious fix for this was to add some kind of CSS selector that the API would wait for before generating the PDF - but this was not possible at the time with Api2Pdf's API.

So the next step, was to add some protective validation that would check if the PDF had any content before then being accepted. If it was blank then an error was thrown for it to be retried.

This worked for a couple of weeks, and then we got some more errors... This time, it had generated the content but the underlying data hadn't yet loaded it. This meant it passed the "blank" check but ultimately hadn't loaded correctly. So I got working on another fix, my first attempt was to take a sample and compare the Buffers and see if they were the same. However, there were edge cases that this could not handle. I then found PDFJS from Mozilla which had a handy feature to be able to get the text of a PDF. I decided to compare the generated PDF's text with a sample I had taken.

Again, this appeared to work for a few weeks - perhaps too well. Although still ultimately had the downside that if it failed we would have to regenerate the PDF and were charged in the process for doing so.

However, after an API release, it appeared as though the page rendering the PDF had slowed down enough that the amount of PDF's getting generated incorrectly had increased.

At this point, I decided I was done with hacky fixes and working with a million libraries not updated in ages - seriously with the amount of NPM packages out there I would have thought there was something decent for PDF's. It was time for the big guns - it was time for Puppeteer.

As you may or may not know, Puppeteer is a beautifully designed API for working with Chromium. It has specific functions for screenshots, modifying the page, etc. But most importantly for us, it could generate PDF's.

Porting over to the API was relatively ease and didn't require a whole lot of modification. What did, was getting it deployable to serverless and AWS Lambda.

Because Puppeteer contains the entire Chromium browser in its install, it vastly goes beyond the size limit for Lambdas (50MB Zipped, 250MB Unzipped) The solution was to use a Lambda layer to contain the Puppeteer stuff and then just the application code in the lambda itself. I used a lambda layer taken from here.

I had to modify my serverless config like so:

layers:
  HeadlessChrome:
    name: HeadlessChrome
    compatibleRuntimes:
      - nodejs12.x
    description: Required for headless chrome
    package:
      artifact: layers/chrome_aws_lambda.zip

functions:
  generate-pdf:
    provisionedConcurrency: 1
    description: Generates PDF from HTTP calls or SQS messages.
    handler: dist/handler.handler
    layers:
      - { Ref: HeadlessChromeLambdaLayer }
    events: ...

Next, I had issues with the PDF working fine, but not displaying any background colors. At first I tried doing the emulatePageMedia function and passing null so that it would not use print based CSS code, but to no avail. Finally, I managed to find a helpful fix on StackOverflow here.

html {
  -webkit-print-color-adjust: exact;
}

I slumped back in my chair after this and breathed a sigh of relief. That was it... or so I thought.

Turns out, in my haste I had set the option of waitUntil: networkidle2 in the .goto() function for Puppeteer. Basically this function is called so that the headless browser will navigate to the page provided. It returns a Promise of the page when the page has loaded. So, it can be awaited until the navigation has finished. But what is considered "loaded" anyway? Well thankfully, Puppeteer takes an array of special strings that tell it what it should consider "loaded". NetworkIdle2, according to the documentation, works as follows: consider navigation to be finished when there are no more than 2 network connections for at least 500 ms. Now I'm not sure why I did this because in hindsight that was bound to go wrong. If the network request that loads the data takes more than 500ms then that means it will consider the page loaded before the content is present. Therefore, we changed it so it waited for networkidle0 which is the same as networkidle2 but waits until there are 0 network connections for 500ms. Additionally, for added safety we added a 5 second "sleep" before it takes the PDF. This probably isn't needed any more, but we wanted to be extra cautious because of all the customer problems that this system had caused.

This was the final piece in the puzzle. For real this time! It was finally stable. It was quite a journey and forced me to conquer some interesting problems with tonnes of various answers that had worked for some individuals. Additionally, we expanded our internal tooling with this project which will help us going forward to resolve customer issues. So overall, despite the chaos, it was a win win. On reflection, I wish we had gone straight to Puppeteer but hindsight is 20/20. In all honesty, I was afraid of it. I saw it as an absolutely mammoth task, having nightmares about trying to implement it years ago at a web dev job, but it was shockingly simple to implement. It's taught me to take a better look at the options and not hold onto opinions from time gone. Technology moves to fast to do so.

I'll conclude this article with a quote that I believe applies here: It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.

Preserving Links whilst Migrating Domains with S3

2020-07-31T00:00:00Z

Domain migrations can be fairly simple things, change the CNAME and bobs your uncle. Difficulties arise when you have two different website systems and existing paths that you want to preserve and the site is run statically with no web server. The latter was the situation we found ourselves in.

The Problem

The product I work on, Koru was purchased by a company called Cappfinity. Koru had a website https://joinkoru.com and Cappfinity had https://cappfinity.com. To consolidate the offerings, the websites were effectively merged.

The setup for the Koru site was as follows:

A static web app hosted on S3 on a subdomain of joinkoru.com
Route53 for DNS
A wordpress site hosted externally for the "main" website
Cloudfront for all SSL and edge distribution

We wanted to redirect *.joinkoru.com to cappfinity.com/koru whilst preserving the paths for the user. Ordinarily, we would have just used an Nginx or Apache rewrite rule. That would have given us much more granular control than the solution we arrived at, but we didn't want to host a whole new web server for the sake of some redirects.

S3 To the Rescue!

We eventually discovered that you can have "Routing Rules" in S3 Buckets. I quickly got to work creating a new S3 bucket. Then under Properties > Static Web Hosting I configured the following Index Document: index.html - this doesn't need to exist, the form element just needs to have content.

The magic happens in Redirection Rules which takes a horrendous XML type configuration. We set it up as follows:

<RoutingRules>
  <RoutingRule>
    <Condition>
      <KeyPrefixEquals/>
    </Condition>
    <Redirect>
      <HostName>koru.cappfinity.com</HostName>
    </Redirect>
  </RoutingRule>
  <RoutingRule>
    <Condition>
      <HttpErrorCodeReturnedEquals>403</HttpErrorCodeReturnedEquals>
    </Condition>
    <Redirect>
      <HostName>koru.cappfinity.com</HostName>
      <ReplaceKeyPrefixWith>404</ReplaceKeyPrefixWith>
    </Redirect>
  </RoutingRule>
</RoutingRules>

So what's happening here?

The first part is the routing condition. For us, we have made it so the KeyPrefixEquals blank which means anything. This handles all the main redirects for existing content. In other cases, you may have it so https://olddomain.com/blog goes to https://newdomain.com/newblog. In that case, you would set the KeyPrefixEqual as <KeyPrefixEqual>/blog</KeyPrefixEqual>.

The second routing rule states that if the document that we redirect to returns a 403 (meaning forbidden) then we will redirect to the same host but replace the Http Error code with 404 so it will go to their Not Found page correctly. This 403 error was happening in a subset of requests due to content not being present in the new system. We left it in for historical reasons.

Subdomains to Suffix's

You'll notice in the rules above that the redirect is not to cappfinity.com/koru but rather to koru.cappfinity.com. I forget the reason exactly for this because we could have used the <ReplaceKeyPrefixWith> tag to redirect to /koru. Nonetheless, we didn't for whatever reason (I hunted my emails down for a answer but couldn't find one). Instead, on the Cappfinity website side, which was newly hosted in Netlify, koru.cappfinity.com was setup as a domain alias and a CNAME was configured for that subdomain to redirect to the Netlify Application.

And hey presto! We were done. Overall, it was a fairly simple migration but used a solution I didn't even know was possible with S3!

Why Backwards Compatibility is Critical

2020-07-03T00:00:00Z

Backwards compatibility is not something I see discussed much in tech circles. It's all new-new-new, fast-fast-fast. Piling features on top of one another and tightly coupling releases between services. Previously Facebook, the 4th most popular site on the internet no less, had the mantra of "move fast and break things". These are the kinds of sentiments I see all around me, particularly from startup and SaaS companies. I've always felt that coupling releases too closely was insanity inducing and seen first hand how corrosive they were to a customer experience of the product.

But the web hasn't always been like this. The core backbone of the internet, in the form of TCP/IP, DNS, HTTP and even HTML and CSS, has been unchanged for many years - or at least changed in manner that doesn't break previous versions. As an example, both Space Jam's website and Million Dollar Homepage both still function on modern browsers, having been created in 1996 and 2005 respectively. So what happened? There isn't one conclusive answer to this, more a prevailing zeitgeist amongst developers and product managers. But in my view, it's due to the large investment that technology has seen over the past 20 years. It's grown exponentially. With that, we have seen bad business practises, ill thought out ideas and customers that are keeping a company afloat. These things existed before, but now manifest themselves in the technology that these organisations build. Additionally, products are architected around small services given a single responsibility. Previously, the web was simple - throw a LAMP stack on a server somewhere and bobs your uncle. There were a lot less moving parts.

Now this isn't going to be a nostalgic post where we reminisce the days of the "good ol' web" or something, because I find that all a bit petty. I want to discuss how we need to build things to last and practical ways to do that in the face of "moving fast" (side note: watch Bryan Cantrill's fantastic talk on the principles of tech leadership here)

But why do developers avoid making things backwards compatible?

This article wouldn't exist if there wasn't at least 1 answer to this question. Primarily it boils down to "it's more effort". If you're making a major change to a service, and you have all other teams that consume or otherwise use this service in some way to also make the needed change then it's reasonable to assume you don't really need the old system. And potentially more work to maintain it. Your team may also not even have a "versioning" strategy in place. I've sat in meetings well over 5 hours of meetings about how to version services with no outcome. A lot of people have opinions about this, and often, developers seem more intent on arguing the others point rather than accomplishing the objective behind the change. Furthermore, there have arguably been a number of failures in attempting to preserve backwards compatibility such as with Java and SQLite3. These challenges can be major roadblocks in creating stability and backwards compatibility in your products services.

Why is it important then?

First we need to clarify that preserving backwards compatibility is not about holding onto legacy. If something is old, busted, broken or unused, then by all means, pave over it and start afresh. There's no need to attach infinite eels to yourself to support absolutely every single use case of your service. Things change as software changes. It's natural. On the other hand, backwards compatibility is about not creating unnecessary work for hundreds of your users every time you make a change. Or coupling releases so tightly that everything has to be deployed at exactly the same time and caches flushed in sync. Or having no deprecation plan and changing external interfaces constantly.

Stripe treads this line very careful (however, I am unaware of the overall experience as a developer there). Being a payments processor, there has to be certain guarantees about how things will be handled. To accomplish this, Stripe use a date versioned API system. You get assigned the latest versioned API when you create an account and can easily update the API if you so wish. But you also have the option to leave it completely. In fact, there are still websites I built a few years ago with now old Stripe integrations that tick along fine. They have a great post about their versioning mechanism here

How to do it

You might assume that because you use a /v1 and /v2 in your endpoints you're all set right? Well, not so fast. Ultimately, as humans we are subject to reason about things that we ourselves cannot fully understand. Therefore, what constitutes a major version bump for some, may not be for others. So how can you do it?

1. General coding practise

If you're changing something minor like the name of a parameter on an interface or the type, then there is the possibility that you can support the old method by casting it to the new type and so forth. There are many general coding practises that allow your code to be bug hardened whilst not introducing lots of spaghetti. Additionally, a good starting point for all backwards compatible changes is to mark the "old" code with a "deprecation" warning of some kind so that other developers in your team know not to use that code any more.

2. Documentation and Deprecations

If you're going to make a breaking change to something, you need a way to communicate that to the consumers of your product and you need to tell them how to update if they absolutely cannot be held on the previous version for some reason. You can do this by giving the customer, plenty of warning via email, an account manager or a deprecation warning in the response. You could have a system whereby, when a deprecated API method is called, it logs it to a table. Each day, the table can then be scanned and you can tell the customer "You called X route which has been deprecated and will no longer receive updates, please see N website for documentation on how to update". Hand-in-Hand with this goes a clear policy on how long you will support "deprecated" routes. Depending on the market you're in that could be a few months or a few years. Either way, be clear to your customers. Again, Stripe does a pretty good job of saying "you can use this near indefinitely" and including that as part of its marketing to developers.

3. Pivoting

If making something backwards compatible has become so incredibly painful that you'd rather play hopscotch on a floor of hot coals, then you need to ask if the service has pivoted to a point where it's potentially a whole new thing. As an example of this, I created a service for a messaging application that kept track of when a customer had last read a particular message. However, it then needed the functionality to manage if the customer had left a group message and then if the customer had muted the messages and so on. Before you knew it, it was no longer an API for managing if the customer had read a message or not but more a fully fledged notifications API. In retrospect, I should have seen this inevitability. But the service had devolved to a point where it wasn't anything like the original. Although it was an internal only service, it's something that looking back I should have redone and gradually migrated over to the new service. Although this may not be preserving the backwards compatibility in a true sense, as long as you provide a sensible upgrade path and don't immediately shutdown the old service then it's ok in my books.

4. Versioning

We've touched on this a few times in this post, and arguments about different versioning strategies has been talked about since time began. I'm not going to provide any guidance on which one is best or which one should you chose - simply decide on one as a team and agree upon clear definitions about what constitutes a version upgrade. Then include this as part of your release strategy. If you work with continuous deployment then perhaps look at something similar to Stripe or a Semver strategy that goes beyond a "/v1" and "/v2" route structure (although may include it). It will depend on a few factors _ Expectations of the market you are in - how long do you customers expect to use your product and forget about the implementation - hint it's often longer than you think _ What is your release cycle like? - if it's daily then you need something to automate the process, if it's each decade where you pack your software onto a disc then it can be something manual * Do you have a lot of third party consumers - if there consumers to your service that go beyond your company then you will have different requirements about deprecation etc.

TL;DR - Pick one and go with it. You can even pick different mechanism for different services!

5. Limit dependencies

Mo' Dependencies, mo' problems - Notorious D.E.V

By limiting the number of dependencies you use, versioning will be easier because you no longer need to provide constant security updates and check if every last version of your software works with it. The Node community is particular bad at this one, and often does provide security and bug fixes down stream and instead just forces everyone to upgrade to the latest version. We can do better than this. But we make our lives a lot easier by reducing dependencies.

Conclusion

There are lots of arguments against backwards compatibility and I can understand why. Personally speaking, I want to build to last. I'd like to think that in 10 years time I could still use my products without having to change the integration. Something about seeing the spacejam website just sort of fills me with a warm glow of a moment in time that is accessible at any point.

Sharpening the Saw

2020-06-26T10:17:00Z

Sharpening the saw is Habit 7 in the cringe inducing book entitled "7 Habits of Highly Effective People". This post isn't yet another book review but rather the work we do to make the rest of my work, better, faster and more consistently. The label Covey gave to this work was "sharpening the saw". It conveys the idea that, given a new saw, one cannot just continuously cut wood all day, every day. Time needs to be given to sharpen the saw, literally, so that the wood cutting rate remains consistent.

As a developer, I interact with hundreds of tools, websites and system a day. Each one has it's own cognitive overhead such as keyboard shortcuts and the like. Furthermore, as technology is so rapidly evolving, there is seldom reason why you shouldn't attempt to iterate your approach to problems. My quest to sharpen the saw started, like most new things, with a question - "How can I make my life easier as less stressful".

1. Keeping better personal notes

Most of the people I admire across a broad spectrum of industries love to keep notes. Learning from your mistakes is very easy to say but not so easy to actually do. And more importantly, implement a system to do so. I solved this by doing the following:

Creating a new GitHub repo called Notes, each note is then a simple text or markdown file. I chose GitHub for storing this so I do not need anything asides from my YubiKey and a terminal - simple and low friction when moving across machines. Also text will never die, platforms will, eventually - even the best
Treating my Todoist Inbox as a brain dump - I run my life on Todoist, so it's a super easy way, if I have a bunch of ideas to just dump them into Todoist. Since this is my task manager, I can process them as if they were a task. If they need action, I will expand them and sort them into their appropriate project. If the idea is a new project entirely (more than 2 actions), I will either do it or look at where I can schedule it
Keeping a currently.txt file open in one terminal window that I update before I context switch to something else. For example, if my wife calls me to help her with something round the house or someone comes to the door, I quickly write what I was "currently" doing. When I then come back, rather than trying to rack my brain to figure out where I was, I just read that document.

2. Improving my Desktop Productivity Setup

I want to expand on this section a bit more in a personal infrastructure post but here is a brief summary of how I got my PC to work for me rather than me working for it.

Store my dotfiles in GitHub and have them auto sync from any of machines when they are edited, this keeps the aliases, functions and programs in sync across the machines
Switch my terminal from Bash to ZSH and install a bunch of plugins to make hopping around the shell easier
Got a USB switcher so I could clamshell my Macbook's (personal and work) and switch between them easily
Got well acquainted with the *nix terminal and keyboard shortcuts
Purchased a solid keyboard as I try to avoid using the mouse at all costs
Analysing all software deeply and attempting to speed up its performance as much as possible

3. Dedicated learning time

Each week, I spend at least 20 minutes learning something new. But it can't just be anything. It has to be something that will further my knowledge in a new paradigm. Learning a new JS framework will not extend my knowledge in a new paradigm. Learning how the TCP/IP stack works in *nix or how a certain part of the V8 engine works will. Personally, I enjoy teaching others so part of my "learning" time can involve teaching others things I already know. The effect this has on the brain and knowledge storage is fairly well documented and I'm not a neuroscientist (yet) so I'll leave it to them to explain why this is so effective at knowledge retention.

Additionally, I find attending meetups (such as LeicesterJS, one I help run), and speaking with other developers fascinating. I like to hear of their problems and throw myself in. It helps improve my questioning ability - something I did not cultivate enough early in my career. In the past, I was very much "open the bonnet and have a rummage" kind of debugger, this approach works for a lot of problems, but there are others such as ones documented where this strategy does not cut it. To combat this, I have invested time in trying to be better at asking questions. As a child we are very akin to simply asking "why" all the time, but asking good questions is more than just why as often things boil down to more than 1 problem. It's about asking questions that carve away at the ultimate nugget of truth based on absolutes. These questions also help me when presenting new ideas. If I present an idea after having read about it on a large companies engineering blog, I would have no foundation of which to base my reasoning on. Instead, by gathering data and arriving at a solution that way, it has a basis to stand on. Don't try to wedge a solution from someone else into a problem you are experiencing, it mostly won't fit.

4. Post mortems and reviews

Part of learning from your mistakes involves reviewing what happened, what went wrong and most importantly why. Although at my workplace we do not have an engineering blog or a culture of writing post mortems for absolutely every issue, I've found it personally beneficial to keep a "war" log of all the big issues I've faced, why they came about and how we solved them. I keep these write ups in my personal notes.

Overview

I'm happy I spent time doing this, and is a process I will continue to follow. Hopefully this helped you practically speaking as there is a lot of wish-wash in this sector. If you have any more suggestions let me know on twitter. You can also following progress of my startup TurboAPI - here

Lightning Fast ZSH Performance

2020-06-19T12:31:03Z

As part of my work to "sharpen the saw", I decided to spend some time improving the performance of various components in my setup. The first target of my attention was ZSH.

Why?

I open new shell instances constantly and try to live in the terminal as much as possible (including for writing this blog - in vim to be precise :wave:), so each second spend loading new shell windows or tabs is a second wasted. Additionally, when things take time to load, you're more distracted and/or frustrated with how long it's taking to load rather than the current outcome you were trying to achieve. In my eyes, speed is one of the 3 major pillars of user experience - along with findability and accessibility. To start on this performance journey, I first needed a quantifiable metric to track my progress as I made changes.

To do this I ran the following command

for i in $(seq 1 10); do /usr/bin/time $SHELL -i -c exit; done

Try running it in your shell of choice now, and you'll get a performance breakdown of 10 runs of initiating your shell! For me the response was this

1.5 seconds 0.8 user 0.7 sys

1.5 seconds may not seem like a lot, but that is more than noticeable to any human. Time perception is a whole other topic for far more intelligent people than I. But after some digging, the consensus seems to be that reaction times are around 150ms (input-to-action) and 13ms for perceptive time (from visual stimuli). Therefore, our benchmark we were aiming for was 0.13 seconds. This is a number I now seek out across all software I use and consume.

Ok so, we've got a benchmark of 0.13 seconds meaning we need to reduce the load times by 1.37 seconds - let's get to it!

Phase one - the big boys

I had a rough idea of what might be taking the time in my zshrc file.

NVM - this is a node version library that I've (unfortunately) used for years. It allows you to quickly swap between node versions. Handy but performance intensive. It initiates for each shell instance so you have access to the nvm command. I don't need this all the time, so it can be removed. Note, by removing this, it still allows me to use the "NVM" node install - just not change it.
ZSH Sourcing - The standard oh-my-zsh config contains the line source $ZSH/oh-my-zsh.sh. We don't need this as I was using Antigen as a plugin manager, which automatically includes oh-my-zsh, so in reality I was sourcing it twice!

After these changes, I ran the test script from earlier and got the following result

0.61 real 0.32 user 0.27 sys

That's nearly a second removed just by removing NVM and sourcing ZSH! A huge difference so happy about that.

The next suspect I had on my list was my prompt, I used this heavy emojified Spaceship theme that was installed via NPM. Most of the time, if something looks good, it takes ages to load. I switched over to the Pure prompt and remeasured the results

0.54 real 0.29 user 0.24 sys

Ok, so not a massive change, but the overall experience of loading a new shell felt instantly snappier. It could have been a placebo but it worked for me, so I plowed on ahead for more culprits. But I needed help...

Phase two - profiling

Now I had removed some "big hitters" to performance, it was time to dive into the nitty gritty. We can do this by profiling our ZSH config so we can see what is taking the time. We can do this with a tool called zprof, which is bundled with ZSH by default. We can add it by putting zmodload zsh/zprof at the top of our ~/.zshrc config and then putting zprof at the very bottom of the config.

Now we simply need to reload our shell and we get a nice breakdown of the time taken for each part of our config.

We can clearly see from the above screenshot that a plugin called set_iterm_tab_color is accounting for a lot of our load time. The plugin wasn't really what I wanted it for anyway (I wanted something like peacock for VSCode). I removed it and a few other Antigen plugins and re-ran the test script again

0.30 real 0.17 user 0.14 sys

That's the load time halved! I was pretty happy to stop at this stage but I kept pursuing that prized 0.13 seconds.

Phase Three - Antigen to Antibody

Here's where things went wrong. As you could see from the screenshot, my second biggest load time item was Antigen itself. Antigen is known as a bit of a big beast. So I started to look for alternatives. Antibody was advertised as a answer to Antigen's woeful performance. I spent around 2 hours diligently porting my config over to Antibody. However, when I eventually got it working and battled through the lackluster documentation, I found it was actually slower than Antigen! A bit annoying to say the least. I didn't write down the test results from then but they were somewhere in the region of 0.8 seconds.

Phrase Four - Remove Pyenv

The last phase I did for now was to remove pyenv and compinit. Loading of pyenv is something that had escaped my notice when originally pruning through my config. I barely do any python development nowadays so this got removed hastily. I reran the tests and...

0.19 real 0.10 user 0.08 sys

Ok, so it's not quite the 0.13 seconds I had hoped for, but not bad.

By this stage, the main load times for opening new tabs was iTerm2 itself (my terminal application of choice). Going forward I am going to switch over to Linux and most likely use Kitty which is a blazing fast terminal program. This should null and void any performance gains I'd get by reducing it down by that 0.05 seconds.

Conclusion

This was a really good use of a few hours of time. It's going to make me happier and more productive at work, which in my mind, is time well spent. I still have plans to speed up ZSH (asides from Kitty) but they will have to wait (convert over to Zplugin and reduce plugins even more). For now, I'll move onto other programs I use regularly like VSCode and Chrome for more performance gains there.

I hope this post helps you speed up your ZSH - let me know the speeds you get on twitter - @joshghent

Speaking of performance, I'm currently working on a side project called TurboAPI. It's a super simple tool designed to monitor your API and Webhook's performance without installing a thing. I'd love if you could check it out here

Personal Infrastructure

2020-06-16T12:01:03Z

After seeing the amazing posts by both Stephan Wolfram and Jess Frazelle, I wanted to chime in on my "personal infrastructure". I've always found stories about how people work, their little scripts and hacks they use and the machines they operate on, to be incredibly compelling - usesthis is a great site dedicated to that very subject.

Principles

I have two principles that I keep close to mind when looking to change or add to my setup.

Productivity - although I dislike the word (it sort of leaves a plastic taste in my mouth), productivity or rather getting the right things done is something that I'm always cognisant of. Because time is always a constraint, I look for ways to reduce the time I actually have to be doing something.
Automation - this sort of links with the above but automation is a guiding principle in my life because the enjoyment I get from having a computer do something I did previously is overwhelming. I am fascinated by small scripts, cron jobs, and simple bots. I utilize all sorts of services to automate tasks. If there is a recurring activity I perform, in all likelihood, I will automate it.

The Daily Grind

Currently, I work as a Senior Software Engineer for Cappfinity. My role is heading up development on a product called Koru which was purchased as a startup. The job, rather generously, purchased a 2019 Macbook Pro with 32GB Ram for development usage. It works great asides from the occasional sluggishness. I'm fortunate as for personal work, I use a Macbook Pro as well so the context switching between the two devices was minimal.

I keep this Macbook primarily "clamshelled" on my desk in a black metal stand I got on Amazon. It is connected via a DisplayLink dock to two 24 Inch Monitors.

I use a Logitech K380 for typing. As much as I would love a mechanical keyboard, my catastrophically frail wrists cannot bear the strain of the key travel. The build quality of the keyboard is not great and is more plastic than an American news presenters face, but it has a great 3 device bluetooth switcher which allows me to toggle between my personal and work laptop's as well as my gaming PC.

I put an MX Master 3 on the right hand side of my keyboard - again it has the same great 3 device bluetooth switcher. And a Magic Trackpad on the left. I occasionally swap the two but generally it stays in this configuration. It sounds odd, but it's nice to have the options and reduce the repetitive movements I do.

During lockdown, my wife was able to spend some time redoing both her art studio and my office. She painted it a subtle green and chose an "urban jungle" theme. I've always loved bringing nature indoors and having lots of plants around really helps me focus and not feel like I'm in an office. Makes it look a more bright and vibrant environment, rather than a dull dreary office.

Health

My wife, kindly, bought me a Herman Miller Aeron for my desk after "relentlessly moaning" (according to her - what does she know) about my previous chair - which was as comfortable as a church pew. The Aeron is like being cradled by 4 angels and has helped my posture and fatigue massively. Yes that's right, fatigue. Sitting down all day can be tiring in a weird way, due to the lack of lower body movement. I've tried to alleviate this by taking regular breaks and working in different environments. It was the best present she's got me to this day and will make a huge difference to my daily life.

Furthermore, I've found the amount of water I drink to be a big factor in my mood. I know attempt to drink around 4L of Pure water per day as well as a coffee or two. I try to avoid caffeine after midday as I find it affects my sleep quantifiable.

Personal Work

Asides from my day job, I am working on a SaaS business, TurboAPI, and a charity, Alex's Wonderland Puzzles (that is still undergoing heavy development).

In the case of the former, I develop the application using Typescript and host on AWS and DigitalOcean with Docker - because ECS is more hassle than it's worth. The frontend is hosted on Netlify and is built using React. I am looking to dabble a lot more in Golang and Rust, so I can work closer to the metal. Golang in particular introduces some new paradigms that are interested to me, and I believe would benefit me. However, my current spare time is dedicated to TurboAPI and Alex's Puzzles so I have that learning on hold for the time being.

For the puzzle company, my role is a little different. I'm in charge of sorting manufacturing, marketing, website development/maintenance and design. For this, I take details notes in Notion as it allows me to create tables, lists, charts and anything else I want. The website was built very simply using React and will soon have a store hosted with Shopify. I do designs in whatever notebook I have to had - usually a moleskin.

On both my work and personal setup, I take notes in a private repo in GitHub. It allows me to jot my thoughts quickly and frictionlessly. I also write post mortems in there from any projects I'm working on. I've never been in an organisation that does public post mortems, but I find them good practise to really nail down the issue that was at the heart of the problem.

Conclusion

I've got tonnes more to talk about here, but I want to keep this post short so I'll bid you all adieu and goodbye. In the next post I'll discuss my personal slack hub and automations I've been working on. Stay Tuned!

How to use Private GitHub Packages on TravisCI

2020-03-09T09:22:03Z

The Problem: It's fairly well documented how to use private NPM packages in a project that uses TravisCI, but what about the GitHub Package Registry?

This was the issue I was facing. I was googling all over the net and finally landed on a solution to solve this problem.

In practise, TravisCI just boots a VM or container that then runs the scripts you have defined. We can leverage the before_install script to setup Travis as a new GitHub Package user. Here is how to do it...

If you're using a team, you will want to create a new user account and add it to your team. This is so we can safely generate a GitHub Token without fear of the person leaving the business in the future etc. Add this new bot user account to your GitHub Team
Generate a new personal access token on the new bot user here. It will need access to write:packages and read:packages
Add the newly generated token to TravisCI's environment variables as GITHUB_ACCESS_TOKEN - you will need to do this for each project that requires usage of the private package
Add the following to your before_install section of your .travis.yml

before_install:
  - echo "//npm.pkg.github.com/:_authToken=${GITHUB_ACCESS_TOKEN}" > .npmrc
  - npm config --global set <YOUR ORG>:registry https://npm.pkg.github.com
  - cp .npmrc ~/

Swap the <YOUR ORG> with your GitHub Organization name without the @ - in our case the line would read npm config --global set k0ru:registry https://npm.pkg.github.com - this org name should also be contained within the package name that you publish, so in the package.json file for your package the name should be @<YOUR ORG>/package-name.
Push up your new .travis.yml file and kick off a build!

Hey presto that should all fit together and download correctly. Short article but it's the article I wish existed when I was searching for an answer!

How to Create a Pinned Gist Bot in 10 minutes with GitHub Actions

2019-11-14T14:58:00Z

Recently I stumbled upon an awesome page I hadn't seen before awesome pinned gists. The premise of the list is small apps that run GitHub actions on a schedule to update a gist that is then pinned to your profile. There are ones for monitoring your Wakatime, your last tweet or even your Strava Metrics.

It inspired me to create my own using RescueTime data - called rescue-box (if you notice, all the apps are appended with -box).

Here is how I created rescue-box and how you can create your own Pinned Gist Bot! This whole process took me around 10 minutes and is a fairly easy process.

1. Pick an Idea

The first and most important step is to have an idea for what data you want to display in your pinned gist. In my case, I chose RescueTime productivity data, but below are some other ideas if you're a bit stuck

FitBit Step Count
JIRA ticket last complete by yourself
Todoist Tasks completed today
Reddit upvotes for your user
PocketCast listening time

2. Get the data

Now once you have chosen your idea you need to find where you can get the data for your app. Most services have a public API, so try Googling <APP NAME> API documentation. In my case, I found the RescueTime documentation here

Inside the documentation, you will be able to find the endpoint you need to call to get the data as well as information about how to authenticate the request.

The RescueTime API Docs I used

Once you have the request, mock it up in a tool such as Postman or Insomnia so you can see how the data will come back from the API.

This is the most tricky part of the process as your idea may not have a public API or the authorization may be too tricky to implement, in this case, try another idea until you find one - this is the most difficult part of the process.

3. Get the foundations

To bypass the boring setup, clone my rescue-box repo as this will give you a great starting point - especially if you are building something in the same format as I did You can do this by running

$ git clone https://github.com/joshghent/rescue-box

Change all the references to rescue-box to your own app name and replacing the RESCUETIME_API_KEY references with your own service's name (e.g., FITBIT_API_KEY). More importantly, update the documentation on how to setup the application - include information and links on where to get the API key for the service you are integrating with.

Now run npm install in your terminal to install the dependencies

4. Modify the API call

Inside the index.js file in the main() function, change the API call URL to be the one for your service. Additionally, change the environment variables loaded at the top with the ones for your service.

Next, inside the updateGist function, change the code from line 30 onwards to be what you want to inject into the pinned gist.

Rescue-box injects a line for each type of "productivity" metric and then has a bar next to each of them out of 100%. However, in the example of a FitBit step tracker, there is no "percentage" for the number of steps you take per day so this can be removed and replaced with other information if you wish.

It is worth noting that you can only have 5 lines displayed in a gist on your profile at a time. In the case of Rescue-box we use the first line to render the date the stats were taken from, leaving 4 lines for the information.

5. Test it locally

By this point, you should have followed the instructions in the setup guide and set yourself up with an API key, a Gist and a Github token. If you haven't already, now is the time to do that Now rename the sample.env to .env and add your application secrets.

Next, run the following command to run the application and update your gist!

$ node index.js

If you go and view your Gist in the browser, this should have successfully updated it with the information you want. If it has worked, then proceed to the next step, otherwise - time for bug fixes! 🐛

6. Publish it to GitHub Actions

Now the automation bit, getting the app to run on GitHub actions.

If you're not familiar with GitHub actions, it's a task runner, similar to Jenkins and can do all sorts of things like publishing to NPM or Docker Hub, building apps and running test suites. For our application, we will be using it to run our app on a schedule every 10 minutes.

The prerequisite to this is to set up your repo (as documented in the rescue-box instructions). If you haven't already created a repo - create one. You will need to add your API key, gist Id and GitHub token into Settings > Secrets for the repo.

Next, modify the .github/workflows/schedule.yml file with the name of the app - this is how the job will display in GitHub actions.

Now update the Update gist action uses: repo. Currently, this should be pointing to joshghent/rescue-box@master but change this to your user name and repo (e.g., joe-bloggs/fit-box@master)

Below, in the env: section, replace the RESCUETIME_API_KEY with whatever it is named in your application (e.g., FITBIT_API_KEY)

Your finished GitHub workflow should look similar to this

name: Update gist with FitBit Stats
on:
  schedule:
    - cron: "*/10 * * * *"
jobs:
  update-gist:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@master
      - name: Update gist
        uses: joe-bloggs/fit-box@master
        env:
          GH_TOKEN: $
          GIST_ID: $
          FITBIT_API_KEY: $

7. Push your app to your master branch

The final step is to push all your code from your machine to the master branch on GitHub!

$ git add -A && git commit -m ":tada: Initial commit" && git push origin master

✨ All done! Now you've got an awesome automated pinned gist application running on GitHub Actions

[OPTIONAL] 8. Add it to Awesome Pinned Gists

The final step is to add it to awesome pinned gists so everyone can appreciate your gist-based genius! - https://github.com/matchai/awesome-pinned-gists

Happy hacking - reach out to me @joshghent if you get stuck at all!

Managing Application Secrets for Terraform across Teams

2019-11-11T15:08:03Z

TL;DR

Terraform stack is deployed via Travis using a script (below)
Secrets are shared by storing an encrypted tar file in Git
The tar is decrypted by TravisCI and another other team member using secret keys stored elsewhere
Variable files and generated dynamically based on the .env

Ok, that's a bit of a wordy title, but that's exactly the challenge I was tasked with solving recently - and it proved to be more of an issue that I expected. The objective was to be able to use application secrets for ECS task definitions inside Terraform and have those be easily shared across the team.

🔐 Encrypting the Configuration

My first approach was simply to use environment variables and then put the environments file in secure storage (i.e., KeyBase) rather than committing it to source control. Simple right? So that works, but in our case, we have TravisCI do the deployments for our stack and so that approach would have been lengthy whenever we wanted to add a new secret parameter.

To solve this, I created a script that encrypts and decrypts a config with OpenSSL encryption. These encryption keys could then be shared across the CI and the rest of the team. It would be ideal to use a GPG key for this but I do not believe this is possible in CI. Plus it's a pain to onboard a new team member as you need to re-encrypt the secrets with their public key - less than ideal.

We have a file called .env in the root of the Terraform repo with the environment variables as you'd usually find them. This is ignored from source control. We then have one script to decrypt and unzip from tar using OpenSSL and another to encrypt the file in a tar. The tar is then committed to source control, it can then be decrypted by Travis and another other team member with access.

# encrypt-config.sh
#!/bin/bash

echo "Compressing config..."
tar czf config.tar.gz .env

echo "Encrypting config tarball..."
openssl enc -aes-256-cbc \
  -in ./config.tar.gz \
  -out ./config.tar.gz.enc \
  -K ${CI_ENC_KEY} \
  -iv ${CI_ENC_IV}

rm config.tar.gz

# decrypt-config.sh
#!/bin/bash

echo "Decrypting config..."
openssl enc -aes-256-cbc -d \
  -in config.tar.gz.enc \
  -out config.tar.gz \
  -K ${CI_ENC_KEY} \
  -iv ${CI_ENV_IV}

echo "Extracting config..."
tar xzf config.tar.gz
rm config.tar.gz

☝ Loading the variables

Terraform environment variables must be prefixed with TF_VAR_, since I didn't want the laborious process of adding this to each variable, I wrote a script to prefix them and then load the variables into the shell environment. The script can be found below:

# loadenv.sh
export $(egrep -v '^#' .env | while read line; do echo "TF_VAR_$line"; done | xargs)

This then meant that my variables could be referenced in terraform by adding them to the root variables.tf like this

variable "REACT_APP_ENVIRONMENT_VAR" {
  type = string
}

...and then passed into the module like this

// main.tf
module "Sample" {
  source                                = "./modules/general-cluster"
  REACT_APP_ENVIRONMENT_VAR                = var.REACT_APP_ENVIRONMENT_VAR
}

In our shell the variable would be loaded as TF_VAR_REACT_APP_ENVIRONMENT_VAR.

🤖 Automating

But I wanted to go a step further, because adding to the root variables.tf each time is a pain. Me thinks, time for a script... so I did some digging and added this to the end of the loadenv.sh from earlier

# Clear the old variables.tf file out
> variables.tf

# Loop through each line of our .env file
egrep -v '^#' .env | while read line;
do
  # Get the first part (before the =) of the line for the variable name
  var_name=$( cut -d '=' -f 1 <<< "$line" )

  # Write it to the variables.tf file
  cat >> variables.tf <<EOL
variable "${var_name}" {
  type = string
}
EOL
done

This now automagically generates a variables.tf file at the root of the Terraform folder. Travis can run this loadenv.sh script based on the encryption keys it has stored in it's own environment. I created yet another script called deploy.sh that Travis runs only on the master branch. As the name implies, it handles the deployments and notifications of such.

# deploy.sh
#!/bin/bash

set -e

if [[ $TRAVIS_BRANCH == 'master' ]]
then
  errorstatus() {
    echo "Error when deploying Terraform config"
    # Slack Webhook Message
    curl -X POST --data-urlencode "payload={\"channel\": \"#deployments\", \"username\": \"Deploy Bot\", \"text\": \":poop: Build #$TRAVIS_BUILD_NUMBER Failed when deployed Terraform Stack. Error log $TRAVIS_BUILD_WEB_URL\", \"icon_emoji\": \":rocket:\"}" $SLACK_WEBHOOK_URL
  }

  # When exiting due to an error, run the error status
  trap errorstatus ERR

  . ./decrypt-config.sh

  # shellcheck disable=SC1091
  source ./loadenv.sh
  terraform init
  terraform validate
  terraform apply -auto-approve
  curl -X POST --data-urlencode "payload={\"channel\": \"#deployments\", \"username\": \"Deploy Bot\", \"text\": \":tada: Build $TRAVIS_BUILD_NUMBER successfully deployed Terraform Stack\", \"icon_emoji\": \":rocket:\"}" $SLACK_WEBHOOK_URL
  echo "Deployment Completed Successfully"
else
  echo "Branch is not master so skipping deployment"
fi

After all this has run, I end up with a nice deployment message in Slack!

And that's all! We are still iterating our approach with Terraform and working to get it running against our entire stack rather than just parts of it. I am enjoying learning it so far, even if it's a bit rough around the edges I'm excited to see where Terraform goes!

Monitoring Git Leaks in Travis

2019-11-08T12:11:00Z

Recently, we've wanted to add Gitleaks scanning into our repos to keep on top of any potential security issues. I checked out a number of tools such as detect-secrets and trufflehog but eventually I decided to use Gitleaks as the format was fairly CI friendly.

There is already a CI version of Gitleaks but it uses a stripped down version of Gitleaks with basic regex. I wanted to use the fully fledged version that was updated a bit more regularly. Additionally, with the CI version you had to configure a few environment variables which I didn't want to do with every single repository.

Since there was not much documentation on how to use it in CI, I decided to post this blog.

Simply add this script in /.ci/leaks.sh This will only audit the current script in the local repo

#!/bin/bash

if [ ! -z $TRAVIS_PULL_REQUEST ]; then
    REPO_SLUG="/${TRAVIS_REPO_SLUG}"

    # Audit the current commit for secrets
    docker run --rm --name=gitleaks -v $PWD:$REPO_SLUG zricethezav/gitleaks -v --repo-path=$REPO_SLUG --commit=$TRAVIS_COMMIT
fi

Next, add this into your .travis.yml. Alternatively just add an additional "script" if you don't want to do different stages

- stage: Leaks
    language: generic
    script:
    - "./.ci/leaks.sh"

Additionally, add docker as a new service in the .travis.yml

That's it! Tweet me @joshghent if you have any problems.

Signal vs Noise - Staying Up to Date

2019-10-16T11:40:03Z

Technology is so fast paced that to stay up to date, you need to be learning on a daily basis. However, the internet is so awash with vast swaths of information of varying accuracy and importance that it's difficult to filter the signal from the noise and only consume that which will be of lasting importance.

This was a challenge that I faced myself in my career of staying up to date and learning about new technologies in a 80/20 fashion. In other words, what is the 20% of content I can "consume" for 80% of the impact.

Why is "staying up to date" important though?

Recently, Corey Quinn, writer of Last Week in AWS, had an interesting point when answering the question of "What's the most consistently wrong thing you see AWS users do?"

Quinn: The most consistent mistake that everyone makes when using AWS—this extends to life as well—is once people learn something, they stop keeping current on that thing. There is an entire ecosystem of people who know something about AWS, with a certainty. That is simply no longer true, because capabilities change. Restrictions get relaxed. Constraints stop applying. If you learned a few years ago that there are only 10 tags permitted per resource, you aren't necessarily keeping current to understand that that limit is now 50.

I've found this to be true in my own career. Often, there will be a workaround I have to do for a particular project, but then a new strategy is released that means there is no need to do that do that workaround any more. Without "staying up to date" I would keep doing this workaround!

The challenge remains however, that you cannot consume everything that is published, even about a specific technology. So how can you filter the signal vs noise?

Subscribing to newsletters

First and foremost, subscribe to these on your work email. Learning is a part of your work and so reading through information on a work relevant topic should be considered as such. Previously, I subscribed to them on my personal email and found myself with a mountain of emails to read through on my way home from work - not fun and I didn't get any benefit from the content.

I find newsletters a great source of learning because they only include articles that particularly stood out with importance. Additionally, each article usually has a short summary that can make you decide whether to read it or not.

There is even a newsletter for HackerNews so you can have it boiled down as a sort of "Greatest Hits" of the week. This helps you feel connected to the community without having to spend all day scrolling.

Go to Meetups

I'm a big advocate of meetups for learning. Why? Because it's dedicated time to focus on one thing. You're not (or at least you shouldn't) be checking Twitter at the same time, you're just listening and learning.

Furthermore, it's a great way to surround yourself with like minded people and build on your ideas. You can gain real insight into how people learned a topic or even got into the industry in the first place.

Join Slack/Discord/Forum Communities focused around a technology or topic

If you go to meetups, they will often have a slack or discord community that you can join and talk with other attendees. I'd highly recommend joining these, but be selective, not joining and participating in absolutely every single one. Instead, encourage conversation by asking questions, even if you think they are dumb - people will help you out. Slack is particularly good for meetups because it allows you to build relationships with the people in the local area. This makes it easier to look for new opportunities, to collaborate and perhaps to secure funding for your next great idea.

Additionally, if you have an interest in a particular technology or area of expertise, these will likely have a community around them also. Personally, I really love the Chaos Engineering Slack as there is always lively discussion and a lot of links being shared on the topic. Again this allows you to filter down articles and gain insights on the topic that you otherwise would not know.

Follow relevant people on Twitter (+ setup Twitter mute keywords)

Despite the hate, I find Twitter a great asset to me in my learning and "staying-up-to-date". But it carries a couple of warnings. Firstly, it's to setup Twitter mute keywords. I have around a hundred or so of these (unfortunately they cannot be imported via CSV or other formats), that filter out everything I don't want to see (basically anything asides from tech). Additionally, I steer clear from politics and other "drama" that springs up in the community. Some may find this to be "putting up the shutters" so to speak, but I get my news elsewhere and my life is a lot happier not reading about other peoples personal lives.

Second is to be selective about who you follow in the community. Go to your favourite "tweeter" and look at who they follow and just follow them all, then if you find the content off topic then you can simply unfollow them. Overtime you can build a highly curated list of people who give you great content.

Listen to Podcasts (selectively) and Books

Listening to content whilst commuting is a great way to learn whilst on-the-go. There are a large array of podcasts that I subscribe to, both technical and non-technical as well as an Audible subscription for a new book each month.

Podcasts used to be a major source of stress for me as they would stack up in my "To Listen" playlist waiting to be listened to. Now I am much more rigorous about which podcasts I subscribe to and listen to. If I am subscribed to a weekly show, I don't care to listen to each episode and just dive in on whatever episode I want to listen to. PocketCasts still says however that since July 28th 2016, I've listened to a staggering 84 days and 3 hours of content. Whilst I listen to this as well as doing other things, such as shopping or commuting, I do suddenly realise two things

I barely remember a single one of those podcasts I've listened to
I could have read/listened to a lot of books in those 84 days that would have had far greater impact.

It was for those two reasons that I have no switched almost exclusively to listening to audiobooks. They give you great insights in on the world whether technical or non-technical. Recently, I've been listening to a number of psychology and history books, which tangentially has given me new perspectives on technology.

I don't really have time to read physical paperbacks (despite loving the tactility), so audiobooks are a great way to read without reading. I find my comprehension and memory better if anything when compared to regular reading as I lean towards auditory learning.

Relax

... you don't need to be learning all the time. Although important, you should not overwhelm yourself with media to consume on a near constant basis. That is why I chose to not include tools like Pocket in this article. For myself, they led to massive FOMO and anxiety around how many articles I had stacking up. I read hundreds of articles previously, and truth be told, I remember almost none of them. What I do remember however, is books. For me, they provide much deeper insights and introduce new world views that allow me to learn in new ways.

Additionally, although it's good to use these different channels, make sure to not be distracted by them and instead allot their own block of time to be focused on individually.

I have set myself the provision that if a tab is open for more than 10 days, then even if I think it might be the greatest piece of literature ever written, then I close the tab. You have to reason, if it is that good, you'll probably hear about it by some other means.

Pentest Aftermath

2019-10-09T14:06:00Z

Recently, Koru had a penetration test done by an independent third party. The actual test took place a little before I had joined but the results only came in afterwards. Having never read through a pentest report, I was curious to see what they would find and more importantly how. Having been listening to copious amounts of DarkNet Diaries, I thought we were in for something along the lines of "we have all your data and locked you out" level of intrusion.

Thankfully, that wasn't the case. Instead we got a 6 page report outlining the findings. Being fairly unfamiliar with the system at the time (it was literally my first week), it was interesting to gain the insights that the document provided. Reflecting on it now, it's not incredibly surprising, we use a set of very tight VPC's to separate each clients data and the only frontend exposure is on a static website; So there are no chances of XSS attacks.

The first discovery which was considered Number 1 priority turned out to be rather humorous. It was discovered that the API returned an "X-Powered-By" header (which tells the client what library or language the service is written in) of "PHP 5.1". I was a bit puzzled by this as the entire stack was Javascript or Python. After some digging I discovered the following code in the Express server declaration.

The express X-Powered-By header code forgery

Sneaky beaky

This was a sneaky tactic... and very funny. This got removed to not return an X-Powered-By header at all.

Additionally, the pentest had pointed out that we did not have a password reset functionality when inside the application. This didn't seem like a security issue but the reasoning behind it was if the account was compromised then the user could take protective action and change their password right there and then. An edge case certainly, but interesting nonetheless. This feature has been more of a pain that we anticipated since Auth0 helpfully doesn't return the "reason" why a password was too weak when changing the password via the API. This is not a nice UX so although the feature has been built, it has not been released.

In connection with this, they pointed out that our password policy was incredibly lenient. Since we use Auth0 as our identity provider, this was a simple matter of upgrading the password requirements, turning on their common password list protection (to prevent highly used passwords like "hunter2") as well as configuring a bespoke list of blacklisted words such as our company name and some industry specific terms.

Another feature we lacked was no Multi-factor authentication. This was an important feature to have, as increasingly enterprise customers are requiring it from their suppliers. Again, since we use Auth0 this was fairly trivial to add, despite bending the UX flow they recommend (which isn't that great).

After this was a number of minor points around the Content Security Policy as well as the headers that were being sent with requests. In the case of the latter, this was solved by the defaults of helmetjs. For the CSP, this required a lot of back and forth configuring all the URL's we let through. In doing so, it made me realise just how much of the web is powered mostly by third parties. Our site has a intercom integration, Auth0, a few JS and CSS libraries and some Google Analytics for good measure, but even not having 1 of those components had ripple effects right across the site. The web is incredibly fragile and it's made me wonder if we need all of it?

I personally have Google Analytics configured on my blog, but I can't tell you the last time I looked at it or even did anything with the data. It simply has been hoovering up data for the big Google machine. The case is similar with the Intercom button, I've personally never once used one and it seems as this recent tweet states, no one else does either...

Here's why these auto-messages with the same text for every visitor simply don't work (over 14K sent, 88% opened, only 3% replied and thus 85% just distracted and probably annoyed): pic.twitter.com/GOAVQAp4BJ
— Yury Smykalov (@ysmykalov) October 10, 2019

Are these things a bad thing? Sometimes yes, sometimes no. I know for a number of companies and even peoples jobs literally revolve around Google Analytics, and that's super cool. But for a lot of a websites and apps, it's just not needed. It's another case of someone saying "oh we should have this so we can track X". Then "X" is forgotten and the person who wanted it in the first place leaves, and then it's just left. Silently slurping data in the shadows.

That was a little tangent, but you get my point. Overall, my experience of having a pentest done was very positive and I'd recommend it for any organisation. Security is something that is always said to be priority number 1 on peoples list, but no action is taken; ergo it's not actually priority number 1. These kind of insights from a third party can quickly and easily give you industry level knowledge on how to prevent issues proactively - which is a hell of a lot cheaper than fixing them retroactively.

Resiliency

2019-10-08T11:43:03Z

At my previous post at CloudCall, I was responsible for the SMS/IM backend. Whilst it was being developed, we made the classic mistake of not worrying about resiliency or testing since we were so stacked with features and had a manual QA department to act as a big bug dragnet.

Once things settled down however, one of my first priorities (for my own sanity and peace of mind) was to focus on resiliency. Here are the steps that I took, that contain translatable principles across any system. In my case, I was working with a suite of NodeJS API's and consumer services hosted on ECS or Lambda.

Goals

Let's start of with some high level goals, because often resiliency can be used to mean "stopping a service from ever crashing". However, the goal of resiliency should be to accept failure, and instead be more focused on handling them gracefully.

I had the objective of making 99% of errors auto recover and be never told about them, but the 1% of errors I wanted to know about immediately. This included not just, is X service healthy, but also is the queue that it listens to being consumed at a rate less than it was being added to and so on.

Making the web more resilient is a certain goal of mine as I (as I'm sure you, the reader, have) been burned by going through a form submission, only for it to go wrong and have to do the entire thing again. Frustrating right? One example of this stands out in my mind where I submitted an application for a bank account. On the confirmation screen, I left it as I got distracted by something else. By the time I went back on the page, Chrome decided to reload it and what resulted was me having no way to login and a credit checking being done on me but no bank account at the end of it! Funny (in retrospect) perhaps but pointed out serious resiliency issues. If you start looking, you'll see them everywhere.

Here's what I ended up doing to make our SMS/IM systems more resilient.

Backing Off

When the primary consumer service (loving named Short Message Event Gatherer or "SMEG" for short), we implemented a retry mechanism that just requeued the message to be re-processed. It was then consumed immediately by another instance which undoubtedly generated the same error (due to 3rd party errors etc).

The correct behaviour is to have a backing off mechanism. This means that in the event of 3rd party failures (databases, external API's etc) it will give them chance to recover before retrying the process.

We implemented a gradual back off procedure that increased 3x with each attempt. So after the first failure it would retry after 1 minute then the second would be after 3 minutes, and then 9 minutes and so on.

Die!

Hosting the application with ECS (Docker) forced us to think in a "cattle" rather than "pets" way of working with servers.

When the applications health check failed or it's heartbeat to RabbitMQ failed, it would immediately kill itself. This would then trigger ECS to boot a new task to take its place.

Health Check on Start

Linked with the above, when the application booted, we did a systems check to make sure it could connect to the database, RabbitMQ and a couple of third parties that we used.

If this process failed on startup, it means that ECS could try to boot it in another availability zone, increasing its chance of success.

We configured alerting on the back of this if AWS could not boot a container that added to our overall picture of the systems health.

Automated end-to-end monitoring

The SMS project was primarily driven as an event-driven microservices architecture with lots of API's and consumer services along the way. And although we had unit and integration test for each of these services, the tests were still isolated to the scope of that particular service. There needed to be a way to guarantee that the whole pipe was flowing not just individual parts and their surroundings.

To do this, we set about creating Node and sometimes Python scripts that would simulate sending and receiving SMS text messages. These scripts admittedly often broke due to the configuration being a bit hacky but other than that, they worked rather well. These scripts were triggered on a CRON basis and could give us a lot of piece of mind that the entire product was ticking over well.

It also covered over failings on the part of monitoring the individual services, although they had health endpoints, often these health endpoints would just be return res.status(200) rather than abiding by the advice I have above. This meant that if one of those services genuinely did go down (which they did in the beginning), the end-to-end monitoring notified us immediately.

CorrelationIds

Due to the microservice architecture patterns, tracing logs through the system quickly became a nightmare. To resolve this we implemented correlationIds that were passed from one service to the next. In our case, they were generated by our API gateway - Kong, so we could trace the call into our infrastructure right from the source.

This is not a unique idea but, looking back in retrospect, would be one of the first things I do. Additionally, as we had setup, I would configure these Id's to be generated at a gateway level as I often ran into issues where requests from a third party to us simply would "disappear".

Fail early

Throughout any system, there will be a number of "failure scenarios" that you have to handle. Previously, we handled all failures the same, just requeued the message and then eventually completely failed. Soon though, we found a number of issues that should not have been retried such as a lack of credit on the account. For these cases, we created a way to categorize the errors, some were critical and others were just warnings. This meant more messages went all the way through (maintaining a minimum viable service level) and fed back to the user sooner on issues that were their fault (sorry!)

Aggressive feature flagging

Although we did not utilize a feature flagging tool such as LaunchDarkly (which in retrospect, we should but didn't know about it at the time), we still aggressively feature flagged everything in the backend. I had a number of features launch ages in advance of when they were actually "turned on" since my pace of work exceeded that of the frontend team. Often, I would create a feature, release it to our development environment, test it and sign it off. Then we would get a bug report or another more important feature, I would then add that on top of the previous feature which would then mean it was a pain to release one without the other. Could that be solved with better release cycles? Perhaps. But often I would not be aware of if Feature B needed to go before Feature A and when Feature A's frontend would be done.

Anyway, the solution was to add feature flags that we then toggled within the applications config that it pulled down from ASM. Easy-peasy. But this simple mechanism allowed code to be released and tested well in advance of when it was actually needed.

Alerting

Being notified of errors is something that is critical in any good system, but in our case we had additional considerations around handling third-parties and since it was driven by a consumer service we needed to make sure that the queue was not backing up to far and auto-scaling policies were working correctly.

Fortunately, were able to setup RabbitMQ to post it's stats to Grafana so we could create alerting from Grafana based on different metrics coming from RabbitMQ - this could trigger auto scaling policies and/or emails and slack messages to the relevant parties.

On one occasion we had an AZ outage where although our service span up correctly in another AZ, some other components that our service required didn't. We soon knew about this thanks to the reporting we had inside the service to report it's health in relation with other services it required.

The end-to-end monitoring also had an alerting component since it notified us if the whole process of sending an SMS message took over a certain N number of milliseconds.

Future Plans

Since I moved to a new company, I did not get chance to execute on all of my resiliency plans.

I had planned to add a new feature into our frontend that would allow people to manually retry messages (like on iMessage or WhatsApp), this would then trigger a new Lambda that would requeue the message based on the data in the DynamoDB table

Additionally, I was looking to implement a feature whereby messages that failed completely (after retries) would be moved to another "Failed" DLX queue. If somehow the app managed to process a message then it would start consuming from the "Failed" DLX queue.

Furthermore, implementing some kind of chaos engineering in production would have properly and automatically tested all the work I had done around handling sudden outages. Although we tested these manually, either by firewalling or taking down the service, we did not test it with each new release, only when the work was originally done so there is the chance it could break in the future. Automating tests like this makes it near impossible to run into these types of issues and uncovers additional flaws in the system.

Summary

Each of these points could have been a blog post in of themselves but I believe it's often good to have a rough overview from a specific view point and then research the implementation separately. I hope you had enjoyed reading about this resiliency work and provided some ideas for how you can make a more resilient and robust web!

Gatsby or Bust!

2019-08-29T11:37:00Z

Recently, I moved my website from a static HTML file on GitHub pages (yes actually static) and my blog from Medium. I decided to combine them both and move over to a Gatsby website.

Why?

I have been wanting to move my blog from Medium for a long time. Since the platform was built, they have struggled to find a viable business model and have resorted to increasingly anti-user friendly ways of attempting to get people to pay for content. I have thoughts on what they should do but that's a topic for another time. Anyway, I wanted for people to actually read my blog and didn't like the mobile experience in particular.

Source: https://medium.com/@nikitonsky/medium-is-a-poor-choice-for-blogging-bb0048d19133

Additionally, I wanted to regain control of my content. I didn't like the idea that a platform could be gaining revenue from content that wasn't theirs. It wasn't as if I was using the platform for free, I had paid $70 or so to get it pointed to my own subdomain (a feature that they later dropped).

The Move

I chose Gatsby for two reasons, it seemed pretty quick and was easy to deploy and add new blog posts to. I could also keep everything inside git and tools that I was already using for development work.

Deployments

I chose to host the site on Netlify and configured auto deployments from new commits on the master branch. I also configured my DNS provider with a CNAME from the root of my domain the the Netlify application.

Along with this, I configured TravisCI to run a spell check on all my blog posts as well as deployment previews for new PR's. This allows me to see new posts before they get merged in the live site.

Development

I started, like most people, with the Gatsby starter blog. I didn't like some of the coding styles, but didn't really care all that much. On top of the boilerplate, I made some additional changes

Added a light/dark mode toggle in the Navbar - heavily inspired by the Overreacted.io blog as well as a myriad of others with the same feature. This was easy enough to do
Changed the color scheme
Added my keybase GPG key for verification of my identity
Changed the style of headings for the blog posts, I found the default headings to be larger than writing on a charity cheque
Added a /now page as inspired by David Sivers and documented the apps and tools I use currently. This needs some improving

Moving my old posts

Setting up the site was relatively easy. The difficult and laborious part was going to be the moving of my old posts. Since Gatsby runs on Markdown, I found a neat NPM app called medium-2-md. I went into all 30 or so of my old blogs and then copied the URL's and then ran the command medium-2-md convertUrl https://blog.joshghent.com/sample-post -f -o index.md

I could have potentially wrote a script to automate it. But...

... It wasn't really worth it

After moving them all to markdown, I then ran the site locally to compare the markdown posts to the Medium posts to make sure they matched and all the content worked.

There was a couple of recurring problems I found

Words often had the * in the wrong place
Paragraphs didn't have enough \n so rendered all as one paragraph
Embed content such as Gists and Tweets didn't work - I had to find Gatsby plugins for this and port over the references to use their format
Image captions did not work - I had to move these over to use <em> tags
Images had to move moved over manually - yup saving each one and importing it correctly. There is now a -i feature you can pass to medium-2-md but this wasn't a feature when I did the original port (a long time before the actual thing launched!)

Review

Overall, I'm really happy with the result. There are some things I need to change but for now, it'll do. It was super quick and easy to get up and running with Gatsby.

A Guide to Leaving Your Job

2019-08-16T12:31:03Z

Recently, I handed my notice in to my previous job at CloudCall after receiving an new offer at Capp&Co. I won't go into why I chose to leave, but handing my notice in did leave me with the challenge of how to uncouple myself as a Developer from the services that I managed. Jamie Tanna has suggested using blogs as a form of documentation, which is exactly what this is, everything you need to do before you leave your job - broken down by time. In my case I had just under 30 days to get everything ready.

30 Days to Go

Take Stock

What products do you own? What does everyone always come to you for? Think about these things and begin to write documentation around all the gotcha's and a FAQ for common issues. Playbooks are often a great resource for the next developer that maintains you systems. I'd suggest writing a playbook on the core functionalities of a system. In my case, I looked after the SMS/IM backend systems, so I documented things such as messages not sending or the messages not being sent to our sync service.

Holiday

Take a look at how much annual leave/holiday/vacation time you have accrued throughout the year. Confirm whether your employer will pay you for any remaining unused annual leave with your last paycheck. If they will not pay you, then best get booking it! Nonetheless, even if they will, now is a good time to take a break so you can prepare for your next role and get other errands done that you'd been putting off.

Organise hand over

Speak with your manager about who will take over the services you manage and make sure to schedule a release of those services (if you do not do CI/CD). Additionally, schedule meetings with these new individuals so you can spend time discussing the systems going through the documentation. I would recommend getting the new developer to read through the documentation and trying to solve any problems that come up whilst you're still there. It "tests" the documentation and means you are still there as a backup if that test fails!

Write no code

Ok maybe, not "No code" but spent the last remaining pieces of time working on anything that will make the systems more resilient as well as tests and "overkill" levels of documentation (better to have more than less - just make sure it's relevant and useful!). If things don't go wrong then it will make the new developers life a lot easier and mean they won't need to suddenly dive in and fix something.

10 Days to Go

Payslips

Download archive if you do not get them paper based. These are useful for your records and may be needed for tax reasons in the future.

Final release

If you don't do continuous delivery then deploy all services you manage alongside the person(s) who are taking those systems over. This will enable them to discover any issues and also it gets all the code you've written out in the wild before you go.

Uncouple Accounts

Over time, as much as you keep it separate, "work" logins and things of that nature seep into your personal devices. Now is the time to sign out of all those services as there will be ones you have forgotten!

In my case I had a list of the following:

Github
Gitlab
Npm
Jenkins/CI
AWS
Removing Email from phone
Removing iCloud from Macbook
Deactivate any work-related IFTTT rules

Logging out

View all passwords stored in Chrome, check if there is anything you need then begin using a "Guest" login to Chrome. Because this won't have any of the passwords, you'll find accounts you need to recover or accounts you need to sign out of on your main chrome login.

Last Day

Take home personal belongings

Remember your mug in the cupboard, the cables that are yours etc.

Security

Remove any SSH/GPG keys from the laptop itself as well as removing them from your Github/GitLab accounts. Additionally, if you use Keybase and your GPG key contains your work email address then remove it and re-push the key to Keybase.

Log out

Along the lines of the previous point, make sure to nuke your browsers cookies/history so you are not signed into any services such as GitHub or Stackoverflow which may have been personal logins but are inherently corporate in nature.

Remove apps or Containers

It's worth removing any apps that have your logins such as Spotify as well as any running docker containers that may contain sensitive data. For example, I ran my LastFm2Slack bot on my work macbook which contained an API keys for my LastFM account.

Take aways

I hope this article can help you in the future if you're moving on from a job. It can be a stressful time and the last thing you want to worry about is becoming a blocker to the rest of your team. Planning ahead, as I have outlined here, allows you to make a clean exit and not have future developers cursing your name for lack of documentation. You want to leave with positive opinions, on both sides of the table - tech is a small world.

Lessons from Battling with Elasticsearch

2019-07-05T12:31:03Z

This is a story of changing requirements being impeded by architecture and software. It’s an age-old adage but I thought it was worth telling this story as a lesson in foresight and blame.

The bug goes as follows, CloudCall developed an instant messaging and SMS application that plugged into their existing application, that previously only handled phone calls. When planning this new messaging system, since we had the requirement to do full-text searches on the message contents, we decided to use ElasticSearch as a data store. No-one had used it before, but we knew the problems it solved and were happy to start development with it.

Later down the line, we began to get bug reports of messages that failed to sync into CRM’s (a primary USP of the product), as well as channels that were no longer visible after refreshing our application. It was a whole host of different bugs that ultimately lay at the doorstep of one simple truth — Elasticsearch is not strongly consistent. You can do your damnedest to try though, which is exactly what we did.

The bugs arose because a user would go to create a channel or send a new message, this would then create the record in Elasticsearch. If the user refreshed immediately, when our API went to look up the channels that belonged to that user, it returned an out of date list — since the new channel was not yet indexed on the shard. Additionally, when syncing messages into CRM’s, we first queried for that message across the shards in Elasticsearch (shards of messages are segmented by calendar month). Again, the document was not available in the shard at the time of syncing — as this process is kicked off right after the Elasticsearch insert.

The solution was threefold. First, where possible we queried for the document on the shard we expected it to be on, and then had a fallback mechanism to do a wildcard search.

In the instance of searching for a specific message, we first query the latest calendar months shard since we assume that messages will only be queried for in a calendar month. If that returns no data, we do a wildcard search across all message shards, where the message will always be available — since we do not insert into previous months message shards and therefore the document has been indexed already if it is present.

Secondly, we tuned the refresh interval for indexing documents from 30 seconds to 1 second. This is a lot more intensive on the boxes we host Elasticsearch on but it’s worth it for the benefits it gives us. This is a simple configuration option within Elasticsearch itself. This means that a shards index is recreated every second to immediately make new documents available on it.

Lastly, there is an option you can pass called “refresh” when doing an insert into Elasticsearch. This tells ES to immediately refresh the index and make the new document available on the shard once it has been created. This again prevents issues around going to get documents that are not yet indexed.

At the outset, I said this was a story about foresight and blame. Picked up the backend services for our messaging application after it has been architected could have led me to curse my predecessors for not thinking about future functionality. And for a time, it did. However, I quickly came to realize that this is a repeated pattern in software development. You can never foresee the future and gaze into a crystal ball made up of 1’s and 0’s.

This thought was inspired by reading http://boringtechnology.club/ and the story of how Etsy built the activity feed and their battles with Memcache’s ephemeral nature. But you know what, after they fixed it, it worked later down the line even after scaling.

I felt the same way about the work I have done on our Messaging backend. I hope that, as Etsy did, they can leave those API’s and consumers to whir away and hum quietly in the background. It is a testament to how good your code is if 20x scale later it is still humming along nicely and a pseudo-metric I aim for with everything I write.

Credit: https://www.osnews.com/story/19266/wtfsm/

It has taught me a lesson not to march in and say “why didn’t they think of this when designing the system??? Time to tear it out and start again”. Instead, take a look at why the design was chosen in the first place, and to work with the design, rather than against it.

Legacy code often gets equated with bad code. But this is seldom truly the case. Legacy code contains bug fixes, resiliency, hundreds of hours of review and many different sets of eyes tweaking and refining. Every line of code written has a reason for being present — even the bad stuff.

Nonetheless, on the contrary, it has taught me vital lessons into thinking at scale and really diving into a technology before using it. Perhaps simply googling “what is bad at” or something similar. This can help you discover the pain points that others have run into. In retrospect, I would have used MySQL as a data store for our messaging service, since we have a vast array of experience in-house with it. We know that MySQL is not good for full-text search but we could use Elasticsearch or something similar to provide just the search functionality rather than the storage of all the data at a later point. Hindsight is 20:20 though, and who’s to say we would not have had issues with MySQL?

There is no real definitive conclusion to this article but here are the main points:

Don’t blame your predecessors for architecture designs or code that was written at the time, have respect for the code and leave it a little bit better than you found it
Go with technologies that people in-house understand and have experience with, you will run into issues you cannot anticipate if not
Hacking around can feel like, well a hack, but there is always a hack in every large bit of software. So don’t be afraid to work with the tools you’ve got to make them work, it’s a lot easier than ripping everything out
Structure your code in a way that makes it easy to rip out one data source and use another
Remember that if a technology has an advantage, there is usually a disadvantage. Especially databases, where there are hardware limitations at play. For example, Elasticsearch has great search functionality, but consistency can be a problem.

Using Grafana for Monitoring you NodeJS Apps

2019-03-06T22:12:03Z

This guide assumes you already have a basic NodeJS API and a Graphite instance configured.

Graphs are a great way to monitor your services, and as an added bonus — they look cool.

I always looked at companies with giant flat screen monitors with pages of various graphs and thought that was all way over my head. Turns out, it’s surprisingly easy.

At CloudCall, we are heavy users of Grafana and a data source behind it called StatsD. We use it to monitor throughput on services as well as general tracking of if a certain service is being used. Here is a guide on getting your shiny API monitored using Graphite.

First thing you need is a Grafana instance. It’s easy to get setup with Grafana — the installation docs are all you will need. You could spin this up locally or run it on a VPS or similar.

After you have picked the route you want to start work on you can run

npm i -s node-statsd-client

This is the statsd client we will be using. StatsD runs on a UDP port and this library simply pushes a string of data via UDP. There are a number of other statsd libraries out there, but I have found this one to be the most reliable — your mileage may vary.

Next up is to create some kind of wrapper around the statsD client as I found it difficult to use (and also not typescript ready).

import { Client } from "node-statsd-client";
import { IGraphiteController } from "../interfaces";

const PREFIX = "MY-API-NAME";

export default class GraphiteController implements IGraphiteController {
  private _client: any;
  // the graphite port will always be this for every environment
  private _port: number = 8125;
  private static _instance: GraphiteController;

  // Check if we are running tests, if so, deactive the graphite logging
  private testing: boolean = process.env.NODE_ENV === "testing";

  constructor(config: any) {
    if (!this.testing) {
      try {
        this._client = new Client(config.statsd, this._port);
      } catch (err) {
        throw new Error(`There was an error connecting to Graphite: ${err}`);
      }
    }
  }

  public static getInstance(config: any): GraphiteController {
    if (!this._instance) {
      this._instance = new GraphiteController(config);
    }
    return this._instance;
  }

  public write(activityType: string, error: boolean = false): void {
    if (!this.testing) {
      this._client.increment(
        `${PREFIX}.${activityType}${error ? ".error" : ""}`
      );
    }
  }

  /**
   * Writes a graphite timing
   * This is used to measure the time a function or piece of logic takes
   * @param  {string} activityType
   * @param  {Date} startDate - a new Date() object. Create this as a variable at the top of the function and then pass it in
   * @returns void
   */
  public writeTiming(activityType: string, startDate: Date): void {
    if (!this.testing) {
      this._client.timing(
        `${PREFIX}.${activityType}`,
        new Date().getTime() - startDate.getTime()
      );
    }
  }
}

Most importantly, we a static *getInstance()* method. But why? We found that, long lived services (anything not serverless) would create a massive amount of UDP connections over time and eventually make it so the service could not create any new connections. We use this getInstance method so we make sure we use a single connection throughout the app.

So how do we implement it?

First we need to import it into our route file

import GraphiteController from “./graphite”;

Next, at the top of the file let’s get an instance of the graphite controller that we can reuse in this file. We create it with the configuration that contains the url for the statsd instance.

const graphiteController = GraphiteController.getInstance(config);

Ok, now we’ve got an instance of the controller we can now record some data.

The first piece of data to record is a simple counter to see how many times a certain route has been used. This is useful because it might be that you need to pull out certain routes into new services so that they can be independently scaled. It may also be that a route was written but never used, so this can be factored out. Let’s add this counter — make sure to add it as the last thing in your router, right before you return the response. That way, if there are any errors along the way then we will not get a count that didn’t succeed.

Here is our router file now

import { Router } from "express";
import { Config } from "./configuration";

// Import or require the graphite controller and activity labels
import { GraphiteController, GraphiteLabel } from "../graphite";

const router = Router() as Router;

// Load your config
const config = Config.getConfig();

// Import the graphite controller
const graphiteController = GraphiteController.getInstance(config);

router.get("/", async (req, res, next) => {

   try {
       const record = await UserController.get(req.query, req.jwt.accountId);

       if (record) {
           // Just before the response is sent, we log the route being called
           graphiteController.write(“GetUsers”);
           res.json({ success: true, data: record });
       } else {
           res.status(404).send();
       }
   } catch (err) {
       next(new InternalServerError("There was an error getting data from the database"));
   }
});

export default router;

Ok, now we’ve setup a log counting the number of successful calls the route has, we should also track any unsuccessful calls to the router. This way, if you see a sudden spike in errors, you can see exactly where that error is occurring.

Add this to the catch block in the router. The second argument defaults to false but when true, signals that this call was an error. In the background, this will append .error to the UDP message, meaning you can filter those calls separately.

graphiteController.write(“GetUsers”, true);

Next we will add time tracking to the route. This way we can see how long a certain route took to execute. This is a great piece of information to have as you can see if you perhaps need to upgrade the machine your API is running on, or where to focus optimization efforts.

To do this you need to record the start time that the route was called. You can do this simply by

const startTime = new Date();

Next, underneath the .write method you added earlier you can add a new call to the write timings method.

graphiteController.writeTiming(“GetUsers”, startTime);

Here we have passed “GetUsers” as the activity or label for graphite and the start time of the route call. In the background, the wrapper calculates the time between the start time and the current time.

And hey presto. You will now be pushing the route timings to StatsD!

At this point, try calling your API endpoints a few times so we have some data to work with. Next, we can move onto to creating the graphs.

Let’s first create the “count” graph that tracks how many times and when the route was called.

Create a new dashboard and you should find yourself on a screen like this. I would recommend creating a new “dashboard” for each API/Service

Move your mouse over to the left and click “Add Panel” in the little menu that pops out. Then click “Graph”

Now, click the graph and click “Edit”. Now we can add a data source for our graph.

Set your data source as your Graphite DB so you can now perform queries for your data. You will need to build up a query like this:

Let’s break this down

* — This is a wildcard query as we do not need to narrow it down just yet
“MYPREFIX” — this our prefix that is configured in the GraphiteController at the top of the file
“GetUsers” — the graphite activity or label, you should use a descriptive name depending on which route you are working on
“Count” — this is the counter, there should also be one for “timer”
“consolidateBy(sum)” — This rounds the values so that you get only integer values rather than decimals
“alias(Get Users Count)” — the alias label for the query, this applies when you show the averages, min and max values.

You should now have a graph like this

For our “error” graph, simply repeat the steps and change the query to add an extra “error” metric — so the full query will be something like * * MYPREFIX GetUsers count error.

A similar process is taken for the timing graph, simply change the query from “count” to “rate” and this will be the timings for the route.

And there we go! Done and dusted! Now you’ve got it working for a single route, you can follow the steps again for all your other routes!

Did you find this guide useful? Would you like to see similar content in the future? Let me know on twitter @joshghent or in the comments below.

Why does NTP Exist?

2019-03-05T22:12:03Z

NTP is one of the most essential and complex systems that never gets spoken about. But why? And what even are they? And why do we need them? If you’re like me, you might have known about NTP servers and known they were important to keep clocks in sync. But don’t computers have clocks already? It was when I visited the Greenwich observatory recently that I realized how complex time was and with a fascination in both computer systems and horology, I decided to dive into the backbone of our lives, time.

What is NTP?

At a high level, NTP or the Network Time Protocol is a network-based protocol and standard by which distributed computer systems can synchronize their time with UTC. The protocol also defines a means by which these systems can passively listen to updates for upcoming leap second adjustments (more on that later).

How do they work?

NTP defines different “stratum” or tiers from GPS or atomic clocks (Stratum 0) down to computers synchronized to a microsecond (stratum 1) and so on until you reach Stratum 16 which NTP defines as unsynchronized.

The stratum number is used to measure the distance between a given device and the “ultimate” time source Stratum 0. This number means NTP can prevent cyclical dependencies too.

As you go down the chain, each “tier” is configured to synchronize with the tier above it. A given device in a tier may sanity check other computers that are in the same stratum (asides from stratum 0). Furthermore, a single computer may query multiple computers from the tier above to gain even more accuracy.

You may think your service needs to be ultra-accurate, therefore you should synchronize based on Stratum-0. Alas, unless you are working in Goldman Sachs, this precision is probably unnecessary. Stratum 1 is used for all primary time servers e.g., time.google.com.

Now how do we actually calculate the time? The client will go and query multiple computers for the time. The client then calculates the offset between its time and the time it received from the computers taking into account the round-trip delay. And hey presto you’ve got time! A computer usually does this sync around once every 10 minutes by dispatching a UDP packet to the desired synchronization servers.

Why do they exist?

This was the big question I was attempting to answer with this research. Why do we need them? Can’t computers keep their own time?

Keeping time is notoriously difficult on something that is not always going to be running like a computer. Furthermore, computers get hot, experience high loads and other factors which could affect it keeping the correct time even when it is on. Additionally, time is relative, literally. If it weren’t for NTP then satellites would not work, since they orbit the globe and therefore experience time at a much quicker rate than us Earth dwellers. There needs to be something else.

On top of all this, the Earth’s orbit is a bit awkward. An Earth “year” is not exactly 365 days. It’s 365 days 5 hours 48 minutes and 45 seconds. Since we can’t just round off that 5 hours, we have to put it somewhere. The Gregorian calendar does this by introducing a 24 hours day every 4 years at the end of February. Ok simple enough.

But now you learn that not only does it not take exactly 365 days to complete an orbit of the sun, but the Earth’s spin slows and speeds whenever it damn well feels like it. There are a lot of factors that can contribute to the Earth speeding up or slowing down but the main one is tidal friction. But other events such as changes in the convection currents within the mantel and earthquakes have slowed the earth down.

Either way, if we just left clocks to it, they would be off by a large margin even after a single year. NTP helps solve this problem by being able to broadcast changes in time and have that synchronize. Google and Amazons NTP servers both use strategies of leap smears, where a second is broken into small chunks and distributed between noon and the following noon UTC. If this strategy is not taken then a double second or skipped second is introduced depending on whether the time is being increased or decreased.

But why is it essential to keeping computer clocks up to date? To name a few:

Stock market — trades need to be nanosecond accurate to guarantee that a certain seller got a certain price from a specific buyer. Exploiting time is how high-frequency trading became so profitable (if you haven’t read about it I highly recommend Michael Lewis’ book Flash Boys).
Filesystems — accuracy about when a file was written or read aids in knowing which version is most up to date and allows the filesystem to set recovery points in the cache. This means when you hit CTRL+Z to undo, it always goes back to the last thing! It also enables things like table and row locking in databases by knowing if a request has completed or not.
Logs — you need them in order, and computers run very fast. Accuracy in time for these logs is important to trace through the exact order of operations
Update times — how many sites have you seen where they have something along the lines of “Last Updated 2 hours ago”. Well if it weren’t for NTP, you wouldn’t have that because if you were in the US and the update came from Japan it would say “Last Updated 5 hours from now”.

There are of course many other use cases for NTP and time synchronization as it underpins the clock of every computer out there. Java brags about being run on over 3 million computers. Well NTP, and all it’s implementations run on every single computer produced since 1985. So take that Oracle.

In any case, NTP sparked my curiosity in low-level systems and protocols and helped me appreciate the foundations of what we today build upon. Understanding them may not improve your code on a meaningful level, but it is vital to have a knowledge of the systems your code run to help you debug and improve performance.

Starting with Why as a Software Developer

2019-02-26T22:12:03Z

Developers are like jackdaws — “oh shiny!”

As you progress as a software developer, you should begin to build an innate sense of when something should be done a certain way, perhaps to future proof it in some way or make it more resilient. It’s an odd feeling, but you just get a sense for something.

I found this happened more frequently as I advanced and learned. But I never stopped to consider *why *I came up with the solutions I did, or a reason why I made a certain decision over another.

I’m a big fan of first principles thinking, and that concept really brought it home to me that to truly progress in my career, I need to start with the why. Understanding the reasons for my decisions and solutions and be able to communicate them in a concise manner.

Let’s give an example of this, we had a few resiliency issues with our SMS system. At the outset, coming up with a solution to this problem appeared to be a black and white exercise. But, I decided to dig a little deeper, why did we want to address these resiliency issues? Is it simply so we would have developer “cred” that our systems are robust enough to handle failures? Partially, but not entirely. The reason we wanted to improve the resiliency of our systems is so that the people using our product would gain faster feedback about the state of their message. If a message failed somewhere along the pipe, we need to make sure we shout back down that pipe to the customer so they know we failed. That is a much better user experience than a message escaping to the ether. Furthermore, we wanted to allow users to retry sending messages, so again this is a consideration in our solution.

Perhaps you might be a product-customer-centric-genius though and think this does not apply, but I bet there are solutions you have come up with where time has not been taken to weight up other options and be able to meaningfully justify the reasoning behind it.

A common example arises when choosing technologies. Developers are like jackdaws, they love new shiny things. Sometimes the new shiny things are amazing, for example serverless technologies would be like a gold ring. But other times, you might encounter things that are like figurative bits of foil. Looks great, and has its place but you already have some nice shiny foil, so is this better? Maybe not. This is not to say that technologies have no merit, but they have to be seriously weighed against the current approach and other options out there. Instead of saying “X will revolutionize how we work, it has Y and Z feature”, go out and investigate what advantages it has over other similar tech, what the learning curve is like and so on. Perhaps even create a PoC if you feel confident. All of these things will help you discover the why. Why do you want to use this technology? Why do you need to change technology in the first place? And so on…

As an example of this process in action, I considered porting a serverless based API written in Node to Golang. I wanted to do Golang because it seemed cool, Docker was written using it and because I had read it was incredibly fast — that was it. However, when I investigated, I infact found that Python, Go and Node have similar cold start times and there would be little advantage in porting. In terms of the processing we were doing, again, Node was just as capable as Go for that particular task and we had the advantage of all our libraries written in Node already. I decided against this and instead poured time into optimizing the existing Node API. This was a far better use of time that porting to some new language I barely knew.

When you’re looking at something new, or dreaming something big — start with the why. You will grow far more because of it.

Architecting the Next Generation of Communication

2019-01-23T22:12:03Z

With the shift to mobile and the statistics of the “younger” generation (hi there) not using phone calls as a means of communication, there is a constant push towards reaching people in a platform agnostic way — via email, LinkedIn, twitter DM, you name it. The challenge arises when you need to create a platform that is scalable demands and flexible enough to hack in any other new communication streams later down the line — maybe we suddenly want support for MySpace messaging.

The architecture I discuss below comes out of experience with this problem first hand, and the solutions we came up with — all to be delivered for a deadline.

Way of Websockets!

Since this needed to be real time in the case of IM or near-real-time in the case of SMS, websockets are the best way to go and a defacto standard for real-time operations on the web. For this, PubNub is a great choice since it already had a lot of the functionality baked in, such as different channels and mechanism to subscribe, send and receive on those channels.

PubNub also had a mechanism called “PubNub functions” whereby any new websocket message on a channel matching a certain pattern would be handled by a function written in plain ol’ javascript. This meant you can fire SMS messages off to other systems that would handle the actual sending of the SMS message and another route that sends WhatsApp messages to Twilio’s API for example. It provided immense flexibility, especially as you expand to different communication methods and channel types.

Although PubNub has it’s own data store in the background, you can only query it through their API, making it difficult to just dive into the DB and find let’s say all channels containing accountId 123. Additionally, you can only bring back 99 records at a time with PubNub which means producing accurate reporting a challenge. The solution was to introduce a second data source. This potentially opens up the problem of many sources of truth. This can be avoided by having an API stood in front of a non-relational database (I would recommend ElasticSearch) which all read operations would go through.

Typescript Time

With a project that is going to span many different API’s and services, Typescript would prove invaluable because it allowed us to reuse a lot of code whilst increasing developer productivity and reducing bugs. Sounds too good to be true right? Well, there were still bugs sure and productivity only took an upwards swing after all the developers got comfortable with it, but overall it was a fantastic move. One of the first things you should do is create a common “type” library that you can share across all of your services and systems that needed them. In this types library was all the interfaces and enums that were going to be used throughout the system. You can store everything there from error codes, to channel types and an interface for how a message was structured. You can then include this library in all your services to ensure consistency.

Different channels

To differentiate the communication types you have, group instant message, direct message, sms message, mass sms message, carrier pigeon etc. you can build this up as part of the channel name. Again, PubNub (who I promise aren’t sponsoring this post) gives great flexibility by allowing channel names to be whatever you want them to be. I would recommend they are built up with the platform, channel type and then a unique identifier .e.g, production.sms.123456. In your pubnub function, you can then check the channel type within the channel name, using regular expression, and handle the message accordingly.

Channels should be created per group of participants per channel type. For example creating a new sms to a contact creates a new channel, sending an sms to the same contact again will not create a new channel. But, creating a group with Bob, June and Sally called “Sales Call” and then another with the same people but called “Another sales call”, would create two different channels. This is how many other chat applications are built which in accordance with Jakob’s Law, is what you want to do.

Now that we have a basic instant message and SMS system, we had a new problem to solve — How do we get notifications to the user? Emit a message of a different type on the existing channels sounds like an obvious solution but it assumes the user is subscribed to channel. Fortunately, one way you can solve this is with a “notification” channel. Each account should be assigned a notification channel. Every time a message is sent, it is also sent to the participants notification channel.

For example, if Bob creates a new group chat with June and Sally, it will send a new message on June and Sally’s notification channels informing our application “hey there is a new channel you need to subscribe to!”. This will then trigger a process in the app to subscribe to that channel in the background. When Bob then sends a message on that channel, it sends another message on both the participants (June and Sally) notification channels. When this message is received by the application you can then pop a desktop or mobile notification depending on the platform.

Additionally, you can use this notification channel to send other kinds of messages like when a channel has been read, or when the user mutes, leaves or hides a channel. Utilizing the PubNub function again, these notifications can be captured and forward them onto a CRUD API which saves them in DynamoDB. This allows us to provide a consistent experience across any device that the account uses.

Some may be wondering why we don’t just call the API directly, but instead go through PubNub, this is to cater for the case that a user has both our mobile and the desktop application open at the same time. Sending the message via the notification channel means if you hide a channel on the desktop, it will immediately be hidden on the mobile app.

Authentication

Authentication can be a big hurdle when breaking up a monolithic architecture into microservices, this is the situation my company found itself in. Prior to developing these SMS/IM systems, users were authenticated to our backend using a username, password and license key. All requests to the API used these parameters. This was not an option when authenticating with PubNub as firstly we did not want to give them access to our accounts database, and second because it’s not an option on their system. A token based system was the only way. We considered a number of different options for token based authentication but eventually settled on JWT because of its flexibility, ease of implementation and security. Combined with this, we had found Kong along with the JWT plugin to be fantastic at handling all the traffic we threw at it.

An enormous amount of work went into not only overhauling the API to accept JWT authentication but also to change all our apps to handle JWT’s. Additionally, we required a refresh strategy for these tokens, for example, if a person remains logged in for a number of days it could be the case that your JWT token we have cached is now expired. This means we need to refresh the token. On any request from our application, we check how long the JWT has until it expires. If it is a day or less away then we refresh the token first.

We leverage the JWT to store information we need for requests, for example, when a request comes into an API, then we will most likely need the accountId, we can find this in the JWT without having to pass anything through with the body of a request.

There is more to tell with the architecture of a deceptively simple system, if you have ever had to architect your own communication platform, how did you do it? I’d be interested to find out and build a knowledge base.

Resiliency By Design

2019-01-19T22:12:03Z

Resiliency by design in your products architecture is a challenging problem that is rarely tested. Building robust platforms are becoming increasingly important as large server providers such as AWS start to show their cracks in addition to good old fashion human error (we had an engineer take down a server by knocking it with his ass). Chaos monkey and other tools have sprung up to pursue down resiliency issues, but despite this, they can still persist. Here are a few things to look out for when designing a new system or analysing existing ones.

Backing Off

Your app tries to contact a critical external service, maybe it’s your database, or perhaps a 3rd party API — whatever the case, it **will **fail on you. A common way to handle this is by setting a timer to retry the call to the service. At CloudCall, we have a stateless service to handle sending SMS messages, whenever it cannot save the message to the DB or send it to our SMS provider (or they return an error), we automatically requeue the message for a set time in the future. If we get the same failure the next time around, we requeue it again, this time for a bit longer and so on, until we get a success.

You may not think this is possible in the world of serverless — but it is! In AWS Lambda, if you throw an exception or use context.fail() then the lambda will retry up to 3 times before giving up. Although with this setup you cannot have the gradual back off, you are still getting the beauty of the retry. However, if you setup the Lambda with SQS you can also configure the lambda to DLX the message which can be set to requeue messages after any time you set.

Reconnection Logic

If a service does lose connection to a service it requires persistent access to, then we need some logic to reconnect to it. We can reuse the same principles from the backing off principles we discussed at the outset. If we cannot connect, try again after a time, then try again after a bit longer, and so on. Simple right?

But when your app boots fresh for the first time, it also needs logic to establish anything it needs in those services. For example, if you have a queue consumer service that maintains a connection to RabbitMQ, when it boots, it needs logic in there to assert all the queues and exchanges it needs. Often, because a queue publisher service has been written previously, **that **service contains all the assert logic. However, when it comes to deploying the queue consumer service, you hit errors because the publisher service was not deployed previously and therefore had not asserted the exchanges and queues the consumer needs. This creates deployment dependencies, which trust me, you don’t want.

Example structure of an app with fail over

Infrastructure Failover

Of course, no matter how much code you write to cope with services being down will have no bearing if your whole server goes down. With the advent of AWS, Azure and GCP, many consider this a thing of the past (99.99% is basically 100% right?), despite this, these services *will *go down. It is essential then to configure automatic failover, unless you enjoy getting woken up at 4am to redeploy an entire environment to another region.

Maybe the entire server doesn’t go down though, it could be that the app is just crashing and you need to restart the container or maybe even the entire server to get it to startup again. In these cases, auto heal mechanisms should be put in place. These mechanisms can restart the service or in some cases, redeploy it elsewhere, should it go down in the primary zone.

Be wary of distributed monoliths

The world of microservices is taking over. The potential it creates in terms of flexibility and reusability are incredible — hence why it is so widely used. Nonetheless, they come with their own trade offs, namely in the structure of them.

One of the main arguments you hear in favour of microservices is that means you no longer have one monolith you are dependant on — like the Death Star for the Sith. But when designing their microservices architecture, the services are just daisy chained together and completely reliant on each other. To negate this, make sure your microservices are exactly that, microscopic. Be wary of clusters of microservices that share a data store, or when changes to one service requires a redeployment of another. Most importantly of all, ensure that the services can scale independently of one another.

Building resilient services can be a challenge and it does take time. Even just configuring the auto availability zone failover in AWS took a long time and consideration by some talented engineers to solve. Like with anything, there is quick wins and acceptable known faults in the system. If you don’t have time to configure auto heal and failovers, make sure you have a process written so anyone can do it manually. They all aid to delivering a optimal system and reliable user experience, but most importantly, you can sleep soundly without getting called up, safe in the knowledge all your servers are humming along nicely.

Let me know any other tips you have for creating resilience in systems!

How to Run a Successful Tech Meetup — even if you’re forgetful

2018-12-22T22:12:03Z

A picture taken from the November 2018 LeicesterJS meetup

LeicesterJS is born out of the rise of Javascript now being the de facto programming language for a majority of developers. Additionally, we aimed to bring together a tech community in Leicester.

I have been running LeicesterJS now for over 4 months and this is just the start. Before the first meetup, I was very nervous about getting everything together and although it went off without any dramatic hitches, I have learned a great deal and continue to do so with each and every event. Here is some of the advice I wish I had before planning my first meetup.

Start with the essentials

The barebones requirements from a meetup is a venue and catering. For LeicesterJS, I was able to partner with my current workplace to host the event and foot the bill for the food and drinks provided. In exchange, we give them a plug at the start of each event and they receive exposure to potential hires. Often meetups will be arranged in this manner as it makes organizing the event a lot easier, since you, the organizer, are at the venue ready to prepare anything that is required. If you are looking to arrange your own meetup, kill two birds with one stone and look to see if your employer wants to help.

Next up is the speakers, for this, I turned to developers in my team and asked around if anyone wanted to give a talk. You can also reach out on twitter or to other developers you know in the area. Worst comes to the worst, do the talk yourself!

And with that, we had a meetup! Except for one key thing…

….People

You want your meetup to be a success and part of that success can be measured by the attendance rate. Here is how I approached building a community of people from the ground up

Get support from co-workers — since they know you, it will be easier to break the ice that having a room of strangers
Link up with other meetups in the area via Twitter — I was very fortunate to have the generous support of PHP East Midlands who cancelled their own meetup to join in with the first LeicesterJS (thanks again guys!). It was great to have them along and again, a few more familiar faces
Connect with a tech Slack group — in my case, TechNottingham has a thriving Slack community, I advertised the meetup in their #javascript channel
Personal social media — in addition to using branded “LeicesterJS” twitter account. I further advertised the meetup with my own personal Twitter and LinkedIn. Additionally, you can encourage your co-workers to put the word out too
Word of mouth — you’d be surprised how many people just come to events because they saw it on meetup.com. From my purely anecdotal data gathering, it seems that people mostly came because they saw the event on meetup.
RSVP’s matter — A day before the meetup starts, it is good to encourage people to update their RSVP status if they are no longer planning on attending. Often times, you will only get 60–70% of the RSVP’s you have on meetup. Make sure you keep track of this KPI and encourage the “regulars” to your meetup

Download the full thing here: https://gist.github.com/joshghent/2bb9c6e2ce616e29aaa3c7a2895cb17d

Processes

After the first meetup, I had a number of people come and ask me for the slides to the talks that had been given. I hadn’t even thought about that! There were a number of these sorts of things that I had to get a process nailed down for.

Talk slides — stealing an idea from NottsJS, I decided to create a Github organization and put the slides in a repo. It was the least frictionless way to distribute them. I considered using Google drive or some other file sharing service but this would give me difficult URLs that audience members may have a hard time finding
Talk ideas — after having a number of inquirers about giving talks I again turned to GitHub. The process is now to submit a talk idea as a Github issue on a repo. I have a set template for the issue so there is not any information left out that I may need at the last minute
Somewhere to talk — despite the bounty of slack groups out there, many requested to set up a Slack group to discuss the meetup and the tech community in Leicester at large. This ticks along nicely. I used Slackin on a free Heroku box to create a site for people to get instant invites
Introductions — the first meet had some very basic introduction slides but I’ve gradually fleshed these out to make sure everyone feels at home. Basic things like pointing out the bathrooms and telling people what time the food will be here and the meetup is expected to end can be very useful to people. Another additional tip is I get people to raise their hand if they with me (and therefore, at the office, we were in), I informed other audience members that they can also go to those ones if they need any help

Other Pointers

After getting through the above, it can seem as if there is nothing more to do — just grow the meetup right? I fell into the same trap too. But a further refinement came first by introducing a code of conduct. In this document, it outlines how not only speakers, but all those in attendance, should behave and having clear processes for people who do not. It’s worth grabbing a document that has already been written because in all likelihood you are not a lawyer (but if you are, fill your boots!). Additionally, it’s good to have a point of contact (an email address and a person named) who should be contacted if someone wants to report something. Although you hope that something like that will never happen, there has been far too many cases in the tech industry alone that have highlighted that problems with intolerance or any sort of bad behaviour towards others is unfortunately not uncommon — often it goes unreported, I hope that never happens at LeicesterJS. Nonetheless, it’s essential to have.

Additionally, you can offer yourself as a resource if any potential speakers need help preparing a talk. Since its beginning, I’ve encouraged first-time speakers to give talks. This can be a nerve-racking experience so give generously to offer support with slides, talk topics etc. The meetup group slack channel can be a great place to do this.

A lesson I learned early on was to encourage audience participation. It makes the meetup more active and motivates people to discuss afterward as well. Perhaps recommend that your speakers open the floor so to speak with a discussion. Asking if anyone uses the technology being discussed, or ask them if they use an alternative.

Starting your own tech meetup is an incredibly rewarding pursuit, but don’t underestimate its work. It’s also worth noting that, if you are in a smaller city as I am, it is better to support any existing meetups than to create your own. Ask the organizers if you can assist in any way, perhaps arranging new speakers or handling promotion of the event, they would welcome anyone to lend a hand.

If you are in the Leicester area, come along to LeicesterJS — the meetup is every 3rd Thursday of the month!

I don’t know what to say…

2018-12-04T22:12:03Z

The issue raised for the event-stream breach. It’s a grizzly flame war that I would not recommend reading

I’m a little late to the party here but after having a couple of conversations at work and with others I wanted to document my thoughts on the recent security issues around the event stream npm package which was used by lots of popular packages such as nodemon.

Surrounding this controversy was many questions about vetting packages more carefully as well as the tendency for node developers to just have a package for basic functionality they could implement themselves (looking at your left-pad).

Whilst all these questions are valid and worth discussing, this is not what I wanted to talk about in this blog post.

I want to talk about giving back to open source.

Most notably around all the discussion around why the original author handed over ownership to “some random”. People questioning how “dare” the author hand over ownership. Others even suggest a conspiracy between the package author and the perpetrator.

I had co-workers approach me saying they couldn’t believe someone did that and how it was “stupid” to hand over permissions to another person.

But was it “stupid” to do so? Well no.

This is open source software that is completely free. It was relied on by hundreds of packages and it alone had over 76M downloads. Yet, despite being depended on so much, it was not financially backed and was a labour of love — as so many open source projects are.

When you are a maintainer of a project, especially if it is popular, there may be many issues but not enough time to fix them. People often simply complain that the free tool they are using often for enterprise software is not working. I myself have handed across ownership of a project and given them full write access. I still take a look at PR’s every now and again but for all intents and purposes, it is their package. I was originally contacted when someone posted an issue about how they wanted to work on it for a class project, I was delighted! Someone wanted to actually spend their time working on a project I originally authored. It never even cross my mind that they would do anything malicious and even if I did, in the politest way possible — it’s not my problem.

Rather than looking at this malicious package and thinking you either need to abandon node js or reject the whole concept of open source, show appreciation for the packages you use. Maybe get your company to give back either by either donating development time or financial resources to it. All other assets in a business are paid for — so why shouldn’t the underlying pieces of your code base be? It is as black and white as this — if the npm library disappeared, a lot of companies would be in serious trouble, yet only a small fraction of libraries receive support.

It is comforting to know that the node ecosystem is not isolated to this issue. Just last year, Equifax blamed Apache struts for a breach to their entire customer base. Apache had fixed the issue but they had failed to update their servers — yet blame was still pinned to that project. From my research, I can find no record of them donating to or sponsoring the Apache foundation — yet their net income last year was $587M.

The message is this, go and check out some packages you use a lot across your projects, see if any of their issues need help. If you do not have development time, check out how you can donate to packages with “npx thanks”. If you cannot do either of those things, leave them a star, there is even an npm package to do this. Just don’t complain about an issue in software you make money off, which you originally got for free.

10 Things I wish I knew before giving my First Tech Talk

2018-11-13T22:12:03Z

Giving the talk — credit https://twitter.com/JamieTanna/status/1029428095223320576

Glossophobia or fear of public speaking is cited as being amongst mankind’s top 10 fears. It related to our inherent fear of failure. Although I have never been afraid of speaking publicly, for even the most experienced speakers, it can be a bit nerve racking at times.

Why did I choose to give a talk then? For one, I wanted the experience, ever since going to my first meetup I thought “that’s really cool to speak about stuff you’re excited about”. In connection with this, I enjoy teaching people, whether that be 1-on-1 or to a group — it’s one of the reasons I contribute to open source, and write blogs. It’s a creative outlet. Overall, my primary objective was simply to share something I’m passionate about and also try and make them laugh — emphasis on the word “try” there.

My first talk was at the NottinghamJS meetup and was titled “Lightning Node Performance”. I’m hugely grateful to the organizers for giving me a platform. Previously they had people from Amazons Alexa division, Microsoft's Machine Learning team and more — so it seemed as if I had big shoes to fill.

But giving the talk is the ending, let’s start at the beginning with things I wish I had known when preparing my first talk.

Preparation took longer than expected

First and foremost, the preparation took a long time. A long time. Initially, I had expected creating the slides and writing the talk to take around 2 days. It actually took over a week — plus all the additions I did late at night and changes to the content of the talk on the day it was supposed to take place. If there is any mistake I made, it’s I severely underestimated the time it would take. It gave me a newfound appreciation for any content I consume, whether that be talks, videos or podcasts. It takes a lot of time to prepare these things. Perhaps why criticism can hurt so much.

Part of the reason the preparation took a long time was I wanted to make sure I was 100% concrete on every last word I said — in case someone picked me up on it and tore the entire talk to shreds. For example, part of my talk was speaking about the NodeJS event loop. Although I know roughly how the event loop works, there were still some questions I did not know. I thought that perhaps someone may ask me about the Node event loop and therefore, I set on down the rabbit hole to explore. This kind of pattern occurred at least 6–7 times when creating the talk and accounted for a large proportion of the time I spent.

The image I created for Node js clusters

Moreover, I wanted to keep the slides almost completely visual. I wanted to keep words off the slides because I have observed people read those rather than listening to you. Finding images for NodeJS clustering is harder than it looks though and so again another time-consuming task was pouring over pages of gifs and images to find one that perfectly encapsulated the subject matter. Often times, I created my own in Photoshop, which again took a large portion of time — primarily due to my appalling photo editing skills.

Choosing a topic is tricky

In connection with preparation time, it also took a long time to come up with a topic. Since I was not from a company, I wasn’t presenting any one particular “thing”. Therefore I went with a more general topic “application performance”. This proved difficult because it’s so broad and had so many subtopics I wanted to cover. For example, I wanted to speak about lambda cold starts, network resilience, asynchronous code in node and much more. Each one in of themselves could have been a talk in their own right. Therefore, a balance had to be struck between covering lots of topics briefly and covering a few topics in depth. I hope I eventually got that balance right, but it’s hard to tell. In the future, I would suggest coming up with a concrete outline in parallel to thinking up a topic.

Not all points are equal

This is a lesson I learned after the fact of presenting the talk. Not every point deserves the same amount of time. Spend more time on the difficult to understand topics and breeze through the small minor points. There is often a sunken cost fallacy at play here, whereby you take lots of time to prepare all the slides so they each deserve their own ceremony. We should try to get rid of this thinking and instead prioritize the points covered. Ordering your points carefully can aid with this. No one wants to be bombarded with lots of heavy topics all in one go, so spread them out and interleave them with smaller, lighter points.

Your delivery can get a bit wooden!

Practice, practice… but not too much

Practising your talk is essential of course but you can practice it too much. At a certain point, your delivery could become too scripted or wooden. Rather than attempting to memorize a script, remember the points you are covering. Then just speak. If you have the subject knowledge then this will produce results. Furthermore, speaking from within rather than from notes will vary your talk in different ways. I found that when practising my talk, I would do it a different way each time, adding anecdotes and talking points and cutting others. This was done at an unconscious level and would not have been achieved if I were reciting verbatim.

There’s an NPM module for that — https://twitter.com/iamdevloper/status/487606612757315584

Don’t fear questions

Questions are fantastic for people to get further insight on what you spoke about and can often reveal places where the talk should have explained a point further or provided a different angle. I didn’t so much fear these questions, more expected the worst. But the questions were about the talk. I did get a couple about technologies I hadn’t heard of but I can hardly be blamed for that — especially in the JS world. Overall, the questions were about the talk and asking me to expand on certain stories I had told about how CloudCall was doing this performance improvement work.

I learned a lot from the whole experience, briefly here are my takeaways.

Upload your slides to GitHub and Slidedeck

One thing people always ask for with talks is where can I get the slides, so make them easily available. Creating a repo called “talks” and upload the file there, and upload them onto Slidedeck for those who may not have powerpoint/keynote.

Visual slides worked well

A picture says a thousand words. Words on slides should be avoided at all costs unless they are used to re-emphasize a point. You can explain much more with visuals. For example, rather than putting a slide with the conclusion from a study, put a nice chart up there with the numbers behind the study.

Avoid lots of code on slides

Code on slides are similar to words on slides. They should be used to make a specific point. Try to keep the code as short as possible, using an extract if possible. It’s not essential that the audience has a complete context around a program.

Slow down delivery

When I gave the talk, I think I rushed a little. It’s a nerves thing I suppose. My advice is to just count in your head 1–5 between points and 1–10 between slides. It will seem like a lifetime from your point of view, but it makes the delivery far more fluid.

Engage with the audience rather than speak to them

My talk was that. A talk. I hope the visuals were enough to keep people engaged but in the future, I will make an effort to ask the audience questions and engage with them further. For example, I may ask the audience if they have any experiences with dealing with X after explaining how I did it.

Since my first talk, I have given a couple of others and want to do more. It’s a good experience but takes a lot of time. Be kind to those who give talks and give constructive feedback, as they have sacrificed a lot of time to deliver this. And if you are interested in giving a talk — do so! Ask the organizers of the event and I’m sure they will be happy to pen you in. If you are in the Leicester, UK area and would like to give a talk, post an issue on the LeicesterJS speaker’s repo and I will get it in the diary — we want to encourage first-time speakers. If you have given a talk, share your experience — it’s good to break down some of the fears people may have.

Networking at Tech Meetups

2018-08-11T22:12:03Z

Tech meetups and talks are a great way to get to know fellow developers in your locality. But it can be challenging if you are introverted by your nature. Although you may not be introverted, some find it challenging to approach people when they first attended a meetup. Networking is a core part of why many attend meetups — whether to find a project to work on, a new job or just a friend. This article is motivated by knowing my past-self and others would benefit from how to network at meetups.

First of all, know who attend meetups. Developers! Rather than being a source for discomfort, you can look around the room in the midst of Gitlab hoodies, beards and sticker covered laptops and breath a sigh of relief. These are your people. Developers are like spiders, they are more afraid of you than you are of them. So just approach them! They won’t hurt you.

A primary source of fear comes from not know what to speak about once you approach someone. At a meetup, beyond the “hey what’s your name” basic stuff, I ask “What project are you most passionate about at the moment? — whether at work or in your spare time”. Or perhaps “Are you going to use any of the tech that was mentioned in today’s talk?”. A safe starter question is just to ask them what they do day to day at work. Be interested in people. People are interesting and always, especially with a developer, excited about something. Find what they are excited about and then drill down on it.

Avoid questions that can be answered with “yeah it’s alright” or “yeah good”. These are Boolean questions. Instead of asking “Did you enjoy the talk” ask, “Is there anything you’re going to apply today from the talk in your work? I’ve not used X technology before but the principles carry over into Y project I’m doing”. The main object of these questions is to get the person talking. People will latch onto you if they do most of the talking.

Often groups can form and so it can seem like you’re butting in on the conversation but don’t fear this! Either look around for a couple of people sitting down or try and ease your way into the group. Don’t feel awkward, just start listening to the conversation. Be careful not to already ask a question that may have been asked previously. Latch onto some new information that is mentioned and ask about that.

Don’t be afraid to end a conversation, you can’t stay at the meetup all night and you want to speak to more than one person. Simply say “It’s been great chatting with you, could we continue this? I have some interesting questions for you. What’s your email and/or LinkedIn?”. Then hand them your phone to put your details in. Keep it professional so try to stick to professional forms of contact (usually email, LinkedIn or Twitter).

After the meetup, try to follow up with an email with a question based on what you spoke about. Keep a template handy that you can fill in for speed.

By even your second meetup, you’ll find it a lot easier to begin speaking to people and the initial barrier will be cut down. Becoming a regular, you’ll also get to know friendly faces so you can check in with them to see how their project is going or their job hunt.

Although these tips are to help you there will be times when you say the wrong thing or blurt something out at the wrong time. But don’t be afraid, we all do it. Even the queen wears underwear as they say. Networking shouldn’t be shied away from for fear of social embarrassment or for looking like a yuppie, it’s a critical part of your careers development and could potentially open many doors for you — it has for me.

If you would like to read more, I discuss similar topics as well as more in-depth technical posts here on my blog. I also tweet here.

Tracking Goals in Todoist

2018-07-27T22:12:03Z

“There is always an app for that” is a phrase I heard repeatedly when I was looking at something to keep tabs on my goals, both short and long-term. But you know what, I don’t want an app! I began to consider using Todoist, my task management app for doing this. After all, it’s one of my most used applications so there is less likelihood of me forgetting about it on the back page of a folder. Instead, it would be right front and center. Whilst I’m sure that all those goal management apps have some useful feature, you might be interested to see how you could use Todoist (or indeed any task management app) to keep track of your goals.

The idea of “task management” may begin to make your goal seem more like a chore but, on the other hand, it makes you think of the goal in more segmented achievable chunks. This really aided me as it helped me apply the age-old advice of being specific and actionable with my goals. Rather than go to the gym, I create tasks to go to the gym on certain days. Additionally, tasks have completion dates so you can set each of those segments against a date to check in with the goal you are attempting to achieve.

Now I’m going to dig into the specifics of how you can set your task manager up to work with your goals. Although this article is geared around Todoist, most task management/productivity type apps will have this. Other apps you could do this with are Omnifocus, Notion, Things, Any.do and Microsoft Todo (formerly Wunderlist).

Create a goal project

The championship is so close!

The first separate is to create a new project to separate your goal based tasks. In Todoist, you can create nested projects so I created a parent project of “Goals” and then child-projects for each goal I had.

Create Recurring Tasks

Using Todoists powerful recurring task functionality you can now create small tasks within each of these goal projects that is set to recur. For example, here is how I set up the “Gym” project.

Notice I have taken full advantage of recurring tasks by creating individual work out tasks. I’ve also made sure to have a task that can be chalked off every day to make sure I’m always chipping away at my goal.

Incorporating some Getting Things Done principles, I’ve created a recurring task to review the progress of this goal. This could be in the form of a journal or a comment I put against the task. It keeps me honed into the goal’s original purpose and focused on the week ahead.

By having this all planned out in advance you will never say “Oh no, I forgot to pack my gym back so I can’t go to the gym” or “I’ve had to get McDonald’s today because I hadn’t made lunch!”. It’s already in your task management app, ready for you. You don’t even need to think about it. This lowers the barrier to entry and reduces the number of “excuse vectors” — the number of different ways you could make excuses for not achieving your goal.

Use Labels and Reminders aggressively to provide context

In Getting Things Done, David Allen discusses context around certain tasks. In our example above, we can’t pack our gym bag at the office, because at that point it’s too late, we need to do it before we leave for work.

Context provides that assistance to highlight the tasks you can and should do right now. Todoist enables this by providing time-based reminders and labels. I would suggest creating reminders for the pack bag task for right after you wake up so you do not forget. Additionally, trigger the press-ups tasks when you arrive at your home location. Location-based reminders are a premium feature but you can just as easily set it for the time you get home from work.

Although not applicable in the example above, labels can provide extraordinary power. If we were learning to code, you could create a task to read a programming article you found and tag it with @out_and_about — this is a label I use for when I’m not physically at home, these are the sorts of tasks I can do on my lunch break. Other tasks such as “Add new API endpoint for getting users” would be a task I label with @home and @deep_work as it will require me to be at home and concentrate on that task for an extended period of time. When looking at what I should do next, labels like these help me to weigh up my motivation and available time.

Start Doing!

As much as all these fancy tools assist in visualizing your goals and the tasks that make them up, there is no replacement for real solid action. There is a vast multitude of facets to forming habits and achieving goals from the practical steps right through to the neuroscience research; that all goes well beyond the scope of this article. All in all, however, just do and be realistic about the time these tasks will take. Now, what are you waiting for? Get cracking!

Lessons from Open Source

2018-07-13T22:12:03Z

Contributing to open source is often touted as a great way to be recognized in the software development community, with many heralding their Github profiles as a resumé of sorts. Additionally, open source software developers find their programming abilities enhanced and motivations for their day-jobs recharged. Beyond these, however, there are further lessons that can be learnt from contributing to open source.

Code Ownership

When I first took over as maintainer for an open source project, I found myself with a good sense of how the code base should be. It was my baby that I had cared for and was trying to improve. When I began to encourage the community to add to the project I began to see strange new solutions that I would not have chosen. Moreover, I was cautious about appointing anyone else as a maintainer to the organisation I had created.

I was not focused on delivering features and bug fixes but instead on how I perceived the codebase *should *be.

“No, it should look like this!”

It highlights an interesting point, you can’t be precious about the approach. Problems can be solved many different ways. There is more than one way to skin a cat as they say — and it could not be truer when it comes to software development.

Your goal as an open source maintainer should be to enable and encourage people to solve these problems however they like. The specifics of the code should be ignored and instead focus should be given to how well documented it is and how well tested it is.

Of course, sometimes a certain approach is more convoluted than perhaps necessary. In these cases, discuss the reasons why the person went for that approach. Perhaps they tried the approach you were thinking of, but it did not work for whatever reason. Don’t go in all guns blazing, as the more the code changes the more it might favour one solution over another.

Communication

Open source by its nature is open to basically anyone with a Github account. Therefore, people who stumble across your project and want to contribute to it may not be from the same time zone or have English as a second language. This can often lead to miscommunications. Therefore, it is best to make a concerted effort to ensure there is no ambiguity with what you are saying. Furthermore, different nations may have certain customs in their language that whilst might offend you, are thought of as nothing from others. This can be the case in reverse too, so be mindful of any language that could offend others unnecessarily.

Emotion doesn’t always travel well on the internet

An important thing to bear in mind when communicating with either maintainers or contributors of open source projects is that they are doing this in their free-time unpaid. With that in mind, you need to be wary of pressuring people into timeframe commitments or being overly critical in merge request comments. Make sure you are kind and considerate throughout. This principle applies to your day to day work, sure there might be times where ruffling some feathers in needed, but buy-and-large it pays to be positive and encouraging.

Writing

Beyond writing code, there is an even more important, yet seldom thought of, form of writing — documentation. Critically in open source, if you want people to use your thing — you gotta tell them how to use that thing!

Where is the “any” key?

This presents an interesting challenge, you need to write your documentation in a manner that suits your target audience. If your application is aimed more towards beginner programmers then gear your documentation towards that. Don’t assume someone already has Node or MongoDB installed, show them how or point them to further guides where they can learn. A good way to hone your documentation writing skills is to discover a new API and write documentation for it. Dig through the source code and find out the usage of that endpoint and what it outputs. Since you’re approaching that project as an outsider your documentation will naturally lend itself to that audience and will provide an outstanding benchmark for how future documentation you will write.

As with all writing, the point is to be clear, concise and easy to understand.

Open source software is a lot more than just adding features and fixing bugs, it’s about the people. These lessons are important to embrace for use in a real-world environment and will prove invaluable. If you’re looking for a place to start contributing Up-For-Grabs is great. Alternatively, find some software you already use and if it is open source, try and tackle an issue on their Github page. Open source has opened many doors for me and furthered my career more than anything else I’ve done — and it can with you too.

The Art of Good Code Review

2018-05-30T22:12:03Z

Code review is a critical part of any software development process. In theory, it is designed to broaden system knowledge amongst the team and ensure that the code is maintainable and easy to read. Perfecting code reviews can be somewhat of an art, it requires a balance of being picky and not sweating the small stuff.

Before we dive into the principles around good code review, we should first define what it isn’t. Many believe that one goal of a code review is to address any styling issues. With the rise of Prettier and ESLint, arguments about code formatting have become moot. Automating formatting with the use of tools removes the personal opinion and sway from any one person. Additionally, this also has the benefit of formatting the code more consistently across all areas of the code base.

Other believe that code reviews should be to catch bugs. Whilst this can be true if you spot something glaringly obvious, it is often challenging to understand the code in a wider context. Bug catching should generally be left to a set of watertight unit and integration tests as well as your top-notch QA department.

Now we know what code review isn’t, what makes a good code review?

Know when to say “it’s good enough”

Although the aim of all software companies should be to deliver A-class code, there is such thing as too nitpicky. There is no need to find fault with every little niggle of code. This will only end up demoralizing the developer. It’s a fine balance between picking at the fine details and not going overboard. Only you can be the judge of that and time with hone your perception of done. I often find that overly-nitpicking comments can come down to personal preference of the reviewer. These types of comments should be avoided whilst instead focusing your attention to actual issues in the code.

Stay Positive

Code review can often seem like the reviewer has marched in and torn to shreds the priceless artwork that you’ve digitally sculpted. Instead of focusing only on the issues, often time it’s good to step back and reflect on the positive attributes of a merge request. Maybe someone has used a kick-ass language feature you had no idea about. Or perhaps they’ve just written an informative function header. Commend them for this type of work. It lifts the spirits of the developer and will mean they are more receptive to the issues you do find.

Look out the output not approach

A key part of code review is analysing the output, not the approach they have taken. There are many different ways of solving a problem, and it’s important to avoid enforcing one solution over another.

In some cases, a developer might not be aware of another approach. For example, let’s say they have written a method that loops through an array to find a matching element in that array. They might not be aware that Array.includes (in Javascript) could solve their problem! In such cases, it’s best to go and talk to the developer in person. It might be the case they cannot use Array.includes. You are most likely tackling this code base as an outsider so it is best to assume that the author is the expert.

Be constructive

When you find an issue in a merge request, which comment do you think would be better?

No. Don’t do this.

Can we change this to not include X as this may cause Z and perhaps do Y instead? What do you think?

In the first approach, the comment is not constructive, not only does it not tell the author the reason for the issue, it also doesn’t suggest an alternative. On the other hand, the second approach suggested using Y instead of X because Z might happen. You might have more experience on a code base to know certain ins-and-outs that the author might not be aware of, your comment will mean the developer will benefit from your wisdom. Additionally, the second approach asked what the author thought. Rather than ruling with an iron fist, it opened up a discussion.

Code review is a fantastic way for all developers of any experience to get their heads stuck into different parts of your companies code base. Furthermore, it’s a great way for developers to gain experience on new approaches to solving problems. If you don’t already, try and include code review into your process and involve the whole team.

Principles of Performance

2018-05-16T22:12:03Z

Photo by Cara Fuller on Unsplash

On the web, speed is everything. But you knew that right? Rather than throwing percentages and statistics at you about site retention rates, let’s take a look at some key principles to bear in mind when looking to improve your app or website’s performance.

These are principles to be used no matter what technology you use and are more broad in their scope. The aim is to make this into a small handbook, not a manual. Just as once you learn to drive a car you, in theory, can drive any other, this article aims to teach the principles and not the implementation.

This is your website

More Network Round-Trips = Slower Load Times

Without a doubt, one of the main contributors to slow page load times is network round trips. By having more assets to download (javascript libraries, CSS modules, images), the more network connections the end-user will have to establish. Regardless of network speed, this will have negative repercussions on page load times — the effect will be more noticeable at slower network speeds.

The first way you can solve this is by compiling your assets. You can try something like Webpack to compile various stylesheets and scripts into bundles — meaning you have a single CSS and JS file that you use across the board. Further, if you have a lot of icons or sprites on your page, it may also be worth putting these onto a single image and then be referenced like sprite sheet — this is a trick used commonly by game developers but can be utilized on the web too.

Another approach is to reduce the assets you are using — question whether you really need that library. There are lots of sites that have sprung up like youdontneedjquery illustrating why you might not need to include that specific library. Additionally, if you do decide you need that library or framework, then often you can import only what you need. In the case of Lodash, you can specify a singular function to save to your dependencies (improving ‘npm install’ time for new contributors) as well as importing only that function thereby not bloating your application with unused code. I find myself keenly aware of this with Bootstrap. Often bootstrap is hastily imported and used for nothing more than it’s easy-to-use grid layout. Truth be told, bootstrap includes a lot of CSS modules for jumbotrons, icons, wells, breadcrumbs and anything else you could think of for building a site. But in 99% of cases, you just don’t need all of its features. With Bootstrap 4 you can use Webpack to import specific plugins. With Bootstrap 3.3 you can get even more granular and create your own customized version of bootstrap, including only what you need.

Note this tip applies only to HTTP 1.1. With the rise of HTTP2 around the corner, it is actually faster to have lots of assets rather than 1 single bundle. However, HTTP2 has yet to see widespread adoption for performance reasons.

Larger Assets = Slow

A fantastic talk by Addy Osmani at CSSConf demonstrated the detrimental effect of having large image assets on your page (especially in the visible viewport). To have a fast running app or website, you have to shed the things that slow you down.

I couldn’t find a photo of Usain Bolt with a bag of sand but here’s the next best thing!

Usain Bolt can run the 100m in 9.58 seconds. That’s very fast. But if he was carrying a bag of sand, he would be a lot slower. It sounds silly but the bag of sand illustrates all those cumbersome libraries and images you are using to try and make the website look sleeker but end up slowing it down. Sure, your image pops in and does a little twirl, but people will have left the site long before that animation library even loads in.

If you decide, no I need that 10Mb 4k image to load on first paint, then look to deliver it via a CDN (such as CloudFlare or cloudinary) and then cache it aggressively at the server and client level. This will benefit recurring users and can reduce page load times by a factor of 10 on the second load. Customers will thank you for respecting their data plans. Excess mobile data charges can quickly rake up and if your app is a major culprit for this, then you may find users dropping off the service.

Feels fast = Is Fast

When loading something, if it “feels” fast then it will be fast. But what does it mean to “feel” fast? Well, when loading a website, for example, prioritise the part that the user can see first — this is called the initially visible viewport. It will vary on a per-device basis but using tools such as Penthouse and CriticalCSS you can bundle and inline the styling that renders the top of your website.

It also means being interactive in the shortest time possible. You want a person to scroll down your website and not hit a load of, what I will call, the “Tasmanian scrollbar devil”. I’m sure you’ve had it yourself, scrolling down a website then an image above the visible viewport loads and pushed the content you were trying to look at down. Incredible annoying UX and takes up valuable CPU time.

You don’t want this guy on your site

To combat this, place invisible div’s with the width and height of images before they load, this will prevent this scrollbar devil from ever arising. You can also perform the same trick with large areas of text or the like, for example, you may have seen Facebook and Jira using Background masks. Here is a great article on specifically how they work.

But what about the rest of the page’s content below the initially visible viewport? That can be lazy loaded. Simple. Utilize async scripts and other methods to ensure your assets are delivered swiftly. Try to defer any background tasks or other assets for later on down the line. For example, if you have a to-do app, I would render main menu bar critically inline as well as placeholder masks for tasks. I’d then prioritize loading the JS that will then request the to-dos from the database. Anything else, such as account settings, the users’ profile image etc. can be saved for later. The goal of the app is to display to-dos. Make that the fastest thing, and forget everything else.

Hosting

Consider your hosting provider as a possible bottleneck for performance. Although we have spoken a lot in this article about initial loading times, it’s worth considering the performance of certain interactions.

Using the to-do app example above, the most important interaction is marking to-do items as completed. It might be the case that you are triggering a serverless function hosted on AWS lambda when the user marks the item complete. If you find this action slow, investigate the bottlenecks. Is it the database connection time? Is it the memory assigned to the lambda function enough? With serverless, perhaps the function is going cold and so has a slow startup time — so it’s better to host it on a long-running server instance. The point is, there are many considerations and possible bottlenecks in even the most simple action.

If you are using a relational database (such as MySQL or PostgreSQL) it’s worth taking a look at your table architecture. Bad database design can necessitate more JOIN’s than would otherwise be needed. Further, joining on un-indexed columns with large datasets will be detrimental to query performance so it’s advisable to take look at what queries you are performing and optimize them. You may even want to consider using Redis or Memcached to cache common query responses.

Set budgets and targets

Chances are, if you are the developer then you will be fairly intolerant to anything slow. In addition, you will have a good idea of how fast things *should *perform as well as the device that people use most often.

Now you have a clear picture of your most common use case, the next step is to create a performance budget. In other words, how fast should it load? Having a clear target will give you something to aim for and to keep a close eye with each new code change that is made. Be vigilant with sticking to that sub 1 second load time, and don’t accept any new code that pushes it over that limit.

Hopefully, you can utilize these principles in the future an apply them to your own website or application. They should be transferable no matter what technology you are using. Let me know any performance principles you have at hola@joshghent.com or comment below! I’m also on twitter @joshghent where I tweet about web performance and more.

LinkedIn For Developers

2018-04-19T22:12:03Z

“Oh, not another recruiter!” – my co-worker said, lazily chucking their phone down. “They just spam!”.

This is an all too common phrase I hear from developers. I disagree with this sentiment because recruiters can get you good jobs and negotiate on your behalf – it’s in their best interest to do so. If you are looking for a new opportunity, LinkedIn can be a great way to connect with people who will start the hunt for you. Here I will break down not only how to optimize yourself for a new job but also hopefully how to remove a lot of the pain points of searching for one.

Clarify Why You Are Looking For a New Position

Before you set out on your voyage to find a new job, you need to know what you want. Further, it is helpful to reflect on your career trajectory and ultimately, to borrow a cliche, consider where you want to be in 2 years time.

After establishing these points you will have a clear goal to aim towards. The tricky part, however, is sticking to that goal. If you want a job that has a foosball table and bean bag chairs, then don’t settle for a company that only has hammocks. It’s a silly example but illustrates the point. In my case, I wanted to cut my commute down and spend less time in traffic in a car. Therefore, finding a company situated right outside a train station was extremely convenient and fulfilled that criteria.

I’d further recommend thinking about what sort of company you would like to work for. That sounds like a vague thing to contemplate, and it is, but this one is not meant to be specific – just in general terms. For example, some would view a workplace like Google as the God-tier level job. With its plethora of perks and benefits such as free meals, on-site gyms and of course, nap pods, it presents a view of how you want to work. Whilst that level of benefits are not available in the vast majority of cases, it’s good to recognize that you hold those kinds of perks in high regard and therefore want to optimize your next job for those sorts of things. Personally, I was looking for a company whilst having a forward-looking growth culture, also recognized that people have lives and families they want to be with. Yours will be different so have a think about what you want.

Now you have those key details, you can include those in your messages and calls with recruiters who can then look for opportunities that fit.

Automate InMail Replies

As I outlined earlier, a big problem is that people receive a lot of “InMail”. These are messages that are blasted out to a wide range of people en masse. Often, the mail is not applicable because the skills for the job they are offering is outside of your knowledge base, other times it may be that the job is not near you or perhaps it’s simply not appealing. In any case, it can be easy to see why these messages would be considered spam.

Nevertheless, InMail can provide a doorway to connect with a recruiter in a meaningful way. I’d suggest composing a few messages that you can simply copy-paste to reply to the recruiter. They will appreciate you having taken the time to reply as it means the recruiter will get back the “InMail Credits” that it costs to send them in the first place. In 90% of cases, I have found that the recruiter would ask me what type of positions I am looking for. In which case, I can reply and suggest they keep in touch with me if they come across a role that fits that criteria.

I have 3 different types of replies to InMail

I’m not interested
I’m not looking
I am interested

Below is my letter I send when I am not looking for a new position.

Hi X, Hope you’re well and thank you for reaching out to me with this opportunity. Unfortunately, I am no longer looking for new positions at this time as I have just accepted a new offer at Z Corp. Thank you for your consideration and best of luck with the business. I will bear you in mind when I look for future opportunities. Kind regards, Y

By having these responses pre-prepared it means I need to spend less time writing and more time gaining useful connections that I can utilize in the future. Furthermore, the friendly reply will set you apart from others they may send this InMail to. People like people, and if you’re a person who is nice, people will be drawn to you. Recruiters are people like you and me, just trying to do their job.

Update your Work History and Skills Sections

Additionally, the number of InMails you get in the first place can be cut down by updating your work history with details description of what you did at that company, what you learned and what technologies you used. The latter is especially useful as recruiters will often target you with jobs related to technologies you used most recently.

Furthermore, the skills section is important as this is one of the factors that recruiters use when sending the InMail campaigns. By pruning out technologies you don’t want to work with, you should receive less mail concerning jobs using those technologies. Put in skills you have and want to use going forward.

To truly go above and beyond and put yourself in the spotlight as a potential candidate, be sure to include any articles you have written, any open source projects you contribute to or maintain. These are all things that good recruiters will look out for, as it will be an indicator that you are a more capable candidate.

Overall these are just a few tips to help you be that little bit extra special (I know you are) to potential companies. My dad always use to say: “There are lots of baked beans on the shelf, but why do people go for the same brand each time? – Because they believe they are the best. Be the best can of baked beans”. The point is that you have to set yourself out from the crowd. There are lots of developers out there and lots of demand but it doesn’t take much to differentiate yourself from the crowd by applying the points above.

Do you have any more tips to share on LinkedIn or perhaps a grievance or two? Discuss it down below or email me at hola@joshghent.com or comment below! I’m also on twitter @joshghent where I tweet about web performance and more.

Solve 90% of Google Pagespeed Insights Issues in 30 Minutes

2018-03-31T22:12:03Z

Source: https://unsplash.com/photos/fxAo3DiMICI

Performance is a critical factor in site retention rates. Time is money, and there is a laundry list of examples that prove people expect near-instant loading and will navigate off a web page if it does not load in under 3 seconds.

Although the Google Page speed Insights score is not a guaranteed stamp that a site will be fast, it does give some indication of this fact. Additionally, it is now one of the hundreds of factors that Google and other search engines use in their SEO ranking algorithms. Truly then, web performance is something to care about from a business standpoint.

Since developers and businesses alike aim for “best bang for your buck”, here I’m going to walk you through three simple steps you can action which will bump your score by at least 20 (if not more). This is not a smoking gun but, applying the Pareto principle, it is better to spend your time doing 20% of the tasks which will give you 80% of the benefit.

Minify your assets

Large page sizes owe their disdain to bloated javascript and CSS assets. An easy way to reduce the sizes of these is to minify them. This can be done via a task runner like Gulp or Grunt. If you are in a hurry then you can use an online tool such as Minifier — however, this will mean you need to re-run this everytime you change the Javascript. I’d recommend setting up an automated task.

If you are feeling adventurous then you can go further with this optimization and use Webpack with tree shaking. This will prune any unused code from your Javascript and therefore reduce the size of the underlying modules that you are minifying.

Delve deeper into minification and you will quickly realize that the best way to reduce asset size is to just have less stuff in them. Therefore, try and reduce the number of Lodash modules you are importing, or Moment.js locales, or perhaps you are importing the entirety of Bootstrap just to use the row and container system.

Images are one of the biggest culprits in the issue of large files. According to the HTTP Archive, as of the 15th March 2018, **over half **of an average sites payload is in images. Therefore it’s crucial to focus your efforts on reducing the number of images you use but also optimizing them. Before putting an image on your page make sure to compress them first. If you are in a hurry then try using this tool called Optimizilla. The solution is to automate this process using your task running (Gulp, Grunt or Webpack) along with a plugin such as ImageMagick or the like. There is even a Wordpress plugin if you are publishing images via a blog.

Cache Assets

Huge savings in speed will come from the client’s browser not having to download the assets in the first place. This will not only drastically improve page load times but also reduce the bandwidth needed for your server (which depending on your hosting provider may reduce the bill). Additionally, if the client is viewing the site on their mobile then they will be able to load your site, safe in the knowledge that they don’t need to be concerned about their data plan.

You can cache your assets in a few ways, the easiest is to set a cache control header on your requests. On an Apache server, you can do that as follows.

Code here: https://gist.github.com/joshghent/fcca761d006ae34a1a2aaa0406a9e0f1

Application caches can also come in very useful. A major issue facing many sites is long-running queries. An easy solution to this is to cache the query response in the application, providing that the query is worth of caching and will not have its response changed a lot. Laravel has this built in and Express can be extended to do this also. This will reduce the server response time and therefore lead to quicker page loads for the clients.

Enable Compression

Lastly, after reducing and minifying your sites payload, you can enable compression which will ensure they are transferred in the smallest possible form. There are two popular algorithms, Gzip and Brotli. The latter is a more recent trend and actual has better compression rates than the long heralded Gzip. Nevertheless, I would still recommend using Gzip as Brotli takes more CPU power (and therefore time) to decompress on the client side.

You can find guides on how to do this around the web that will be up to date long after this blog post is published but here is a good one for Apache (which I assume will stay the same!).

I hasten to add, that if your site must support Netscape 3 and below then compression will be redundant here as they only support HTTP/1.0 which does not send the Accept-Encoding header. Nonetheless, with less than 1% of people worldwide using anything other than “the big 5” browsers — I think you’ll be in the clear.

Performance should be considered a feature and whilst it would be great to spend lots of time on performance, companies tend to prioritize other tasks ahead of it. Whilst that is not ideal, using the 3 tips above (which should take less than 30 minutes per point) you can quickly improve the performance of your site or application in addition to having more potential leverage for time to be allocated for furthering performance.

Do you have any other performance quick tips? Let me know at hola@joshghent.com or comment below! I’m also on twitter @joshghent where I tweet about web performance and more.

📱 Zen iPhone

2018-03-26T00:00:00Z

Photo credit: https://unsplash.com/photos/Dl6jeyfihLk

Ever since the smartphone arrived in our hands, people everywhere have been utterly entranced by them. Spending never more than a moment without being bathed in white digital light.

I consider myself in that crowd. But recently I have been more inclined to try and spend time on my phone in a more purposeful manner. I found that I was just scrolling endlessly on various apps just to avoid doing other things — as a form of procrastination. I’m not a maverick to the extent that I will abandon a smartphone entirely and resort to a Nokia 3310 or carrier pigeon but I did “refactor” my phone. And in doing so found a few helpful points which may help others do the same.

I treat my phone as my virtual desk and although it has long been touted that genius’ generally prefer their desks messy, to the vast majority of people, order and tidiness is preferred. In this way, be mindful about organising your home screen in a way that is conducive to things you want to accomplish and deters from things you don’t want to get distracted by.

Many will say to completely delete distracting apps. Generally, social media and/or games are a way to relax on the train home or perhaps to scroll through whilst waiting in line and so deleting them seems like a step too far. As a solution to this, you can put “distracting” apps on the second page of a folder.

For example, have a social media folder that on the first page has just LinkedIn. On the second page of that folder, is Instagram, Twitter, Snapchat and more. Not only does this mean it takes an extra action to access these distracting apps, but also keeps the screen less busy; Every app prefers a different color theme, Twitter is blue, Snapchat is yellow and Instagram, like a child trying to pick which ice cream they want, went for a loud 3 color gradient. Keeping these out of sight keeps your phone a more serene place.

In line with this “less busy” theme, I have found simple two-color linear gradient backgrounds particularly helpful. I found single colors too boring but a gradient gave it that little extra twist. Further, the menu bar (or whatever you call the bar at the top of the screen) was a particular source of chaos. This was solved this by removed the battery percentage so that it is only icons on my menu bar — again giving it an air of peace and serenity.

The plethora of large red badges that just beg you to click on them are seldom left untouched. I removed all of these with two exceptions, my unread email count and the Todoist tasks I have due for the day.

Out of sight, out of mind — proverb

Without a doubt, one of the most useful features you can activate on your phone is automated Do not disturb. As a programmer, I like this automated. Therefore, I have it configured to turn on my do not disturb between 10 pm and 6 am. This means I am not distracted by incoming emails or other messages as I am winding down for bed.

Before bed, it is optimal to ensure your phone has blue light shifted such as Night shift (iOS) or Twilight (Android). The science on it’s benefits to sleep are sketchy at best but from personal experience, it does reduce strain on my eyes compared to staring at a blue screen. When you’re used to a bright blue display, you don’t know any different. But trying to go back is impossible for me now.

In my quest for a more peaceful digital experience, I stumbled across a tool called Moment. It’s an app that tracks your phone usage. It works by looking at your battery usage, which you are asked to screenshot each week. The photos are then automatically scanned by the app.

My Homescreen

As you may already know, I’m a huge fan of making data-driven decisions and believed this was a great way to gain some insight into how I am using my phone. Because I have only been using this for the past month, the data set is fairly limited and I think I have some false positives from leaving my phone unlocked and listening to audio from Youtube (I have yet to configure the excluded apps list). Nonetheless, over time this will build into a substantial data set I can draw sensible conclusions from. It will unlock answers to questions such as “What apps do I go to first when I unlock my phone?” and “What percentage of my time is taken up by social media in my 8-hour workday?”. Questions like these lead to inherently biased answers when asking one’s own self — because we all want to seem like a good person. Consider the “I am a better driver than you” experiment and it quickly becomes apparent that we are not good judges of our own character. Automated applications will allow you to quickly gain insights and patterns you would never be able to see.

The final conclusion I came to with all of these apps and settings is that ultimately I needed to be more mindful of the purpose I had each time I went for my phone. I never considered my phone usage an issue; More that I could be using the time more effectively. Thinking about the task you want to accomplish will enable you to have a clearer vision of how achieve that objective. It doesn’t always have to be productive. Sometimes, the purpose of unlocking your phone will be to have a mindless scroll down the DailyMail app (a guilty pleasure of mine that is so mind-numbing they have started using it as an anaesthetic in some hospitals). Other times it will be to add a new task to your todo list. In any case, it’s fine. Just have something.

Do you have any other tips for achieving zen in the busy world of devices we now have? Comment below or tweet me @joshghent

Understanding PHP hatred

2018-03-05T22:12:03Z

Pictured: The PHP developer in their natural state of silent contempt

It’s an age-old joke to hate on PHP. But why do people dislike it so much? After all, PHP powers 80% of the web (a large majority of that is credited to Wordpress, but still). In this article I break down the main gripes of PHP development and share advice on language and system design.

Inconsistent method naming

The biggest problem people see when they first look at PHP is the inconsistency of the standard language methods. When PHP was first released in 1994 it did not have namespacing which meant all methods had to exist globally at the root level. When namespaces were finally introduced in PHP 5, the damage had already been done. Methods that ordinarily have been namespaced under it’s particular category (such as **String** or **Array**), were just plonked and prefixed with the category instead.

This led to names such as array_map and str_repeat. Now that’s all well and good, but the problem is that the prefix + underscore method was not always used. Soon, there was a whole host of methods named things like strtolower and ucfirst that broke those rules.

Additionally, these method names had inconsistent usage of snake_case — as is the case across most of the string methods. You have functions such as strtotime and str_split — why is it not str_to_time? Who knows.

Furthermore, another minor inconsistency that had escaped my notice until studying the list of PHP methods, is the usage of ‘to’ and ‘2’. In some cases ‘2’ was substituted into method names, presumably to look like a teenager texting on a Nokia 3310 in the early 2000’s.

As a result, we now have methods such as ‘bin2hex’ and ‘deg2rad’ as well as strtotime and strtolower.

With all that said, what’s the take away for you and me? In software design, and most things, consistency is key. By having a consistent interface for programs to use both programmatically (with consistent and logical API endpoints and parameters) as well as visually (with easy-to-use but also functional UI’s), we enable more logical and clear usage for people using our UI’S and developers integrating with our API’s.

Not to be too hard on the PHP developers, but it is clear that seldom thought went into planning the language or thinking about its future scope. Don’t make the same mistake. Drill down into all the little features and quirks of your system as well as ones you may add in the future. It is impossible to gear up for every eventual outcome but it is worth having that forward thinking view, so you are less likely to get tunnel visioned into “the” product. Software always changes.

Inconsistent argument orders

Another inconsistency is that of argument ordering. Arrays, dictionaries, hashes, whatever you call them, they are an integral part of any language that developers using that language will use on a daily basis and form a core part of storing and manipulating data on any system.

You’d think that, being such an important part of the language, that they at least would be consistent. Unfortunately, you’d be wrong.

If you’ve ever done PHP development you may have run into this issue. You’ve got an array of numbers that you want to double and then return into a new array. No problem! You say.

“I’ll use array_map!”.

So you write the code and then run it and then…

What? You say. After much debugging, thinking there may be an issue with your method, you finally resort to the PHP docs.

There you discover…

Documentation for array_map https://secure.php.net/manual/en/function.array-map.php

It’s callback first. Not callback last, like you had just done with array_filter.

I don’t know how many times I’ve done this but each time you can’t help but slightly curse the name Rasmus Lerdorf.

Besides the importance of consistency as we have already spoken about, what else can we learn. Well for my money, it’s helpful error messages. Rather than spitting out something vague and meaningless like in this case, write something helpful and actionable. I read a great article about this very topic — you can find it here. I’d highly recommend reading it. Ideally you want to make your UI (including visual and programmatic UI) as intuitive as possible but account for cases where someone makes a mistake (we all do) and handle it gracefully by guiding the user to the correct course.

Frustrating usage

Asides from being poorly named, the usage of these method is also frustrating.

Explode is a method that takes a string and a delimiter and breaks up that string on the delimiter into an array. Simple right? You’ve probably seen this in Javascript with String.split() and other languages. The quirk here is that passing empty string (“”) or null as a delimiter will cause the method to return false. Rather than simply treat it like every other language (empty string returns every character as an element of the array or null to return the string in its entirety), PHP decides to treat it as an error condition. But because it does not throw an error you are forced to check it manually.

Another aggravating method usage case is when manipulating arrays. Sort() and all the other array sort methods (there are a lot, all more confusingly named than the last) in PHP operate the on the array in place and do not return a new manipulated array. They simply return true or false. This prevents you from method chaining and makes the code you write with array manipulation that bit more verbose than it would otherwise be. Further, array_reverse (in the same category of array manipulation) does return a new array but this again means more inconsistency (even though in this case, the inconsistency is good).

Without doubt however, the trifecta of annoyance comes from finding a string within a string. This could not be more simple. Every language has a method like this, and they all work the same. A needle (the string you want to find) and a haystack (the string you want to find it in) are accepted, the method then returns the haystack index at which this needle was found and returns -1 if it wasn’t found. This is the case for Javascript, C and most other languages. PHP however, being the language hipster that it is, decided that this wasn’t good enough and decided to break the status quo by returning false if the needle wasn’t found. That doesn’t sound so bad (although inconsistent with every other language in existence) but if you loosely compare false in PHP, it becomes a 0. Now this is an issue with the following code

Unfortunately, because the developer was expecting a response of anything but -1 from the strpos method, this code will return true even though the needle is evidently not in the haystack. I find this one of the the most glaring oversights in PHP because it’s so easy to get wrong when programming something that depends on this as well as being again, inconsistent with other languages.

Bad error messages

Error messages are a major problem with PHP. I distinctly remember my first gripe with PHP — debugging. Being unfamiliar with PHP at the time, I did not think to use a 3rd party tool to debug my code; that should be built in — right? Surprisingly (and unfortunately) not. I spent a while googling around to find out I had to turn on errors with some specific variables and debug levels. If you look at the search results for “How to turn on php errors” or “PHP blank screen, no error”, the issue quickly becomes apparent.

Now, you may have got error messages **_actually _**working but sooner or later you come across this gem.

**PHP: **Parse error: syntax error, unexpected T_PAAMAYIM_NEKUDOTAYIM

You: A what?!

PHP: A Paamayim Nekudatayim of course…

For the uninitiated, paamayim nekudatayim is a romanized version of the Hebrew word for “twice colon” which is referring to the scope resolution operator (::). The kind you would use to call a static method such as this

This was originally introduced by Israeli-built Zend Engine back in PHP 3. Now that’s fine for people who speak Hebrew, but English is widely accepted as the lingua-franca of programming and the internet at large. Again, it all relates back to ease of use. After finding out the meaning, in a way, I kind of like it as a fun quirk of PHP with an interesting backstory but it is very confusing to new PHP developers (or developers full stop).

This error message still lives on today in PHP 7.

The main issue with PHP error messages are the detail and specificity. At some point, because you’re not a robot (or maybe you are — if so please fill out this captcha before continuing) you’ll miss type something, perhaps missing a bracket or quotation mark. Maybe your code looks something like this.

Result of the code above

If you’re a battle hardened PHP developer then you may have spotted that the closing quotation mark on line 5 is missing. Yet PHP considers it helpful to return the message from line 8.

Fast debugging is a hugely crucial issue for programming languages. Not only the frustration for the developer, but by having ambiguous error messages, it means the developer spends more time debugging which costs the company and/or client. With the above example, it may seem that this is a mountain being made out of a mole hill, but imagine this in a large application and quickly the issue becomes increasingly worse and aggravating.

Method duplication

Last but not least is the issue of method duplication — as in the case with die and exit as well as implode and join. Now, this may not seem like the biggest sin. After all, die came from Perl and will therefore be easier for programmers with that background to use, and exit came from C, again, allowing them to have an easier transition.

The problem with this for new programmers or programmers without a C/Perl background, it doesn’t become easier, just more confusing. You end up questioning which to use? Is one better than another? Should enforcement of one over another be in a style guide? All valid questions that leave the developer going down a rabbit hole of syntax quirks rather than actually working on the task at hand.

Helpfully the PHP manual clears up the differences

Yeah… perhaps not. As a takeaway from this point, it is important to not pollute your documentation of a codebase with legacy stories of reasoning behind one decision or another (as has been done here). On the other hand, question whether there should be those stories in the first place.

Although there are many more quirks from interesting to downright insane, I have found PHP very useful, as have thousands of developers worldwide. If you are a PHP developer with a startup idea, don’t wait to learn a new language or framework to build it. Just do it in PHP. Software can always be iterated on, and to worry about “What programming language should I do it in”, is even more insane and redundant that some of PHP’s quirks. All languages will be capable of doing anything (asides from system specific native apps of course but you get the point) — some are just different to others, which is why we have so many. But the speed gains you get from writing an app in a different language rather than doing it with what you know will be negated 10 times the amount due to the time it takes you to learn that new language.

The age old adage is “Only a poor workman blames his tools”. Now write the damn thing in PHP.

What Tracking My Expenses for a Year Taught Me About Personal Finance

2018-02-26T00:00:00Z

2017 was the year I tracked my finances. I started doing this because I wanted to remove the mystery of where my money was going. I wasn’t overspending per se, but found it challenging to know exactly what I had spent across credit cards, debit cards, cash, etc.

Although there are a lot of personal expense tracking software, uncharacteristically, I chose a low tech option — a simple Google Sheet. I then have a tab for each month and columns to track the Date, Shop, Amount, Notes and then a category of the expense (I have the option of fuel, entertainment, food, shopping, gifts and other).

Now we’re in 2018, I now tracked my specific expenses for over a year. Here is what I’ve learnt.

Money isn’t going where you think

Ok sure, you might have a rough idea. But how much did you spend on lunches last month? For me, it was way more than I expected. Whilst, how much I spent on fuel was a lot less. This enabled me to adjust my budget accordingly and also be more mindful of spending in those areas. It’s all well and good to *say *you have a budget, but are you really sticking to it?

In my case, I wasn’t, but that’s not a problem. Now I have the data, I can make informed decisions about where my money is going.

It also enables me to plan around larger spending that may happen on an annual basis. Car tax for example is something I pay yearly, however, in my budget it is included as a monthly expense. I then put this money into a specific account that I can later use on that date. I apply the same principle for clothing as again, it is something I do bi-monthly rather than specifically each month.

The bank should work for me

A common misconception, at least one that I believed, is that banks do not really owe you anything. They do! My previous bank that I had held for the previous 8 years paid me very low interest and didn’t offer any other benefits. I then started searching other bank accounts and found many paid 5% interest (Nationwide Flexdirect), or offered store points when you use them (Tesco Current account).

The other misconception I had was that it would be difficult to change accounts. Most banks in the UK will be registered with the current account switching service which means they handle transferring direct debits, standing orders and forward any incoming funds to your old account to the new account. This process usually takes 7 days.

I personally have it setup so I get paid into 1 bank account and then have standing orders out to all my other accounts (saving, spending etc). This enables me to switch bank account a lot with relatively low friction. Changing regularly does negatively affect my credit score but since I am not applying for a mortgage or loan any time soon then this is not an issue (and by the time I want to, it will have improved).

The point is, don’t just stay with your current bank. Martin Lewis keeps a great updated list of the best bank accounts and breaks down their details as well as all the ins-and-outs of qualifying for them.

Automation is key

Without doubt the best thing I did to optimize my finances this year was putting it on autopilot. Finances and money I find interesting but, all in all, they are a drab subject. By automating everything, I am free of an ambiguity about where money is going or will go and I don’t need to spend my own time dealing with it manually.

Here’s what happens when my paycheck comes in:

Credit cards are paid
Bills are paid
Standing orders send money to my spending account, saving(s) accounts and ISA — I would recommend setting up different “pots” as I spoke about earlier for different financial jobs. For example I have an account for my emergency fund, one for car fuel, one for annual car expenses (tax and insurance) and another for general saving (I use a bank that allows me to create “pots” within this account for holidays and any other items I am saving for).
Money left in my primary bank account (the one that gets the paycheck) is left until the end of the month and then moved into my savings account.

My investments are also all automated as I use a service called Nutmeg to balance my portfolio according to my risk profile.

If you struggle with setting aside money to save, try a service such as Moneybox which takes weekly deposits from your account as well as rounding up change from purchases.

Any other tips that you use?

Comment below or tweet me with any other personal finance advice you’ve applied. I’m not a qualified finance professional by any stretch of the imagination, so your circumstances may vary, but overall the principles of automation and tracking you money can be utilized no matter what your situation. By applying the advice above, I have been able to stick strictly to my spending budget, increase my savings and pay off my debts — and you can do that too. All it takes is an free afternoon and a memory of all those internet banking passwords.

Beginners Database Design Primer

2018-02-14T00:00:00Z

Your boss has just got off the phone with a client who wants a bespoke social network site targeting a niche market. And they want you to head up the project. You’ve never built a social network. Your mind goes to Facebook, Twitter, and Instagram. They’re built by thousands of people with genius level IQ’s and degrees. How could you compete?

Well that’s at least the situation I found myself in.

My task was to first to design a database for this social network client. This is the advice and help I would have wanted to read. Please note: This applies generally for all database design. A social network will be merely used as an example.

But why did I feel that the database design was so critical?

Bad database design leads to laborious and slow queries
Duplicate data and incorrect column data types occupy more disk space

Think about your features and the relationship between them

What are the features of the site? Really break it down. The more you break it down, the less “Oh yeah, let’s add a table for that” you’ll get later down the line. I suggest using a tool like draw.io to map out your tables before putting them into the actual database.

As we are building a social network, the leading feature is letting users posts and allowing other (authorized) users to comment. Here we have a one-to-many relationship, because a user will be able to have many posts and comments.

We will have other instances such as with user “likes” (such as liking a page, or group) that will require a many-to-many relationship because a user will have many likes, and those likes will have many different properties.

One of the primary objectives of good database design is to remove redundant data and increase the integrity of that data. Often, people combine tables that have one-to-one relationships. For example storing users data with their address. Let’s see why:

The example on the left combines the users table with the address for that person. On the right we have separated it into different tables. But which approach is optimal?

Well, as with most things, it depends. When I was building this particular application, the address was an optional field, therefore it was most appropriate to store these locations in another table. Furthermore, it was under future consideration to allow for many addresses for a single user. Therefore, in my case, storing them in the same table would restrict me to a 1-to-1 relationship only, thereby limiting me on future features.

Performance may be a factor here as the more “relationship” tables you have, the more joins you need to make, thereby slowing query time. This could have repercussions as you scale.

Think about what data you will need

That leads onto my next point about think through what your application will display. Think about the query to get that data in your head. If, to extract that information, you have to make tons of joins and subqueries and all sorts then perhaps you need to rethink your design.

I’m most keenly aware of this principle when thinking about analytics. For example, in this social network application, we want analytics for where the application is used. If we are to do this purely via the address fields we store for users (without any IP location wizardry) then we may chose to store the counties list in a separate table and then link that with the main addresses table. This would enable us to quickly query for certain countryid’s and ascertain how many users are registered there. We _could leave the country as a text input for the user but this may lead to incorrect spellings of counties and other duplicate data that would lead to false statistics.

Overall, understand what your application requires and design around that whilst still being flexible for future expansion.

Think about the data stored in the columns

An underestimated point of database design is that of column data types. Often columns are given simple VARCHAR or INT data types, this is not always the most performant or memory efficient way of designing them and changing the data type after the fact can lead to corrupted data in those columns.

Become familiar with the different data types, these are almost universal across all programming languages and databases and will allow you to think at a lower level of abstraction to the data you are handling.

As mentioned previously, one of the main problems with inefficient data types is that they occupy too much space on disk. For example, let’s say we are storing a flag checking if a users post has been deleted or not. Because the only 2 values of this column should be 0 or 1 (the former for undeleted, and the latter for deleted), then we do not need the 4 bytes it takes to store a whole integer (which can be any number from -2147483648 to 2147483648). We merely need the TINYINT data type (or BOOLEAN in MySQL), which occupies a mere 1 byte. That’s a whole 4 times smaller! Now 4 times smaller than nothing is still nothing so this may seem like a needless reduction that will save a fraction of disk space. And in most cases, you are probably right; But if the service were to scale, to hundreds of thousands or even millions of rows, then your boss would be thanking you for saving them a lot of money in drive space by going for the TINYINT option. Take the time to think of the most performant and lean design for your database. Act as if you are designing for Facebook-level scale — it will pay off.

Avoid storing calculated columns

A common mistake when designing a database is to store redundant information that can be calculated by your application. Using my example, we want to display the users age on their profile. It may be possible to store the users age directly in a column, but if we have their birth date then we can simply calculate their age on the fly. If speed becomes an issue where calculations cannot be done, then it would be best to cache it in your application or in a key-value database such as Redis before finally resorting to storing the calculated value. Other examples of where calculated information storage might be suggested is for exam grade averages or the number of orders a person has placed.

What other tips do you have for database design?

I tried to target the “Pareto problems” here — meaning that, in my view, these 20% of tips will solve 80% of basic database design problems. That being said, there is a lot more to learn, and if you’re interested check out database normalization rules (NF) once you have fully taken in the information from this article.

👋 I am available for hire as a freelance application consultant/developer. Contact me at hola@joshghent.com if you would like to discuss any projects you have in mind.

How to Attend Your First Programming Meetup

2018-02-06T00:00:00Z

Attending your first programming meetup can leave you a little apprehensive. I felt the same! So, I thought it may be useful to break down my first meetup experience and how you can start attending meetups yourself.

First meetup #nottsjs pic.twitter.com/2IQ0vImjxW
— Josh Ghent (@joshghent) June 13, 2017

Why did I want to go to a programming meetup in the first place?

The primary reason was the people. I think it’s always a good idea to network with people who you could potentially be working with and learn from them. There is always something you can learn from someone, regardless of their ability — and I embrace this principle heavily at these meetups by attempting to talk to as many people as possible.

The most fascinating aspect, in my view, is hearing about how companies are working on a structural level and the technologies they are utilising. For example, I spoke with someone who worked at a bank and was working on rewriting their entire stack using Node. I found this intriguing because Node is a relatively new technology. I quizzed him on a how they were handling the key areas for banking software, scale and security. Since I myself had never used node in such a critical environment, it was fascinating to gain such insights.

Talks are also a great way to learn about new technologies. My first meetup had a talk entitled “Machine Learning for Muggles” and walked you through how to use Azure’s Machine Learning capabilities as well as a broad overview on how machine learning works. Being on the very bleeding edge of technology, it was amazing to get industry leading teaching and far outweighed any content I had found online. Furthermore, having an expert on the subject delivering the talk allows you to ask questions that may be challenging to find answers to online.

The third and final reason, is that any good meetup will have free pizza! 🍕

How did you find your meetup?

There is an easy answer to this: meetup.com. I looked up all programming meetups in the area and found one that looked active and had a history of talks I was intrigued by. I signed up for a few, I would recommend any “first time meetup” folk to do the same — spread yourself out, you never know what you may like. Despite being a PHP programmer at my present job, I ended up attending a Javascript meetup! Don’t limit yourself to a meetup targeted at a language you currently use. Take a look at the talks and see if they interest you. As I say, my first meetup was the Nottingham Javascript group but the talk itself was about machine learning, far related to Javascript.

Generally, talks will be geared around a certain technology, let’s say Docker for example — with tips targeted at that meetups language. As an example, a talk about Docker at a PHP based meetup may be titled “Setting up a PHP development environment in Docker”. Even if you don’t know PHP (or whatever language the meetup aims at), you’ll find the talks valuable and lots of people who, like you, don’t use that particular language.

What should I do at a meetup?

Now you can only go for the talk but that’s only half of the experience. I would highly recommend (after grabbing some pizza of course) just approaching people and introducing yourself. Programmers are generally a shy bunch, but ultimately, we all share a common interest so there will be plenty to talk about.

Initially I found that everyone seemed to be in their own little circle talking amongst one another; if that is the case, just go up to them and say “hi, how ya doing? I’m Josh” (obviously substitute your name for mine but you get the idea). Ask about people’s jobs and what they are doing there, what exciting technologies they are using, and if they found the talk interesting. Almost every programmer you will talk to will have some kind of side project they are working on — talk about that! That can lead to some of the most exciting discussion as usually people are experimenting with cutting edge technology that they would not be able to use day-to-day.

Be thinking all the while about what questions you can ask them based on what they are saying, it show you are listening to them and are genuinely engaged in the conversation.

Other talking points may include, asking them about the company they work for, how big is it, where are they based and so on.

When wrapping up a conversation, don’t make excuses about going to the bathroom (unless you actually need to), simply say “I’m going to introduce myself to some other people now but I’d love to continue this conversation, perhaps I can take your email and we can talk”. Since this is work related, people may get a little cagey about handing over their phone number, so opt for business related avenues of communication, email, twitter and LinkedIn.

All in all, don’t be scared. These are your people, just as shy and nerdy as you are. So put yourself out there and see who you meet. There are lots of interesting people out there! If all else fails, see if you can convince a coworker or friend to come along.

👋 I am available for hire as a freelance web and application developer. Contact me at hola@joshghent.com if you would like to discuss any projects you have in mind.

Bulletproof Node — Security Best Practises

2018-01-23T22:12:03Z

Make your Node app like this guy

System breaches are now commonplace. Stories of IoT devices being compromised, entire countries credit history leaking online as well as thousands of other systems compromised, hacked, infiltrated and destroyed.

Now it may seem that from all these stories, that any attempts to improve system security is fighting a losing battle. And in a way, you’re right. But, think about it this way, your house (or apartment) is not impenetrable. However, you still have a lock on your door and make sure to secure the premises before you leave. Security measures such as locks, alarms and perhaps even CCTV cameras are preventative — not guarantees of complete security. Web application security is the same, the more barriers we put up, the harder it is for attackers to exploit different “vectors”.

Here is a quick guide on changes you can make to your application right now without large code changes.

Use Synk to monitor security vulnerabilities

Nowadays, modern web applications use many dependencies, those dependencies in turn use even more dependencies. It’s dependencies all the way down. Either way, it’s unfeasible to know every single dependency and keep up to date with security news. Synk is a handy tool that allows you to automatically scan security vulnerabilities in your web applications, it supports a wide range of languages from NodeJS, Python, PHP and Ruby as well as many others. Additionally, if you just have a NodeJS application, Github now comes with automated integrated CVE security alerts too.

Add Helmet for all requests run through Express

A chain is only as strong as its weakest link, therefore make sure **all **API routes are secured. Additionally make sure that all those routes are used! By reducing the surface area, there is less chance of an exploit being found.

Helmet is a NodeJS tool, that bolts onto Express and acts a middleware. It takes any outgoing requests and adds various headers that help to keep the request secure.

Keep NodeJS and all dependencies up to date

Although you don’t want and/or need to update the latest major version of NodeJS, it is important to update to any minor version that include security updates. The same applies with project dependencies. The main push back on this has always been that you can’t trust semver. I wholly agree, but with a handy tool called Next Updates, you can run your test suite against new dependency versions automatically. Now this is not a guarantee that new versions of dependencies will work as it will vary on how broad and thorough your tests are; But, it does automate a large portion of work. In line with automating processes, you can configure Greenkeeper to submit a new pull request for new versions of dependencies that your app uses. By submitting a pull request, this should flag up any problems as it runs your test suite.

Monitor for multiple invalid requests, and any other potentially malicious traffic

Your routes could be as secure as Fort Knox but attackers could still potentially bring down your site by DDoSing it or brute forcing login forms. You can configure monitoring of your site to log out to Papertrail or Logstash that will then notify you if a certain type of log (I recommend having a “malicious traffic” category) that will then notify you directly (via SMS or Email for example).

Pair this with running your server with foreverjs which will automatically restart the server if it crashes or gets timed out.

Monitoring

This is, in my opinion, the most important aspect of them all. By implementing monitoring of your applications usage, you can potentially pick out malicious activity. Here are a few recommendations of what you can monitor:

Multiple failed login attempts for both the application and the server itself (FTP, SSH etc.)
Logins from new IP address — many services have automated emails go out to the user if this event occurs. They can then click through and report malicious activity themselves.
Attempt to access application resources directly (e.g., environment variable files)
Changes to user details (email, password etc) — this is to cover the case where people may have access to the person’s computer and want to hijack the account.
Attempt to login with hacked credentials — a new common hack is to take details from other breached services and apply them to other services (because most people use the same password for multiple services). This one sort of ties in with multiple failed login attempts but adds a new angle in what a potential attacker is trying to do.
Attempt to do SQL injection or other XSS attacks — if you see a particular user attempting to do any of these sorts of attacks, most likely no action will be necessary, as your app should be secure and the likelihood is that they are just messing about. Nonetheless, it may be worth keeping track of these users and the IP address as a sort of “black book”.

Me talking to my API routes

You may have noticed the general theme going here — automation. I had a plethora of other tips for this article that I cut, as a) you can find them in articles elsewhere and b) data is the only way you will be able to find weak points. A chain is only as strong as its weakest link. For example, perhaps your application (targeted at a less than tech-savvy audience who don’t use crazy high entropy pass-phrases with a password manager) has a password policy which means many people are writing their passwords on post-its and putting them on their desk. This may lead to someone spotting the password and using it. Without data and monitoring, you would never be able to see that the users account was accessed from a new IP. The point is, there is no “one-size fits all” solution to security. Take a look at how your app is being used and prioritize security methods to help those use cases first.

And that’s a wrap. Let me know which tip you found most useful or implemented yourself!

👋 I am available for hire as a freelance web and application developer. Contact me at hola@joshghent.com if you would like to discuss any projects you have in mind.

How to Learn a Programming Language in Record Time

2017-06-28T22:12:03Z

Note: This article is aimed primarily at beginners who perhaps know a single language but are looking to start learning another.

When picking up a new programming language your first port of call might be the doc ument ation, maybe it’s reading through some code on a project you admire or perhaps you learn most effectively by building. Whatever the case, we can apply the Pareto principle to learn 80% of the language from 20% of it’s features. If you’re coming from a background where you know design patterns and common programming features (control flow, loops et cetera) then this is more than possible.

When I initially thought of this idea I didn’t think that it would be possible to boil down a language to such a degree. But then again, when was the last time you used clz32 or bzflush? Programming languages have grown over time to implement features that, in day to day development, you mostly won’t need. Learning a new programming language can therefore seems a daunting prospect — but it need not be.

I applied this exact pattern of learning when trying to learn Java and it worked relatively well. There was things I didn’t know from this, such as exact patterns of inheritance but at a very basic level, I could hold my own — and that’s the purpose. As you dive deeper into your new programming language of choice, you will get to know the nuances, why and how it solves specific problems and what’s best practise. This will, at the very least, give you a good grounding in a language in an efficient manner.

Here’s my list of things to prioritise so you can pick up a new language in record time:

Variable creation — if it’s a strongly or statically typed language, then this extends to how to declare variables of different types (integer, string, object, array). If the language has the feature then we can learn how to create a constant too.
Loop ’n’ number of times — In Javascript this would be achieved by — for(var i = 0; i < n; i++) {}
**Loop over a key:value store **— Key:value stores are called Objects in Javascript in other languages they are called Hashes (Ruby) or Dictionaries (Python). Nonetheless, they are all the same, and usually there is a particular method to iterating over them because they are referenced by “keys” and not index numbers (as with arrays).
Referencing items in array — In javascript you can reference arr[1] for the second item of an array. In addition to basic referencing, there may be special methods like end() to get the final element of an array.
Functions — How to create them, with or without arguments.
Add to an array — How can we add an element to an array?
Remove from an array — Likewise, how can we remove a particular item (of index ’n’) from an array?
Class creation and constructors — This is where I find languages differ wildly in particulars of the syntax. PHP, for example, has a special __construct function that you must use to construct the class.
StdOut method — In javascript this is console.log in PHP it’s print. This is probably my most used method when debugging.
Comparison operators — How do you check if a variable is false or true? How do you compare a larger number against a smaller number?
Length of a string — A must have for any language. I find myself using this all the time but a common use case is checking whether we should truncate a string before displaying it to a user.
Length of an array — Crucial when working with loops as 99% of the time, you will be iterating over an array for however long the array is.
Public and Private methods — All languages (especially those with classes) should have this, and is essential when you want to disallow access to functions outside the class.
Try…catch blocks — I only ever use these when integrating Stripe payments but they can be handy other times, perhaps when you are testing for a bug and want to capture it for your bug tracking software.
Returning from functions — Not all languages use return! (See Rust).

And that’s it! This will by no means teach you a language per se but it will provide a good base level to become familiar with the syntax.

This learning pattern relies on knowing concepts and design patterns; It goes to show how your learning can be transferred from one language to another!

Remember the core concepts and features of a language are what matters.

Additional Reading/Resources

https://tim.blog/2009/01/20/learning-language/ — The original article this is inspired by.

https://learnxinyminutes.com/

What programming language should I learn: or Why it doesn’t matter

2017-06-23T22:12:03Z

Searching ‘What programming language should I learn’ will return you over 7 million results. The first one says Javascript, the next PHP, another extols the virtues of Java and statically typed languages. What even is a statically typed language you might ask? I just want to make apps for my phone!

This is the struggle of beginners learning to program — and even myself, when I was looking for a second language to learn. There is so much noise and no clear path. Now I’m not about to add to all those articles and recommend you another language.

I’m instead going to promote a new idea — IT DOESN’T MATTER. No really! Here me out.

There is no one programming language to rule them all. Although there are many people who vouch that Javascript can be used anywhere and is therefore the best, with React Native for native applications on both mobile and desktop, Node JS on the server and regular plain ol’ Javascript baked right into the browser. It sounds a very compelling argument. And it certainly is. I think Javascript is a great language to learn for anyone. Having said all that however, let’s look at the history of programming languages and frameworks.

According to the TIOBE index, Java is by far, firmly positioned in the number 1 spot. Javascript is on track to eclipse that, but it wasn’t that long ago, that the coolest language on the market was Perl or even Fortran. My point here is that, even though a language seems popular right now, technology is an extremely vicious landscape. Ever changing as preferences ebb and flow and new technologies are favoured for older ones.

Even within the Javascript ecosystem specifically, only a year ago Angular was the thing. Then React came along, and everyone jumped ship. Angular 2 was later released with little interest — Angular 1’s usage continued to dwindle. It can flip in an instant. A number of people were rightfully aggravated that they had spent all this time learning a framework that was now as good as dead.

My point here is this…

Languages and frameworks, come and go. Concepts are here to stay.

Let’s take React as an example here. I am of the opinion that although React is not here to stay. The concepts and underlying principles behind React are definitely here to stay. The way React promotes a hierarchical component structure and flows data through that structure; The way it only re-renders the parts of the components that have had a state change, is all amazing.

Now whether you chose to learn React or not is your choice, but the concepts behind it will soon be applied in other languages. And at that point, if you do understand React, you will be able to adopt that way of thinking when building your application in that languages version of React.

In the wildly successful game Portal, the narrator comments “Now you’re thinking with Portals”. The idea of the saying is that now you’re thinking with the technology you have, in this case the Portal gun which the player users to create two ends of a portal that link between each other.

We can apply this saying to the technology we work with. When you finally crack the concept of how data can travel from your models through to your views, you can say “Now I’m thinking with MVC”.

You’re not a REAL programming if you’re not doing this #obvi (source: https://xkcd.com/378/)

Trying to get into this space and become a developer can be a scary place, but fear not! Programming is about the problem solving skills you develop. Not about what shiny syntax you happen to use. Many people use a language that they never used before that job (myself included). When learning to program, you will naturally acquire these skills. It’s something I was told when I first started to learn to code, when I thought to get a job you had to be this NASA-level genius that had made 10 apps and half a dozen website (hint: you don’t need that — it certainly helps though!), I didn’t believe what they had to say. I simply couldn’t fathom this indescribable skill you just ‘acquired’ as if by some miasma. Well, it is true. Trust me.

My piece of specific advise would be to start building things as soon as possible. They don’t have to be amazing, they don’t have to be original. They just have to be something you’d like to try out. Freecodecamp does this excellently in their curriculum. One of my first projects was a calculator. I came to that project because I know how calculators work and I know how add, subtract, divide and multiple work. From my learning of programming, I knew that to keep track of the buttons the user had pushed I could store it in a variable. I knew I would need to hook up the buttons to trigger a function that would do various things. I had almost no idea of the inner workings of the application. But overarchingly I knew the concepts of how it should work. Apply this same principle to things you build and you’ll go far. It’s also great because you get to solve actual problems as if you were on a job. And debugging, like you will in a job. 90% of programming is debugging what you’ve written — so get used to it! Further, doing a project yourself will give you an understanding of what the code does. Many tutorials adopt a “type this, do that” mentality, with little explanation given to what or how this thing works. Take the time to read documentation and truly understand what you are writing.If you don’t right away, that’s ok. Take some time, chew on it, and come back to it later. That’s part of learning these concepts.

As a bonus to all this project building, at the end, you get something cool you can show your Mum, Dad, Cat or beloved family member/pet.

Keep learning. Don’t get discouraged. And have fun, make some cool stuff, programming is like digital wizardry, use your powers!

Additional reading

https://www.reddit.com/r/learnprogramming/wiki/faq#wiki_which_programming_language_should_i_start_with.3F