agility – Codemanship's Blog

An Ode To “It Can’t Be Done”

Yesterday, I blogged on its 25th birthday about the Manifesto for Agile Software Development.

Whenever I mention Agile (as she’s known to her friends and to her enemies), inevitably there’ll be at least one comment along the lines of “It can’t be done” or “Nobody’s ever done it”.

They may qualify that by citing organisational or technical obstacles to applying the values and principles laid out in the document, like entrenched management or unskilled developers or a risk-averse, command-and-control culture.

For sure, these are all obstacles I’ve faced many times. Fair comment.

Having said that, these are also obstacles that I’ve overcome in 90% of instances.

It’s possible to manage upwards, winning the support needed for – at the very least – your customer to actively engage with an agile development process.

You don’t need to change the whole organisation to create an island of self-organisation and rapid feedback. This is why I focus on teams and individuals, and don’t get involved with organisation-level “agile transformations”. From what I’ve seen in the last 25 years, They. Just. Don’t. Work. Not on that scale. But that’s a whole other post.

As for developer skills, well there’s a reason I necessarily ended up becoming a trainer and mentor in skills like usage-driven analysis & design, TDD, refactoring and continuous integration. To some extent, I’ve had to train most of the teams I’ve worked with going back to the late 1990s.

My response when friends complain that the team they’re stuck with “don’t know what they’re doing” is to ask “And what are you going to do about that?”

Command-and-control cultures are often a product of lack of trust. And in many cases, that lack of trust has been earned by past performance. Many disappointments, many broken promises. So organisations micromanage, and if anything’s guaranteed to end an experiment in software agility, it’s that.

To earn trust, teams need to deliver rapidly, reliably and sustainably. To gain the autonomy needed to deliver rapidly, reliably and sustainably, teams need to be trusted. Catch 22.

Here’s the thing, though. Nothing’s stopping you from having a conversation about that. There’s no law that says you can’t spell out the impasse to the managers you’ll be needing that trust from.

There’s a window of opportunity here for someone coming in to the organisation with fresh eyes to break the cycle. And that’s been my specialty for most of my career.

This is where it can help to have some courage, as well as a healthy disrespect for hierarchical authority.

Does it ruffle feathers? You betcha! You’ll struggle to find a project manager who has a good word to say about me.

Did we actually deliver? Almost always, yes.

Often begrudgingly, organisations have to acknowledge that the medicine is working after the team’s delivered and delivered and delivered, and users are making positive noises about the quality of it.

The fact that it’s hardly ever your code that wakes the head of engineering up at 2 am definitely sends a message.

The bargain with the Devil, of course, is that you have to deliver, and deliver, and deliver. Results are a much easier sell than promises. You really need to know what you’re doing.

While the rest of the engineering department may still be stuck in micromanagement hell, my experience has been that a reliable track record – suitably trumpeted to make sure the right people notice (dev teams need good PR) – will usually encourage management to back off.

Not always, of course. Sometimes it really is just about status and control, and I’ve used more ruthless tactics in those situations.

To borrow from comedian Stewart Lee, I can do office politics. I just choose not to.

I’d much rather have an honest conversation, but in leadership roles, my job is to remove the obstacles standing in the way of the team (and the customer, and sometimes the customer is the obstacle, so some tough love may be required).

If the choice is between creating value, and preserving the status quo, you can probably guess which door I normally go through.

Some teams are working in industries that are heavily regulated, where development processes are highly prescribed. There are a lot of boxes that need to be ticked before software can be released.

Here’s a little secret that I’ll let you have for free: you can fit a more iterative process inside a less iterative process without the high-ups and the box-tickers realising. As John Daniels and Alan Cameron Wills once said to me when we hired their company to do some specialised custom work for my client, “Yeah, we can make it look like that.”

The manual demands “Big Design Up-Front”? We stick a UML reverse-engineering step into our build pipeline. And for once, they get an architecture model that accurately reflects the actual design! We don’t need to mention that we already wrote the code.

(I would normally do something like that anyway, because an up-to-date-ish bigger picture helps us steer the architecture more effectively.)

More generally, when teams I’ve worked with have faced obstacles to rapid, reliable and sustainable development, we’ve almost always found ways to remove them or go around them or dig a tunnel under them.

9 times out of 10, the solution is very simply not to ask for permission. Just f**king do it.

Do we need the boss’s approval to write the tests first? No. If we’re delivering, what business is that of theirs?

Do we need C-suite buy-in to check-in changes in smaller batches? Again. Who’s gonna know?

It’s a happy accident that some of the changes we could make that have the highest leverage – the biggest impact for the smallest investment – are also the least visible outside the team. We don’t need to get budget to start writing unit tests. We don’t need the CTO’s signature to run a linter.

It’s usually only when the change you want to make will require people outside the team to change that buy-in is really required. This is where cohesive cross-functional teams shine best. The less we need from other teams or from management, the more autonomous we can choose to be.

And becoming agile under the radar reduces the risk of attracting the attention of organisational antibodies. We can be delivering very visible results, and the mechanisms involved can be largely invisible.

I’ve worked in teams where we’ve done everything from setting up our own token-ring network because the client wouldn’t allow us on theirs (obviously, don’t do this if you work for MI5), to inventing a team member so we could use their PC as a build server after they refused to give us a machine.

If it’s in the business’s best interest, we can usually find a way. Just takes a bit of creative thinking, a pinch of courage, and a little dash of charm. “Wah, wah, wah, we want a build machine!” is less persuasive than you think. At the very least, be ready to explain why it’s in their interest. A lot of the time, “It can’t be done” really means “I couldn’t persuade them”.

I’d guesstimate that more than half the successes I’ve been involved with doing software development in an agile way have been despite the hand we were dealt.

And never underestimate the bargaining power of a united team. Those 10% of times when greater agility stayed firmly, stubbornly out of reach were when I was the lone voice in the wilderness. In those situations, I cut my losses. And I’ve grown much better at recognising the signs before I engage.

By 2000, I was usually in a position to change the makeup of my team, too. Extreme Programmers of a feather and so on.

The poker metaphor is very appropriate here, since the real benefit of agile software development is minimising risk. We don’t let uncertainty pile up into big batches and big releases, and managers usually realise that their picture of actual progress is far more realistic, enabling them to make more informed decisions grounded in the reality of user feedback about working software.

So not only can it be done, it has been done, and successfully, many times. I’ve seen it, and I’ve lived it.

You might as well tell me that my house “can’t be built”.

Psst. Did you know that, as well as delivering high-quality training and coaching in agile development skills like TDD, refactoring and CI, I’m also available as a fractional principal engineer. I’ve worked with clients on an ongoing basis helping dev teams to turn “It can’t be done” into “We did it!” Let me have those difficult conversations with management and absorb the heat coming from the project office.

The Agile Manifesto at 25 – The Most Talked-About Unread Document In Software

Feb 11th 2001 – exactly 25 years ago – was the first day of a 3-day meeting at the Snowbird ski resort in Utah that gave birth to the Manifesto for Agile Software Development.

Like all important documents, most folks who think they know what it says don’t appear to have actually read it.

There’s no mention of “sprints”, or “story points”, or “user stories”, or “build pipelines” or “Kanban boards” or any of the other details of specific Agile methodologies. The Manifesto isn’t a “how-to” guide. It’s a “why-to” guide.

It lays out a set of values and principles for teams who wish to be more responsive to changing user and business needs, through continuous learning and adaptation in a highly people-centric – and especially customer-centric – process.

On the front page of the manifesto’s website, it states:

“We are uncovering better ways of developing software by doing it and helping others do it.

Through this work we have come to value:

– Individuals and interactions over processes and tools

– Working software over comprehensive documentation

– Customer collaboration over contract negotiation

– Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.”

The values and principles of Agile Software Development describe what it is like to be agile (with a small ‘a’). We do not “do” agile. We do not “adopt” agile. We do not “implement” agile.

We become agile.

The manifesto itself is, of course, a laudable but compromised attempt to bring multiple iterative and “lightweight” development methodologies – different “brands” – under one banner.

One of those brands came to dominate in the minds of software developers.

Another came to dominate in the minds of managers and executives, and ultimately came to dominate economically. This was a mistake.

“Agile Software Development” is doing software development in an agile way. It’s not a management thing.

Agile teams are self-organising, because putting decision-making power where (and more importantly, when) decisions need to be made – and therefore understood – is one of the keys to unlocking greater agility. A chain of command almost always becomes a serious bottleneck.

The other key message that got lost in translation is the need for strong technical chops. Teams who can’t deliver rapidly, reliably and sustainably get micromanaged, and that ends their experiment in agility.

Without development teams with strong technical discipline who are empowered to make decisions, Agile Software Development becomes little more than Agility Theatre. And that’s the reality for the majority of teams today – command-and-control waterfall development wearing an Agile Halloween costume, while the technical capability withers from lack of investment.

In this respect, Agile Software Development’s reputation – like the reputations of so many ideas in software development – is built on the experiences of teams that weren’t applying the values and principles. Most developers and managers have never even seen software agility in practice.

I recommend reading the manifesto again (or for the first time). Perhaps do it as a team. Maybe talk about how the values and principles could or are helping, and identify where your organisation might just be play-acting

We Don’t Out-Deliver The Competition. We Outlearn them.

I’ve long considered software development as a process of removing uncertainty.

The customer asks us for “Instagram, but for cats” – which could have infinite possible interpretations – and our job is to whittle those possibilities down to a single interpretation. Computers kind of insist on that.

How could this process be represented more essentially, so I can see the wood for the trees of such a complex thing?

Let’s play a game.

There are two teams, A and B, and they are both tasked with guessing a random 4-digit number. Their guesses must be submitted with numbers they have to carve on to stone tablets.

Team A guesses all 4 digits at a time. They painstakingly carve “0000” on to a tablet and submit that to learn whether it’s right or wrong. In this case, “0000” is wrong. So they painstakingly carve another 4-digit number, “0001”, which is also wrong.

If the guess is wrong, the tablet is destroyed, and they have to start all over again. Let’s say that the time to carve one digit is 1 hour. So it takes team A 4 hours to make one guess.

Team B take a different approach. They guess one digit at a time, still carving them into stone tablets. They start by guessing that the first digit is “0”, which is wrong.

When their guess is wrong, the tablet is also destroyed, and they must start a new one.

Which team – A or B – would you bet on to guess the 4-digit number first?

Worst case, team A could take 40,000 hours. Worst case for team B is 40 hours. The odds of team A guessing right in the first 40 hours are 1,000:1 against. I’d bet on team B.

Now, let’s 10x team A’s “productivity” by giving them a machine that can carve one digit in just 6 minutes. Each guess now takes them 24 minutes instead of 4 hours.

Which team would you bet on now?

The odds of team A guessing right in the first 40 hours using the 10x machine are 100:1 against. I’d still bet on team B.

We’d be mistaken to confuse “numbers guessed” with “numbers guessed correctly” as our measure of productivity here.

What’s giving team B such a massive advantage is not the speed at which they produce tablets, but the speed at which they reduce uncertainty.

When team A make their first guess, the odds of it being right are 1:10,000. On their second guess, having ruled out one 4-digit number, they are 1:9,999.

When team A make their first guess, their odds of guessing all 4 digits correctly are also 1:10,000. But on their second guess, having ruled out all 4-digit numbers beginning with “0”, they are 1:9,000.

Basically, with each guess, team B reduce the uncertainty by a factor of 10%. Team A reduce it by a tiny fraction of that with each of their guesses. To put it another way, team B outlearns team A, even as team A out-delivers them by a factor of 10.

The takeaway for me is that – in software development, considered as a process of reducing uncertainty – it’s the batch size and the feedback loops doing the heavy lifting.

If your team wants to build the skills they’ll need to outlearn the competition, solving one problem at a time in tight feedback loops, visit my training site for details of courses and on-the-job coaching.

Five Boring Things That Have A Bigger Impact Than “A.I.” Coding Assistants On Dev Team Productivity

Here are 5 factors that make a bigger difference to software development outcomes than “A.I.” coding assistants, but teams don’t address because they’re “old news, granddad!”

Smaller teams are better value/$ spent
More frequent releases accelerate learning what has real value
Limiting work in progress – solving one problem at a time – increases delivery throughput
Cross-functional teams experience fewer bottlenecks and blockers than specialised teams
Empowered, self-organising teams spend less time waiting for decisions and more time getting sh*t done

Now, I appreciate that every one of these is a can of worms that many organisations simply do not wish to open. They all have deep implications, and require foundational changes not just to the way we work, but the way we think.

For example, smaller, more frequent releases implies software’s in a shippable state more often, which implies faster build & test cycles… and down the rabbit hole we go: into testing pyramids and separation of concerns and micro-cycles with continuous testing, continuous integration, continuous code review and… Come to think of it, the stuff I teach 🙂

Another example, empowering teams requires a pretty high level of psychological safety. When people are afraid to fail, they’re afraid to try – to make calls, to take initiative, to just f-ing do it! The culture of an organisation, which may have evolved over many years, is a hard thing to reshape. There’s often a lot of unspoken rules – sure, you say your door is always open, but… It takes much work and many iterations to shift those underlying patterns in the way we interact.

But waiting on the other side of that long journey is a high capability to rapidly and sustainably create and adapt working software that meets rapidly-changing business needs. Software agility Nirvana.

We already know from the data (e.g., DORA) that “A.I.” coding assistants don’t unlock that door.

The A-Z of Code Craft – N is for Non-Blocking

By bike, it takes about 15 minutes to get from my house to Wimbledon Village in South West London. In a sports car that’s 10 times as fast as a bicycle – let’s call it a “10x” mode of transport – it still takes about 15 minutes to get from my house to Wimbledon Village.

When we travel on London’s roads, the journey time’s mostly determined not by the performance of our vehicle, but by how much time we spend waiting. Waiting at traffic lights. Waiting at junctions. Waiting at pedestrian crossings. Waiting to join roundabouts. It’s mostly waiting.

During rush hour, the average journey speed in London is just 9 miles/hour, whether you’re in a Porsche 911 or on a bicycle. This is not a limit of your vehicle, this is a limit of the system your vehicle has to work within.

We see a similar effect with software developers. Take any “10x” developer and put them in a 1x system, and you’ll get 1x performance out of them every time. (Yes, even if they use an “A.I.” coding assistant!) Fitting a jet engine to your car isn’t going to get you to Wimbledon Village any sooner.

If you really want to get more value sooner out of a dev team, don’t focus on the performance of the developers, focus on reducing the time they spend waiting – the time they spend blocked from creating value by the system they’re working within.

Sadly, blocking behaviours are rife in our industry. Pull Request code reviews are a good example, where a developer’s changes sit on the shelf waiting to be approved before they can make it into the end product.

There are many other examples of blocking behaviour, such as waiting for customer input, waiting for the UX designer to provide wireframes, waiting for a QA team to test the software, and so on. The average team spends most of their time not moving forward, but sitting at proverbial traffic lights.

In concurrent programming, we have a concept of “non-blocking” processes that can continue without waiting for another process to finish. Maximising the non-blocking parts can hugely improve the performance of the system as a whole.

There are so many common blockers that it’s beyond the scope of this little essay to discuss them all, but I can offer some general advice on unblocking your development teams:

Trust and empower teams to make more of the decisions
Encapsulate the knowledge and skills needed to deliver user/business outcomes within teams
Limit the amount of work in progress. It can be hard to see the bottlenecks and blockers when the team has a bunch of half-finished features in play.
View all handovers and sign-offs with hostile suspicion. They are the traffic lights of your dev process.
If something’s hurting (e.g., merging feature branches), do it more often. If possible, do it continuously. This is especially true about communication.
Java Jane doesn’t have to wait for the DBA to add a column if she can do it herself. T-shaped developers get blocked far less often.
More traffic = slower traffic. Team size has a similar, very well-known effect.
Dependencies are not your friend. If adding a new feature involves every team, you’re going to be doing a lot of waiting.
Remember that traffic lights exist for a reason. The speed demons in our teams are as likely to cause accidents as the speed demons on our roads. When unblocking processes, remember to make sure safety isn’t compromised. Not testing code before a release might seem faster…
And don’t forget – it’s the speed of the system we’re optimising. Focus more attention on that, not on individual developers.

If you’re serious about building your team’s capability to rapidly, reliably and sustainably evolve software to meet rapidly changing business needs, my Code Craft and Test-Driven Development live remote training workshops are HALF PRICE until March 31st 2025.

The A-Z of Code Craft – M is for Modularity

The ultimate goal of code craft is to be able to rapidly and sustainably evolve working software to meet rapidly changing business needs.

The key to this is short delivery lead times, and the key to that is making sure the software’s shippable at any time.

The key to software being shippable at any time is continuous testing, and the key to continuous testing is tests that run very fast.

If it takes 8 hours to sufficiently test the software, we’re at least 8 hours away from it being shippable (in practice – because you can introduce a lot of bugs in 8 hours – a lot longer).

If it takes 80 seconds, then a potential release is much, much closer to hand (and the bug count’s likely to be much, much lower – it’s a win-win).

So fast automated tests are the key to agility. And the key to fast automated tests is good separation of concerns in the architecture; otherwise known as modularity. (See “E is for Encapsulation“)

In an effectively modular design, different aspects of the system can be changed, reused and – most importantly – tested without needing to change, reuse or test other aspects. So we can test the calculation of the mortgage interest rate without having to involve, say, a database or a UI in that test.

Most programming languages and tech stacks have their mechanisms for encapsulating code at multiple scales from individual functions or classes, all the way up to distributed services and systems of systems. But the same principles apply at every level.

Well-designed modules:

Have one reason to change
Hide their internal workings
Are easily swappable
Have interfaces designed for the cient’s needs (not “What does this module do?”, but “What does the client need to tell it to do?”)

In particular, when it comes to achieving test suites where the vast majority run very fast, we need to cleanly separate our application’s logic from external concerns like accessing files or databases, calling web services, and so on. (See “H is for Hexagonal“).

I’ll repeat it one more time, for the folks at the back: the key to agility is modularity.

The A-Z of Code Craft – J is for Just-In-Time

Supermarkets are famously cashflow businesses. They don’t buy a million bottles of shampoo and stick them in a warehouse to sell throughout the year. They buy just enough bottles for the week, and those bottles will find their way on to shelves and into shoppers’ baskets very quickly.

Their supply chains are what they call “just-in-time”. The product goes from the supplier’s factory to the supermarket checkout as quickly as possible. Their systems and processes are streamlined to make lead times short.

The amount of money a business has invested in, say, stock or parts or ingredients is called “working capital”. Businesses of all kinds seek to minimise their working capital and to maximise their cashflow so there’s enough money to “keep the lights on” while they realise the return on their investment.

Software development’s an investment that our customers hope to see a return on in a similar way. And the flow of that return can be crucial to keeping the lights on. I’ve seen a lot of start-ups fail, not because what they were creating had no value, but because they ran out of cash to pay staff and creditors before that value could be realised.

The working capital in software development – the shampoo sitting in warehouses, if you like – is all the work that the development team has done that has yet to make it into the hands of users. Software that can’t be used has no value. Let’s call it “work in progress”.

Development teams who care about keeping the lights on seek to minimise the amount of work in progress, and to maximise the flow of value out into the business. They don’t design and build 50 features and then release the software after 12 months. They design and build one feature and release that as soon as it’s ready to be consumed. From the factory to the checkout lickety-split!

And in the same way a supermarket’s supply chains are optimised to get the product from the supplier to the checkout as soon as possible, the best dev teams optimise their processes to make the lead times on getting feature and change requests into production as short as possible, and reliably as possible.

Just-in-time delivery processes have another major advantage. If your supermarket bought a year’s supply of a particular brand of shampoo, and halfway through the year a new brand is launched with a massive ad campaign, you’re potentially stuck with a tonne of bottles that are suddenly “so last year”. If you buy them one week’s worth at a time, you can switch brands and capitalise on the buzz.

The practices of code craft – Continuous Testing, Continuous Integration, Test-Driven Development, Refactoring, Modular Design – are enablers of limiting work in progress, shrinking lead times, maximising the flow of value and improving responsiveness to changing needs.

The A-Z of Code Craft – A is for Agility

When the idea of software development as a craft became popular, a lot of the talk was about “professionalism”, “mastery” and “beautiful code”. But this missed the point of craft, and arguably helped to alienate business stakeholders, giving them the impression software craft was some kind of esoteric pursuit with no value to businesses.

Nothing could be further from the truth. When we shift the focus from what craft means to us as developers, we can consider how practices like continuous testing, refactoring, modular design and continuous integration impact businesses.

The disciplines of code craft enable continuous delivery – the ability to ship working software that does what the customer wants at any time, and to do that reliably and sustainably.

No delays. No need for an “acceptance testing” or “stabilisation phase”. No downtime or frantic bug fixing after release. The software’s what was agreed, and it’s ready to go at the push of a button. Again, and again, and again, for as long as the business needs.

This dramatically shrinks lead times on new features and changes, making software much more responsive to changing business needs.

The mistake a lot of organisations have made is believing that the key to agility is what they’ve heard of as “Agile” – Scrum, Lean, Kanban etc. Nu-uh! The key to /real/ agility is code craft, and you build your “Agile” processes around those technical disciplines.

Because if the software’s not fit for release at any time, all the stand-up meetings, burndown charts and Jira tickets won’t amount to more than “Agility Theatre”. This is why so many “Agile transformations” failed. You can’t manage your way to software agility without building that capability in your dev teams.

Measuring Inner-Loop Agility

When I teach teams and managers about the feedback loops of software development, I try to stress the two most important loops – the ones that define agility.

Releases are where the value gets delivered, and – more importantly – where the end user feedback starts, so we can learn what works and what doesn’t and adapt for the next release.

The sooner we can release, the sooner we can start learning and adapting. So agile teams release frequently.

But frequency of releases doesn’t really define agility. I see teams who release every day or every week, but feature requests still take months to get into production.

That feature-to-production lead time is our best measure of how responsive to change we’re really being. How soon can we adapt to customer feedback?

For a portfolio of software products, a client of mine plotted average feature-to-production lead times against the average time it took to build and test the product.

We see a correlation between that feature-to-production lead time and the innermost loop of software delivery – build & test time.

Of course, this is a small data set, and all the usual caveats about “lies, damned lies and statistics” apply (I would love to do a bigger study, if anyone’s interested in participating).

But I’ve seen this distribution multiple times, and experienced it – and observed many, many teams experiencing it – in the field.

Products with slow build & test cycles tend to have much older backlogs. Indeed, backlogs themselves are a sign of slow lead times. I explained the causal mechanism for this in a previous post about Inner-Loop Agility. When we want to optimise nested loops, we get the biggest improvements in overall cycle time by focusing on the innermost loop.

Now, here’s the thing: everything that goes on between releases is really just guesswork. The magic happens when real end users get real working software, and we get to see how good our guesses were, and make more educated guesses in the next release cycle. We learn our way to value.

That’s why Inner-Loop Agility is so important, and why I’ve chosen to focus entirely on it as a trainer, coach and consultant. I can’t guarantee that you’re building the right thing (you almost certainly aren’t, no matter how well you plan), but I can offer you more throws of the dice.