TrenchOps 🐎

Insights from the tech trenches

Cover Image

📌 Welcome @ TrenchOps 🐎

About TrenchOps: An IT Support Engineering & Tech Career Blog

TrenchOps is a blog about IT support engineering, technology careers, and management in the software industry - written by a Lead Support Engineer with 20+ years in enterprise software, from 1st- to 3rd-level support and escalation through technical account management, program management, and consulting.

The name: "TrenchOps" means "operations in the tech trenches" - hands-on frontline IT support, technical operations, and enterprise software work, not military combat. "Power horses" 🐎 is our term for the dependable engineers who carry a team's load.

Who it's for

  • Engineers (support, SRE, DevOps, security, architects) growing their careers
  • Managers and team leads building real processes and preventing burnout
  • Senior leadership wanting visibility into the mismanagement that quietly kills retention, quality, and revenue

What it covers

Practical career and leadership guidance plus anonymized stories from enterprise IT support: promotion strategy and burnout, imposter syndrome and boundaries, retaining high performers, leadership anti-patterns, support as a revenue advantage, documentation and KCS, and IT resilience without single points of failure.

Categories

Field notes for the engineers, managers, and leaders who keep enterprise software running.

Cover Image

Your Brain Only Commits the Diffs

Why five years at the same job feels like a long weekend

A colleague once told me he had fifteen years of experience. His manager, in a rare moment of honesty, replied: "You have one year of experience, fifteen times."

Brutal. Also, physiologically accurate.

Here is the mechanism. Your brain does not store duration. It stores change. It works like a version control system with aggressive deduplication: if today is byte-identical to yesterday, no commit gets written. Same alarm, same commute, same standup where six people say "nothing from my end", same lunch, same evening scroll, same bed. Nothing new to encode, so nothing gets encoded. And when you later ask your memory "how long was that?", it does not count days. It counts commits.

Empty log, short history. That is time compression - and it is why a week of travel feels longer in hindsight than the four months of office routine that funded it.

Researchers have a name for the small-scale version: the oddball effect. Drop something unexpected into a stream of repetitive stimuli and the brain rates the surprise as having lasted measurably longer than the identical repeats around it. Attention spikes, uncertainty spikes, encoding spikes. The stretch is on the order of tens of milliseconds in lab conditions - tiny, but it proves the machinery exists. Scale it up to weeks and you get the reason your thirties evaporated.

Read more
Cover Image

The Day Management Banned German in the German Support Team

And other lessons from running teams you don't understand

I once worked in the German unit of an international support organization. Next to us, both on the floor plan and spiritually: the French team. Everyone else on the planet got English support.

Why German support? Not because Germans can't speak English. They can. They just believe they can't, and not every German wears their accent as proudly as Werner Herzog. The market was simply too important. Why French support? France has the Toubon Law - product information for French consumers has to be in French - and in practice companies extend that to support, partly because, let's be diplomatic, the English proficiency curve in France has a certain shape.

Management hated both teams. Openly, structurally, consistently. Two reasons:

First, we were hard to replace. German-speaking engineers outside Germany were rare, and remote work wasn't a thing yet. Leverage. Managers despise leverage they don't hold themselves.

Second, our metrics looked "bad" and couldn't be fixed with a pep talk. German customers are demanding, so closure rates and CSAT ran lower. My favorite artifact: our 1-10 post-case survey collected an avalanche of German 5s. Not from dissatisfaction. A typical case: customer reports our software behaving "flaky," we analyze it, prove their storage paths are dropping, hand them a clean action plan, close the case. The German mind computes: "Friendly, competent, but my issue is technically not resolved. A fair 5." The same conversation gets a 7 or 8 in the anglosphere. Same work, same outcome, two points of CSAT difference - purely cultural. Any dashboard comparing those teams side by side was measuring national character, not support quality. Nobody upstairs wanted to hear that, because the dashboard was easier to read than the customer. On a side note: we also had a lot of German customers who gave of 1's and 2' because in the German school those are excellent grades, just like A's and B's. On a scale from 1-10 those values are, however, perceived as "catastrophic". Later, they changed the survey system to a smiley system which was harder to misunderstand.

Read more
Cover Image

Why vibe-coded tools are obsolete before release

And why the next top hire thinks three problems ahead

A team I once worked with built an internal tool in eleven days. That used to be a quarter's work. It was a beautiful ticket-triage dashboard with auto-tagging, severity prediction, a heat map and a dark mode nobody asked for. They used AI-assisted "vibe coding," which means describing what you want in plain language and letting the model write most of the code. The demo was flawless.

Then a quiet engineer in the back raised his hand. He was one of those people whose calendar is suspiciously empty because he actually works.

"Why do we get these tickets at all?"

Silence. Someone checked. About 40% of the category came from a single cryptic error message in the installer. He rewrote the message that afternoon and added one link to a knowledge article. The ticket volume collapsed within weeks.

The dashboard was now a very elegant way to watch an empty room. Eleven days of work went obsolete before its release party. The cake was still good, though.

Nobody did anything wrong. And that is exactly the problem.

When building becomes cheap, the bottleneck moves

For decades, building software was the expensive part. Thinking about what to build was considered cheap, almost a luxury. You could afford a fuzzy problem statement because implementation took so long that reality corrected you along the way.

Read more
Cover Image

The Corporate Cicada Cycle

Why support is powerless in some years and decides renewals in others - and how to know which year you are in

Cicadas spend most of their lives underground. Then, on a schedule nobody voted for, they climb out, make an unbearable amount of noise, and vanish again. Some species run on a 13 or 17 year clock. Prime numbers, apparently, because predators can't sync to a prime.

Corporate culture has a similar rhythm, just shorter and less elegant. It swings between top-down and bottom-up. And whichever half of the cycle you are standing in determines something most support leaders never account for: whether your work influences revenue at all this year.

Not how good your work is. Whether it counts.

The two weather systems

In a top-down phase, decisions are made somewhere with a nice view and a beverage. The prevailing theory of human nature is that people are fundamentally slacking off and need to be measured harder. Surveillance software gets a budget line. Presence gets confused with output. The message to everyone below director level is: be grateful you have a job, do your job, don't editorialize.

In this phase, support is a cost center. Full stop. You are a number in a spreadsheet next to "cloud spend" and "office plants." Your KPIs are cost-per-ticket and time-to-close, which is another way of saying: "please handle customers faster and think about them less."

Read more
Cover Image

What Nobody Warns You About Before Your First Invoice

Three things kill small businesses that never appear in a business plan: the room you work in, the stories you absorb, and the money that was never yours.

I have been self-employed a few times. Before my enterprise career, between two of them, and on the side. None of it made me rich. All of it taught me things you cannot learn from a startup podcast, because podcasts talk about scaling and my problem was that a customer's dog did not like me.

Let me hand over the three lessons that actually cost me something

Your business has a physical address, even when you think it doesn't

Everyone starts in the bedroom, the garage or the study. Fine. Nobody plans the other rooms.

One of my ventures was classic IT service. Persuading stubborn printers to acknowledge the existence of WiFi. Removing pre-installed bloat from our favorite corporate operating system. Explaining, for the fourth time, that a tablet needs updates. Mostly on-site, at people's homes.

Some clients objected to travel fees. Reasonable. They wanted to come to me. Slight problem: I had no office and no presentable workshop. So I invited them into my dining room - statistically the least private room in the house, right up until a stranger sits in it holding a coffee cup and telling me about his divorce while I diagnose a random reboot.

Read more
Cover Image

Your Brain Doesn't Learn From Success

It Learns From Being Wrong

I once watched a senior engineer - twenty years of experience, could read a stack trace like a bedtime story - try to learn to juggle at a company offsite. Three balls, two hands, zero dignity. She dropped everything for twenty minutes straight while interns filmed it.

Six months later she told me that offsite was the week she finally cracked a certification she'd been stuck on for a year. She was convinced it was the juggling. She might have been right.

Here's the mechanism, and it's backwards from what most people believe: the adult brain doesn't rewire because you feed it information. It rewires because you feed it errors. When your predictions about the world fail hard and repeatedly, your nervous system releases the neuromodulators - dopamine, norepinephrine - that mark active circuits for change. No errors, no chemical signal, no rewiring. Recent neuroscience even points to a dedicated brain-wide "model failure" alarm system (the locus coeruleus) whose whole job is shouting "your predictions are garbage, update everything."

That feeling of frustration? That's not a bug. That's the smell of the signal being released. The brain games industry sells you comfortable puzzles you're already decent at. Comfortable means low error rate. Low error rate means the alarm never fires. You're paying a subscription to not learn.

Read more
Cover Image

The Strategy That Never Won a Single Match - And Won Everything

What a 1980 computer tournament explains about your worst customer, your dumbest feud, and your exit interviews

Around 1980, a political scientist ran a tournament for computer programs playing the iterated Prisoner's Dilemma. Game theorists, mathematicians, economists, the whole intellectual heavyweight class, submitted strategies. Some were baroque: probabilistic betrayal, opponent profiling, deception layers built to squeeze a few extra points out of the naive.

The winner was a handful of lines of code, submitted by a mathematical psychologist. Cooperate on move one. After that, do whatever the other player just did. That's it. Tit For Tat. Shortest entry in the field. It won again in the second tournament, where everybody already knew it was coming and had specifically designed strategies to beat it.

Here is the detail people always skip: Tit For Tat never won a single individual match. It cannot. By construction it can never score more points than its opponent, because it only ever defects in response. The best possible outcome against any given opponent is a tie.

It won the tournament by being the strategy that everyone did well against, including itself.

Read that twice if you measure your quarter in individual wins.

Read more
Cover Image

Count the Rounds

Why a five-stage interview process tells you everything about the job - before you even get it

The job post says "fast-paced environment." The process says recruiter screen, hiring manager call, take-home assignment, technical deep dive, second technical deep dive, "culture add" panel, psychometric questionnaire, and a final chat with a VP who will ask you where you see yourself in five years.

That's eight rounds. For a company that claims to "move fast".

Believe the process, not the post.

Times are rough. Plenty of good people lost their jobs to the AI wave, and plenty more are quietly refreshing job boards at their desks. My personal bet is that once the AI honeymoon ends, a lot of companies will discover that their institutional knowledge left in a cardboard box, and they'll be rehiring. Until then, desperation is a terrible career advisor. It tells you to say yes to every round, every test, every "just one more quick call with a stakeholder."

Don't. Here's why.

What an interview is actually for

A competent company reads your CV and can map your skills to its needs. With a little effort, it can verify most of your history. So the interview only has three real jobs.

Read more
Cover Image

Your AI Agent Reads 50,000 Tokens to Learn One Thing

Compression as via negativa: the cheapest token is the one you never send

Take any coding agent session and open the raw request log. Not the pretty transcript - the actual payload. What you find is a JSON search result with 100 hits where 3 mattered, a log file where the interesting line is buried in 800 lines of "INFO: still fine," and a directory listing of a repo the model already walked twice.

Then apply the uncomfortable ratio: cost of the finished output versus the raw inputs actually required to produce it. The model needed maybe 2,000 tokens of real signal. You paid for 55,000. In manufacturing terms, that is a part made of solid gold to hold a plastic clip in place.

I spent a few weeks running everything - coding, research, boring text work - through a compression proxy called Headroom, and the interesting part was not the money. It was watching exactly how much of what I "sent" to the model was never information in the first place.

What it actually does

It sits between your agent and the API and squeezes everything the model reads - tool outputs, logs, retrieved chunks, conversation history - before it goes upstream. Compression happens locally, nothing gets shipped off to a third party to be shrunk. The model can pull the original back if it needs it.

Read more
Cover Image

Be Employee Obsessed

Everyone says "customer first". Almost nobody means it.

Years ago, I shared a few carbonated drinks with an SVP whose management philosophy fit on a bar napkin: "Be employee obsessed."

I was skeptical to the point of rudeness. So I asked him straight: "with that approach, how do you make sure customers get world-class service?"

He didn't blink. "If you hire top performers, you never have to worry about customer satisfaction. I have never once hired a talented support person who wasn't already customer obsessed. Nobody ends up in this job by accident. It's ingrained, or they'd have picked a role where humans don't call you when their week is on fire."

So the model was simple: hire people who are obsessed with customers, then spend your own energy protecting them. Not managing them. Protecting them. From three specific things: corporate nonsense, burnout, and what he cheerfully called "stupid customers".

Let me unpack all three, because each one had teeth.

Burnout is rarely about volume

His claim: information technology has one of the worst burnout rates in white collar work, and the cause is usually misdiagnosed as "too much work". Sometimes it genuinely is too much work. But more often, he said, people are either doing the wrong things, or doing the right things in a broken flow imposed on them by the company.

Read more