Rupert Foggo McKay

The Hidden Complexity of Barcodes

2026-05-29T00:00:00Z

What is a barcode encoding?

The black and white stripes are an optical encoding of a decimal number. A scanner reads the pattern of bars and recovers the digits.

For retail products, the dominant formats are UPC-A (12 digits, common in the US) and EAN-13 (13 digits, common everywhere else). Every product you buy in a supermarket has one of these.

A brief history

A corner shop with a few hundred products can function with a cashier who knows the prices. A suburban supermarket in 1970 might stock 10,000 SKUs and serve hundreds of customers a day. At that scale, manual price-keying at checkout wasn't just slow — the error rate across millions of daily transactions became a real cost.

The inventory side was arguably more significant. Without item-level sales data, replenishment relied on periodic physical counts and buyer intuition. The barcode promised to turn the checkout lane into a data collection point: every sale automatically recorded, stock levels updated in real time.

The timing was right. IBM had been building mainframes for large retailers through the 1960s, so the back-end infrastructure existed in prototype form. The missing piece was a reliable way to get an identifier off a tin of beans and into a database without a human in the loop.

In 1970, US grocery trade associations formed the Ad Hoc Committee on a Uniform Grocery Product Code — a voluntary industry body with no regulatory authority — and invited technology companies to submit proposals.

After evaluating submissions from IBM, RCA, Litton and others, they chose the IBM proposal, designed primarily by George Laurer, in 1973. This became UPC. The first product ever scanned was a pack of Wrigley's chewing gum, at a Marsh supermarket in Troy, Ohio, on 26 June 1974.

RCA had pushed a competing bull's-eye circular barcode that could be scanned from any angle. It lost out partly because it was harder to print reliably on packaging of the era.

In the US, UPC was administered by the Uniform Code Council. Europe responded with EAN in 1976, administered by the International Article Numbering Association. The two organisations coexisted for decades.

UPC-A and EAN-13

When EAN was designed in Europe in 1976, there was already a large installed base of US UPC scanners. The designers needed EAN to be backward compatible.

Their solution: EAN-13 is a 13-digit superset of the 12-digit UPC-A, with one extra leading digit. European codes use non-zero values for that digit — the Netherlands prefix 870, for instance, starts with 8. For US products, the leading digit is 0, which means a UPC-A code becomes a valid EAN-13 simply by prepending a 0. An EAN scanner seeing a code that starts with 0 can drop that digit and treat the remaining 12 as UPC-A — the mapping is bijective.

This meant European scanners could read American barcodes from day one.

GS1

In 2005, the two organisations merged to form GS1, a non-profit that now serves as the global authority. The system works hierarchically:

GS1 allocates 3-digit prefixes to national member organisations (GS1 UK, GS1 Netherlands, etc.)
Each national body allocates company prefixes to businesses that pay to join
Each company then assigns the remaining digits to their own products

So an EAN-13 code like 0 01234 56789 X breaks down roughly as:

001 → GS1 US prefix (starting with 0, as established above)
234 → company prefix (assigned to a specific business)
5678 → product reference (assigned by that company)
X → check digit

This federated design means no central body needs to know every product in the world. They just carve up the number space and delegate.

Prefix allocation

GS1 prefixes are 3 digits, ranging from 000 to 999, giving 1,000 possible values. Not all are assigned — large portions are reserved for future use. In practice there are around 120 active member organisations.

The US alone occupies a huge chunk: 000–139. The Netherlands gets 870–879, a block of 10 prefixes. Smaller countries often get just one or two.

The structure within a prefix

Once you're past the 3-digit GS1 prefix, the remaining 9 digits (before the check digit) are split between a company prefix and a product reference. Crucially, this split is not fixed.

GS1 allocates company prefixes of varying lengths depending on how many products you need:

Company prefix length	Product reference digits	Max distinct products
7 digits	5 digits	100,000
8 digits	4 digits	10,000
9 digits	3 digits	1,000
10 digits	2 digits	100
11 digits	1 digit	10

Larger companies with more products get shorter prefixes, leaving more room for product references. Smaller companies get longer prefixes and can identify fewer products.

The 12-digit code space gives 10¹² = 1 trillion possible identifiers. That sounds like a lot, but every digit is decimal — 0–9 only, no hex, no alphanumeric encoding — which makes the space around 4,700 times smaller than it would be with base-36 characters. The constraint was baked in during the early 1970s, when systems needed to be operable by humans reading and typing numbers.

The check digit

The final digit is a checksum mathematically derived from the preceding ones — alternating multiply-by-1 and multiply-by-3, sum the results, modulo 10. This catches most scanning errors and accidental transpositions. A misread digit almost always produces an invalid check digit.

Collisions

In theory, the namespace partitioning guarantees uniqueness — if everyone follows the rules. In practice, a few things go wrong:

Fraud — counterfeit goods sometimes copy a legitimate barcode from a real product.

Recycling — GTINs (the formal term for barcode numbers) were historically permitted for reuse after a 48-month window. GS1 prohibited reuse entirely in 2019, but old assignments still circulate in legacy systems.

Unofficial barcodes — small sellers sometimes generate barcodes without a GS1 prefix, using randomly chosen numbers that may clash with real products.

In-store barcodes — codes starting with 2 are explicitly designated for internal store use (loose produce, deli counter items). These are not globally unique and only mean something within that store's system.

Shortcomings

No GTIN history — GS1's registries reflect current assignments only. Historical data exists only as a byproduct of services that happened to cache it — crowd-sourced databases like Open Food Facts, web archives, commercial lookup tools. No one designed a system to track reassignment history; the assumption was that reassignment would be rare enough not to matter.

Prefix exhaustion — If a company exhausts its product reference space, it can't get its prefix shortened — that would renumber all existing products, cascading through every retailer and system that holds those GTINs. So they acquire a second prefix and assign new products from there.

Mergers — Two companies bring two prefixes. There's no clean way to consolidate, because renumbering would require updating every downstream system. Merged companies typically carry both prefixes indefinitely, which is why products from the same brand sometimes look like they come from different companies.

Corporate identity coupled to product identity — the constraint underlying all of this. The GTIN system ties product identification to corporate structure, which causes pain whenever corporate structures change — and they change constantly.

Digital Link

The 1D barcode is showing its age. It encodes only the GTIN. Modern supply chains also want to encode expiry dates, batch numbers, serial numbers, and sustainability data on the same label. Right now that typically means printing multiple barcodes on the same packaging.

GS1's answer is Digital Link — a QR code that encodes a URL, pointing to a rich product data page. The URL structure embeds the GTIN, so existing systems can still extract the product identifier, but the code can carry much more.

GS1 has been pushing a target called Sunrise 2027: by that point, major retailers' point-of-sale systems should be capable of scanning 2D codes at checkout. Walmart, Carrefour and others are publicly committed to this.

The transition strategy for suppliers is straightforward: print both. Legacy 1D barcode for scanners that aren't yet 2D-capable, Digital Link QR code alongside it for those that are. Suppliers can adopt now without waiting for universal retailer readiness.

The 1D barcode won't disappear in 2027. Vending machines, self-checkout kiosks, and embedded scanners in smaller retailers have long replacement cycles. The magnetic stripe on payment cards was superseded by the chip in the 1990s and lingered for decades. Expect something similar here.

Take care,

Rupert

Death

2024-03-24T00:00:00Z

My mum was born 22nd November 1946. In 2005 she died of cancer.

As of January 2023, she died more than half my life ago.

We were in New Zealand in 2003. One day she began to bleed from the vagina. I remember her on the toilet with the door open, crying: "I just want to stop bleeding". The doctors diagnosed her with uterine cancer. She had a hysterectomy not much later. But later still when she had trouble breathing, the doctors found cancer in her lungs. You can remove a uterus, but you can't remove the lungs.

12th of May 2004 21:55 Wednesday

Hello today. Did I tell you mum’s got cancer? Well she has. She seems to be responding well the to chemo and is likely to survive. She takes it on the chin. I try harder to help around the house for my some what incapacitated mother. Life goes on. Good night.

She was put on a course of chemotherapy. They put poison into her blood. It is a treatment which attacks all the cells in the body. The theory goes that if you do it for long enough the cancerous cells will die, at which point you stop putting poison in the blood, and hope the other cells will recover. Hair falls out as the follicle cells die. The stomach lining also suffers especially. Usually patients on chemotherapy lose a lot of weight. If any other medical treatment had even a half the side-effects, it would not be allowed on humans. But chemotherapy is reserved for cases of desperation.

10th of April 2005 Sunday 23:51

Mum came back from London weeping, she could be dying soon but is not afraid of that but is sorry that she never got further in life and afraid that she’ll never see me and felix grow up.

One day after school my mother texted me to come home. Her hair was coming out. We sat together on the couch, gently pulling it out in clumps. It felt like peeling velcro.

My mother didn't like being bald. She was self-conscious in public about it. She tried wearing wigs for a while, but felt self-conscious about wearing a wig too. Eventually she would become adept at styling her outfit with a hat or headwrap.

19th of August 2005 Friday 23:25

I’m back from a ten day holiday in Holland and mum has begun another course of chemo-therapy. This has put her in an ill state.

She was so tired all the time. The chemotherapy puts the body in a constant state of fighting to recover from the damage it does. We were on the train one time; I had to help her walk. She needed to sit down to rest. We found a little nook near the kitchen carriage. It was quiet there and she fell asleep sitting beside me. But soon a group of five lads came and sat nearby. They were drinking and joking and laughing. Loudly. I could tell my mother couldn't sleep but she needed to. I wanted so desperately to tell the men to shut up and let my dying mother rest for just one little moment. But I was afraid of them. They were loud adult men. I was just a scared little boy, and I didn't know how to protect my mother.

1st of November 2005 Tuesday 23:06

Mum had problems with breathing this morning and I was pulled out at lunch to drive her to the hospital, she’s now staying there for a couple of days on a ventilator. She’s caught a chest infection.

The cancer in her lungs caused the tissue to become useless. As her lung capacity melted, she had to be put on an oxygen mask. As her lung capacity went further down, so the oxygen concentration had to go up. The high oxygen concentration dries out the lungs. Periodically they had to remove the oxygen mask, and use a different tool. Something like an inhaler but specifically for spreading moisture in the lungs. She had to inhale and exhale through this thing for a minute. The doctor would stand there and time every second. She cried the whole time, every time, because she couldn't breathe. She felt that she was suffocating every time. We couldn't do anything.

My brother Felix didn't live with us during this time. He had been studying in Coventry, and then after graduation, moved to London. I remember the first time he saw this process of suffocating my mother for 60 seconds. He cried in the chair beside her bed. I watched him cry, and in that moment I realised how numb I was to all this. I had gotten used to it.

3rd of November 2005 ~~Wednesday~~ Thursday 00:35

Visited mum in hospital, she says it’ll be better for me if she dies sooner so that I can get over it before my A2level final exams.

I'm not "over it".

4th of November 2005 Friday 00:01

Mum’s on a nebuliser, she doesn’t have a chest infection, she doesn’t have a clot in her lung, she doesn’t etc. etc. but still she can only barely breath without aid.

I was given "power of attorney". If my mother became a vegetable, I had the legal power to decide whether to pull the plug. I didn't want that power. But they had to give it to someone.

6th of November 2005 23:52 Sunday

So Mum’s now on 15 litres of oxygen an hour (the maximum) and a constant morphine drip.

Mum could die any time now. She's just not breathing right.

Every week or so the doctor would tell us that they had increased the oxygen concentration she was receiving. Pure oxygen is not fit for humans however. There is a maximum limit that is safe. So it was that one day the doctor said: "We have reached the maximum oxygen concentration we can administer". He didn't say more than that. What he didn't say was "Whatever hope you had up until now, forget about it. Her death is now guaranteed". I looked around the room at the other adults. I wanted to understand if they had heard the thing the doctor hadn't said. But they didn't say anything either.

With the maximum oxygen limit and the lung capacity decreasing bit by bit each day, my mother died of an excruciatingly gradual suffocation. In the last two days of her life, she was largely unconscious the whole time.

The call was put out to all friends and family, this is it, come say your goodbyes now. They flocked to us. They flocked to her. Some 20 or so people seemed camped out in our home, and in the hospital ward. Some came all the way from New Zealand.

7th of November 2005 Monday 22:18

Got pulled out of school at 10 o’clock cos mum’s going comatose.

Spoke to mum in private on one occasion (you cant speak with, only to) and cried alot. Becky came back down late in the day and Dad came. I’ve been at the hospital since 10 and up til now. (12 hours) Toni and Simon are staying the night with mum.

I remember speaking to her that day. I was left alone in the room with her. I told her about school, and my saxophone lessons. I tried to keep talking, because I was afraid of what would happen when I ran out of things to say. But I didn't know what to say. I couldn't bare my own silence. But I couldn't bare to leave the room either. I needed someone to come in and relieve me from having to confront this reality. Eventually, with great guilt I left the room, just briefly to tell the others in the waiting room, that someone else should go be with her now. Immediately afterwards I stepped to one side in the corridor, and wept facing the wall. I couldn't see through my own hands and tears. But an anonymous nurse slipped in to hug me. I never saw their face. In that moment they were literally just a shoulder that I cried into. I owe them so much for that moment in which I was able to sob. Neither of us said a word during. I hope I at least mumbled a "thank you" afterwards, but I don't remember.

9th of November 2005 Wednesday 03:03

Got a call from simon at 7 last morning saying mum odd only had a few hours left. We got their and she was weaker than ever. Its been a fucking long day. Linda met us at the hospital and brought a tiny little locket of Manley beach. She’s hanging in though. We all know its time for her to go soon but she’s clinging to something.

We had been staying more or less non-stop in the hospital ward. Watching her die. That evening someone recommended that Felix and I go home and sleep just one night in our own beds to recover. So we did.

9th of November 2005 Wednesday 23:38

Got a call from Simon at 10 in the morning saying that mum’s breathing had become distressed and as we prepared to set out he called us again to say that she had died. We all still went to see her

Neither Felix nor I were surprised. But I felt guilty. Later I would lie to people that asked "of course I was there when she died". I would feel guilty about the lie too.

We travelled to the hospital to visit her. I had expected to see my mum in a hospital bed, just the same as yesterday. But her limp jaw was so twisted to the side. This was not my mother. This was a corpse.

Just as fast as the swarm of well wishers arrived, so too did they leave. Even Richard and Felix returned to London. Felix encouraged me to visit often, but I didn't like to leave home. So it was that I came to living alone in a three bedroom house. I don't know how I survived that time. I was not ready to live alone. I was not ready for the world.

The Skull

2023-09-23T00:00:00Z

There's a skull on my shelf. I'm not kidding. It's a real human skull.

I used to spend every summer with a family in Holland. The grandfather of the family was called Henk Pelser. He was a medical student in Amsterdam when World War II broke out. To the Nazis he was an Aryan, and he could simply have kept out of trouble in occupied Netherlands. But he saw Nazism for the evil it was and spent all his time being a thorn in their side, at great personal risk.

There's an organisation in Israel called Yad Vashem, who are dedicated to the remembrance of the holocaust and those who resisted. Here is their page on Henk Pelser.

Henk died in 2007. He seemed to die by choice. His wife Sara had died of throat cancer shortly before. Around the time she died, Henk had been recovering from a hip replacement. He was doing simple exercises everyday to get used to walking again. But when Sara died, he just stopped trying. I remember him staying in that chair. His daughter, Simcha, would speak quietly to him, but he was adamant, there was nothing left for him without Sara.

At his funeral they played a piece of music from a dutch nursery rhyme. It was the tune he would whistle outside Sara's window when they were just teenagers in Amsterdam. She would come to the window even when her parents wouldn't let her out, just so that they could see each other every day.

After his death, we had to go through the sombre process of looking through all his things and seeing what ought to be kept. It was then that we found the skull. No one knows how or when he got it. But it had a pair of pins embedded on either side, which hinted at our best guess: it may have originally been part of a complete preserved skeleton arranged on a scaffold and used as a teaching aid for medical students. You don't often see these around anymore, and even when you do, they tend to be made of something synthetic. Originally however these were made from the remains of a real human.

So there it was. A skull.

No one else wanted it. They ummed and ahhed. It couldn't possibly be thrown away, but no one had the stomach to keep it. I had been respectfully silent all day. I'm not a blood relative after all, and was only there as a spare pair of hands. But finally I suggested that I could take it. To my surprise no one disagreed.

Since then the skull has stayed with me. First I had I took it in hand luggage to England, and then it travelled with me again when I moved to the Netherlands permanently. On both occasions I did not declare it and security never seemed to notice it.

The skull still sits on my shelf. Some nights before I go to bed, I'll rest my hand on the crown. I'm not sure what I intend by this. But I have this almost desperate sense of wanting to understand what it means for this artefact to represent the remains of a human story. I know nothing about this story. I do not know the skull's age, gender or origin. But as I rest my hand on the crown, I will sometimes rest my other hand on my own head. In that moment I try to capture the feeling that I'm just a big bag of bones. That my two hands contain only superficially different contents. Somehow this calms me. I sleep easier when I can clear my mind of the day to day.

How Phi and Fibonacci are connected

2023-07-09T00:00:00Z

Definition: Fibonacci Sequence

The Fibonacci Sequence is a sequence starting, 0, 1, 1, 2, 3, 5, 8, 13..., and each subsequent number is the sum of the previous two.

We can define the Fibonacci numbers recursively as:

F_{0} = 0

F_{1} = 1

F_{n} = F_{n - 1} + F_{n - 2}

Definition: Phi

Phi is the "Golden Ratio". Two quantities $a$ and $b$ are in the golden ratio $φ$ if:

\frac{a + b}{a} = \frac{a}{b} = φ

Given this definition, we can derive the following additional relationships:

1 + \frac{1}{φ} = φ

φ^{2} = φ + 1

Connecting the two together

The values of the Fibonacci sequence increase in size very quickly. But how fast exactly?

Lets assume it can be approximated by some exponential curve of the form:

F_{n} = x^{n}

Where $x$ is some constant.

If so, then from the recursive definition of the Fibonacci numbers:

F_{n} = x^{n} = F_{n - 1} + F_{n-2} = x^{n - 1} + x^{n - 2}

x^{n} = x^{n - 1} + x^{n - 2}

And if we divide through by: $x^{n - 2}$

Then we get:

x^{2} = x + 1

This last relationship is exactly the relationship we identified earlier in the definition of φ. And so we can conclude: $x = φ$

But wait! We aren't done. This conclusion was predicated on a very shaky assumption "lets assume the Fibonacci sequence can be approximated by some exponential curve".

Despite this assumption we can conclude rigorously that if any consecutive pair of Fibonacci numbers were exactly in the Golden Ratio, then all subsequent pairs would also be.

So we've identified that the Golden Ratio is "stable" with respect to the recursive property of Fibonacci numbers.

But that's not useful if we never happen to hit the Golden Ratio in the course of the Fibonacci pairs.

Lets consider what happens when the current ratio is not $φ$ . Lets model it as having a ratio of $φ + δ$ , for some non-zero $δ$ .

Then the next ratio will be:

\frac{1 + φ + δ}{φ + δ} = \frac{1}{φ + δ} + 1

This last term is similar to $1 + \frac{1}{φ} = φ$ but with the added $δ$ term. As such if $δ > 0$ , then our first ratio was greater than $φ$ , but our second ratio will be less than $φ$ .

On the other hand if $δ < 0$ , then our first ratio was less than $φ$ , but our second ratio will be greater than $φ$ .

From this we can conclude that ratios of Fibonacci pairs will "oscillate" either side of $φ$ . But you may also note that because of the position of the $δ$ term in the denominator the absolute distance from $φ$ reduces from every ratio to the next.

We have therefore concluded two things:

Fibonacci pair ratios will "oscillate" either side of $φ$ .
Fibonacci pair ratios will converge on $φ$ .

Notice also that these two conclusions did not depend on the starting values chosen for the Fibonacci sequence. They follow solely from the recursive property of the Fibonacci numbers and the definition of $φ$ . As such these conclusions would still hold no matter what starting values are chosen.

Take care,

Rupert

Counting the Hard Way

2023-06-25T00:00:00Z

In this post we'll explore the weird world of alternative base number systems.

Let's start by counting to ten:

0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, ...

Congratulations.

But let's look at that sequence of symbols above. Isn't it weird that for most of the numbers I used just a single symbol, but for the last one I used two symbols (1 and 0)?

In base 10 we have 10 distinct symbols from 0-9, but we have to break out into multiple symbols for numeric values larger than 9. Let's hold that thought and move onto a different base number system.

Binary Base 2

Binary is a numeral system using just the digits 0 and 1. It is used extensively in computing and digital technology.

In binary, counting to ten looks like this:

0, 1, 10, 11, 100, 101, 110, 111, 1000, 1001, 1010, ...

General Rules of Positional Notation Systems

Now that we've seen two examples, lets consider the general properties of Positional Notation Systems.

When we read a number like 423. We interpret this as three separate values added together:

4 * 100
2 * 10
3 * 1

We could also write this as:

4 * 10^2
2 * 10^1
3 * 10^0

We do almost exactly the same in binary. Consider 1011:

1 * 8 = 1 * 2^3
0 * 4 = 0 * 2^2
1 * 2 = 1 * 2^1
1 * 1 = 1 * 2^0

In both cases the individual digit values are multiplied by some power of the base value.

With this understanding it should be possible to understand how any integer greater than 1 can be used as a base, and what counting in such a system would look like.

But now that we know the rules... what if we start to break them?

First up: what if the base was negative?

Negabinary Base -2

Suppose we have a base of -2. The powers of -2 are:

-2^5 = -32
-2^4 = 16
-2^3 = -8
-2^2 = 4
-2^1 = -2
-2^0 = 1

So how can we use this to count to ten?

The first two numbers are easy: 0, 1. But already the next number presents a problem. Using just the powers available to us above how can we represent two?

The trick here is to notice that we can use:

-2^2 = 4
-2^1 = -2

i.e. 110 is two in Negabinary.

We make three by simply adding one to two.

i.e. 111 is three in Negabinary.

Then the whole sequence to ten is:

0, 1, 110, 111, 100, 101, 11010, 11011, 11000, 11001, 11110, ...

But what if we now also decide to count down?

Counting negatively:

0, 11, 10, 1101, 1100, 1111, 1110, 1001, 1000, 1011, 1010, ...

Notice an interesting property here:

All the positive values have an odd number of digits
All the negative values have an even number of digits

At no point did we need a minus sign to represent negative values!

We can intuitively understand this interesting property by recognising that the minus symbol has become redundant since it is "prebaked" into the base itself. Our "menu" of powers of -2 offers us both negative and positive values to choose from.

But we aren't done yet! What if our base was a complex number?

Quarter-Imaginary Base 2i

The Quarter-Imaginary numeral system was first proposed by Donald Knuth in 1960. The trick with this base is recognising that powers of 2i "rotate" through the Gaussian plane.

Powers:

-1: -(1/2)i
0: 1
1: 2i
2: -4
3: -8i
4: 16
5: 32i
6: -64
7: -128i
8: 256
9: 512i

With this base we need four symbols: 0, 1, 2 and 3 to represent all values.

Positive real integers:

0, 1, 2, 3, 10300, 10301, 10302, 10303, 10200, 10201, 10202, ...

Negative real integers:

0, 103, 102, 101, 100, 203, 202, 201, 200, 303, 302, ...

Positive imaginaries:

0, 10.2, 10, 20.2, 20, 30.2, 30, 103000.2, 103000, 103010.2, 103010, ...

Negative imaginaries:

0, 0.2, 1030, 1030.2, 1020, 1020.2, 1010, 1010.2, 1000, 1000.2, 2030, ...

Some other examples:

1+i = 11.2
-1+i = 113.2
-1-i = 103.2
1-i = 1.2

In this way we can represent all complex numbers without needing a minus sign, nor a sign for i.

As before you'll notice that the modulo of the length of a number tells us something about where on the Gaussian plane our value is. Another interesting feature is the use of 0.2 to give us -i.

That's all for today.

Take care,

Rupert

Loose Equality violates Transitivity

2023-03-09T00:00:00Z

In JavaScript:

"0" == 0; // true
0 == []; // true
"0" == []; // false

These three statements demonstrate that loose equality in JavaScript is not transitive and therefore is not a valid equivalence relation.

Despite this, the above is not a bug. It conforms exactly to ECMAScript: IsLooselyEqual.

Equivalence Relations

An Equivalence Relation is the mathematically formal term for what we intuitively understand as "equals". In order for a relation to be a valid Equivalence Relation, it must satisfy the following three properties:

Reflexivity: a == a.
Symmetry: "If a == b then b == a".
Transitivity: "If a == b and b == c, then a == c.

By the way, NaN == NaN; // false violates reflexivity. But there are reasons for that decision, which I will not get into in this post. If you are especially interested, I recommend: Wikipedia | Comparison with NaN

The Easy Fix

The easiest way to alter loose equality (==) to fix transitivity would be to treat all cross-type comparisons as false. This is exactly what the strict equality (===) operator in JavaScript does. Reference: ECMAScript: IsStrictlyEqual

But that's boring. Let's entertain the question: are there ways we can preserve at least cross-type equalities, while also preserving transitivity?

The Pitfalls

As soon as we allow even one equality across types we create a cross-type equivalence class.

An Equivalence Class is the set of all elements which are equivalent to one another.

Once you have a cross-type equivalence class, you have to be very careful when considering new members of that class. It is not sufficient to check our intuition against just one member of the equivalence class, since equivalence with any member, implies equivalence to all members (via transitivity).

The above implies two statements:

Distinct values of the same type must never be members of the same equivalence class.
A maximal equivalence class has exactly one value from each type.

A Silly Proposal

Given the boolean type has only two values, there can exist only two distinct equivalence classes including booleans. We can use this as the basis for a set of maximal equivalence classes.

The false equivalence class:

String: ''
Number: 0
BigInt: BigInt(0)
Boolean: false
Undefined: undefined
Symbol: Symbol.for('')
Null: null

The true equivalence class:

String: 'true'
Number: 1
BigInt: BigInt(1)
Boolean: true
Symbol: Symbol.for('true')

The false class includes seven values, exactly one value from each primitive type.

The true class has five values. The only missing types are undefined and null, which cannot be included without merging the two equivalence classes. I really don't like the use of 'true' as the canonically true string, but this is what we get for pushing an idea to its limit.

In reality prohibiting cross-type equality is a perfectly sensible decision and very easy to reason about. It's no wonder that === is greatly preferred to ==.

Take care,

Rupert

Langton's Ant and Unanswerable Questions

2022-11-21T00:00:00Z

Langton's Ant is a cellular automaton first described by Christopher Gale Langton in 1986. It looks like this:

The "Ant" exists on an infinite grid. Each cell of the grid can be either white or black, but initially all are white. Each "generation" the Ant moves according to the following rules:

Turn left if the current cell is white, else turn right
Change the colour of the current cell
Step forwards

If we allow the ant to run for some time, it initially produces some very beautiful symmetries. For example:

Generation 96 displays an "S"-like pattern:

Generation 184 looks like a pokeball about to open:

Generation 368 is like a four tentacled monster:

But then it produces chaotic patterns for many thousands of steps.

We might have guessed that this chaos would proceed forever. But then at around 10,000 steps, something magical emerges:

The Ant has begun to build a structure known as the highway. The highway is an infinitely repeating pattern. It takes the ant 104 steps for each iteration of the highway, and each iteration extends the pattern along a diagonal trajectory.

But we can change things. Instead of starting with a blank grid, we could set some cells to black at the beginning. If we then run the Ant it will generate different patterns. When people try this, they find that the highway always seems to emerge sooner or later. It is widely believed that the highway is inevitable, and yet this conjecture has never been proven nor disproven, despite many years of effort.

Such a simple system to define, and yet we are unable to answer the question "Does the highway always appear, no matter the starting state?"

Rice's Theorem

Henry Gordon Rice proved the following theorem in 1951:

All non-trivial semantic properties of programs are undecidable.

A trivial property of a program is one which is either always true or always false for every possible program.

Semantic properties are about the behaviour of the program, rather than syntactic properties which are about the source code.

A property is undecidable if it is not possible for there to exist an algorithm which correctly identifies the property for all possible inputs.

A famous example of such a non-trivial semantic property is: "Does this program halt?". We know from the Halting Problem that there can never exist a systematic way to identify whether any program halts or not.

Once again, we find that such a simple question cannot be answered.

Rationalism and Empiricism

Rationalism and Empiricism are two different philosophical stances on how best to pursue knowledge.

A Rationalist believes that reason is the primary way to gain knowledge. Truth is found by reasoning from first principles.
An Empiricist believes that experience is the primary way to gain knowledge. Truth is found from direct observation and experimentation.

Neither view is always correct. Different fields of knowledge lend themselves more to rationalism while others to empiricism. Mathematics for example is exceedingly rational, whereas Biology is exceedingly empirical.

Computer Science is founded upon Mathematics, with the aim to reason about computation and in so doing to derive concrete proofs. And yet examples like Langton's Ant and Rice's Theorem demonstrate that even very simple problems can elude solution.

There is a tension between Computer Science as an academic pursuit vs Software Engineering as a professional discipline. At heart, I am a rationalist. I crave proof and certainty. But I am employed as a professional. When trying to find truth in any matter, think critically about what is provable from reason and what is not. For the unprovables, be humble and open minded. Observe, experiment and adapt to your circumstances.

Take care,

Rupert

Default Options Pattern in TypeScript

2022-10-24T00:00:00Z

Here's a pattern I find myself reusing in a lot of projects:

type FormatOptions = {
	color: string;
	fontWeight: string;
};

const format = (
	message: string,
	{ color = "black", fontWeight = "medium" }: Partial<FormatOptions> = {},
) => {
	// Implementation not important
	// for this blog post
};

Explanation

Imagine you have a function which can display some text formatted in a particular colour:

const format = (message: string, color: string) => {
	//
};

// Format the text red
format("My message", "red");

But most of the time you don't want to have to specify a color, so it would be nice if the color argument were optional:

const format = (message: string, color = "black") => {
	//
};

// A red message
format("My red message", "red");
// A message using default color
format("My black message");

But now suppose I also want to support a font-weight option. We could provide two optional arguments, one after the other:

const format = (message: string, color = "black", fontWeight = "medium") => {
	//
};

format("My medium black message"); // This works nicely!

// 🚨 I wanted bold fontWeight...
// 🚨 but this is read as a color
format("O no this won't work", "bold");

The problem is that we want a way to make color and fontWeight both optional, but if they are just optional parameters, then order matters. So we have to pick one to go before the other, and then we lose the ability to specify a later optional argument without also specifying an earlier one.

Instead we should wrap them up in an options object:

const format = (
	message: string,
	{
		color = "black",
		fontWeight = "medium",
	}: {
		color?: string;
		fontWeight?: string;
	},
) => {
	//
};

// This works!
format("default formatted message", {});
// Also works!
format("A red message", {
	color: "red",
});
// This works too!
format("A bold message", {
	fontWeight: "bold",
});

This is a perfectly good point to stop... however I don't entirely love having to pass an empty object in the case where we want all the defaults. What if instead the whole options object were itself optional?

const format = (
	message: string,
	{
		color = "black",
		fontWeight = "medium",
	}: {
		color?: string;
		fontWeight?: string;
	} = {},
) => {
	//
};

// Now this is fine too!
format("A plain default message");
format("Formatting still works too", {
	color: "blue",
	fontWeight: "light",
});

Perfect! We get support for sensible defaults, while also allowing users to provide specific options only for the bits they care about.

The only thing I might do in this position is pull out the type of the options object for clarity:

type FormatOptions = {
	color: string;
	fontWeight: string;
};

const format = (
	message: string,
	{ color = "black", fontWeight = "medium" }: Partial<FormatOptions> = {},
) => {
	//
};

Which brings us to where we started 😀

That's all for today

Take care,

Rupert

Accessing Unknown Objects in TypeScript

2022-09-11T00:00:00Z

There's a utility function I find myself adding to almost every TypeScript repo I work on.

It goes like this:

const isObject = (
  unknown: unknown
): unknown is Record<PropertyKey, unknown> =>
  typeof unknown === 'object' && unknown !== null

Usecase in Error Handling

Occasionally we have to deal with unknown values. The most common example of this is in catch blocks. Often we want to assume that the caught error is some kind of native Error object, which would have a message property.

try {
  somethingDangerous()
} catch (error) {
  // `error` is of `unknown`
  console.log(error.message)
  // We get a type error on `error.message`
  // because `Object is of type 'unknown'`
}

Since TypeScript 4.4 caught errors are unknown by default.

But by using our isObject utility, we can narrow the error to something which has keys or not:

try {
  somethingDangerous()
} catch (error) {
  console.log(
    isObject(error)
    ? error.message
    // 👆 safe access within this branch
    : error
    // In this case `error` is something different
    // So we'll just log it directly as is.
  )
}

Usecase in Composing Type Predicates

Learning to create your own type predicates is an important skill in TypeScript. There's a lot you can do by hand, but I will very often use isObject as a starting point for more involved type predicates.

Here's a very simple example:

type UserApiResponse = {
  userName: string
}

const isUserApiResponse = (
  userResponse: unknown
): userResponse is UserApiResponse =>
  isObject(userResponse)
  && typeof userResponse.userName === 'string'

If we need to go any deeper than one level, this approach becomes very cumbersome; at which point I recommend reaching for a schema validation library such as Zod.

Here's how the same code above would look with Zod:

import { z } from 'zod'

const UserApiResponse = z.object({
  userName: z.string()
})

type UserApiResponse = z.infer<typeof UserApiResponse>

const isUserApiResponse = (
  userResponse: unknown
): UserResponse is UserApiResponse => 
  UserApiResponse.safeParse(userResponse).success

That's all for today.

Take care,

Rupert

Simulating Dice

2022-08-13T00:00:00Z

In this post I will demonstrate two approaches for simulating rolls of a die of a given size, using only rolls of a die of a different size.

Throughout this post I will use dn terminology to refer to a die of size n. For example a d6 refers to the classic six-sided die.

Using base number conversion

Given an input die of size n and a target die of size m, this approach works by expanding the output space of the dn by rolling it repeated times, and interpreting the rolls in base n mod m. Some rolls however must be discarded to prevent bias in the outcomes.

To present this approach, I will start with some small examples before building up to a fully generalized algorithm.

Simulating a d6 using repeated coin flips

Given a coin, we can flip it as many times as necessary for the number of outcomes to exceed the target number.

For a d6, we would need to flip the coin at least 3 times to produce more than 6 outcomes. If we only flipped 2 times, we could only output 4 outcomes. At 3 flips we can output 8 outcomes. Given that we have 8 outcomes, we need to discard 2.

Here is one scheme for doing that:


HHH	1
HHT	2
HTH	3
HTT	4
THH	5
THT	6
TTH	DISCARD
TTT	DISCARD

Note that the ordering here is arbitrary, but I have used binary ordering, treating H as 0 and T as 1. Although arbitrary, this ordering will prove convenient for generalization to other die sizes.

Additionally notice that in this case both of the discarded results start with TT, so we can restart after only the second flip if we have seen TT so far.

We can combine the above two notes to produce a method of arriving at results without needing a lookup table:

Flip a coin twice
If the flips were TT, discard and restart
Otherwise flip a third time
Treating H as 0 and T as 1, read the flip outcomes as a binary number and add 1.

Simulating a d6 with a d4

The above method of using a coin is equivalent to repeated use of a d2. A coin is a d2. We can extend this method to any other die size.

With a d4, we need two rolls to produce a large enough outcome space to simulate at least 6 results. The outcome space is much larger than we need though, having 16 outcomes.

But we can map a given target result to multiple rows in our table, like so:

First roll	Second Roll	Result
1	1	1
1	2	2
1	3	3
1	4	4
2	1	5
2	2	6
2	3	1
2	4	2
3	1	3
3	2	4
3	3	5
3	4	6
4	1	DISCARD
4	2	DISCARD
4	3	DISCARD
4	4	DISCARD

Note as before that we have an opportunity to terminate early. In this case when the first roll is a 4.

In order to produce a result without a lookup table we can use the same approach as before, but instead of interpreting d2 rolls in binary, we instead interpret d4 rolls in base 4, but also needing to take care to adjust for out by one errors caused by our base number system starting at 0, but the numbers on our dice conventionally starting at 1.

Precisely, one must follow these steps:

Roll a d4.
If the outcome was 4, discard and restart.
Roll the d4 a second time.
Subtract one from both outcomes.
Concatenate the two results and read as a base-4 number.
The computed value at this point will be between 0 and 11.
Add 1 to this value and then apply mod 6.

We can now extract from this a pattern to this process, in which any die can simulate any other die.

Given some pair of naturals n and m, We can simulate a dm using a dn as follows:

Find the smallest natural k such that n^k >= m.
Roll the dn repeatedly, k times.
Read the rolls as digits of a base n number.
Discard values greater than the largest multiple of n that is less than or equal to m.
Mod the result by m and add 1.

See below for an implementation of this process in TypeScript.

[Click to expand] Implementation

/**
 * Given `n`, simulate rolling a `dn` once.
 *
 * The possible outcomes are the set of naturals from 1 to n inclusive.
 */
const d = (n: number) => Math.floor(n * Math.random()) + 1;

const produceOneResult = (n: number, repeatCount: number) => {
	const rollResults = new Array(repeatCount).fill(0).map(() => d(n));

	// This reads the results as a single number in base `n`
	const result = rollResults
		.map((roll, i) => (roll - 1) * n ** (repeatCount - i - 1))
		.reduce((a, b) => a + b, 0);

	return result;
};

/**
 * Given a `dn`, simulate the outcomes of a `dm`
 */
const simulateMwithN = (m: number, n: number) => {
	/**
	 * `repeatCount` is the number of times we will need to roll the smaller die.
	 */
	const repeatCount = Math.ceil(Math.log(m) / Math.log(n));
	/**
	 * The maximum valid result is the largest multiple of n that is
	 * less than or equal to the size of our target space
	 */
	const maxValidResult = Math.floor(n ** repeatCount / m) * m - 1;

	let result: number;
	do {
		result = produceOneResult(n, repeatCount);
	} while (result > maxValidResult);

	return (result % m) + 1;
};

simulateMwithN(6, 4);

Try in the TypeScript Playground

Using recursive unit interval segmentation

Given an input die of size n and a target die of size m, this approach works by mapping to and from the unit interval. First segment the unit interval by m. Then treat rolls of the dn as "narrowing in" on a range of the unit line segment. We start from a range inclusive of the whole unit line segment: 0 to 1. Then for example we can treat a roll of a 1 on a d4 as narrowing from the range 0-1 to 0-1/4. If we rolled at again and it came up 2 we would then narrow to the second quarter of the current range, i.e. 0-1/4 to 1/16-2/16. At each stage we can check whether the current range fits entirely within a single 'segment' of the output die. If so we output the value on the target die associated with the segment. Otherwise we continue narrowing.

Let's show a complete example to demonstrate.

Given an input d4 and target d20.

The target segments are 20 equal sized segments on the unit interval.

The current input range is the full 0-1.

The first roll of the d4 yields a 3. The current input range is narrowed to 2/4-3/4. The current viable output values are 11-15. We need to narrow further.

The second roll of the d4 yields a 1. The current input range is narrowed to 8/16-9/16. The current viable output values are 11-12. We need to narrow further.

The third roll of the d4 yields a 4. The current input range is narrowed to 35/64-36/64. The current viable output values are 12. We halt with the output 12.

See below for an implementation of this process in TypeScript:

[Click to expand] Implementation

/**
 * Represents a range on the unit interval
 */
type Interval = [number, number];

/**
 * Given `n`, simulate rolling a `dn` once.
 *
 * The possible outcomes are the set of naturals from 1 to n inclusive.
 */
const d = (n: number) => Math.floor(n * Math.random()) + 1;

/**
 * Let the unit interval be segmented into `m` segments of equal size,
 * label the segments from 1 to `m`.
 *
 * Then given some `n` on the unit interval, output which segment of `m` it belongs to.
 */
const fromUnitNtoSegmentsM = (m: number) => (n: number) => {
	if (n === 0) return 1;
	return Math.ceil(n * m);
};

/**
 * Given a die, map a single result to the unit interval
 */
const fromDNtoInterval = (n: number) => {
	const dieResult = d(n);

	const lowerBound = (dieResult - 1) * (1 / n);
	const upperBound = lowerBound + 1 / n;

	return [lowerBound, upperBound];
};

/**
 * Given an interval and a die size,
 * narrow the interval using a single role of the die
 */
const narrowIntervalWithDN =
	(n: number) =>
	(interval: Interval): Interval => {
		// This generates a new unit interval
		const [innerLower, innerUpper] = fromDNtoInterval(n);

		// But we need to map it into our existing interval
		const currentIntervalSize = interval[1] - interval[0];
		const newLowerBound = interval[0] + innerLower * currentIntervalSize;
		const newUpperBound = interval[0] + innerUpper * currentIntervalSize;

		return [newLowerBound, newUpperBound];
	};

/**
 * Given a `dn`, simulate the outcomes of a `dm`
 */
const simulateMwithN = (m: number, n: number) => {
	// We'll use a tuple to track a range on the unit interval
	let interval: Interval = [0, 1];

	const mapToDM = fromUnitNtoSegmentsM(m);
	const mapFromDN = narrowIntervalWithDN(n);

	// In this loop we repeatedly narrow our output interval
	// Until its lower and upper bounds fit within the same segment of `dm`
	while (mapToDM(interval[0]) !== mapToDM(interval[1])) {
		interval = mapFromDN(interval);
	}

	return mapToDM(interval[0]);
};

/**
 * A test of the distribution of a simulated dice
 *
 * Outputs a record where each key is a die result,
 * and each value is how often that result occurred
 */
const test = new Array(10000)
	.fill(0)
	.map(() => simulateMwithN(6, 4))
	.reduce<Record<number, number>>(
		(acc, curr) => ({ ...acc, [curr]: (acc[curr] ?? 0) + 1 }),
		{},
	);

/**
 * If the test is successful, each value should be roughly equal
 */
console.log(test);

Try in the TypeScript Playground

Comparisons

Both approaches could theoretically never halt. In the first approach, we might always hit a DISCARD result. In the second approach we might always narrow in on a segment that straddles the boundary between two possible output values. However in practice the probability of this happening for either approach tends to zero for sufficiently many rolls of the input die.

One might also consider the practicality of either approach. In the presence of a computer there are much more convenient methods for simulating a die. However in the absence, the first approach can be somewhat pre-computed by drawing up lookup tables for a given input die (such as the ubiquitous d6) and a selection of output die.

I can't find a comparative advantage to the second approach compared to the first... and yet I can't help but shake some gut feeling of preference for it. Something about "zooming in" on the unit segment appeals to me.

That's all for this post.

Take care,

Rupert

Rupert Foggo McKay

The Hidden Complexity of Barcodes

# What is a barcode encoding?

# A brief history

# UPC-A and EAN-13

# GS1

# Prefix allocation

# The structure within a prefix

# The check digit

# Collisions

# Shortcomings

# Digital Link

Death

The Skull

How Phi and Fibonacci are connected

# Definition: Fibonacci Sequence

# Definition: Phi

# Connecting the two together

Counting the Hard Way

# Binary Base 2

# General Rules of Positional Notation Systems

# Negabinary Base -2

# Quarter-Imaginary Base 2i

Loose Equality violates Transitivity

# Equivalence Relations

# The Easy Fix

# The Pitfalls

# A Silly Proposal

Langton's Ant and Unanswerable Questions

# Rice's Theorem

# Rationalism and Empiricism

Default Options Pattern in TypeScript

# Explanation

Accessing Unknown Objects in TypeScript

# Usecase in Error Handling

# Usecase in Composing Type Predicates

Simulating Dice

# Using base number conversion

# Simulating a d6 using repeated coin flips

# Simulating a d6 with a d4

# Using recursive unit interval segmentation

# Comparisons

What is a barcode encoding?

A brief history

UPC-A and EAN-13

GS1

Prefix allocation

The structure within a prefix

The check digit

Collisions

Shortcomings

Digital Link

Definition: Fibonacci Sequence

Definition: Phi

Connecting the two together

Binary Base 2

General Rules of Positional Notation Systems

Negabinary Base -2

Quarter-Imaginary Base 2i

Equivalence Relations

The Easy Fix

The Pitfalls

A Silly Proposal

Rice's Theorem

Rationalism and Empiricism

Explanation

Usecase in Error Handling

Usecase in Composing Type Predicates

Using base number conversion

Simulating a d6 using repeated coin flips

Simulating a d6 with a d4

Using recursive unit interval segmentation

Comparisons