Is Matrix Multiplication Ugly?

4 Replies

A few weeks ago I was minding my own business, peacefully reading a well-written and informative article about artificial intelligence, when I was ambushed by a passage in the article that aroused my pique. That’s one of the pitfalls of knowing too much about a topic a journalist is discussing; journalists often make mistakes that most readers wouldn’t notice but that raise the hackles or at least the blood pressure of those in the know.

The article in question appeared in The New Yorker. The author, Stephen Witt, was writing about the way that your typical Large Language Model, starting from a blank slate, or rather a slate full of random scribbles, is able to learn about the world, or rather the virtual world called the internet. Throughout the training process, billions of numbers called weights get repeatedly updated so as to steadily improve the model’s performance. Picture a tiny chip with electrons racing around in etched channels, and slowly zoom out: there are many such chips in each server node and many such nodes in each rack, with racks organized in rows, many rows per hall, many halls per building, many buildings per campus. It’s a sort of computer-age version of Borges’ Library of Babel. And the weight-update process that all these countless circuits are carrying out depends heavily on an operation known as matrix multiplication.

Witt explained this clearly and accurately, right up to the point where his essay took a very odd turn.

Continue reading →

Picturing Mathematics

3 Replies

I’m a great believer in low-tech math. I don’t like to rely on things a computer tells me; what if there’s a bug in the code? I prefer trusting things that I can check for myself. At the same time, I’m keenly aware of the limits of my imagination even when it’s aided by paper and pencil. Sometimes I need a computer to show me things I can conceive of but can’t see.

Continue reading →

Randomness Made to Order, part 1

1 Reply

As a member of the Advisory Council for the National Museum of Mathematics (“MoMath”) over the past decade, I’ve had a number of unique opportunities, such as the thrilling chance to improve the Museum’s datebase via my smartphone and watch exhibit-content update in real-time, and the less thrilling opportunity to break an exhibit on the museum’s opening day (buy me a coffee and I’ll confess to you that shameful episode from my past). But the opportunity I’m writing about today is one that’s still playing out: the chance to play a role in creating a new kind of Math Thing, namely, a programmable quincunx.

If you look up the word “quincunx”, you’ll find that one definition is this arrangement of five dots:

Image
A quincunx as seen on a standard die.

But I’m talking about the kind of quincunx that looks like this:

Image
A quincunx as seen in some science museums. From p. 261 of “Lady Luck” by Warren Weaver (permission pending).

You’ll notice that the balls piled up at the bottom form a bell-shaped curve, reminiscent of the normal curve from statistics:

Image

This isn’t a coincidence; the quincunx was designed to illustrate statistical principles in general and the Gaussian distribution in particular. Many science museums have a quincunx, but MoMath was unique in having an adjustable quincunx in which a lever allowed users the chance to introduce biases at the junctions, making it more likely for balls to go to the left or to the right, and correspondingly shifting the bell-shaped curve to the left or to the right.

I wrote “MoMath was unique …”, not “MoMath is unique …”, because when MoMath’s lease at its old location on 26th Street ran out, it moved to a temporary smaller location on Fifth Avenue, and the adjustable quincunx wasn’t included in the downsized museum. But in 2026 a bigger-than-ever MoMath on Sixth Avenue will feature something new under the sun: a customizable quincunx, in which each junction will have its own individual bias, and in which the distribution of the balls at the bottom won’t necessarily be a Gaussian at all, but a curve of your own devising.

Continue reading →

When .999… Isn’t 1

3 Replies


In ordinary math, the infinite decimal .999… is defined to be the limit of the terminating decimals .9, .99, .999, …; that is, it’s defined to be the real number that the fractions 9/10, 99/100, 999/1000, … approach in the ordinary sense. And that limit is most definitely 1, not some real number that’s a tiny bit less than 1. This is not an approximate truth; it’s a 100% accurate, rigorously established mathematical fact. It’s a part of how the real number system works, and it’s a feature, not a bug.

But if you’ve read my essay “Marvelous Arithmetics of Distance”, you already know that there are number systems in which things that look like rational numbers behave differently1, and you won’t be too surprised to learn that there’s another new number-game to play. It’s the q-deformed real number “game” of Sophie Morier-Genoud and Valentin Ovsienko, and in the context of their work, it becomes true (in an admittedly somewhat arcane sense) that 9/10, 99/100, 999/1000, etc. do not approach 1 but rather approach something smaller. Except that it’s not the numbers themselves that behave in this ill-bred way; it’s their avatars in the q-deformed world, avatars that Morier-Genoud and Ovsienko write as [9/10]q , [99/100]q , [999/1000]q , etc.

In this essay, without going too deeply into the underlying theory, we’ll take a concrete look at the q-deformations of the numbers 1/2, 2/3, 3/4, etc. and of the numbers 2/1, 3/2, 4/3, etc. In the ordinary real number system, all these fractions approach 1 as the numerators and denominators get big, regardless of whether the bigger number is on top of the fraction or on the bottom. But looking at these rational numbers through the spectacles that Morier-Genoud and Ovsienko have given us, we’ll see that we get two different limits according to whether the numerator is bigger than the denominator or vice versa. In particular, we’ll find that, while the q-deformations [2/1]q, [3/2]q, [4/3]q, etc. approach [1]q, the q-deformations [1/2]q, [2/3]q, [3/4]q, etc. approach [1]q’s evil (or maybe not so evil) twin.

Continue reading →

Remembering Kelly

2 Replies

This past week I was saddened to learn of the death of mathematician and teacher David C. Kelly, the founder of the Hampshire College Summer Studies in Mathematics program (HCSSiM). “Kelly”, as everyone called him, had a huge impact not just on my career but on the careers of people spanning several generations.

I knew Kelly for nearly fifty years. At the time we met I was a high school student who’d done well enough in inter-school math competitions to earn a spot on the Nassau County team, and when I and my team-mates went to the Atlantic Regional Math League competition, Kelly was there, spreading the word about HCSSiM. I thought he looked remarkably like Kurt Vonnegut, though not everyone agreed. Judge for yourself:

Image
Not Kurt.
Image
Not Kelly.

The “17” in the background in the former picture is important, as you already know if you read my essay “Will ’17 Be the Year of the Pig?” And if you haven’t read that essay, and you’ve wondered why I post my blog on or around the 17th of each month … well, read that essay.

I don’t have anything to write about the summer program that I didn’t already write back in 2017, but I do want to share my two favorite Kelly stories. I’m sure I’ll get some details wrong, and alas, Kelly isn’t around to set me straight, but I think he would agree that my versions are true in spirit. (And if any of you have corrections, please post them in the Comments!)

Continue reading →

Math and the Museum

4 Replies

“I couldn’t help but wonder…” — Carrie Bradshaw (in every episode of Sex and the City)

The best birthday party I ever had as a kid was a trip to the Museum of Natural History in New York City with half a dozen like-minded friends and my indulgent parents. The huge dinosaur skeleton in the main hall was impressive, but I was even more enchanted by the exhibits at the Hayden Planetarium. How intriguing it was to see a ball roll round and round the inside of a curvy funnel, evading its fate for what seemed like an eternity before finally falling into the hole in the middle, and how fun to wonder how the rules governing our universe not only allowed but mandated this behavior! How intriguing it was to see how much I would weigh on different planets, and how fun to wonder what that would feel like!

But as much as I enjoyed the planetarium, my childhood love of science was already secondary to my passion for math – the skeleton of the universe, you might call it. I probably would’ve enjoyed a trip to a math museum even more than a trip to a natural history museum. The trouble is, New York City didn’t have one.

That’s not true anymore. New York City now boasts one of the best mathematics museums in the world, the National Museum of Mathematics, informally called MoMath. With 19,000 square feet containing over three dozen exhibits, MoMath became a major attraction to NYC-area schoolchildren and tourists from all over the world when it opened in 2012. Sometime in 2026 it’ll be moving to a new location at 635 Sixth Avenue, where it’ll occupy either 36,000 or 46,000 square feet.

The square footage depends partly on you, as I’ll explain.

Continue reading →

Is 1 Prime, and Does It Matter?

16 Replies

If you ask a person on the street whether 1 is a prime number, they’ll probably pause, try to remember what they were taught, and say “no” (or “yes” or “I don’t remember”). Or maybe they’ll cross the street in a hurry. On the other hand, if you ask a mathematician, there’s a good chance they’ll say “That’s an excellent question” or “It’s kind of an interesting story…”

Some people treat the non-primeness of 1 as a mathematical fact and nothing more, but those people are missing out on something important about the nature of mathematics.

Continue reading →

How Pi Almost Wasn’t

5 Replies

Since you’re reading this essay, you probably already know about the mathematical holiday called Pi Day held on March 14th of each year in honor of the mystical quantity π = 3.14…. Pi isn’t just a universal constant; it’s trans-universal in the sense that, even in an alternate universe with a different geometry than ours, conscious beings who wondered1 about the integral of sqrt(1–x2) from x = –1 to x = 1 would still get — well, not 3.14…, but exactly half of it, or 1.57…. Therein lies a catch in the universality of pi: why should 3.14… be deemed more fundamental than 1.57… or other naturally-occurring2 pi-related quantities?

I suspect that, even if we limit attention to planets in our own universe harboring intelligent beings that divide their years into something like months and their months into something like days, many of those worlds won’t celebrate the number pi on the fourteenth day of the third month. It’s not just because 3.14 is a very decimal-centric approximation to pi (is there a reason to think that intelligent beings tend to have exactly ten fingers or tentacles or pseudopods or whatever?). Nor is it just because interpreting the “3” as a month-count and the “14” as a day-count is a bit sweaty. And it’s not just because holding a holiday to celebrate a number is a weird thing to do in the first place. It’s also because, in our own world, we came close to having a different multiple of pi serve as our fundamental bridge between measuring straight things and measuring round things.

Continue reading →

Dedekind’s Subtle Knife

2 Replies


“Think about the knife tip. That is where you are. Now feel with it, very gently. You’re looking for a gap so small you could never see it with your eyes, but the knife tip will find it, if you put your mind there. Feel along the air till you sense the smallest little gap in the world…”

– Philip Pullman, “The Subtle Knife”


The modern era of mathematics began on November 24, 1858, when mathematician Richard Dedekind proposed the first firm foundation for the real number system and, even more importantly, established a new way to think about – or maybe I should say a new way to avoid thinking about – the ultimate nature of mathematical reality.

You may remember from last month’s essay that Dedekind had come up with an axiom called the Completeness Axiom which, in combination with the rules of algebra, will let you prove all the important properties of the real number system that calculus needs. This is the postulational approach: list the properties you want numbers (or points or lines or whatever) to have, postulate that those properties hold, and see what consequences follow. But there’s a problem with this method: how do we know that the postulates are true? As Bertrand Russell wrote, “The method of ‘postulating’ what we want has many advantages; they are the same as the advantages of theft over honest toil.”

For instance, consider the assertion that twice the cube root of 2 equals the cube root of 16. If we postulate the existence of a number x that satisfies x3 = 2 while also obeying the ordinary rules of algebra, it’s easy to show that (2x)3 = (23)(x3) = (8)(2) = 16. But how do we know that such a number x exists to begin with? It seems a bit like circular reasoning.

Dedekind found a way out of the circle, relying on the gimmick that you can sometimes build a new number system by hitching a ride on an old one.

Continue reading →

The Real Line versus the Fakes

2 Replies


There’s a pretty thought experiment that’s sometimes attributed to Democritus though it’s actually due to a later popularizer of the atomic hypothesis1 and it goes like this: Suppose we use the world’s sharpest knife to cut a block of cheese in half, leaving two small blocks where before there was one large one. If cheese is made of atoms, some of the atoms end up in one half and the rest end up in the other. But if cheese is a continuous substance, and the block of cheese is analogous to a line segment in Euclidean geometry, then what happens to the points that are exactly lined up with the knife’s edge? Do they get duplicated? Do they vanish? Does the knife somehow slip to one side or the other of those knife’s-edge points? None of these options seems satisfying, but if we’re cutting something that’s truly continuous, what other options do we have?

In some ways the slipping-knife option, in which the symmetry of the block of cheese gets broken, seems the least satisfying to me. It suggests that you can never truly cut something into two equal pieces, not because of the imprecision of human instruments but because of something inherently strange about space – a strangeness that affects even the bare-bones one-dimensional kind of space that Euclid didn’t find interesting enough to say much about. Yet curiously, the third option in the parable, in which the knife must slip either to the left or to the right, received a kind of vindication a century and a half ago when German mathematician Richard Dedekind set out to think deeply about what real numbers are.

Continue reading →