These days, web projects involve many of the same steps: installing node_modules, running linters and tests, starting dev servers, and compiling files with esbuild or Rollup.

The default choice for automating these steps is often npm/yarn scripts: little shell commands written into your project’s package.json file. More complex projects sometimes evolve into using tools like Gulp/Grunt, or even full-blown Docker builds.

But I find make often fills many of the same needs without as much fuss.

Its age and simplicity means it gets a lot of things right:

• It’s already available everywhere. Most systems install make when you first set up developer tools. There’s rarely extra steps to get it working.
• It’s fast. People routinely point out that npm/yarn scripts are shockingly slow to start. make routinely runs commands ~30x faster, which is nice, because fast software is good software.
• It’s language-agnostic. Since make is already installed and works off shell scripts, it doesn’t require commitment to one language or toolchain. You can use it for Go, PHP, Rust, or Node projects equally. And you can shell out to language-specific tools when needed.
• It’s simple, with room to grow. Makefiles have a reputation for getting out-of-hand on large projects. But for smaller projects, the simple file structure, and dependency tracking, and support for multiple commands hits a nice sweet spot.

A consistent interface to project scripts

Thanks to these qualities, make shines as a way to create a consistent interface to commands across projects.

Most projects I work on require some kind of build process. To get to work, I need to re-learn the project’s tools, and the multiple commands needed to download dependencies, build assets, and start a dev server.

Building a site with Kirby¹ I need a mix of php, tailwindcss, esbuild, and rsync commands. For a Next.js site, I’m using their built-in CLI. And with Fresh, I don’t even have a package.json file — I’m using deno run and deno fmt. It’s annoying to track these commands down again and refamiliarize myself with where they live: in a JSON file? In a set of script files? In the README?

A Makefile acts as a conventional home for these tools and commands. It’s a single file at the root of the repository, that works with any language, and is easy read/add new commands to.

I leverage this conventionality to create consistent commands between different projects. For example, in every new project, I add a make dev command, which auto-downloads dependencies, starts a dev server, and watches for changes, no matter the language.

For reference, here’s a list of common commands I’ll add:

• make dev starts a development server with live reloading
• make build builds a production-ready binary or set of files
• make deploy tags a release for CI to build, or rsyncs files to a server
• make format formats all code to a standard style, using prettier or gofmt
• make lint runs code quality checks, like eslint or golanglint-ci
• make test runs a full set of test suites. Sometimes I’ll include lint scripts in here so there’s a single command to run in CI
• make clean removes all build artifacts and downloaded dependencies
• make help lists all the commands in a Makefile (discussed here)

This “one command” and “one interface” is highly beneficial for teammates too. They can get started quickly, and if they want to learn more or tinker, the commands and dependency relationships are all documented in a single place.

Recipes

Other articles cover the basics of how to write a Makefile in great depth,² so I’ll leave syntax aside and instead focus on sharing some common techniques that I use:

• Tasks without dependencies
• Referencing node_modules
• Auto-installing node_modules
• Skipping re-runs of slow tasks
• Using environment variables
• Configuring tasks with default variables
• Self-documenting Makefiles
• Parallel dev servers
• Hermetic environments

Tasks without dependencies

By default, make rule names refer to actual files to be built. If you write a rule like this:

dev:
	@echo "Hello, world!"

And then later create a file named dev, make doesn’t run the commands.

$ touch dev
$ make dev
make: 'dev' is up to date.

For one-off scripts that you just want to run when you tell them to run, this default is a little annoying.

In these cases, add .PHONY: <name> to the end of the target definition. This tells make to skip the dependency checks and run the command every time:

dev:
	@echo "Hello, world!"
.PHONY: dev

Now:

$ touch dev
$ make dev
Hello, world!
$ make dev
Hello, world!

I usually bundle the .PHONY line directly with the command it’s associated with, which makes it easy to tell which commands are “tasks” and which ones are instructions for building files:

dev: node_modules ## Start a dev server
	@./node_modules/.bin/next dev
.PHONY: dev
 
lint: node_modules ## Lint files for code quality
	@./node_modules/.bin/next lint
.PHONY: lint
 
format: node_modules ## Format code to a standard style
	@./node_modules/.bin/eslint --fix 'src/**/*.{js,jsx,ts,tsx}'
	@./node_modules/.bin/prettier --write 'src/**/*.{js,jsx,ts,tsx}'
.PHONY: format
 
node_modules: package.json
	@yarn install

Referencing node_modules binaries

In web projects, you’ll often want to reference tools installed via yarn or npm, like next, prettier, or eslint. In npm/yarn scripts, you reference these utilities by name:

{
  "scripts": {
    "format": "prettier --write 'src/**/*.{js,jsx,ts,tsx}'"
  }
}

Under the hood, these scripts are stored in the node_modules/.bin directory. When you run yarn dev, it adds that directory to your $PATH, so running yarn next refers to the local installation instead of any global ones.

In Makefiles, we can reference these tools by being explicit. Write ./node_modules/.bin/<name> before commands you want to use. So instead of prettier, you write:

format: node_modules
	@./node_modules/.bin/prettier --write 'src/**/*.{js,jsx,ts,tsx}'
.PHONY: format

This explicitness has the benefit of throwing an error if the local prettier isn’t installed (but it should be if you set up node_modules as a dependency).

There are techniques to modify your $PATH inside Makefiles, but they don’t seem to work consistently with the version of make that ships with macOS.³ Better to keep things simple.

Auto-installing node_modules

When working with teammates unfamiliar with Node, I’ll sometimes help troubleshoot errors like this:

I ran yarn start and got this error:

$ yarn start
$ run-p start:*
/bin/sh: run-p: command not found
error Command failed with exit code 127.

This message means they tried running yarn start without running yarn install first. But it’s not terribly clear that’s how to resolve the error unless you know what you’re looking for.

Instead, with make, we can express the relationship between all these dependent tasks at once so we don’t need to think about them again.

For example, if we create a task to build node_modules when package.json changes, we can leverage that in all the other scripts:

dev: node_modules
	@./node_modules/.bin/next dev
.PHONY: dev
 
build: node_modules
	@./node_modules/.bin/next build
.PHONY: build
 
node_modules: package.json
	@yarn install

Running make dev will install node_modules first, then run the dev command after. And on subsequent runs, it’ll skip the install step and just run the dev command.

Skipping re-runs of slow tasks

For slow tasks, you can save time on subsequent runs by adding known dependencies to your rules.

For example, when working on a Next.js app locally, Next runs data fetching functions like getStaticProps on every page navigation. Sometimes, when I’m working on a feature that requires me to navigate between pages quickly (like transitions between pages), I run a production build to avoid the delay of the data fetching when previewing my work.

With make, I do this by defining the dependent files and folders, which we should rebuild the output for when they change. For this Next.js example, it means we can run a production build and start up the server. When running the command again, if the production build is up-to-date, we can just start up the server without the delay of rebuilding.

In order to get a list of all the files in a directory, we can use the $(shell <cmd>) expression and the find command:

start: node_modules .next
	@./node_modules/.bin/next start
.PHONY: start
 
.next: node_modules next.config.js $(shell find src -type f -name "*.ts" -o -name "*.tsx")
	@./node_modules/.bin/next build

Now running make start:

$ make start
# ... next.js build output
ready - started server on 0.0.0.0:3000, url: http://localhost:3000

Running it again skips the build and just starts the server, since the files haven’t changed.

$ make start
ready - started server on 0.0.0.0:3000, url: http://localhost:3000

You can add different file extensions by adding more instances of -o -name "*.<ext>". To add CSS files to the mix:

.next: node_modules next.config.js $(shell find src -type f -name "*.ts" -o -name "*.tsx" -o -name "*.css")
	@./node_modules/.bin/next build

Using environment variables

To use env variables you’ve defined in a local file, like a .env, you can use the following block at the top of your Makefile:

include .env
export

I find this makes it easy to store secret token for one-off tasks like API calls or just avoiding installing a dotenv dependency into my projects:

SSH_USER="user"
SSH_HOST="example.com"
SSH_DIR="public_html"

include .env
export
 
content:  ## Sync content from the server to the local environment.
	@rsync -avz --delete "$(SSH_USER)@$(SSH_HOST):$(SSH_DIR)/content/" content/

Configuring tasks with default variables

Using ?= you can set default variables that you can override outside of make. This lets you expose some configuration options to Makefile users.

PORT ?= 9000
 
dev: node_modules
	@php -S localhost:$(PORT) index.php
.PHONY: dev
 
node_modules: package.json
	@yarn install

Now you can customize the port (while falling back to a default value) like so:

$ PORT=2000 make dev
PHP 8.2.1 Development Server (http://localhost:2000) started

Self-documenting Makefiles

Makefiles can grow to contain lots of commands over time. One trick I’ve seen is to use this make help target to document important scripts:

help: ## Show this help
	@echo "\nSpecify a command. The choices are:\n"
	@grep -E '^[0-9a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "  \033[0;36m%-12s\033[m %s\n", $$1, $$2}'
	@echo ""
.PHONY: help

Using this, you can document commands with a ## comment at the end of a line. For example:

dev: node_modules ## Start a local development server
	@./node_modules/.bin/next dev
.PHONY: dev
 
format: node_modules ## Format all source files
	@./node_modules/.bin/prettier --write 'src/**/*.{ts,tsx}'
.PHONY: format
 
node_modules: package.json
	@yarn install

Adding this target means running make help will list all the targets labelled with the ## comment. Use this to call out important commands for yourself or your teammates.

$ make help

Specify a command. The choices are:

  dev          Start a local development server
  format       Format all source files
  clean        Clean all built files
  help         Show this help

Parallel dev servers

I’ll often want to run multiple long-lived processes while working on a project. For example, a recent website I worked on uses:

• A php -S dev server
• A tailwindcss --watch process
• An esbuild --watch process

Unfortunately, like npm/yarn scripts, make doesn’t have meaningful tools for running long-lived parallel tasks. There are technically a few ways you could get parallel servers running, but each technique gets a little fiddly for my tastes:

• Using make -j3 <a> <b> <c> doesn’t treat processes as a group. So if one subprocess fails (eg. a syntax error stops the Tailwind --watch process), the others keep on going. You lose your live-updating changes without noticing until things are broken.
• Shell background jobs with <command> & suffer the same process group issue, plus can sometimes leave processes hanging around after you Ctrl+C out.
• External binaries like GNU’s parallel command, or foreman work, but they need separate installation steps.

Thankfully, make doesn’t care: we can just pop into language-specific tools that are suitable for the job. In Go projects, I’ll use a custom Go script. In Node projects, I’ll use the concurrently npm package:

yarn add --dev concurrently

PORT ?= 9000
 
dev: node_modules
	@./node_modules/.bin/concurrently \
		'php -S localhost:$PORT index.php' \
		'./node_modules/.bin/esbuild src/*.ts --bundle --outdir=assets/ --watch' \
		'./node_modules/.bin/tailwindcss -i src/index.css -o assets/index.css --watch'
.PHONY: dev
 
node_modules: package.json
	@yarn install

You still get the nice make dev interface and dependency tracking, but proper parallel dev server handling behind-the-scenes.

Update: as of recently, I’ve been using tandem, a tool I built for running multiple commands as a process group. You pass it multiple commands and it runs them in parallel, shutting down the whole group if one fails.

It’s designed to be easily embedded in Makefiles with a one-line installation script. Add .cache to your .gitignore file, and use the following block:

PORT ?= 9000
 
dev: .cache/tandem node_modules
	@.cache/tandem \
		'php -S localhost:$PORT index.php' \
		'esbuild src/*.ts --bundle --outdir=assets/ --watch' \
		'tailwindcss -i src/index.css -o assets/index.css --watch'
.PHONY: dev
 
.cache/tandem:
	@mkdir -p $$(dirname $@)
	@curl -fsSL https://raw.githubusercontent.com/rosszurowski/tandem/main/install.sh | bash -s -- --dest="$$(dirname $@)"
 
node_modules: package.json
	@yarn install

The first time you run make dev, it’ll download and cache it for future uses.

Hermetic environments

Building on the auto-installing node_modules technique: you can get fancy with dependencies to download local versions of your entire toolchain. This way everyone runs the same version of everything, without the slowness of using Docker.

This snippet reads yarn and node versions from files called yarn.rev and node.rev, downloads local copies into the tool, and uses those instances for all your scripts.

16.13.1

1.22.19

export CACHE_PATH := .cache
export PATH := ./tool:$(PATH)
 
dev: node_modules ## Run a local development server
	@./node_modules/.bin/next dev
.PHONY: dev
 
clean: ## Clean all build artifacts
	@rm -rf ./tool
	@rm -rf ./cache
	@rm -rf ./node_modules
.PHONY: clean
 
node_modules: package.json tool/yarn tool/node
	@yarn install
 
# Reads a version number from a file called `node.rev`
tool/node: node.rev
	@mkdir -p tool
	@mkdir -p $(CACHE_PATH)/node
	@$(eval OS=$(shell uname -s | tr A-Z a-z))
	@$(eval ARCH=$(shell uname -m | sed -e "s/x86_64/x64/" | sed -e "s/aarch64/arm64/"))
	@read -r REV <$< && \
		cd $(CACHE_PATH)/node && \
		curl -L -o node.tar.gz https://nodejs.org/dist/v$$REV/node-v$$REV-$(OS)-$(ARCH).tar.gz && \
		tar --strip-components=1 -xzf node.tar.gz
	@echo "#!/bin/sh" > $@
	@echo 'exec /usr/bin/env PATH="$(CACHE_PATH)/node:$$PATH" "$(CACHE_PATH)/node/bin/node" "$$@"' >> $@
	@chmod +x $@
 
# Reads a version number from a file called `yarn.rev`
tool/yarn: yarn.rev
	@mkdir -p tool
	@mkdir -p $(CACHE_PATH)/yarn
	@read -r REV <$< && \
		cd $(CACHE_PATH)/yarn && \
		curl -L -o yarn.tar.gz https://github.com/yarnpkg/yarn/releases/download/v$$REV/yarn-v$$REV.tar.gz && \
		tar --strip-components=1 -xzf yarn.tar.gz
	@echo "#!/bin/sh" > $@
	@echo 'exec /usr/bin/env PATH="$(CACHE_PATH)/yarn:$$PATH" "$(CACHE_PATH)/yarn/bin/yarn" "$$@"' >> $@
	@chmod +x $@

This example probably pushes the limit of what should go directly in a Makefile vs. being broken out into a separate shell script or tool, but hopefully it demonstrates the extent to which you can automate your dev environment with Makefiles.

We use something similar to this at Tailscale, and it greatly simplifies getting new teammates started, and upgrading versions of core dependencies like Go, Node, or Yarn.

Limitations

While make is great for many projects, it’s not always the right thing to reach for. A few cases where you should avoid it:

• If you’re working with Windows. Makefiles usually rely on a lot of UNIX tools and conventions, like ENV=val environment variables, or awk, sed, and grep, which don’t work on Windows. It’s possible to make cross-platform Makefiles, but if this is important to you, it may be better to use a tool that abstracts away platform differences.
• If you’re automating complex builds. make gives you simple tools for simple builds. With involved chains of dependencies, Makefiles get unwieldly fast. If you start mucking with file modification timestamps, dealing with weird shell quoting rules or anything to do with automake, you’ve probably gone too far, and would be better served by another tool.
• If you already have a setup you’re happy with. For simple projects, npm/yarn scripts can be enough! Don’t change if you’ve got something working. That said, I find Make’s speed, simplicity, and little bit of extra flexibility helpful, and suggest you give it a try.

Reference Makefiles

• The stripe-cli has some great snippets, including a // TODO: comment finder, a Git tag generator, and Git hook installation step.
• This Tailscale docker desktop extension shows using a Makefile to make a nicer dev experience. Rather than manually building containers, installing them, and configuring dev server options, I composed them into some commands that made working on the project a lot easier.
• This Kirby CMS Makefile which has commands to update a vendored dependency, sync content with a remote server, and deploy via rsync to a PHP server.

Know other good web-centric Makefiles? Let me know!

Recently, I felt an odd longing for seasons.

San Francisco has only two: pleasant and foggy. Trees keep their leaves year-round, and temperatures are consistently in the sweater range. It’s a city of eternal mild weather.

This past December I went back to Toronto when winter was in full force. My phone told me it was -30°C. The downtown core was like a wind tunnel, blasting my face with high-speed frigid air. Why does anyone put up with this? I thought. But here I am, five months later, sitting in the California sun, longing for some crappier weather.

It’s not the Canadian winters that I miss so much as the way seasons naturally divide the year.

Living in a season-less city makes time move at a scary pace, like how as a kid, summer vacation would end before you know it. Between work, travel, and a few personal projects, weeks pass by in an instant. I’ve been here for two years, but it’s only felt like a couple months.

Few tools have been as important to humanity as calendars. Shortly after writing systems developed, calendars were found. It makes sense: in agricultural days it was important to know when in the year to plough, to seed, to harvest and when to let fields lie fallow.

Every culture developed their own local calendars to track the time, often sharing the definition of day (an easy one), but varying from one region to the next in concepts like weeks, months, and years.

As societies urbanized, these measurements were standardized. Over time weeks and months were defined and consolidated into either a moon-phase-based lunisolar calendar, or a sun-path based solar calendar (the latter being the origin for the Gregorian calendar we use today). Standardizing weeks and months was important: by having a shared language for time, you can now talk with the town over about if winter is coming early this year, or when the next holy day is, or even just speak meaningfully about the future.

There’s no shortage of writing on our mechanization of time, but McLuhan is particularly concise:

The clock [or calendar, in our case] dragged man out of the world of seasonal rhythms and recurrence, as effectively as the alphabet had released him from the magical resonance of the spoken word and the tribal trap.

Like written language, the development of a standard calendar enabled stronger coordination and control for countries and societies. And also like written language, standard calendars introduced shared metaphors and patterns of thought. The week, the month, the year, sure. But also the financial quarter, the nine-to-five, the holiday.

This language shapes how we think about time and hence how we think about our lives and work.

For instance, my roommates work at larger companies, and have projects which are constrained to the current financial quarter. They often come home frustrated, feeling like they can’t make meaningful progress because of the short cadence. It encourages decisions that quickly cater to metrics, rather than working towards about more meaningful, holistic changes. The chunks of time are too small, too constraining.

The inverse happens every New Year’s Eve. People set resolutions that are achievable within a year, but as the days pass, lose themselves in how quickly they go by. The cadence for reflection and review is often too long, and the goals become disconnected from everyday experience, nebulous and intangible.

Both of these experiences point to, in part, a failure of how we conceptualize and represent time. Using years and quarters to divide time often feels quite … arbitrary.

Standardized calendars are no doubt great for accounting, communication, and coordination¹ but they fail at more personal functions like doing deep work, or maintaining cycles of interest and curiosity.

”Phases”

A friend of mine has a unique method of journaling.

He documents moments from his life via a mishmash of words, events, ideas, and screenshots, and groups them into discrete phases in iPhoto. Each phase is roughly two weeks long — but it’s defined by his living situation and activities, not the days of the month. A week-long bike trip across Southern Europe might be one phase, but two months of heads-down focus on school work could be another.

It’s a great way to remember things. In conversations, a topic or feeling will come up, only for him to say, “oh man, that was so phase 37.” By chunking his life out into phases it becomes easier to grasp, both in language and in memory.

Contrast that with a more standard journaling experience: writing a couple hundred words every few days. At the end of a month or a year, you look back on a sea of sentences, unfiltered and uncategorized. So much is written that it’s hard to extract anything meaningful from it without jumping around to specific days.

By eschewing a standard period for a self-defined one, my friend has inadvertently created a personal unit of time.

Thinking in Decades

Consider another function of calendars: planning.

You open up Google Calendar and you get a view of the next 7 days. You can block out the next 168 hours of your life in 30 minute events.

But unless your life is a long series of back-to-back 30 minute meetings, does this interface make sense? Do most people actually live that way?

In contrast, James Fisher’s long calendar idea sounds really compelling:

How long will you be in your current house? How do you plan out your thirties? When will you move to Spain, as you’ve always imagined you would? When might you have your first child, and what job would you like to aim for by then? […]

These life questions require a different kind of calendar. They operate on a different timescale, where the month is probably the most granular unit of time you might want. These questions work on a personal human epoch — “I am 29”, not “it is 2017.”

How might seeing your whole life, laid out on a single surface affect how you live and plan?

Small Seasons

In Kenya Hara’s book White, he briefly talks about sekki, small seasons used in ancient agricultural China and Japan to divide the year. Each sekki corresponds to a shift in the climate, not the calendar, with poetic language to match the changes.

Where spring is our “large” season, lasting roughly from March to June, the sekki calendar breaks it down into six small seasons: start of spring, rain waters, the going-out of the worms, vernal equinox, clear and bright, and rain for harvests.² Both describe a similar period of time, but one does so with a precision that feels much more human.

These days, I increasingly find myself using sekki as a calendar.

On one hand, it’s nice to have a calendar so explicitly based on the surrounding environment; you’re encouraged to look up and notice what’s different today, that the crickets are no longer chirping and that time has gone by.

But it also better matches my own ebb and flow of interest, attention, and general mental headspace. Two weeks of deep focus on something is a healthy amount before I want to move on to a different aspect, or a new project all together.

Sekki, phases, and the long calendar are just a few examples of more human and purposeful calendaring systems. Heck, even a seasonal vegetable chart for an area is an unexpectedly useful time-keeping tool. Why shouldn’t your calendar help you decide when to make a minestrone soup, oxtail stew, or a pico de gallo?

You can imagine some bigger questions along the same train of thought:

• How can calendars inspire or inform daily living?
• What does a communal calendar look like? How can it afford community action and a sense of collective identity?
• How can calendars encourage thinking beyond our own lifespans? (Peter Bïlak’s 100-year calendar is an interesting example of this)

I’m interested in more conversation and links about this.³ It seems like there’s merit to finding more personal ways of chunking out time. Especially in an era with such powerful means to represent time, finding novel and more meaningful timescales seems like fertile ground for exploration.

But, if nothing else, I’m happier just to notice the time passing a little more often.

Further reading

• Small Seasons
• In Search of Personalized Time
• A Line Moving Across A Window Once Every Year
• Austin Kleon’s notes on seasons, including Seasonal Time

Travel is one of those perennial human problems, and over the years we’ve devised increasingly complex solutions to move people and things from one place to another. Bikes, trains, cars, planes, and now roller-coasters inside tunnels. Each new approach lets us travel quicker than before, but also requires more maintenance, safety equipment, and regulations.

In university I was introduced to the writings of Paul Virilio, a French cultural theorist. One of his most known ideas is the integral accident, which is often summarized in this quote:

When you invent the ship, you also invent the shipwreck; when you invent the plane you also invent the plane crash.

It’s a meaty idea that digs into humanity’s infatuation with technology. But when I hear it, all I can ever think is what it means for objects moving at high speeds.

Moving faster inherently creates more risk. Even when the risks are well-controlled (plane rides are statistically safer than driving, right?) they are never gone. A rocket will always have more dangerous potential than a car.

Greater velocity equals greater consequences when things go wrong. There’s no getting around that nature, no matter how many safeguards you implement.

It’s always interesting to see innovative approaches to age old problems. But, in my head, complex solutions never seem quite as appropriate as ways to use simpler ones, like walking, biking, and creating more dense urban spaces.

Last week I discovered that ffffound shut down.

I didn’t visit it often, but when I did, I was amazed by the variety of stuff on there. The site was an archive full of architectural photography, abstract art scans, and all kinds of other images. Seeing it gone made me a little sad.

Back in February, Vine also shut down. Only a few months later, people shared similar doubts about SoundCloud’s future.

Often supported entirely by ad money, these community-companies on the web seem to struggle to stay online.

When conversations about internet shutdowns come up, people often ask, “what was their business model?” But, isn’t the web supposed to be an open, global community? Why is a business model a necessity for someone to make a meaningful site?

More and more, the web feels like a playground for businesses than the information superhighway it was originally conceived as.

Early Days

The computer pictured above is the world’s first web server. Built to share documents amongst scientists at CERN, it was the first use of the HTTP protocol, the system that drives the internet. Colleagues of Tim Berners-Lee could access files on his computer from anywhere else in their local network.

On its casing is a sticky note warning: “This machine is a server. DO NOT POWER IT DOWN!” Makes sense. If the server is down, nobody can read the files on it.

This little sticky note is a clue to a fundamental concept behind HTTP: it’s a request-response protocol. A “client” makes a request, and the “server” sends a response. One computer has a file, and many others ask for it. One server, many clients.

These Days

The web has grown a lot since its early days. Instead of sticky notes on computers, internet infrastructure looks a little more like this:

How we use it has changed too. We’re not just passing around text files, but images, videos, articles, and applications.

But the shape is still fundamentally the same. A domain name points to a single server. One server fields requests, responding to many clients.

The one-to-many shape leads to some interesting consequences.

For one, its enabled the rise of the “platform web.” Not everyone can or wants to run their own server, so people make platforms where others can contribute without owning a server: writing blog posts on Tumblr, sharing images on Flickr, putting life events on Facebook, etc. But running these platforms has costs, and over time, some platforms win out over others.

The platform web has become a marketplace. The only way to survive is to get as big as possible, hence our current world, where a few platforms can account for the majority of all web traffic.

And when one person or organization hosts all the code and all the data, the power dynamic changes.

• Facebook owns the server, so only they can see the code that decides what news to show you. To everyone else, its a black box
• Google owns the servers holding people’s search history, making it easy to surveil without the knowledge or consent of the people putting that data there
• Twitter owns the server, so Twitter gets paid to show you ads
• The easy scalability of computation makes it possible for huge monopolies to form, resulting in more billionaire individuals than ever before in history.

The assumptions of HTTP, expanded to a global scale, enable the centralization information, money, and power in an unprecedented way.

We trust these tech powerhouses to act in our best interest. Computers and networks will lead us to a more open, connected, and democratic future, right? But over and over again, we see ethically questionable decisions being made by these groups. Over and over again, these groups aren’t held accountable for the power they wield.

Distributed Web

For all that negativity, there’s tons of exciting work these days to find alternative shapes for the web. Shapes that benefit people, not corporations.

One exciting area is distributed web technologies, protocols vying to replace HTTP as the way we pass information across networks. There’s a few projects trying to fill this space. Some bigger ones: IPFS, Zeronet, and Dat.

Many of these alternatives share a resemblance with BitTorrent: one person starts to host something, and everybody re-hosts it as they view it. The distance between server and client collapses. No one person owns the servers. Instead, everyone is both a server and a client.

If the internet we’re moving towards today is the “corporate web”, distributed web tech tries to move towards a cooperative one: everyone works together to re-host things they’re interested in. Everyone’s responsible for keeping it online, sure, but everyone owns a piece of it.

What may seem like a simple change — a shift in how digital files move about — can have huge impact when applied at scale:

• How does a company make ad money if they can’t control what gets shown to people?
• How do governments track and surveil people if the information was scattered across millions of computers, rather than neatly compiled by one organization in one place?
• How do you monopolize (attention, data, etc) when the only way to grow is with the help of others?

Distributed web protocols no longer give easy solutions to those challenges. The decisions behind decentralization make forming communities easy and forming companies hard.

All the alternatives I listed above are still in their early stages. There’s tons of challenges around usability, financing, and public interest/adoption. But for all the challenges, there’s some amazing opportunities too:

• What if a “like” wasn’t a gesture that told companies what ads to serve you, but instead helped creators host their work?
• What if the users of a platform could own and invest in its future, rather than VC firms looking to make a profit?

Anyways, the big point here is that we don’t need to continue to live out the consequences of technical decisions made 40+ years ago.

We have a responsibility as citizens of the web to challenge the politics embedded in our tools, the futures they’re leading us towards, and to question who they’re really serving.

Further Reading

• What is the P2P Web?
• Beaker Browser
• So you want to decentralize your website