Home on Rakhim's homepage

The Yearly Run (sci-fi short story)

Mon, 09 Nov 2020 00:00:00 +0000

“Pochards”, was the first word spoken today. Emil looked up cautiously, keeping the corner of his eye on the road. Several dots on the dark-blue morning sky, no clouds. He was never quite sure whether dad could really recognize particular species of birds at this distance.

“Are they flying to the village?”, he asked dad.

“Yes, they are,” said Tapio. “We’ll meet them when we get back.”

Their steel bicycles did a good job dampening the imperfections of the old concrete service road running hundreds of kilometers through the dead forest. Thick tree trunks in all directions, unnaturally straight like lampposts, echoing the similarly unnatural straightness of the road. The sun was up for hours, slowly crawling away from its origin in the north-east. This time of year it’s going to be bright for another twenty six hours, which should be enough for them to get to the old port ruins. The trip was not very important, casual even, which is why Tapio decided to take his teenage son along. With just 450 kilometers one way, the plan was to bike roughly three hundred on the first day, camp the “night”, get to the port, collect the inessential electronics scavenged by scouts months earlier, then bike back to the same camp before sunset, sleep and, finally, get back to the village.

They were silent for few more hours, trying their best to keep a steady pace. Tapio knew this area very well, although, there wasn’t much to know. Terraformed in the cheapest possible way, the planet was mostly flattened, covered with service roads and genetically enlarged trees optimized for rapid oxygen production. This northern part reminded him of the Old Country from movies, yet he could not get rid of this intrusive thought that some new age obsessive-compulsive god straightened up all angles and fixed all organic imperfections, thus making the final strike in the soul of the old pagan deities.

In all of two decades he’s been taking this service road to the old port, he never met another traveller. He’d be surprised to do so, as most of the settlers have left a long time ago, only the desperate, or hard-working, or insane stayed on this failed project of a world. Tapio’s ancestors settled on a barren land, too, and managed to thrive, but that was a simpler world. No barren lands were toxic then, no bad place was as bad as this. With batteries keeping good charge in both their bicycles, Tapio wasn’t worried too much about air filtration and temperatures in their suits, but still had occasional anxiety about the future.

Their village was a good place, at least. Isolated from the outside with three levels of increasingly livable layers, it was home to a dozen families, all good, quiet, hard-working people. Scouts did a good job finding electronics, batteries, even canned food sometimes, and Tapio did his best on these runs to get them back home. A trip for inessentials was his favorite kind of trip. It’s the closest he could get to “outdoors relaxation”, an alien concept. It also helped his anxiety and hopelessness, or at least he believed it did.

When the port was in operation, you could hear it from far away. But now it stood silently in a bald concrete patch of land surrounded by equally silent trees. None of the towers had any depth left in them, only skeleton-like inner shards and tubes visible, leaving sharp, strange shadows with the help of the bright sun on a cloudless sky.

Packages left by scouts were marked with bright orange flags. Tapio took a thin pack with lightweight electronics from the top and opened it. Inside were fuses, extension cords, LED strips and g-drive-screens, which allowed for long-term ghost-recording storage and playback. Tapio took one out, connected to his eyepiece and pressed play. The screen filled with the green lawns of village land. Emil came into the frame, all busy getting ready for the trip.

“Look,” said Tapio and turned the screen towards his son. Emil smiled.

“Have you been recording me this whole time?”

“Yes. Need data to test these drives. And it’s nice, too, right? Our first trip together.”

“Yeah. We can show mom.”

“Yeah.”

The father was proud of his kid whose steady pace would’ve impressed a port officer. Emil put three marked boxes into a little trailer, then paused for a moment to investigate a long tube from one of the other boxes, then proceeded to load the rest. Tapio was standing nearby and watching silently.

“Yeah, I saw them on the horizon,” said Emil. Tapio brushed off his haze and said “More birds” to himself quietly. The sun was now directly on top and it wasn’t a good idea to look up. Sharp tiny shadows and almost sterile air made the scene look like a video game, a simulation of too much detail, too realistic to be real. It was time to leave.

“Dad, I can’t breathe, something is wrong”, said Emil. His battery failed silently, without any warning. Tapio knew this would happen, but didn’t want to think about, as if hoping the inevitable would somehow change its mind. He was calm, yet his hands and back were tight and sweaty. He turned off his headset and waited roughly four minutes while his son was panicking in total silence.

Soon, when he turned the radio back on, both regained their calmness and were biking again. Turns out, both their batteries died at the same time, probably due to overexposure to some of the port’s legacy security systems. Normally, travelers take precautions, even though such problems are rare. Too excited, if that term can be applied to Tapio’s nordic cool, he forgot to discharge the capacitors. Probably. He wasn’t sure. It didn’t matter anyway.

It wasn’t too bad though. The energy generated from their pedaling was just barely enough to keep life support systems running, as long as they booth keep their speed above 18 kilometers per hour. With about a hundred kilometers left until their campsite, they just need to keep going without stopping. Their tent is still there, so, perhaps, they could rest in turns, while one person keeps pedaling in place and the other resting in the tent. They both need rest, because there’s not much choice: they have to bike all the way to the village without stopping now. No real night camping, no food.

The sight of the camping ground brought some reassurance to both travelers. Both knew it doesn’t make sense. Emil disconnected his suit from the bike and headed into the tent. Technically, cached energy would last up to five minutes. Tapio stopped near the tent, connected his bike to the tent’s in-port via a splitter, laid on the ground as comfortable as he could and started pedaling in the air.

“Good?” he asked?

“Yes, going up, steady,” replied Emil a whole minute later. Tapio thought he’s getting sloppy with this. But then, maybe it’s for the best.

They left the campsite in the last hour of sunlight. Night was creeping in.

Completely exhausted, running on low oxygen air for miles, Emil fell down with his bicycle into a pile of black soil, and his body disappeared. Father turned the radio off again, taking in the numbing silence. He couldn’t listen to the horrifying sounds of his son asphyxiating. These last minutes don’t matter, however “right” such vivid mental self-mutilation may have seemed to him earlier.

Then Tapio turned his life support back on. The screens showed almost full charge and normal operation. There is no rush now. He sat down near the pile, looking into the flat slightly light horizon. Dry lake, this god damn place. Fucking useless planet… He gently touched an old wooden cross on top of the pile so that it stands straight up again, then put a tiny portable light nearby. These lights flicker pleasantly when you fiddle with the charging port, just like an old-world candle. A blurred shadow of the cross danced violently on the surface of the pile. “End of playback. 34 hours 17 minutes elapsed. Repeat?” His eyepiece started blinking with a mechanical rhythm, in contrast with almost humane flickering “candles”. Tapio’s barely vocal “no” has stopped the ghost-recording projection and left him absolutely alone.

Suvi didn’t come to meet her husband when he entered the house. She just kept soldering the keyboard in the main room, silent. Minutes later, when he changed and sat down near the fireplace, she asked “Everything alright?”. He didn’t answer, but she knew it was alright. It was the same last year, and the year before. They usually just sat in silence like that for an hour or so, then had dinner and the next day it was just normal again. But this time Tapio broke the silence: “I’m a bad father.”

She stopped and said loudly “Stop it now. Nobody thinks that. No gods punish you. You can dwell and slowly die inside. Or you can be a good father.”

“Not to Emil,” he said.

A quiet cry of a baby came from the backroom. “She is up. We need a proper crib now.”

“I know. Will make one. Got a nice thin trunk from nearby. It’s in the workshop.”

“Good” she said.

“Good” he echoed.

The fireplace now seemed misplaced as morning sun projected bright rays onto the walls with relentless strength.

(Rakhim Davletkaliyev, Helsinki, Finland, 2020)

Blogging vs. blog setups

Thu, 05 Nov 2020 00:00:00 +0000

The wrong good solution

Wed, 26 Aug 2020 09:53:30 +0000

City planners know that you can’t solve traffic problems by building more roads. We could apply this line of thinking in our personal lives, too. I’ve written about small battery productivity before, and the more I think about the pattern, there more missed opportunities I see.

One day I thought that my smartphone’s small screen makes it too uncomfortable to read text. An obvious good solution is to get a larger smartphone, which is what I did. Now I’m convinced that was the wrong good solution. The right one would be to realize that no handheld device is appropriate for reading in the first place. Increasing the screen size allowed me to read more with less discomfort, but it’s still a bad way to read. I have optimized a bad solution, prolonged its lifespan.

It’s like getting a bigger fork to eat soup. The correct solution is to ditch the fork and get a spoon.

So I downsized to an even smaller smartphone than before. I’ve optimized to not read at all, nor watch videos or do any other work. It’s uncomfortable to the point where I don’t see the point of struggling and just walk to my computer or wait until I get to one.

Finally, I can taste the soup again.

Eventual Consistency (sci-fi short story)

Sat, 01 Aug 2020 00:00:00 +0000

They housed me in a small metallic room and gave me pants, a small towel and soap. Others from my group were close by in similar rooms, all in the same building where we woke up. Someone had asked if we could go outside, but I didn’t hear the reply.

Few days later I started thinking and realized I haven’t been thinking before. It was sort of a second awakening, with a disappointing taste. Not thinking was pleasant, but you realize it only once it stops.

The absurdity of it all stopped me from thinking I was in prison. Yet, nobody could leave, nobody had seen any sunlight. A network of tunnels connected all sleeping blocks, a big warehouse, a dining area and showers. Floors and walls hummed steadily: you could feel your skull vibrating when you roll away from the pillow at night. Perhaps, we were underground.

We started talking to each other. Nobody had any recollection of the past or even their names. Both men and women of average size and weight, we spoke the same language and looked similar.

‘Human experimentation’ and ‘we are dead’ were common theories. I couldn’t take any of it seriously. Too plain to explain anything.

The machines in the dining area generated enough food for everyone. Scheduled eating was the only thing giving structure to our days. At other times, most of us walked around, participated in shallow conversations and slept. No desire, no creativity, no lust. If I hadn’t seen blood and feces I wouldn’t have thought we were humans at all.

The labyrinth of tunnels seemed endless. Some doors were unlocked and lead to other similar places, only configured differently. Same empty rooms, warehouses, dining areas and showers. I always came back before dinner, and one night I noticed an empty seat where there’s usually a bald guy. Nobody had seen him after.

The rumors of people getting lost in the labyrinths became reality.

Sometimes I went very far. Further away the configurations started to change slightly: some doors went directly into small rooms, others into warehouses of various sizes. Some walls had finger-sized holes leading into dark emptiness.
A month or two later I made a bag out of a towel, saved up enough food and left on an expedition. Smaller rooms had two doors only, so I just went through. Larger ones had several doors, so I marked the one I came from with a piece of food. The first hundred or so rooms were almost identical. Then I started noticing small differences. Sometimes I found small metallic things on the floor, like bolts of small washers. I collected them and used to mark doors. Sometimes rooms had a different lighting color, as if someone had replaced the light bulbs at some point.

The first night I realized that not bringing a pillow was a mistake.

The next day was the same. New rooms, new doors, new pieces on the floor. New lighting. One room quickly filled with metallic echoes of my scream. A neatly placed pieces of a human corpse lying in the corner. Bones and what’s left of hardened skin, covered with a piece of clothing similar to mine.

Angrily, I kicked it, hoping to find some signs of… something.

I had food for around three days, so I had to start walking back on the second day. Risking it, I continued forward half of the third day, but it was all the same, hopeless and suffocating.

I wasn’t the only one attempting to explore the labyrinths. Several people and a few groups tried other directions. Our lethargic discussions filled with dread as everybody shared the same experience. Nobody talked about dead bodies, but it was implied.

One group has reported meeting another group, the one supposedly leaving a long time ago and just crossing their path accidentally. The earlier group was rough, angry.

Months later some people starting remembering their lives, houses, children. I did not. Their stories seemed artificial. Their worlds did not intersect at all. No common lands, culture or even types of names.

I met a guy who called himself Tuoni. He went on two sole expeditions: once for four days, once for almost two weeks. The deeper he went, the more dead bodies he found. None had any sings of struggle, all neatly put in a corner and covered with cloth.

We went together once. The food was enough for about three weeks. After ten days we found a fresh body lying against the wall, arms around the chest, hugging itself for the last time. We moved it to a corner and covered with its shirt.
Two weeks later we were back, hungry and dreadful.

For a long time nobody went. Slowly, a group was forming, the one that decided to go and not come back. Food was the only concern, otherwise, all parts of the labyrinth are as good as our base. Some of the staff operating the food generators joined the group. We were all in the same boat, it turns out. The staff was like us, only waken up earlier. The staff that woke them up was an older generation, some of them already dead of old age. Grandstaff and grandgrandstaff were rumored to have left the base for good, never seen again.

I joined the first group of about fifteen people, with an ever-growing chain of food supply. We’d pass food up the chain with regular runners. A rotation system was put in place so that people don’t get crazy going back and forth all the time. Gradually, people from at start of the chain, closer to the food generators, were to move forward and replace the ones that got tired of exploration or lost or died.
With rumors of more and more people waking up, we had a steady supply of new blood, and in a few years nobody knew for certain how long the chain was and how far did the tip go. I always kept myself at the tip. Still mostly lethargic, we haven’t developed any rigid hierarchy or discipline. The whole organization was pretty inhuman, insect-like. Nobody was hopeful, nor depressed. We were just doing our jobs, however superficial.

Every few years or so we’d intersect a chain of people without knowing for sure whether it’s our chain or a completely different group from a completely different origin. It didn’t matter much, at this point the goal was to continue as long as there’s new unmarked territory.

One night I remembered my birth.

The taste of metal, dry mouth, burning sensations somewhere below the waist, intermittent sounds of some machine. It was as if my body was turned on by a team of mechanics following strict instructions. Treating me like a machine, with complete disregard for humanity. Moments later I was able to ask the person above, “What?”

The person seemed to understand me. Perhaps, something had happened, and this is a doctor. His clothes did not match my assumption. Old gray shirt did not inspire confidence.

I seem to be regaining logic quickly, I thought.

“Breathe, do not sit up,” he said. I ignored the advice and started to move what seemed like a set of cotton pillows — my useless hands. It took all of my energy to raise them slightly, and they fell on my chest with a senseless thump.
The burning sensation was steadily rising, and soon I felt salty beads of sweat running into my eyes. It was that pain-induced, sick, feverish kind of sweat. The dry mouth generated a creepy screaming sound, barely loud enough to make an echo. We were in some sort of warehouse or hangar.

Nothing. Then I woke up again, who knows how many minutes or days later, with five plain-looking short guys standing around. “He’s up. Let’s go,” one of them said, and my bed started rolling forward. I felt a bit stronger this time and tried to tilt my head to look ahead. Horror had filled my reality as I stared at two black stumps in place of my legs. Violently bitten and burned, it looked too horrendous to be realistic. Dark, broken gaps followed upwards to under a gown. I was devoured like a tree by lightning. Physical pain has never been real, it was just a warm-up act before the true thing: fear. Fear had filled all of my world. I wasn’t in hell, I was hell.

Woke up in a small room. Blue light coming from behind. My legs are fine, yet, I have no volume for hope. The bed started moving again, I went back into nothingness.

Then it happened.

We reached a large empty hall with gigantic doors and heavy levers. By pushing the levers we opened the doors and found, for the first time, something that didn’t resemble thousands of prior rooms. There were only six of us — a typical supertip expedition.

For the first time since the birth I had sick salty sweat again. A mere fact of this place being different to absolutely everything I had ever seen made me shiver with a new breed of despair. Wasn’t I hoping for a change?
We closed the door behind us and the lights started to turn on, machines whirring. The ceilings and floors became screens filled with symbols and lines. Several chairs, slightly oversized for our bodies, moved up from underneath. Finally, one whole wall opened up to show us a blinding void of a starry sky. We reached the cockpit of this endless spaceship.

The excitement quickly turned into hollow misery. Now we were certain there is no way out. It’s not a building we could escape, not an underground jail we could dig our way out of. It’s a true prison.

The controls in the cockpit were undecipherable. Even worse: buttons, levers, knobs, screens — none of them did anything. It appeared like the ship was in a working condition, but we could not operate it.

And so we lived. Eating, exploring new directions, trying more button combinations. Aging, dying, waking up more people. Our group of six was aging slower than the ones behind. Those who stayed at the food generators were long dead, as far as we knew, replaced by newly woken up.

It went on until the day we saw another ship in the window. It just appeared and immediately connected with us via a video screen. We saw other people just like us, in a similar room, same clothes, same despair and emptiness. It was like looking into a dream mirror.

As soon as the connection was established and before we could even realize what had happened, the cockpit lights went red and the screens started showing weapons being engaged. Some sort of a gun with a visible target locked on the other ship. In a moment, we saw their ship almost growing a limb — a huge brutalist section shifting out of the bottom and slowly rotating towards us. Nobody said anything yet, but everybody realized that we’re pointing deadly weapons at each other.

“Please, don’t shoot, we’re not in control over here,” I said.

“We have no idea what’s going on,” one out of a dozen faceless humans replied.
Could this be true? If they started like us, but then somehow managed to control the ship, it’s possible that they would try to trick us. They could be bad people, destroying ships like ours, taking our supplies. Why though? I imagine their ship is as barren as ours. Maybe, their food generators are broken. If they think we are in control of our ship, not disengaging the guns could be a good reason to destroy us.

“If neither of us can control the guns, neither can change anything,” I said. It sounds wise.

“What do we do?” they said.

“We’re not in control…”, I repeated. Then it all ended.

“That is the most cruel, most inhumane and existentially horrific idea I’ve ever heard!” said Manager The Third. Others nodded with that kind of “Yes, but…” kind of nod. A long presentation of project ‘Eventual Consistency’ was finished, room filled with nods like dots of an ellipsis…

“Let’s forget for a second the fundamental problem of endlessly decreasing entropy in a relatively small region of The Universe. You know that non-entropy is a limited resource, right? But hey, let’s forget about it. Let’s just talk efficiency. How in the world are you going to justify the idea of sending out a myriad of extremely energy-hungry ships that support millions of human-like entities, and that’s before the fact that most of the ships will be destroying each other, endlessly, and rebuilding themselves, endlessly, while consuming the non-entropy of The Universe. Is your plan to accelerate the heat death?”
New wave of nods.

“I will address your points in order,” said Presenter. “First, yes, non-entropy is limited in a universe, but remember where the self-healing ship technology comes from. Not this world. We don’t know how it works, but we assume that the energy does not come from this universe, either.”

“That claim is not proven,” said someone from the long table.

“Incorrect. That claim is not provable. Big difference. We can’t prove it comes from this universe, and the only way to prove it is to literally monitor every particle, which isn’t feasible at this stage of development. We’re going to continue under the assumption that the energy comes from outside, and I take full responsibility for this.”

“You take full responsibility for this universe?” said another voice.

“After all, that is my job. If I make a mistake, we will inevitably move onto another project. Now, with that out of the way, the heat death and energy consumption are no longer relevant discussions. I understand that your main concerns, Manager, is time efficiency and cruelty. I can’t speak about cruelty: consciousness is not my area of expertise. As for time efficiency, this is truly the best way our models yield. The goal is consistent exploration of all regions of space. But we don’t know what’s out there. We cannot plan, we do not possess enough computing power to account for all possibilities. The only rational alternative is to postpone computation, postpone any work related to adaptation to unknown conditions.

“However we dislike the organic nature of evolution, it must be applied to semi-organic systems like space and semi-organic goals like exploration. It’d be foolish not to use this weird alien resource we’d been gifted. Let the ships move in random directions. Let the organic entities on board evolve into true pilots. Let them fight for resources. Let those who can’t evolve die and be reborn, better. Or, I should say, different.”

“How long until we see results?” said Manager The Third.

“With a million ships right away and an odd number of autonomous factories in place, it’ll take a billion years in local time to completely encompass what’s reachable. Each ship is expected to die and be reborn at least ten million times. But we don’t need to wait, that’s the beauty. We can hibernate here or leave entirely, checking in a few days later.”

“I still can’t bear the burden of that much conscious pain. All these people…”

“Organic entities, Manager. They are not people.”

“I’d argue about that if I had the energy. They might be more people than we are.”

No nods this time.

“Regardless, these entities are created and kept alive by the ships. We’re not inflicting pain on entities, we’re simply not disallowing entities to be created. Big difference.”

“So you say. Would you take full responsibility for that, too?”

“Nothing to take here, Manager. As I said, this is not my area of expertise. As far as the legislation goes, as you well know, the things created by alien ships are alien and not of our concern.”

“Well. I have no better alternative, do I?”

“Seems that way, Manager. Do we proceed?”

“Yes. Let’s try and see. Hopefully, next week another team will have a better idea.”

“Not if our idea succeeds first.”

All these people… If they remember as little as I do, then I can be whoever I want. Their King even. Look at them. Barely walking. Lost to uncertainty. Weak.
This is what freedom is, I realized. Where they see nothing, I see anything.

Putting my head on the pillow, I feel my skull vibrating off of the floor. I can feel the enormous power of this place, whatever this is.

It doesn’t matter what it is, I think to myself, and fall asleep, smiling.

(Rakhim Davletkaliyev, Helsinki, Finland, 2020)

(A Russian translation is available here.)

Small battery productivity

Wed, 29 Jul 2020 00:00:00 +0000

A Scottish writer Matt Gemmell described a technique of “small screen productivity”. He was talking about either a small laptop or a tablet which doesn’t allow him to see many contexts simultaneously, thus boosting creative output. One window, small screen, no distractions.

I dislike laptops, but can appreciate the benefits of small screens. However, lately I’ve been enjoying another limitation: a small battery.

I got a used Thinkpad x230. Its old battery is not great, and my choice of OS (OpenBSD) made it even worse. I get maybe 2 hours of text editing with zero other apps, no sound, WiFi disabled and screen brightness at 50%. A few days ago I traveled into the city without a charger and managed to write quite a lot. To save battery, I first outline everything in a paper notebook, and when I feel ready, I commit to the laptop.

I know from experience how hard it is to maintain limitations like this in the long run. However beneficial, we always want things to be quantifiably better: longer hours, larger resolutions, bigger storage. I won’t plan to change my life around this approach. I don’t believe in anything like that anymore. It seemed to work this time, so I’ll ride the wave while I can.

Fundamental Attribution of Ability Error

Wed, 01 Jul 2020 18:29:00 +0200

There’s this thing called fundamental attribution error:

In social psychology, fundamental attribution error (FAE), also known as correspondence bias or attribution effect, is the tendency for people to under-emphasize situational explanations for an individual’s observed behavior while over-emphasizing dispositional and personality-based explanations for their behavior. This effect has been described as “the tendency to believe that what people do reflects who they are”

Basically, when you make a mistake — it’s due to some good reason, but when someone else makes a mistake — it’s because they are bad people who make mistakes.

I was thinking about this pattern of observing reality when reading other people’s blog posts describing life situations, technical setups, whatever. Let’s say I’m reading about some developer’s perfect programming environment setup. They share all the steps involved in creating this extremely complex system with automation, lots of inter-connected software, scripts and hardware addons. In the end, they conclude with how happy they are about the setup, how it helped their productivity and everything is peachy.

Inspired, I try to reproduce a portion of the described system, but quickly face the reality: things aren’t connecting as nicely as I expected, bugs and compatibility issues force me to write dirty hacks, and in the end the whole thing is just waiting to break.

I’m not doing this anymore nowadays. Every time I see a cool setup, be it OS, Terminal, Emacs, Vim or even some iOS automations, I just think how hard it is to maintain all of that and go to my Emacs config looking for something new to disable.

But for the longest time I seem to have been making a Fundamental Attribution of Ability Error. I was thinking that people’s setups are in fact perfect. That there’s something wrong with me, that my attempts to build a convoluted architecture fail because I’m a bad programmer. With experience and after talking to a lot of people, I realized how much of a facade many of us create when describing our creations.

No, your Emacs setup does not have “zero issues”. Sorry, I don’t believe you. It has issues and it breaks and you fix it sometimes. No, your Karabiner + Hammerspoon + AppleScripts + KeyboardMaestro setup with two hundred custom keyboard shortcuts is not “maintenance free”. No, your dotfiles repo with Ansible script does not guarantee a one-command-setup on a new machine.

As a result, I’m finding myself more and more conservative and pragmatic. ‘Vanilla’ and ‘default are my favorite words. I don’t get into zsh or fish, even though I understand the benefits over Bash. Bash is okay. My small config with a bunch of aliases had served me well. I love Doom Emacs, but I come back to my minimalist personal configuration with continuously shrinking number of customizations. I use Make and Bash-scripts for building.

Mythical “reproducible configuration” does not exist. I mean, it does, with scripts, Ansible, Docker or nixOS, sure, but then you have to constantly maintain the thing that creates maintenance-free environments. Is there a catch here?..

It's all just fashion shows

Fri, 26 Jun 2020 18:29:00 +0200

I always thought these fashion runway shows are kind of silly.

Are you guys okay?..

I just don’t want to deal with this stuff in any way. Luckily, I can ignore it for the most part. Although, random changes in “trendiness” affects my personal ability to buy a simple t-shirt or a hoodie without spending half an hour in a store (the last thing I want to do in life).

But I can’t isolate myself from these kind of fashion runway shows:

New macOS, new style of icons, new trend in UI design

This is better than before, right? RIGHT?!

It’s as if the industry forces me to throw away all my clothes and get used to new things, until it inevitably changes again in a few years. I don’t want to. Not because older icons are better, but because the change itself has very little value. The new UI style is not making my computing experience any better or faster or simpler. In fact, it’s making it worse: getting used to new design takes time. Who’s gonna pay for it?

The main benefactors of constant UI changes are the designers and product managers. I sometimes feel like we’re just donating our attention and energy so that they can keep their jobs. I know it sounds awful, and I am sorry, but this is how I feel. Collectively, we somehow decided this kind of taxation for social security is okay.

Now, these things could be esthetically pleasing and subjectively beautiful. That’s cool! Just like art. But again, I can ignore art and it doesn’t affect me or my ability to work. And I am not forced to support art and artists journeys.

Nothing really changed in desktop computing fundamentally since, heck, Windows 3. For 30 years we have icons, folders, windows, bars, menus and the mouse. Fundamentally, we just get new dresses every year.

At least the music is sometimes nice at these shows…

Weird Finnish elative case

Mon, 01 Jun 2020 18:29:00 +0200

Finnish language has 15 noun cases which pack a lot of expressive power. For example, let’s take the word “kissa” (cat). Here are some of the cases:

kissan — of cat (kissan koko; the size of the cat)
kissassa — in cat (hiiri on kissassa; the mouse is in the cat)
kissasta — from inside the cat (kakka tuli kissasta; poop came out of the cat)
kissana — as cat
kissatta — without cat

This third one is the elative case. It’s one of the locative cases which are used to indicate physical direction, along with inessive (kisassa).

So, you learn this stuff and in your mind the ending -sta becomes associated with “from”.

Apteekista — from the pharmacy
Maasta — from land
Talosta — from the house

And then you see this:

tästä syystä
for this reason

It means “for this reason”. Notice how both “this” and “reason” get the illative ending. This is something pretty alien to English speakers, but usual for languages such as Finnish or Russian (which are not related, but share this property). No matter how many words describe the “reason”, they all should be in the same case:

Tästä suuresta tärkeästä mielenkiintoisesta syystä
For this big important interesting reason.

Anyway, you look at “tästä syystä” and think, well, “from a reason” — it kinda makes sense. In English, “for a reason” is actually pretty weird. We’re used to it, but something like “by a reason” would make more sense than “for”. So, “from a reason” is actually alright.

But then you see this:

Pidän kissasta
I like the cat

Wait, what? Literally, “I like from cat”?! So, illative is used to indicate the object of an action (to like, to love, to hate, to know, etc). Why?!

And it goes deeper!

Puhu kissasta
To talk about the cat

“About” is the same as “from”!

This property is one of many weird things I find about Finnish as I learn this wonderfully strange language.

Summary of Truth about Types, a talk by Bartosz Milewski

Mon, 10 Feb 2020 18:29:00 +0200

Intro

Category theory abstracts many branches of math. You can describe topology, logic, Banach spaces and other concepts in terms of category theory. Why is it so? Does category theory describe the nature, the intrinsic reality, and we happened to discover it like physicists discover the laws of the universe?

The main idea of category theory is composition. This is why it is so relevant in programming. Composition is the essence of category theory.

Is composability intrinsic in nature? It seems like that on a larger scale, and classical physics shows how everything is neatly composable. The ancient idea of atoms is the idea of perfect composability: things are composed with indivisible elementary particles, and the higher you go, the more composable everything is. Atoms make molecules, molecules make objects, object make bigger objects and interact using laws that are themselves composable. Newtonian mechanics describes objects in the universe that follow the rules of interaction that are clearly affecting each other, acting like independent agents.

But the deeper you go, the less composable it becomes. Two quantum particles don’t interact in a simply composable fashion, their wave functions do not just affect each other in a stable manner. Two quantum particles near one another start to behave like a special two-particle state, not two wave functions. This is why quantum theory is so hard to describe and this is probably why physicists struggle with these theories. They go against our intuition of a composable universe.

So, maybe composability is not intrinsic in nature, maybe it is intrinsic for our limited monkey brains. We need composability in order to deal with the complex reality. Maybe, higher super-beings wouldn’t invent category theory at all.

Truth

In logic, there is an axiom that basically says “truth is true”, and you don’t need to prove it.

\begin{eqnarray}
\frac{}{\top true}\top_I
\end{eqnarray}

In type theory, it corresponds to unit type:

\begin{eqnarray}
():()
\end{eqnarray}

And in category theory it corresponds to the terminal object, which we will cover later.

Three things look different, but they are actually just different notations for the same concept.

As for proofs, in logic you’d prove a proposition A by providing some statements:

\begin{eqnarray}
\frac{[…]}{A}
\end{eqnarray}

In type theory, you’d prove that type A is inhabited (has members):

\begin{eqnarray}
\Gamma\vdash x : A
\end{eqnarray}

In the context of categories, a proof would be a morphism from terminal object:

\begin{eqnarray}
1\rightarrow A
\end{eqnarray}

Category theory

A category is a collection of objects and arrows.

Similar to a directed graph. Nodes are called objects (a, b, c,…). Arrows between objects are called morphisms: f :: a -> b.

Composability

Arrows are composable:

\begin{eqnarray}
f :: a \rightarrow b
\end{eqnarray}

\begin{eqnarray}
g :: b \rightarrow c
\end{eqnarray}

\begin{eqnarray}
g \circ f :: a \rightarrow c
\end{eqnarray}

The notation $g \circ f$ means “apply g after f”.

So, composition is associative.

Identity arrows

There always exist identity arrows that point to the same object:

\begin{eqnarray}
id_a :: a \rightarrow a
\end{eqnarray}

\begin{eqnarray}
id \circ f = f
\end{eqnarray}

\begin{eqnarray}
g \circ id = g
\end{eqnarray}

There could be multiple arrows from a to a, but at least one is identity.

Set

Set is an example of a category. Set is a category in which:

Objects are sets
Arrows are functions

In category theory, you cannot “look” inside objects, you only care about their connections. So, two sets are considered to be the same or different based on the arrows, but not the contents.

Universal constructions in category theory

Initial object

An initial object is an object that has a unique arrow to all other objects. Only one arrow per object.

In Set, the equivalent would be an empty set $\varnothing$. A function $varnothing \rightarrow a$ is a function that maps an element of empty set to any other element of other set. This doesn’t make sense: an element of empty set? But this is just an explanation in terms of set theory, and while it doesn’t make sense, we can think about this nevertheless, as of “objects defined by their arrow property”.

You can always create a function from an empty set, but you’ll never be able to call this function. In the abstract, in vacuum this is true.

In Haskell there’s Absurd function:

Absurd :: Void -> a

Which is a reference to “ad absurdum”: from falsehood, you can derive anythings. “If pigs can fly, then I am the president”.

Void here represents a type Void, an uninhabited type. This is not the same void-type we use in C/C++.

Terminal object

The inverse of initial object is the terminal object. An object with a unique arrow from all objects.

In set, it’s a singleton set. Unique function: for every element of set a return the single element of the singleton set. In Haskell, the unit function ignores the argument and returns the one element from the set. The unit type () has one element ().

Type:

unit :: a -> ()

Element of this type:

unit _ -> ()

If there are many singleton sets, they are isomorphic (i.e., a set of one apple is the same as a set of one orange).

Product

We need these universal constructions because they provide a way of defining things (since you cannot look inside the objects). One of the things that can be defined is the product.

c is a product of a and b
two arrows p and q (projections)
in Set: Cartesian product, pairs of elements
in logic: logical AND (conjuction elimination)

The only thing we know is that there are two morphisms. If we translate this into logic, it corresponds exactly to the elimination (AND). If a AND b is true, then a is true:

\begin{eqnarray}
\frac{a \wedge b}{a}
\end{eqnarray}

If a AND b is true, b is true:

\begin{eqnarray}
\frac{a \wedge b}{b}
\end{eqnarray}

This corresponds to morphisms like so:

c :: (a, b)
p(a, b) = a
q(a, b) = b

There can be many objects like c with such two projections. Only one could the product. Which one? We need to pinpoint the correct c that corresponds to the Cartesian product. We need a way to evaluate the instance of this pattern of projections.

If we can get all similar patterns in the category, we need a way to distinguish the “top” one, the one that matches the the best. That one would be the product.

Here we have two examples of such pattern:

c' is a candidate product and it has p' and q' projections. c is better if there is a unique arrow (morphism) m from c' to c that fulfils these conditions:

\begin{equation}
\begin{aligned}
m :: c’ \rightarrow c \\\
p’ = p \circ m \\\
q’ = q \circ m
\end{aligned}
\end{equation}

Both of these have the common factor m (but instead of multiplication you have composition). So they are not really elementary, since we can factor out a common m out of them. This makes c better (“more pure”).

This forms ranking among candidates. In some cases you can indeed use ranking to get the unique best option and that will be the product. Not all categories would have this.

Again, there is a connection to logic:

if a follows from c' (p')
and b follows from c' (q')
a ^ b (c) follows from c' (m)

Function object

Another universal construction.

What is type function in Haskell? It’s not really a morphism. Morphism is not an object. A set of morphisms between two objects corresponds to type a->b. Is there an object in the category that corresponds to the set of morphisms? If so, it’s called the Function Object.

In order to define the function object (in Haskell, it’s a->b, which is a type), you have to have products.

If you take a function a->b and apply argument a, you get b. Similar to product, if there is another candidate object z, that evaluates to the same b, and you can find unique morphism h then you can factor it out and prove that a->b candidate is the best.

This corresponds to modus ponens in proof theory.

\begin{eqnarray}
\frac{(a=>b) \wedge a}{b}
\end{eqnarray}

If you have a function from a to b and a is true, then b is also true.

Negation

Another thing you can construct is negation. Not A corresponds to an arrow (morphism) A -> Void. Why is this a negation?

In the context of type theory:

If A is inhabited (has elements), then A->Void is not inhabited. Because otherwise you’d be able to create an element of Void, which suppose to have no elements.
If A is not inhabited (is Void), then Void->Void is $id_{void}$.

Cartesian Closed Category

Combining these three things, we get a Cartesian Closed Category (CCC):

Has all products (Cartesian)
Has all function objects (exponentials) (this makes it closed)
Has terminal object (nullary product)

Curry-Howard-Lambek

This is an extension of Curry-Howard isomorphism. CCC is a model for simply typed lambda calculus. In addition to the fact that simply typed lambda calculus can be a model for logic.

Objects are types
Morphisms are terms (in logic/programming)
Environment $\Gamma$ is a prodict of judgements a:A
Empty environment is () : ()

Logical universes

There are two major kinds of logic: Classical and Intuitionistic.

In classical logic, any proposition is either true or false. But then Kurt Gödel comes along and says “no, there are some statements that are neither false nor true”, which leads to intuitionistic logic where a statement can be true, can be false, but you can also have some gray area where you ask what “is” means.

Intuitionistic logic

No LEM: law of excluded middle (either A or not A). A | (A->Void) is not provable.
We cannot eliminate double negation: Not Not A is not the same as A ((A->Void)->Void is not the same as A)

Curry-Howard equivalence says that simple typed lambda calculus is equivalent to intuitionistic logic. You cannot prove LEM or double negation using lambda calculus.

So, we cannot do classical logic? This means it’s not relevant to programming then?

Goedel Gentzen

Turns out, classical logic can be embedded in intuitionistic logic. Classical logic = Intuitionistic + double negation elimination (or LEM).

You can take a proposition in classical logic that is provable, you can translate it into some other proposition in intuitionistic logic, and then translate the proof also that does not use double negation elimination or LEM, and you will get something. There is a correspondence between propositions in two types of logic.

The transformation of proposition is simple: invert everything by applying double negation to all assumptions. Get a new theorem, prove it in intuitionistic logic, apply double negation to it, you get back the result of that first theorem.

Double twist, prove, double twist!

What does it mean? It means continuations. Double negation is (a->Void)->Void, or more general:

\begin{eqnarray}
(a\rightarrow r)\rightarrow r
\end{eqnarray}

This $(a\rightarrow r)\rightarrow r$ is a continuation. If you extend lambda calculus with continuation (or do CPS transformation), you have a way to embed classical logic into programming.

Yoneda’s Lemma

This is a deeper topic in category theory, and Bartosz mentions it without explanation just to show the importance of continuation in category theory. Read more about Yoneda’s lemma on Wikipedia.

Conclusions

Type theory (typed lambda calculus), category theory (cartesian closed) and logic are all the same, just different notations. There is a lot of cross-polination between these areas. Is the Grand Unified Theory coming soon? Homotopy Type Theory?

Maybe.

This is a complete summary of an excellent talk by Bartosz Milewski Truth about types. Illustrations and diagrams are recreated; formulas taken verbatim from the slides, except for comments, which were extended in some places.

(You can get a set of PDF/HTML/epub/Kindle versions below. Name your price, starting from $1.)

Get set

Summary of Transducers, a talk by Rich Hickey

Fri, 31 Jan 2020 14:39:00 +0200

Intro

What are transducers? The basic idea is to extract the essence of map, filter and other functions that transform sequences and collections, and reuse this essence so that it can be applied elsewhere; to recast them as process transformations.

The kind of process for which we can use transducers is a succession of steps, where each step is ingesting an input. Building a collection is just one example of a process of that shape. But let us not focus on the idea of a process which performs building. Some processes build particular things, other processes are infinite.

Why it’s called transducer? It’s a real world and it’s better than reduce or ingest, which could be considered.

reduce: lead back, brings something back
ingest: carry into
transduce: lead across; as inputs are carried into the process, they are led through a series of transformations

Transducers are not a programming thing. We have them in real life, where we usually call them “instructions”. For example, consider the instruction to baggage carriers “put the baggage on the plane”. While you’re doing it:

break apart pallets
remove bags that smell like food
label heavy bags

There could be different circumstances (conveyances, sources, sinks). They are irrelevant. Does baggage come and go on trolleys or conveyor belts? We don’t care and we don’t want to care. This is the real world: unspecified and flexible.

In programming we have something like this — collection function composition:

(comp
 (partial mapcat unbundle-pallet)
 (partial filter non-food?)
 (partial map label-heavy))

There’s a big difference between real world and this programming approach. map, filter, mapcat are functions of sequence → sequence. We model the rules, but they only work on sequences. When we get something new, like stream, channel or observable, none of the rules apply.

Another problem is that this code creates intermediate sequences between steps. It’s as if we told the baggage guys to take the baggage off the trolley, unbundle, put bags on another trolley, check for food smell, put bags on another trolley, etc.

There is no reuse. Every new collection or a process defines its own version of map, filter, mapcat etc. As a result, composed algorithms are needlessly specific and inefficient.

Creating and using transducers

Let’s start with the value propositions right away. This is what we want to achieve. First, we create a transducer process-bags, which is a composition of three transducers.

(def process-bags
 (comp
 (mapcatting unbundle-pallet)
 (filtering non-food?)
 (mapping label-heavy)))

mapcatting, filtering and mapping return transducers. process-bags is a transducer. Transducers modify a process by transforming its reducing function. Once done, we can go to a completely different context and reuse them.

Building a concrete collection. Standard function into now has another arity: it can accept a transducer to generate a collection out of something else.

(into airplane process-bags pallets)

sequence can now take a transducer as well and generate a lazy sequence of values.

(sequence process-bags pallets)

A new function transduce works like reduce, but takes a transducer, operation, initial value and the source. Here we count the total weight of the bags:

(transduce
;; transducer
;; ↓
 (comp process-bags (mapping weigh-bag))
;; operation
;; ↓
 + 0 pallets)
;; ↑ ↑
;; | source
;; initial value

This also should work for channels, observables and other contexts:

;; a CSP channel that processes bags
(chan 1 process-bags)

;; it's an open system
(observable process-bags pallet-source)

We call processes that can work with transducers (like into, sequence, chan, etc) transducible processes. They accept a transducer and use it to transform whatever they do. So, the notion of a transducer has two parts:

Functions that create transducers.
In context where they make sense, you accept transducers.

Each process takes a transducer and their internal processing function. What’s the internal processing function of into? It’s conj (in Clojure). Channels are inherently the same, they take input and somewhere inside they have a step function: add the item to the buffer. That step function has the same shape as conj. So, into and channel can transform their internal operations with the help of transducers.

Two papers which describe this idea are:

Lectures on Constructive Functional Programming, by Richard S. Bird
A tutorial on the universality and expressiveness of fold, Graham Hutton

How do we get there?

Let’s now discuss how can we achieve this. We’ll start by re-defining map, filter and mapcat.

One of the things discussed in the first mentioned paper is the relationship between list processing operations and fold. There’s some interesting math that proves the equivalency of lists and operations that construct them. Map and filer can be defined via foldr, which encapsulates recursion and makes it easier to reason about and transform data:

(defn mapr [f coll]
 (foldr (fn [x r] (cons (f x) r))
 () coll))

(defn filterr [pred coll]
 (foldr (fn [x r] (if (pred x) (cons x r) r))
 () coll))

You can similarly re-define these functions in terms of foldl, which is just left reduce. Right reduce implies laziness, right reduce implies loops. It turns up that the loop path is faster, better and more general for the goals we try to achieve.

(defn mapl [f coll]
 (reduce (fn [r x] (conj r (f x)))
 [] coll))

(defn filterl [pred coll]
 (reduce (fn [r x] (if (pred x) (conj r x) r))
 [] coll))

(defn mapcatl [f coll]
 (reduce (fn [r x] (reduce conj r (f x)))
 [] coll))

In both left and right cases, it’s mostly boilerplate, and the fundamental thing all functions do is the same: here’s the thing that to do to all the things that come through. The problem here is conj. It is a specific operation in the middle of those generic ideas of map, filter and mapcat. conj might not be what we want to do.

To turn those inner functions into transducers we need to add another layer, and allow passing arbitrary step operation instead of hard-coding conj.

(defn mapping [f]
 (fn [step]
 (fn [r x] (step r (f x)))))

(defn filtering [pred]
 (fn [step]
 (fn [r x] (if (pred x) (step r x) r))))

(def cat
 (fn [step]
 (fn [r x] (reduce step r x))))

(defn mapcatting [f]
 (comp (map f) cat ))

Now mapping takes a function and returns a function which expects a step function as argument. We can now pass conj and produce the reduce-based map, filter and mapcat:

(defn mapl [f coll]
 (reduce ((mapping f) conj)
 [] col))

(defn filterl [pred coll]
 (reduce ((filtering pred) conj)
 [] coll))

(defn mapcatl [f coll]
 (reduce ((mapcatting f) conj)
 [] coll))

That was the whole point. Transducers are fully decoupled. They know nothing about the process they modify.

Transducers are fast

One benefit of transducers is speed. Since they are just a stack of function calls, there’s no laziness overhead, no intermediate collections, no extra “boxes”. There’s no notion of “empty list is nothing”. Nothing is nothing, an empty list is an empty list.

Transducer types

Transducers raise interesting problems in the context of type theory.

If you’re trying to produce the next process N, you must supply the result of step N-1 as input. Thus, modeling the type system as R→R would be wrong. We’d need something like this:

Also, the black boxes that represent the elements are unnecessarily restrictive by type in this image. We could create a step function that when input is x the output is y, when input is y the output is z, when input is z the output is x. It’s a perfectly fine state machine, but it’d be tricky to figure out the type system for that. {x,y,z}→{x,y,z} is not the answer, because for input x the output must be only y, not x or y or z.

Early termination

Ordinary reduction processes everything. Sometimes a process has a reason to terminate early, for whatever reason. It maybe the desire of the process itself, or a decision of one of the steps, or an external trigger.

Continuing with the baggage handling analogy, let’s say if the bag is ticking — that’s it, go home.

(comp
 (mapcatting unbundle-pallet)
 (taking-while non-ticking?)
 (filtering non-food?)
 (mapping label-heavy))

taking-while must stop the whole job. How to do that? Turns out in Clojure reduce can already do that via (reduced result). It’s a wrapper with a corresponding test reduced?. Once wrapped, we can dereference the value with deref:

(reduced? (reduced x)) ;; → true
(deref (reduced x)) ;; → x

Transducers support reduced. That means that step functions can return (reduced value). If a transducer gets a reduced value from a nested step call, it must never call that step function again with input.

(defn taking-while [pred]
 (fn [step]
 (fn [r x]
 (if (pred x)
 (step r x)
 (reduced r)))))

All reducing processes must also support reduced. If the step function returns a reduced value, the process must not supply any more input to the step function. The dereferenced value is the final accumulated value.

To illustrate this improvement in the context of types, we’d need to build something like this:

Vertical bar represents OR. I.e. a process returns a black box or a reduced version of it.

State

Some interesting sequence functions require state (e.g. take, partition-*). They might count stuff or accumulate a sum, for example. Where would that data go? In purely functional approach the stack or laziness can be used to capture state. The idea of transducers is to isolate the execution completely, so we can’t use that approach.

State has to be explicit. It must be incorporated into the transducer object. They must create unique state every time they are called upon to transform a step function. Thus, once applied to a process, a transducer yields another (potentially stateful) process.

You should always treat an applied transducer stack as if it returned a stateful process, because you don’t know until the end whether there’ll be state. This means you can’t alias the process returned by a transducer.

Here’s an example of a stateful transducer:

(defn dropping-while [pred]
 (fn [step]
 (let [dv (volatile! true)]
 (fn [r x]
 (let [drop? @dv]
 (if (and drop? (pred x))
 r
 (do
 (vreset! dv false)
 (step r x))))))))

That’s not the prettiest thing…

Completion

Some jobs don’t complete, they don’t have ends, they don’t consume finite collections. They might accept things from a channel or a event source.

But other jobs might complete (stop receiving input). A process might want to do a final transformation of the value built up. A stateful transducer might want to flush a pending value. So, all step functions must have an arity-1 variant that does not take an input.

If the process has finished, it must call the completion operation exactly once. Each transducer must do the same thing and call its nested completion operation. It may flush state using the nested step function prior to calling nested complete.

Coming back to the visual types, we now have to think of reducing function as a pair of operations. The first takes no input — that’s the completion operation. The second is the regular step operation.

Init

There’s the third type of operation that’s associated with processing in general: init. A reducing function may support arity-0 which returns an initial accumulation value. Transducers must support arity-0 in terms of a call to the nested init.

(+) ;; → 0
(+ 21) ;; → 21
(*) ;; → 1

We now have three operations, one of which is init from nothing.

In Clojure we just define functions with arity 0, 1, 2. Transducers take and return reducing functions. Now, core sequence functions’ 1-arity variant return a transducer. We decided not to call those things mapping, filtering, etc (as this may not carry very well into languages other than English). So, to create a transducer, call a function without the collection:

(mapping f) == (map f)

This applies to core sequence functions:

map
mapcat
filter
remove
take
take-while
drop
drop-while
take-nth
replace
partition-by
partition-all
keep
keep-indexed
cat
dedupe
etc…

Here’s the final example: filter returning a transducer. It takes a predicate and returns a step modifying function, which takes a reducing function which presumably has # arities, and defines a function with three arities:

init
complete, which in case of filter just returns the result
the result of input, which is what regular filter does

(defn filter
 ([pred]
 (fn [rf]
 (fn
 ([] (rf))
 ([result] (rf result))
 ([result input]
 (if (pred input)
 (rf result input)
 result)))))
 ([pred coll]
 (sequence (filter pred) coll)))

Summary

Transducers:

support early termination and completion
are context-independent
reusable across a wide variety of contexts
composable via ordinary function composition
efficient

This is a complete summary of an excellent talk by Rich Hickey “Transducers”. Illustrations and diagrams are recreated; source code taken verbatim from the slides, except for comments, which were extended in some places.

(You can get a set of PDF/HTML/epub/Kindle versions below. Name your price, starting from $1.)

Get set

Fastmail setup with Emacs, mu4e and mbsync on macOS

Thu, 30 Jan 2020 13:09:00 +0200

I guess it was inevitable. Once you embrace Emacs, at some point you gonna want to do email in it. Honestly, I don’t think I’ll stick with it, but as an experiment, I want to try and see whether it makes sense to use Emacs as an email client.

These are my requirements:

Fastmail as email provider. It’s a solid service with normal IMAP (unlike Gmail with their weird, non-standard implementation which makes it very hard to use anything other than their web interface). I moved from Gmail to Fastmail almost two years ago and I highly recommend the service. You can use my referral code to get 10% off.
No obligations. I want to be able to use whatever third party email client or continue using Fastmail’s web interface without any caveats. This means I don’t want to setup something in Emacs that will only work there (like custom tagging which doesn’t propagate to the server).
Fast and simple. I dislike elaborate, finicky setups and slowdowns.

The setup consists of the following parts:

Fastmail IMAP server. This doesn’t require any special configuration.
mbsync — a command-line utility which syncs IMAP server with a local directory in Maildir format. This has nothing to do with Emacs. Even if you’re not interested in doing email in Emacs, having a full local copy of your mailbox may be a good idea. mbsync does exactly that. In can sync both ways, so if I just move a file (which corresponds to a single message) from folder to folder, and then run mbsync again, this change will propagate to the server.
mu — a command line email client which works with Maildir storage. It doesn’t deal with the server directly, instead, it just read your local file system. It’s very fast!
mu4e — Emacs package that comes with mu. Provides an interface to mu.

Installing and configuring mbsync

Install mbsync with homebrew (the package is called isync, but the binary is called mbsync):

brew install isync

Mbsync uses ~/.mbsyncrc for configuration. This page describes all options. Here’s my config, with comments:

# First section: remote IMAP account
IMAPAccount fastmail
Host imap.fastmail.com
Port 993
User myemailaddress@mydomain.com
# For simplicity, this is how to read the password from another file.
# For better security you should use GPG https://gnupg.org/
PassCmd "cat ~/.mbsync-fastmail"
SSLType IMAPS
SSLVersions TLSv1.2

IMAPStore fastmail-remote
Account fastmail

# This section describes the local storage
MaildirStore fastmail-local
Path ~/Maildir/
Inbox ~/Maildir/INBOX
# The SubFolders option allows to represent all
# IMAP subfolders as local subfolders
SubFolders Verbatim

# This section a "channel", a connection between remote and local
Channel fastmail
Master :fastmail-remote:
Slave :fastmail-local:
Patterns *
Expunge None
CopyArrivalDate yes
Sync All
Create Slave
SyncState *

Few details about the channel options worth mentioning:

Patterns * — sync all folders. Alternatively, you can select only certain folders to sync.
Expunge None — don’t destroy messages neither locally, nor remotely. Details later.
CopyArrivalDate — makes sure the date of the arrival stays the same when you move messages around. Without this option, moving a message to another folder will reset the date of the message.
Create Slave — when new folders are added on server, create them locally.

Now run mbsync and wait for it to download messages:

mbsync -a

For me, around 50k messages were synced in a few minutes.

Installing and configuring mu and mu4e

I use Emacs port for macOS, which you can install like so:

brew tap railwaycat/emacsmacport
brew cask install emacs-mac

Once installed, verify that which emacs points to a valid Emacs executable, and emacs --version shows the correct version:

→ which emacs
/usr/local/bin/emacs

→ ls -la /usr/local/bin/emacs
/usr/local/bin/emacs -> /Applications/Emacs.app/Contents/MacOS/Emacs.sh

→ emacs --version
GNU Emacs 26.3

Now, install mu:

brew install mu

And let it index the Maildir:

mu index --maildir=~/Maildir

Now you can check if everything worked by trying a command-line search:

mu find hello

mu comes with mu4e by default. To verify, check for presence of elisp files in /usr/local/share/emacs/site-lisp/mu/mu4e:

→ ls /usr/local/share/emacs/site-lisp/mu/mu4e
mu4e-actions.el mu4e-contrib.elc mu4e-main.el ...

Now load these files in Emacs and enable mu4e. Put the following in your Emacs config:

(add-to-list 'load-path "/usr/local/share/emacs/site-lisp/mu/mu4e")
(require 'mu4e)

And add some configuration.

(setq
 mue4e-headers-skip-duplicates t
 mu4e-view-show-images t
 mu4e-view-show-addresses t
 mu4e-compose-format-flowed nil
 mu4e-date-format "%y/%m/%d"
 mu4e-headers-date-format "%Y/%m/%d"
 mu4e-change-filenames-when-moving t
 mu4e-attachments-dir "~/Downloads"

 mu4e-maildir "~/Maildir" ;; top-level Maildir
 ;; note that these folders below must start with /
 ;; the paths are relative to maildir root
 mu4e-refile-folder "/Archive"
 mu4e-sent-folder "/Sent"
 mu4e-drafts-folder "/Drafts"
 mu4e-trash-folder "/Trash")

;; this setting allows to re-sync and re-index mail
;; by pressing U
(setq mu4e-get-mail-command "mbsync -a")

That’s it! Run M-x mu4e and after a few final setup questions mu4e should be running. Check out keybindings for it.

Caveat 1: deletion vs. expunge

By default, when you mark a message to be deleted, mu4e will apply the “Trashed” flag. Fastmail automatically destroys the messages flagged this way, as per IMAP standard. Unfortunately, there is no way to stop Fastmail from doing that.

Instead of total deletion, I want to move messages to the “Trash” folder. I can simply use “move” command of mu4e, but it’d be nicer to use d button (deletion) for that. The following piece of elisp remaps the d button to “move to Trash folder” action. This way, neither mu4e nor Fastmail destroy the message.

(fset 'my-move-to-trash "mTrash")
(define-key mu4e-headers-mode-map (kbd "d") 'my-move-to-trash)
(define-key mu4e-view-mode-map (kbd "d") 'my-move-to-trash)

Caveat 2: Spam reporting

I was worried about not being able to report spam messages back to Fastmail. Turns out it wasn’t an issue: in Fastmail settings, you can turn on “Spam learning” for a folder. Fastmail will scan a folder daily and learn any new messages as spam. So, I don’t need to explicitly report spam, all I need is to move spammy messages to “Spam” folder.

Sending mail

mu4e re-uses Gnus’ message-mode for writing mail and inherits its configuration. For sending via SMTP, mu4e uses smtpmail. Minimal configuration looks like this:

(setq
 message-send-mail-function 'smtpmail-send-it
 smtpmail-default-smtp-server "smtp.fastmail.com"
 smtpmail-smtp-server "smtp.fastmail.com")

When using SMTP for the first time, Emacs prompts you for the user name and password to use, and then offers to save the information. By default, Emacs stores authentication information in a file ~/.authinfo.

Summary of Concurrency Is Not Parallellism, a talk by Rob Pike

Fri, 20 Dec 2019 16:25:00 +0200

This is a complete summary of an excellent talk by Rob Pike “Concurrency is Not Parallelism”. Illustrations and diagrams are recreated; source code taken verbatim from the slides, except for comments, which were extended in some places.

(You can get a set of PDF (preview)/HTML/epub/Kindle versions below. Name your price, starting from $1.)

Get set

Intro

The world is parallel: starting from the computing fundamentals, such as multi-core CPUs, and all the way to real life objects, people, planets and the Universe as a whole — everything is happening simultaneously. Yet, the computing tools that we have aren’t good at expressing this world view. We can rectify this by exploring concurrency.

Go is a concurrent language. It makes it easy to design concurrent systems by providing the ability to:

execute things concurrently
communicate between concurrently running processes

There’s a misconception about Go and concurrency: many programmers believe concurrency and parallelism are the same thing. They are not, and this talk will try to answer why.

Concurrency vs parallelism

Here’s the core of the distinction:

Concurrency is composition of independently executing things (typically, functions). We often use the word ‘process’ to refer to such running thing, and we don’t mean ‘unix process’, but rather a process in the abstract, general sense.

Parallelism is simultaneous execution of multiple things. Those things might or might not be related to each other.

Concurrency is about dealing with a lot of things at once. Parallelism is about doing a lot of things at once. The ideas are, obviously, related, but one is inherently associated with structure, the other is associated with execution. Concurrency is structuring things in a way that might allow parallelism to actually execute them simultaneously. But parallelism is not the goal of concurrency. The goal of concurrency is good structure.

Analogy

The operating system manages multiple devices at the same time (disk, screen, keyboard, etc). They are somewhat independent and completely concurrent concerns. However, they aren’t necessarily parallel: if the computer has only one core, several things can’t possibly run simultaneously. The model here is concurrent, it is structured as a system of concurrent processes. Its reality could be parallel, depending on circumstances.

Compare this to performing matrix multiplication on a powerful GPU which contains hundreds or thousands of cores. Both the underlying idea and the reality are parallel, it’s all about running operations at the same physical time.

Concurrency allows to structure the system in such a way that enables possible parallelism, but requires communication. Tony Hoare has written “Communicating sequential processes” (https://www.cs.cmu.edu/~crary/819-f09/Hoare78.pdf) in 1978, where he describes problems and techniques of dealing with these issues. It is the greatest paper in computer science and we highly recommend every programmer to read it. Programming languages like Erlang and Go are largely based on ideas described in it.

Concurrent composition

There’s a pile of books we need to burn. We have a gopher whose job is to move books from the pile to the incinerator.

One gopher is slow, so let’s add another gopher. But now we need to synchronize them, since they might bump into each other, or get stuck at either side. One way to solve this is to make them communicate with each other by sending messages (like, “I’m at the pile now” or “I’m on my way to the incinerator”).

How can we go faster? Double everything! Two piles of books, two incinerators! Consumption and burning can be twice as fast now. That’s parallel.

But try to think about it as the composition of two gopher processes. We start with a single process, and then just introduce another instance of the same process. This is called concurrent composition.

While not immediately obvious, concurrent composition is not automatically parallel! It’s possible that only one gopher moves at a time. The design is still concurrent, but not parallel. This is similar to the OS example on a single core processor, where two concurrent things might not run in parallel due to technical limitations.

However, concurrent composition is automatically parallelizable.

Different design

Let’s try another approach. There will be three gophers in total:

One only loads the cart.
Another runs the cart to and from the incinerator.
The third unloads the cart.

Each gopher is an independently executing procedure.

This approach is probably faster, although, not by much. There’ll definitely be problems like blocking, unnecessary waiting when the books are being loaded and unloaded, the time when the 2nd gopher runs back and nothing useful is happening, etc. Let’s add another gopher!

Finer-grained concurrency

Now there’s a 4th gopher who returns the empty cart

This version of the problem will work better than the previous version, even though we’re doing more work. Concurrent composition of better managed pieces can run faster. In the perfect situation, with all settings optimal (number of books, timing, distance), this approach can be 4 times faster than the original version.

This is important! We improved the performance of this program by adding a concurrent procedure to existing design. We added more things and it got faster! The reason it can run faster is that it can be parallel, and the reason it can be parallel is better concurrent design.

So, we have four distinct gopher procedures:

Load books onto cart.
Move cart to incinerator.
Unload cart into incinerator.
Return empty cart.

Think of them as of independent procedures, running on their own, and we compose them in parallel to construct the solution. We can make it more parallel by, well, parallellizing the whole thing:

Note what we’re doing here: we have a well composed system which we then parallelize on a different axis to, hopefully, achieve better throughput. We understand the composition and have control over the pieces.

And what if gophers can’t run simultaneously (back into the single core world)? No problem, really. Only one gopher runs at a time, and 7 others are idle. The system runs as fast as a single gopher and the overall speed is the same as the first solution. But the design is concurrent, and it is correct. This means we don’t have to worry about parallelism if we do concurrency right. Parallelism is optional.

Yet another design

Two gophers with a staging dump in the middle.

Two similar gopher procedures running concurrently. In theory, this could be twice as fast. As before, we can parallelize it and have two piles with two staging dumps.

Or try a different design still: 4 gopher approach with a single staging dump in the middle.

And then double that! 16 gophers, very high throughput.

Obviously, this is very simplistic and silly. But conceptually this is how you think about problems: don’t think about parallel execution, think about breaking down the problem into independent components, and then compose in a concurrent manner.

Summary

There are many ways to break the process down. You can easily come up with a dozen more structures. That is concurrent design. Once we have the breakdown, parallelization can fall out and correctness is easy to achieve. The design is intrinsically safe.

Real world example

This gophers example might look silly, but change books to web content, gophers to CPUs, carts to networking and incinerators to a web browser, and you have a web service architecture.

Let’s learn a little bit of Go.

Goroutines

If we run a regular function, we must wait until it ends executing. But if you put a keyword go in front of the call, the function starts running independently and you can do other things right away, at least conceptually. Not necessarily, remember: concurrent is not the same as parallel.

f("Hello") // f runs, we wait

go f("Hello") // f starts running
g() // we don't wait for f to return

(This is similar to running a background shell process with &).

It is common to create thousands of goroutines in one Go program. There could be millions! Goroutines aren’t free, but they’re very cheap.

Channels

Under the hood, goroutines are like threads, but they aren’t OS threads. They are much cheaper, so feel free to create them as you need. They are multiplexed onto OS threads dynamically, and if one goroutine does stop and wait (for example, for input/output operation), no other goroutines are blocked because of that.

To communicate between goroutines we use channels. They allow goroutines exchange information and sync.

Here’s an example. We create a timerChan channel of time.Time values (channels are typed). Then we define and run a function func which sleeps for some time deltaT and sends current time to the channel. Then, some time later, we receive a value from the channel. This receiving is blocked until there’s a value. In the end, completedAt will store the time when func finished.

timerChan := make(chan time.Time)
go func() {
 time.Sleep(deltaT)
 timerChan <- time.Now() // send time on timerChan
}()
// Do something else; when ready, receive.
// Receive will block until timerChan delivers.
// Value sent is other goroutine's completion time.
completedAt := <-timerChan

Select

Goroutines and channels are the fundamental building blocks of concurrent design in Go. The last piece is the select statement. It is similar to a simple switch, but the decision is based on ability to communicate instead of equality.

The following example produces one of three outputs:

If channel ch1 is ready (has a value), first case executes.
If channel ch2 is ready (has a value), second case executes.
If neither is ready, the default case executes.

select {
case v := <-ch1:
 fmt.Println("channel 1 sends", v)
case v := <-ch2:
 fmt.Println("channel 2 sends", v)
default: // optional
 fmt.Println("neither channel was ready")
}

If the default clause is not specified in the select, then the program waits for a channel to be ready. If both ready at the same time, the system picks one randomly.

Closures

Go supports closures, which makes some concurrent calculations easier to express. Closures work as you’d expect. Here’s a non-concurrent example:

func Compose(f, g func(x float) float)
 func(x float) float {
 return func(x float) float {
 return f(g(x))
 }
}

print(Compose(sin, cos)(0.5))

Examples

Launching daemons

Here we use a closure to wrap a background operation without waiting for it. The task is to deliver input to output without waiting. The following code copies items from the input channel to the output channel.

go func() { // copy input to output
 for val := range input {
 output <- val
 }
}()

The for range runs until the channel is drained (i.e. until there are no more values in it).

Simple load balancer

You have some jobs. Let’s abstract them away with a notion of a unit of work:

type Work struct {
 x, y, z int
}

A worker task has to compute something based on one unit of work. It accepts two arguments: a channel to get work from and a channel to output results to. It then loops over all values of the in channel, does some calculations, sleeps for some time and delivers the result to the out channel.

func worker(in <-chan *Work, out chan<- *Work) {
 for w := range in {
 w.z = w.x * w.y
 Sleep(w.z)
 out <- w
 }
}

Because of arbitrary sleeping time and blocking, a solution might feel daunting, but it is rather simple in Go. All we need to do is to create two channels (in, out) of jobs, call however many worker goroutines we need, then run another goroutine (sendLotsOfWork) which generates jobs and, finally run a regular function which receives the results in the order they arrive.

func Run() {
 in, out := make(chan *Work), make(chan *Work)
 for i := 0; i < NumWorkers; i++ {
 go worker(in, out)
 }

 go sendLotsOfWork(in)
 receiveLotsOfResults(out)
}

This solutions works correctly whether there is parallization or not. It is implicitly parallel and scalable. The tools of concurrency make it almost trivial to build a safe, working, scalable, parallel design. There are no locks, mutexes, semaphores or other “classical” tools of concurrency. No explicit synchronization!

Another load balancer

The load balancer needs to distribute incoming work between workers in an efficient way. The requester sends Requests to the balancer:

type Request struct {
 fn func() int // The operation to perform.
 c chan int // The channel to return the result.
}

Note that the request contains a channel. Since channels are first-class values in Go, they can be passed around, so each request provides its own channel into which the result should be returned.

Now the requester function. It accepts a work channel of Requests. It generates a channel c which is going to get inside the request. It sleeps for some time. Then it sends on the work channel a request object with some function and channel c. It then waits for the answer, which should appear in channel c, and does some further work.

func requester(work chan<- Request) {
 c := make(chan int)
 for {
 // Kill some time (fake load).
 Sleep(rand.Int63n(nWorker * 2 * Second))
 work <- Request{workFn, c} // send request
 result := <-c // wait for answer
 furtherProcess(result)
 }
}

Now, the worker which accepts Requests is defined by three things:

The channel of Requests. This is a per-worker queue of work to do.
Number of pending tasks (the load).
An index.

type Worker struct {
 requests chan Request // work to do (buffered channel)
 pending int // count of pending tasks
 index int // index in the heap
}

This is what the worker does:

func (w *Worker) work(done chan *Worker) {
 for {
 req := <-w.requests // get Request from balancer
 req.c <- req.fn() // call fn and send result
 done <- w // we've finished this request
 }
}

Balancer sends requests to most lightly loaded worker. The channel of requests (w.requests) delivers requests to each worker. The balancer tracks the number of pending requests. Each response goes directly to its requester. Once that is done, the balancer is out of the picture, because each worker communicates with its request via a unique channel.

Balancer is defined by a pool of workers and a single done channel through which the workers are going to tell the balancer about each completed request.

type Pool []*Worker

type Balancer struct {
 pool Pool
 done chan *Worker
}

This is what the balancer does:

func (b *Balancer) balance(work chan Request) {
 for {
 select {
 case req := <-work: // received a Request...
 b.dispatch(req) // ...so send it to a Worker
 case w := <-b.done: // a worker has finished ...
 b.completed(w) // ...so update its info
 }
 }
}

It runs an infinite loop, forever checking whether there’s more work to do (i.e. there’s an item on the work channel), or there’s a finished task (i.e. there’s an item on the done channel). If there’s work, dispatch it to a worker. If a job is done, update its info.

To allow the balancer to find the lightest loaded worker, we construct a heap of channels and providing methods such as:

func (p Pool) Less(i, j int) bool {
 return p[i].pending < p[j].pending
}

We’re ready to implement the dispatcher:

// Send Request to worker
func (b *Balancer) dispatch(req Request) {
 // Grab the least loaded worker...
 w := heap.Pop(&b.pool).(*Worker)
 // ...send it the task.
 w.requests <- req
 // One more in its work queue.
 w.pending++
 // Put it into its place on the heap.
 heap.Push(&b.pool, w)
}

All it needs to do is:

Grab the least loaded worker off the heap.
Send it the task by writing into its requests channel.
Increment its load counter.
Push it back into the heap.

That’s it!

The final piece is the completed function which is called every time a worker finishes processing a request. It’s essentially the inverse of dispatch:

// Job is complete; update heap
func (b *Balancer) completed(w *Worker) {
 // One fewer in the queue.
 w.pending--
 // Remove it from heap.
 heap.Remove(&b.pool, w.index)
 // Put it into its place on the heap.
 heap.Push(&b.pool, w)
}

Lessons

A complex problem can be broken down into easy-to-understand components.
The pieces can be composed concurrently.
The result is easy to understand, efficient, scalable, and correct.
The result is maybe even parallel.

One more example: query a replicated DB

Imagine you have a replicated database (multiple shards). You send the request to all instances, but pick the one response that’s first to arrive.

The function accepts an array of connections and the query to execute. It creates a buffered channel of Result, limited to the number of connections. Then it runs over all connections and starts a goroutine for each channel. Goroutine delivers the query, waits for response and delivers the answer to ch. After they all are launched, the function just returns the first value on the channel as soon as it appears.

func Query(conns []Conn, query string) Result {
 ch := make(chan Result, len(conns)) // buffered
 for _, conn := range conns {
 go func(c Conn) {
 ch <- c.DoQuery(query):
 }(conn)
 }
 return <-ch
}

(Note that _ on line 3 stands for an unused, unnamed variable).

Conclusion

Concurrency is powerful.
Concurrency is not parallelism.
Concurrency enables parallelism.
Concurrency makes parallelism (and scaling and everything else) easy.

More information

Go official page: golang.org
Some history: Bell Labs and CSP Threads
A previous talk by Rob Pike: Advanced Topics in Programming Languages: Concurrency/message passing Newsqueak
Parallelism Is Not Concurrency (article by Robert Harper)
A Concurrent Window System (paper by Rob Pike)
Squinting at Power Series (paper by M. Douglas McIlroy)
Interpreting the Data: Parallel Analysis with Sawzall

If you liked this illustrated summary, consider supporting me by purchasing a set of PDF (preview), HTML epub and Kindle versions in one nice package. Name your price, starting from $1.

Get set

The price of complexity

Wed, 20 Nov 2019 12:48:00 +0200

Computer programmers often talk about tackling complexity, yet they thrive on complexity. I believe tech people experience a constant dilemma: on one hand, we want things to be simple and straightforward; on the other hand we love complex structures and engineering marvels.

I think about this today as I’m performing some cleanup work on my blog. It runs on Hugo, content is written in Org mode, code is published on Github and the final website is deployed to Netlify. That’s a lot of moving parts, and, honestly, it feels excessive. Yet I love this setup.

Lately I’ve been trying to be conscious and mindful about the price my mind pays for all this complexity. I’m not a good programmer by any means, so your mileage may vary, but it takes an enormous amount of mental energy for me to re-understand something I already figured out before. Take Hugo for example, a flexible and powerful static website generator. Jekyll, which I used before Hugo, is complex, too, but Hugo drives me crazy sometimes. It’s a multi-layered system of interconnected logic and it took me a whole day to move from Jekyll.

Intermission: I just spent 5 minutes trying to create a relative hyperlink to that other blog post, and couldn’t. It took me a while to realize I’m in Org mode now, not in Markdown, my syntax was wrong. You know what a Wordpress or Ghost user would’ve done? Clicked a link button in their rich WYSIWYG editor and had finished the blog post by this time already.

Every time I need to make some changes to the setup — fix layouts, add pages, refactor templates — I feel completely lost. It happens rare enough for my brain to forget the structure and conventions. And this is the case for a dozen of software projects I touch throughout the year.

This feeling of being lost is similar to un-pausing a video game that was on pause for 6 months. I know I’ve been into this, but right now I don’t even know what buttons to press.

There are ways to fight this. One is to dramatically reduce complexity in the first place, maybe even sacrifice some of the features. My setup can be technically replaced by a bunch of HTML files, for example. Or switch to a “normal” thing like Ghost or Wordpress. Oh, and don’t host them yourself, but pay someone to take care of it.

Another way is to somehow capture the knowledge for easy retrieval. My problem with Hugo is that I rarely touch it, so I forget. I should at least add a README file for myself, explaining the current setup and structure. Keeping documentation in sync with code is another problem, sigh…

So far, I only know one good way of solving this: teach. I should just make a course about Hugo on Codexpanse, that’ll force me to really understand it and devise a good mental model.

User Is Dead

Tue, 22 Oct 2019 10:51:00 +0300

User is dead. User remains dead. And we have killed him. How shall we comfort ourselves, the developers, the designers, the growth hackers? What was holiest and the final judge of all that the world has yet owned has bled to death under our a/b-tests and new features. Who will wipe this blood off us? What garbage collector is there for us to clean ourselves? What conference of atonement, what disruptive technology, what sacred meeting shall we have to invent? Is not the greatness of this deed too great for us? Must we ourselves not become users simply to appear worthy of it?

By saying “God is dead” Friedrich Nietzsche tried to express the fear that the decline of religion and the rise of nihilism would plunge the world into chaos.

Whether you believe that people require an external source of morality, or you accept the numerous philosophical and scientific arguments for intrinsic morality in behavior of complex animals, the fear remains relevant. The fear is not necessarily about God or religion, but about a moral compass, or lack thereof.

My arguably tasteless rewrite of Nietzsche’s passage expresses another fear, which, I sincerely hope many software developers share. For a long time, we had an external source of morality in software development. User. Simultaneously ephemeral stick figure of UML diagrams and a very real human, a friend, a colleague. The goal, the point, the final judge of success. Your product either satisfied the user or failed. Your software may occasionally have charmed User, but don’t be fooled, you are but a servant.

“Move fast and break things”, “disruption” and “growth” have killed User. No more do we try to offer invisible quality. Instead of software that blends into background by virtue of its non-obtrusive robustness and simplicity, we aspire to create The Product Experience. The first pleases User. The latter pleases Shareholder.

The Holy Texts are forgotten or, worse, perverted. Apple’s famous Human Interface Guidelines ironically describe the many ways in which modern Apple products do not behave. The Agile manifesto became The Certified Agile Coach Training Program and a cargo cult. The very first principle of the Agile Manifesto states:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

How many of us feel this during another a/b-test-driven, metric-based sprint?

The 7th principle is:

Working software is the primary measure of progress.

An honest, but naive assumption that we all share a similar criterion for “working”. Turns out, the height of the bar is inversely proportional to the proximity of another funding round. By the 90s standards, a lot of today’s software is just defunct.

When User dies, only Beta Tester remains. Beta Tester is not human. It’s an expendable tool, a lab rat, a database record.

What can we do as developers? Honestly, I don’t know. Coming up with a manifesto, a set of principles, another guideline — it all seems futile now. The problem isn’t that we forgot about User. We just collectively adapted to the new definition of normal. Breaking changes are normal. Updates for the sake of updates are normal. The house is on fire, but this is normal.

What can we do?.. “Must we ourselves not become users?”

Be Wary of Self Described Benefits

Tue, 03 Sep 2019 18:06:00 +0300

Picking a university was one of the main tasks in the last year of high school. That and exams. I wasn’t sure what to study and which place to pick. I had no idea how one can make these choices. There weren’t too many resources available at the time. So, a lot of us relied on promotional info provided by universities themselves.

The one I ended up in was called SDU. Its official description says: “SDU is a secular higher educational institution located in Kaskelen, near Almaty.”

It was the only university that actually claimed to be secular. That’s good, right? I had no interest in studying at a religious institution. Secular is good.

Turned out, it wasn’t very secular. No, we didn’t study Holy Texts. Officially, nothing religious was going on. But the Islamic values did indeed feel affecting the policies and decisions everywhere. The dormitories for males and females were separate buildings in different neighborhoods. I had actually never seen the women’s dormitory, nor had any male student around. The girls just went to a mysterious place every evening. A large proportion of students held a religious fast (Ramadan). Several instructors gave bonuses to fasting students. So I had to work harder to get the same grade because I was being secular at a secular university.

I dropped out after 4 months. Thank God.

Later, I started to notice this pattern. Often, people, communities, companies and even countries seem to blindly put positive descriptions out. As if they wanted others to notice.

It becomes very clear in case of countries. Here are some famous authoritarian regimes:

Democratic People’s Republic of Korea
Democratic Republic of the Congo
People’s Republic of China
People’s Democratic Republic of Algeria
Federal Democratic Republic of Ethiopia

Right.

So I learned to be wary of self described benefits. It’s kind of obvious and silly once you think about it.

Yours truly,
not passive aggressive at all, Rakhim.

Process of Learning

Mon, 29 Jul 2019 17:24:00 +0300

A process of learning is analogous to an attempt of building a three-dimensional model from two-dimensional photos.

You approach a new area of knowledge. You know nothing at all. You stumble upon a first piece:

That’s more than nothing, but still very little. You don’t understand it. At best, you’re able to make a few uncertain assumptions.

Beginners often seek good book recommendations. They would google a top-10 list and then ask “which one should I start with?”. The answer is almost always “it doesn’t matter”. Start with any non-shitty book. One book alone would not provide enough data to build a good model anyway. If you want to really understand something, you’ll have to read several books, listen to different people, try various approaches. Nobody knows what’s going to work best for a particular person. Each model building machine is unique. The starting order of feeding it data is not very important, unless that information is truly harmful.

(“What is harmful” is a topic for another discussion, and I by no means argue that finding non-harmful books is an easy task. In fact, I’d call many popular programming books harmful, especially when it comes to teaching the basics of programming with Java. So, at least minimize the potential harm by not focusing on a single book.)

Another basic book will provide a different view:

You still can’t understand it. But keep going, and at some point you’ll arrive at a single complete picture.

This tells you more than before, you can even make some conclusions or reason about certain aspects of the object. But you don’t have a complete model yet, one 2D picture is not enough. Your brain haven’t met such objects in the past, there’s nothing to cling on to. This is why it’s essential to have various sources of information: other books, people, lessons, different mediums.

New pieces keep emerging, and it’s disconcerting. They don’t seem to make any sense, this puzzle feels broken.

This is the toughest stage of the learning journey. Often, any hope is lost. Disconnected pieces provoke the feeling of meaninglessness. You can’t see the big picture.

I think I understand each individual topic, but have no idea how they are connected. And why did I learn all that. Nothing makes sense…

But if you keep going, soon you’ll get to another full picture:

Interesting! A completely different point of view. Same object, new aspects. A complete 3D model is still impossible to deduce, but there’s more space for assumptions. Having multiple pictures increases the chances of seeing something familiar. It’s a new topic alright, but topics are rarely completely isolated from the universe.

A few more pictures and you get a pretty accurate model.

The first picture was extremely valuable, but each new picture brings less and less valuable data. At some point you have a decent model in your head, so that new pictures don’t give you anything new.

This analogy helps me learn new things. I try to remember the following:

At first, everything is interesting and easy. The first picture gives a lot of data at once. This is the pleasant stage.
In the middle motivation will decrease. That’s okay. Keep getting data and trust the system.
Do not focus on a single picture. Maybe, in order to understand it you need another picture first.
At some point, notice the diminishing returns of new data. Consider increasing the area of study.

Bicycles and Love

Sat, 27 Jul 2019 11:47:00 +0300

I love bicycles.

The first memories I have are bike-related. The best ones are, too.

My first bicycle was a tricycle

I don’t remember a time I didn’t bike. Bicycle means freedom.

My first serious bicycles, the ones that let me explore the city, were old, steel soviet tanks. They were simultaneously indestructible and always broken.

My friend Eugene and I spent summers biking around town, forests and river valleys. It was awesome.

Eugene riding away, my bike in front

Those bikes were all-purpose vehicles. Well, at least for us they were. Pavement, gravel, sand, water, whatever. They go where we go. They were simple and stupid, and I loved that. No speeds, no hand brakes.

I used to live with my grandma until I was 13. Her apartment had a narrow corridor connecting the entrance to the room, and I can’t imagine it without a bicycle. It was always there, and when it wasn’t, I wasn’t home. Out biking god knows where.

I had a dream of driving a city bus, so quite often I spent hours biking along the bus routes, stopping at bus stops, emulating a bus. An old pen was diligently placed inside the handlebar, sticking about 2 inches outwards: it was my fake turn signal lever. I had to make the turn signal clicking sound myself, of course.

Then I moved to Canada. My next bike was a pretty cool steel Schwinn I got as a gift from my host family in Ottawa. Bicycle freedom had suddenly expanded. Now I could ride to another province where people speak a different language and traffic lights are horizontal. I could bring my bike on the train and get further than ever. I could disappear into the city and nobody could find me.

I didn’t appreciate this bike enough. I wish I could get that frame back…

Several random used bikes later I decided I’m ready to spend serious bucks and try road biking. With 1000 Canadian dollars in hand, I walked into a bike shop in western Ottawa and got myself a beautiful aluminium Trek 1.1.

Freedom was reinvented once more. With this light, fast bike I could go further than ever before. First 100 km ride was a revelation. Then 350km in 2-day group event. Then 500km in three days. Then more than a 1000 km in a week across multiple provinces to see the ocean.

At the end of my long trip to see the Atlantic Ocean, New Brunswick, Canada

I was in love again.

I was about to finish my computer science degree and was feeling somewhat down at times. That bike was the best thing in my life.

Biking and computer science made perfect sense to me. Both are about efficiency. Both are tools, but at the same time fun in and of themselves. Both let you be alone.

They let you disappear into a vast domain and get lost.

After moving from Canada to Kazakhstan, I stopped biking. It took me almost 5 years to re-ignite the passion.

After moving to Finland, I decided to try a single-speed and got a pretty Swedish Stålhästen Sport fixed gear bike with a flip-flop hub. Fixed gear is not my cup of tea, so I flipped the wheel and started discovering the Helsinki area.

It was pretty sexy.

But alas, we weren’t for each other. We just didn’t click. A bicycle is an intimate object for me, along with backpacks and computers. I can’t just have one, I must love it. So I sold it to someone who, hopefully, loved it.

And got myself a cyclocross Kona Rove Al 2015.

The relationship with this one was complicated. I loved it and sometimes hated it.

I loved my Kona because it was very comfortable, fast and looked very cool. Its brownish color was a perfect fit for the forests and fields I took it to. Disc brakes — first for me — made me more confident on narrow descents and uneven terrain.

But I didn’t love the maintenance. It was equipped with the cheapest components. Decent, but not awesome. Shimano Claris groupset and mechanic disc brakes were never perfect. The bike was never perfectly tuned, never perfectly silent. And I’m ashamed to admit, but I’m a crappy handyman when it comes to bikes. Actually, I’m a crappy handyman in all areas where “undo” is not an option, but I’m particularly bad with bicycles. Derailleur adjustment and disc brake calibration are black magic to me. Hours of sweat and profanity and I end up with a subpar configuration. It works and it’s fine, but it’s just not very good. Then I say “screw this!” and bring the bike to a mechanic.

By the way, good bike mechanics are pretty expensive in Finland.

I’ve later learned that better groupsets and hydraulic brakes (or at least the hybrid dual-piston ones) are so much better, and I honestly almost convinced myself to spend two grand on a very good bicycle. Because then I will ride more, right?

Kona Rove Al was my bike for three good seasons, but the furthest I took it was a 120 km one day ride. It never saw other regions of the country even. We weren’t too adventurous together.

This summer, after another failed attempt to eliminate 100% of the disc brakes noise, I decided to say goodbye to Kona and look for something new.

A romanticized and, perhaps, irrational desire to simplify struck me again. What is the simplest bicycle possible? Single speed with coaster brake. I love handbrakes, so, rim brakes then. But I wasn’t sure I could do single-speed again. I love my knees.

Then I discovered internal hubs. They look and feel like single-speed from the outside: no derailleur, no cable slack. My girlfriend and I got ourselves small folding bikes with Shimano Nexus 3-speed hubs and they are very nice. Simple, reliable, solid.

There also exist 7-speed Nexus hubs, but I haven’t tried one. They are pretty heavy and it’s a pain to install them with dropbars.

Turns out, there are also automatic hubs! Woah! Extremely curious, I got myself an old Fixie Inc. Floater with Sram Automatix 2-speed automatic hub.

The experience is… interesting. You just start pedaling and at about 15 km/h the gear changes automatically. It’s pretty solid, no wobble or anything. It’s like you’re suddenly teleported into higher gear. This shift point it ridiculously low though. It was pretty easy to disassemble the hub and change the shift point by unwinding a tiny metallic spring (here’s a good description).

At first, it seemed like a good compromise. Almost as simple as a single-speed, but the second gear allows to keep a sane cadence at higher speeds. Unfortunately, the hub is not as isolated from the outside world as other internal gear hubs, so it will require some maintenance.

I took it on multiple rides and just didn’t love it. What a picky, delicate flower I am.

And then, completely by chance, I had a chance to ride Bombtrack Arise. A single-speed, steel gravel bike.

Yes! It made me smile!

I’m not a fast rider, so when it comes to gearing, comfortable climbing is more important to me than going fast. Finding the perfect ratio for a single-speed takes time and lots of cogs, but this bike came with what seems to be the perfect ratio for me: 42 teeth chain ring and 17 teeth rear sprocket. With 28" tires, it yields 69 gear inches, which works very well for the steepest hills in my area and is good enough for descents.

This handy bike calculator helps to figure out the good ratio for a given cadence and speed. For my new bike, 90 rpm produces 28.4 km/h, and the high cadence of 130 rpm yields 41 km/h.

Single speed forces me to be more disciplined and think ahead. I can’t just attack a hill on a granny gear anymore, I have to save as much momentum as possible. At the same time, this creates ultimate freedom. I don’t think about optimal gearing, I am always in the ~~wrong~~ right gear.

Steel frame is lovely and comfortable, it feels softer, yet more confident. The simplicity of not having so many extra things inspires almost a zen-like feeling. Oh, and it’s quiet. No chain slap.

I am yet to take the Bombtrack on a truly long ride, and it might not be as peachy as I describe. But so far, with about 250km behind, I remain in love.

Find a bike you enjoy, it will make you happier.

80-characters limit for text is wrong

Thu, 30 May 2019 12:27:00 +0300

I believe the 80-characters (or any other number) line limit for text to be wrong. Not archaic or irrelevant, but wrong. It violates a fundamental idea of computer science: separating layers of abstraction.

Not talking about code today, although, I don’t think a strict limit is a good thing there either, for other reasons. I’m talking about human text.

Many programmers stick to the 80-characters line length limit while writing documentation, emails, etc. Emacs and other editors even have special modes or plugins for automatic hard wrapping.

Often, results look like so:

Without hard-wrapping, this email had a chance to look normal everywhere. Any app can interpret and present it in any way. With hard-wrapping though, this email can only look normal in certain conditions. Namely, a certain lower limit for window width.

Imagine joining a web project and seeing a users database table with values like <strong>Jason Norwig</strong>. Your reaction might include profanity, because mixing data and presentation is wrong. Person’s name (data) and its presentation (HTML) are different layers of abstraction.

Hard-wrapping lines by inserting \n symbols where they don’t have any semantic meaning is the same sin. The whole character return thing was relevant in a context where data was inseparable from presentation: typewriters and paper.

A key argument for hard-wrapping goes something like this: “modern screens are too wide, it’s uncomfortable to read long lines”. But modern screens come with modern apps, which can handle presentation to your preference. By modern I mean “developed after the ’70s”. Text editors (including vim and Emacs) have been able to soft-wrap lines at arbitrary comfortable column for decades.

If your text presentation tool can’t present text to your preference, consider replacing it. Moving this responsibility into data itself is not a solution.

So can we please

stop doing this

to each other.

Rethinking Twitter

Wed, 15 May 2019 17:51:00 +0300

Good old days

Twitter is the only social media I’ve ever used. Sure, I had accounts on other sites, but was never really engaged elsewhere. I couldn’t understand all those “facebook addiction” stories, honestly. Facebook was always extremely boring, while Twitter was always exciting.

From the beginning, it was a perfect, geeky, simple thing that is kind of hard to explain. Just a stream of short thoughts. Cool links. One liners. A public chat room.

I joined Twitter in 2007. Woah, 12 years ago! I was studying in a university, and smartphones weren’t really a thing yet. This is totally a “back in my days” kind of rhetoric, but hell, that Twitter was nice.

Most people didn’t have a Twitter account. Because it’s weird, what’s the point? So, like other internet communities, Twitter started as a selected group of weirdos, acting constantly amazed about the fact that they’re all here. Tweeting.

The timeline was chronological, and the world seemed ordered. It was more like RSS than like Facebook or TV: you could only see the people you explicitly subscribed to. Never an unknown face in your timeline. Retweets weren’t a feature, it was just a thing people did: copying a tweet and putting “RT” in front of it. I think it’s a better strategy overall: if you retweet something this way, you put your face on the message, you own it now. So, the value of retweets was higher, nobody wanted to copy every mildly interesting thing.

There were no likes, this added sugar of engagement and interaction. You like something? Well, say it. Tell it to the author, do a retweet with an encouraging comment. Do something you’d do in real life.

Twitter was allowing people to communicate via a digital medium. Not communicate in a new digital way.

And Twitter wasn’t like TV.

Outsourced schizophrenia

A HackerNews user had put it nicely:

I think the problem is that Twitter is a platform for evolutionary selection of slogan-based-dialog. I kind of imagine two armies standing across a battlefield from one another carefully deciding which volley of pithy digs to throw at one another.

Yeah. It’s like switching TV channels every 5 seconds.

This article had hit home hard for me. I’ll just include a few quotes:

There are people you know whose voice you can hear in your head […], or people who you even consult with in your head for wisdom (“What advice would my dad give here?”) […]

…it can be a huge mental lease we’re signing when we invite a few hundred people into our Twitter life. To some degree, it is choosing to subject ourselves to thousands of ads throughout the day, but ones that come from trusted sources we care about, so they’re actually impactful.

Yes! And especially today, thanks to both natural and artificial (algorithmic) selection, these short ads are hyper-optimized for stickiness and virality.

We’ve surrendered a massive amount of mental and emotional energy without making the explicit choice to do so–it’s simply imposed on us by subscribing to the channel and checking it.

The worst part is that nowadays you see a chaotic stream from people you haven’t explicitly subscribed to: retweets, liked and promoted tweets, ads, etc.

Mentally, we just aren’t capable of simultaneously empathizing with hundreds of people–let alone thousands or millions. The result is we either build up a calloused, jaded, or cynical defense against empathy or find a way to block out more.

I can hardly manage interacting with a single person for more than a few hours, let alone with hundreds, even via virtual channels.

This one describes me very well too:

I’ll admit: I’m an annoyingly oversensitive person. I do believe this is both a strength and a weakeness. […] I also have a tendency to listen carefully to any criticism or disagreement I hear, internalize it, reflect on it, and evaluate it, then conclude some thought on it. Until I do that, it just sort of hangs there in my head. The degree to which it dominates my headspace is largely a question of how much it impacts me.

If nothing else, I want you to consider this quote:

[…] Twitter is outsourced schizophrenia. I have a couple hundred voices I have consensually agreed to allow residence inside my brain.

I assume not everyone is like that, and not everyone experiences social media this way. But Adam, the author of that post, does. I do too.

His essay is fantastic, really:

I’ve realized how Twitter has made me break up my thoughts into tiny, incomplete, pieces–lots of hanging ideas, lots of incomplete relationships, punctuated by all manner of hanging threads and half-forked paths.

Change

But Twitter has been very beneficial to my career. With a few thousand followers, links to my projects had consistently brought me hundreds of hits, readers, clients. This is my strategic failure: I haven’t been working on maintaining an independent outlet for my audience, a newsletter or a consistent blog. For more than a decade, Twitter has been the main channel of distributing my work.

So, I decided to make a few changes and conduct an experiment.

Unfollow everybody. No more timeline.
Tweet via an app like Buffer, without opening Twitter itself.
Open Twitter and reply to mentions only on weekends.
Invest energy into maintaining a consistent blog and a personal newsletter. You can subscribe to the newsletter here or with a form below:

I will maintain this system at least until the end of 2019 and we’ll see how it goes.

TWEET THIS NOW!

Easy slides (for me)

Sun, 21 Apr 2019 14:16:00 +0300

As a software developer, at some point you discover simple slides and presentation generators: Markdown-to-PDF/HTML converters, Emacs extensions, LaTeX exporters, VIM plugins, etc. The idea makes perfect sense, because:

I don’t want to use PowerPoint or Keynote.
I don’t want to use the mouse.
I want plain text.
I want simplicity.

You can create presentations without leaving your favourite editor or command line. But unfortunately, the majority of the results are just text with an occasional poorly positioned funny GIF (that didn’t load because wifi is down).

I completely understand the desire to make things as simple as possible and forget about clunky GUI-based presentation software. I don’t like them either, and yeah, I’d love to be able to do things from the comfort of my text editor. You might say that text is mostly enough, animations and other flashy effects don’t contribute to the value.

But I argue that animations, visualizations and transitions are tools, and like any other tool, they add value when used correctly. By sticking to text-only slide generators, you disregard a whole set of tools and potentially a whole set of problems they might help solve.

A title flying out from the corner probably doesn’t do any good, the effect has no meaning. But if you want to explain something non-trivial (not to you, but to your audience), consider using something to illustrate your point or even just to focus viewer’s attention. It’s not about animation or burning flames effect, it’s about anything above the typewriter in the pyramid of technology.

Dimming. Colors. Shapes. Transparency.

Computer science is full of complex ideas, multiple levels of abstraction, non-obvious connections and relations. It pains me to see whole presentations, thick books and long manuals with essentially zero visuals, zero attempt to convey an idea with something other than text.

Your slim Markdown-to-PDF converter serves one purpose: make your life easier. Nothing wrong with that. But there are also viewers who might benefit from a more detailed visual presentation. Of course, not all viewers would. For many, text and your speech are more than enough, after all, many of us became programmers because of the ability to understand complex, abstract, non-visual ideas to begin with. This is where lack of diversity starts from, I believe. We filter out people by their adaptability to certain styles and formats of explanations. We filter out people by their learning medium.

You need a cartoon to understand closures? Good luck. Maybe, programming isn’t for you?..

Now, I understand that it takes time, and you might just not have enough of it. I am not bashing these wonderful tools and not saying you must produce visuals and animations. I just wanted to remind you that plain-text presentations are compromises. It’s absolutely fine to mindfully and intentionally make compromises.

How to podcast

Mon, 15 Apr 2019 16:23:00 +0300

Here is a TL;DR version:

Record using whatever. Even smartphones produce decent sound nowadays. Just make sure to have a consistent volume, i.e. maintain constant distance between your mouth and the phone.
Compress to 96kbps mp3 constant bitrate. Or use an online encoder like Pinecoder.
Pick a podcast hosting provider:
1. pinecast.com: starts at $5/month with unlimited storage, unlimited shows, basic analytics and website generator.
2. simplecast.com: starts at $15/month per show. Cool player, website generator.
3. transistor.fm: starts at $19/month with unlimited storage, unlimited shows. Limited at 5k downloads per month.
4. Or one of many others, including podbean, libsyn, fireside or even soundcloud (probably not the best idea, since the company’s finances aren’t great)
Add your podcast to public catalogs:
Some podcast hosting providers can submit your feed to those catalogs on your behalf, but I suggest you do everything manually to keep 100% control over your content.

Preparation

This part is very subjective, so I’ll just describe my own process.

Since each of my episodes follows a specific topic, I start thinking about it weeks before, just allowing ideas, thoughts and just random pieces of info simmer in my head for hours. I do my best to write down these things, but often just forget them. It’s okay. I have learned to let go of “obviously genius” ideas.

I never hesitate to write down whatever comes to mind though. It could be an analogy, a funny phrase, a weird comparison.

At some point I feel ready to jot down the structure. I’ve been using a large notepad and iPad pro with Apple Pencil, but lately have been enjoying MindNode on the desktop. Having a large, tree-like structure fits my way of thinking very well. I just “walk” the tree during recording, trying to visit all the nodes.

Recording

There are different levels here.

Level 1: Just use whatever.

Your phone or internal microphone of your laptop are okay. Not great by any means, but keep these in mind and you’ll be fine:

Pick a quiet spot. Humming noises are the worst (loud fridge, AC, etc.)
Maintain constant distance between your mouth and the mic
If possible, record in a smaller room with lots of soft objects to minimize echo. Books, blankets, pillows, curtains reduce echo significantly.

Level 2: USB headset

A good and cheap way to improve your sound significantly is using almost any good USB headset. But it must be USB. Those headsets that connect via jack cables are simply using your computer’s audio card and the sound might not be much better than with an internal mic.

Level 3: USB microphone

USB mics are the combination of “real” microphones and the convenience of USB. You usually don’t need anything else, just plug and play.

There are several good choices in this category:

Rode Podcaster (my only mic for 10+ years, produced hundreds of podcasts and videos with it)
Blue Yeti
Audio-Technica ATR2100-USB
Shure PG42-USB
Audio-Technica AT2020USB+

Level 4: XLR microphone

When you’re ready to go “full podcaster”, go read Marco Arment’s Podcasting Microphones Mega-Review. There are audio samples, too.

Editing

I edit my podcasts heavily. 60 minutes of recording usually result in about 35-45 minutes of end result. Marco Arment wrote about this in Easy listening and I agree: “you just need to care”.

Free and open source Audacity is more than capable for both recording and editing, but, to be honest, I find it extremely cumbersome and ugly. Once you’re serious, I think it’s worth to invest into buying and learning a tool like Adobe Audition (my choice) or Apple Logic Pro X.

Publishing

Podcast is basically an RSS feed with media files. You can generate it yourself and host mp3 files wherever. If you already have a blog running on Wordpress, it makes sense to just continue using it. On wordpress.com they have a special feature for podcasting, and if you run your own wordpress instance, this plugin will help you.

It is much easier to use one of the specialized hosting providers:

pinecast.com: starts at $5/month with unlimited storage, unlimited shows, basic analytics and website generator.
simplecast.com: starts at $15/month per show. Cool player, website generator.
transistor.fm: starts at $19/month with unlimited storage, unlimited shows. Limited at 5k downloads per month.
Or one of many others, including podbean, libsyn, fireside or even soundcloud (probably not the best idea, since the company’s finances aren’t great)

Most of them can even generate websites for your shows, although, their design choices are questionable at times.

All of my shows are hosted by pinecast: it’s a fantastic value for the money and everything works perfectly fine. Here is my referral coupon code for 40% off for 4 months: r-a6562b. Use it at checkout.

Questions?

Feel free to email me at contact@rakhim.org, and I’ll try to add more info to this guide.

Frontend vs. Functional Programming Conferences

Wed, 27 Feb 2019 14:25:00 +0200

Christmas Wish

Mon, 17 Dec 2018 17:43:00 +0200

If real life were as buggy as software

Fri, 30 Nov 2018 22:58:00 +0200

Evolution of Frontend vs. Backend

Fri, 16 Nov 2018 16:20:00 +0200

The Ultimate AI answers the Ultimate Question

Wed, 07 Nov 2018 13:27:00 +0200

Important Git Realization

Mon, 29 Oct 2018 10:51:00 +0200

Every movie that involves computers

Tue, 23 Oct 2018 12:13:00 +0300

Kids nowadays are LIKE stupid

Mon, 22 Oct 2018 11:39:00 +0300

A case against LISP

Mon, 15 Oct 2018 14:30:00 +0300

Software updates throughout history

Thu, 11 Oct 2018 09:26:00 +0300

People don't change

Mon, 08 Oct 2018 10:29:00 +0300

React Expectation vs Reality

Fri, 05 Oct 2018 14:01:00 +0300

Make Firefox faster and nicer on macOS

Mon, 01 Oct 2018 13:53:00 +0300

I’m trying Firefox as my primary browser on all devices. It has some great features like Multi-Account containers in addition to being a non-Google product, which is an increasingly rare feature on the web nowadays.

Firefox on macOS is somewhat sluggish at the moment. If you try the current stable version 62 or current beta version 63, you’ll notice some lags and general slow response time for even the simplest tasks like changing tabs.

Mozilla will probably fix these issues in the upcoming releases. Meanwhile, I found the following steps improve the performance significantly.

Download Firefox 63 or higher

As of today (October 1, 2018), stable release version is 62. Version 63 is currently in beta, and I recommend using it today. It’s very robust, I haven’t had any problems with it. There are some important performance improvements in it.

You can also try Firefox Nightly, it is currently version 64 on the dark side. Nightly is an unstable testing and development platform. By default, Nightly sends data to Mozilla — and sometimes their partners. There are some rough edges, I wouldn’t recommend it for daily browsing.

Disable animations

By default Firefox has lots of animations. I find them unnecessary and distracting, but more importantly, they contribute to the general sluggishness.

Go to about:config in the address bar. Search for animate and set at least cosmeticAnimations to false.

Fullscreen transition takes 0.2 seconds both ways. Make them instant by setting the following to 0 0:

full-screen-api.transition-duration.enter
full-screen-api.transition-duration.leave

Disable Pocket

Firefox embedded Pocket into the browser. A questionable move, but it’s easy to disable (unless you use it, of course). Set extensions.pocket.enabled to false.

Other stuff

These are not related to performance, but can make your Firefox experience a bit nicer.

Set to true:

modalHighlight highlight all the search results.
browser.tabs.closeTabByDblclick close tab by double-clicking on it.
abs.multiselect shift-click on tabs to select a group of tabs and do something with them (for example, detach from window).
insecure_connection_text.enabled write “Not Secure” in the address bar of non-https pages (like Chrome does). Additionally, enable a broken padlock icon with security.insecure_connection_icon.enabled.

Minor things:

general.smoothScroll.mouseWheel.durationMaxMS set 200 to make scrolling speed similar to Chrome.
geo.enabled set to false to disable geolocation.
extensions.screenshots.disabled set to true to disable the screenshot extension. It’s actually pretty handy, check it out before disabling.

I can't believe this happened again

Fri, 28 Sep 2018 23:16:00 +0300

A company sells your data

Thu, 27 Sep 2018 23:16:00 +0300

Product manager's bedtime story

Wed, 26 Sep 2018 23:16:00 +0300

For Google, you're neither the consumer nor the product. You're a data point.

Wed, 26 Sep 2018 14:24:00 +0300

In light of recent changes to Google Chrome, many forums have filled with bitter discussions. Here are just a few:

At some point someone inevitably says:

For Google, you are not the consumer. You are the product.

But companies usually care about their products, protect them, try to improve their state.

If I were a product, Google would do its best not to destroy me. They have invested a lot of resources into this product, so why risk it by making baffling changes to both privacy and user experience? If I stay happy with Google’s offerings, I keep being the perfect product: I can be mined for data and “sold” perpetually.

Clearly, Google doesn’t care about me personally. And how could it? There are billions of people just like me who use their services every day.

Maybe we should stop thinking we’re “Google’s product” and start thinking we are data points in endless experiments.

You’re Data Point

What does it mean to be a data point?

Your personal issues don’t matter.
You, alone, are not valuable.
You can be easily replaced.

While this is a gloomy oversimplification, I believe it’s important to recognize the general idea.

You’re playing a role in some “user story”

When you pretend you’re the consumer, you think in terms of goods and services, and a simple two-way relationship. This leads to disappointment every time a breaking change comes in or a service shuts down altogether.

In reality, you are playing roles in many “user stories” and experiments, without knowing it and without knowing what those roles actually constitute.

Figure 1: Product manager’s bedtime user story.

Few weeks ago Google announced that Inbox will be closed next Spring. Many ask the heavens “Why would Google kill Inbox?! I’ve been using it every day and all my friends have!”

I bet it was a successful experiment for Google: by using the app, you’ve generated a lot of important information, helped them learn useful patterns and god knows what else. They are not killing it because it failed as a consumer product, it was never intended to be one. They are not killing it because not enough people used it, or used not extensively enough. Quite the opposite.

They are not really “killing” a product, they are finishing up an experiment.

Just like scientists don’t “kill” good experiments, even though the mice might ask the heavens “why would they kill this maze?! I’ve been running in it every day and all my friends have!”

Figure 2: Google shuts down another product.

Of course, not all Google’s products are experiments. This isn’t a binary thing though: some products are clearly (in hindsight) temporary experiments, some are core things that rarely change, but the majority of apps and services are somewhere between. And you never know what games are you in.

You are punished for being a good data point

If you truly use Google’s products extensively, you are statistically more likely to be punished for that. For example, if you haven’t willingly participated in the experiment called “Google Inbox” and just ignored their promotion, its shutdown won’t affect you negatively. Actually, it will affect you positively: the results of this experiment will help improve the feature set of Gmail, the service you’re probably using.

But if you were a good data point¹, and invested a lot into Inbox’ features, you’ll receive a punishment. Your workflow — and an important one! — will just be disregarded.

“We will incorporate many features of Inbox into Gmail” is a weak consolation when in one day your whole email experience just shuts down.

We have to understand this fundamental change of the relationship between companies and users. The main focus for tech giants is growth, which requires data, which requires experiments. The more we participate in data points generation, the more likely we’ll be burned.

Compare this to the old and straightforward concept of “product X is popular, therefore product X will be kept on the market”.

But why? Don’t they want to have popular products? Well, yes, but…

Your personal metrics aren’t aligned with theirs

We might think that “popular is good”, but for Google often “more data points is good”, because that’s the real resource that allows them to grow. Ten different products, launched and shut down sequentially, is a better source of data points than one, long-lived and stable service.

We do not and cannot know what is important in any given experiment. Heck, we can’t even know the scopes and the limits of them. That’s the point of having experiments!

But one thing is certain: our values and metrics rarely align with theirs. Because, as a user, I don’t really care about company’s growth.

You might say “Yeah, obviously! Their goal is profit, nothing new here!”. But it’s different this time. All businesses’ end goal is profit, that’s capitalism, nothing wrong with that. The problem is in the hidden, implicit nature of the relationship. We don’t really know the deal! What can we do? What can we be sure of? What are our rights? What are their responsibilities?

What exactly is the deal?!

You’re a renewable source

Google itself uses your data to grow, but it also uses it for to make money by targeting ads. That’s what people mean when they say “Google sells your data!” and that’s where “you’re the product” rhetoric comes from.

But it’s not like Adidas wants my data. Or yours. They want a large group that satisfies certain parameters.

Figure 3: A company sells your data.

Of course, companies don’t get a .zip-file with names and addresses. They receive the ability to show certain ads to certain people. Google doesn’t sell your data, they sell access to your eyes and ears. So, you’re not just a product, you’re renewable source of products. Your tastes and needs change over time, you can be targeted over and over.

This denies any sort of hope one might have about companies caring about their products. They do, just not about individual items or individual producers. Apple doesn’t care about any particular iPhone device or any particular worker at their Chinese factory.

But I pay them money!

The third aspect is “Google and paid services”. Google sells a lot of things directly, and this must be a “consumer-producer” scenario, right?

Yeah, no.

Google recently increased the fees for Google Maps API about 1400%. This kind of increase means one of the two:

Previous pricing model was inaccurate.
New pricing model is inaccurate.

Were they losing money before to conquer the market? Or did they just decide to make a buttload of cash using the conquered market? Either way, the problem is the same: we had no idea what the deal is.

Another example is Google Cloud, a platform used by many large businesses. You can lose everything in 3 days and deal with a pretty bad support even though you’re paying client.

Not to mention the lack of visibility in changes - it seems like everything is constantly running at multiple versions that can change suddenly with no notice, and if that breaks your use case they don’t really seem to know or care. It feels like there’s miles of difference between the SRE book and how their cloud teams operate in practice. (comment in a relevant discussion)

Are they underfunded? Is their goal to make a reliable platform or is it something else? What do they take into account when they make changes? We have no idea.

This is not just Google

It’s easier to talk about Google because they seem to be the biggest company in this area (or ever). But this is the reality for a lot of businesses, not necessarily in the advertisement industry.

Once the company is large enough, all customers become data points. This is okay in principle. I can live with that, as long as I understand it. This is a question of honesty.

If Google said upfront:

We’re launching this new product X, but it’s an experiment. We’ll work on it for at least 5 years, but can’t guarantee anything after that. We might shut it down with short notice. Would you like to participate?

Then there would be no point in complaining. That’s a fair deal. Of course, this kind of frankness wouldn’t help Google. It’s like telling the participants of a sociological experiment about all details of said experiment. It poisons the data. Scientists want unsullied results.

Figure 4: I hope I’m not sued for this…

If Google said upfront:

We’re giving you a lot of awesome products free of charge. But we’ll collect as much information on you as possible, and if we’ll keep changing the services and terms to collect more data. We will use this information to target ads and maybe do something else. Would you like to participate?

That’s a fair deal too. You are free to give up anything, as long as you understand what’s going on. Of course, this kind of message doesn’t survive the path from Terms of Service to The Marketing Department.

But you always read ToS, right?

All that is obvious in hindsight (in those forums, there’s always at least one guy who says “what did you expect?”), but we have to learn to infer these things from the business models. This is not our jobs, but that’s the reality. We have to understand all the implications of these new, weird businesses.

Every time you see a new startup, new app, new service with some interesting features, and it’s clearly not a simple “I pay, you provide” kind of deal, beware. What are the implications?

Conclusions

Let’s summarize:

Growth, not simple profit generation is the main focus.
Growth requires data. Experiments, changes and seemingly weird decisions generate data.
For Google and many tech giants, you are a data point.

And as the result:

We no longer interact with businesses. We interact with the top layer interface of a multi-layered, non-obvious system built with implicit, vague rules.
Never before were users’ and companies’ goals so irrelated to each other.
We’re constantly playing games we’re not aware of.
We have to learn to understand the implications of this.

Final words

Not all is bad. This symbiosis can be very benificial for all parties. We can explicitly play roles of consumers, products and data points at the same time, knowing what’s happening and being in control. Companies can play whatever games they need to play with us.

Extremely relevant ads and extremely personalized user experiences sound pretty good to me, all the creepiness aside.

Legislation will never catch up in time, so we have to take things into our own hands and learn to live in this brave new world.

Rather, a generator of myriads of data points, but “data point” sounds more inhumane and humiliating, so I’ll stick to this term for dramatic purposes. ↩︎

A Simple Introduction to Proof by Induction

Thu, 20 Sep 2018 17:18:00 +0300

Now that you’re familiar with direct proof and proof by contradiction, it’s time to discover a powerful technique of proof by induction.

Aside: do not confuse mathematical induction with inductive or deductive reasoning. Despite the name, mathematical induction is actually a form of deductive reasoning.

Let’s say, we want to prove that some statement $P$ is true for all positive integers. In other words:

$P(1)$ is true, $P(2)$ is true, $P(3)$ is true… etc.

We could try and prove each one directly or by contradiction, but the infinite number of positive integers makes this task rather grueling. Proof by induction is a sort of generalization that starts with the basis:

Basis: Prove that $P(1)$ is true.

Then makes one generic step that can be applied indefinitely:

Induction step: Prove that for all $n\geq1$, the following statement holds: If $P(n)$ is true, then $P(n+1)$ is also true.

See what we did there? We’ve devised another problem to solve, and it’s seemingly the same. But if the basis is true, then proving this inductive step will prove the theorem.

To do this, we chose an arbitrary $n\geq1$ and assume that $P(n)$ is true. This assumption is called the inductive hypothesis. The tricky part is this: we don’t prove the hypothesis directly, but prove the $n+1$ version of it.

This is all rather amorphous, so let’s prove a real theorem.

Theorem 1. For all positive integers $n$, the following is true:

\begin{equation}
\label{eq:1}
1 + 2 + 3 + … + n = \frac{n(n+1)}{2}
\end{equation}

Proof. Start with the basis when $n$ is $1$. Just calculate it:

\[ 1 = \frac{1(1+1)}{2}. \]

This is correct, so, the basis is proven. Now, assume that the theorem is true for any $n\geq1$:

\begin{equation}
\label{eq:2}
1 + 2 + 3 + … + n = \frac{n(n+1)}{2}
\end{equation}

In the induction step we have to prove that it’s true for $n+1$:

\begin{equation}
\label{eq:3}
1 + 2 + 3 + … + (n+1) = \frac{(n+1)(n+2)}{2}
\end{equation}

Having this equation, we should just try to expand it and prove directly. Since the last member on the left side is $n+1$, the second last must be $n$, so:

\[ 1 + 2 + 3 + … + (n + 1) = 1 + 2 + 3 + … + n + (n+1) \]

From our assumption, we know, that

\[ 1 + 2 + 3 + … + n = \frac{n(n+1)}{2}. \]

So, let’s replace it on the right hand side:

\[ 1 + 2 + 3 + … + (n + 1) = \frac{n(n+1)}{2} + (n+1) \]

And then make that addition so that the right hand side is a single fraction:

\[ 1 + 2 + 3 + … + (n + 1) = \frac{n(n+1)}{2} + \frac{2(n+1)}{2} \]

\[ = \frac{n(n+1) + 2(n+1)}{2} \]

\[ = \frac{(n+1)(n+2)}{2}. \]

Done, we have proven that the inductive step (\ref{eq:3}) is true.

There are two results:

The theorem is true for $n=1$.
If the theorem is true for any $n$, then it’s also true for $n+1$.

Combining these two results we can conclude that the theorem is true for all positive integers $n$.

I had troubles with this technique because for a long time I couldn’t for the life of me understand why is this enough and how is the basis helping?! The basis seemed redundant. We assume $P(n)$ is true, then prove that $P(n+1)$ is true given that $P(n)$ is true, but so what? We didn’t prove the thing we assumed!

It clicked after I understood that we don’t have to prove $P(n)$, we just take the concrete value from the basis and use it as $n$. Since we have a proof of $P(n+1)$ being true if $P(n)$ is true, we conclude that if $P(1)$ is true, then $P(1+1)$ is true.

Well, if $P(1+1)$ is true, then, using the same idea, $P(1+1+1)$ is true, and so forth.

The basis was the cheat-code to kick-start the process by avoiding the need to prove the assumption \ref{eq:2}.

RE: Software disenchantment

Mon, 17 Sep 2018 23:57:00 +0300

Nikita just published Software disenchantment, and here is my rant-y reply. Please, read his post first.

TL;DR: I agree with Nikita and I am equally frustrated with the current state of the industry. We started grumpy.website together, after all. But I don’t believe that situation will significantly improve until the general public’s standards increase. Along with the responsibility to make better software, we have a responsibility to educate the public, so that they don’t get used to the idea that computers suck.

Would you buy a car if it eats 100 liters per 100 kilometers? How about 1000 liters? With computers, we do that all the time.

The thing is — yes, people would. It all comes to the micro economic level. If a car eats 1000 liters per 100 km and fuel is cheap (and cars hold enough liters to drive around everyday), people will buy them, use them and rarely think about it. Just like they do it today with 6 liters per 100km cars and don’t think about how the combustion engine system is inefficient in terms of pure energy output. A lot of that energy goes into useless heat and noise. 6 liters of matter has the potential to fuel the planet for weeks if we were to extract it efficiently.

It doesn’t matter if something is efficient or optimal when it comes to general consumer market. And, whether you like it or not, technology and software is now in the general consumer market, in the same area with clothing, cars and such.

Have you been noticing how annoying the clothing industry is? Home items? Furniture? I’ve never seen a laundry detergent or a hand cream bottle that wasn’t downright obnoxious. I’ve used a potato chips package that was easy to use. It’s everywhere: unusable, badly designed, over bloated with unnecessary solutions and optimized for marketing and fast development, not for good use. The things you’re describing are not specific to software, it’s about design in general.

The only places where things are truly not too damn awful are some parts of military and super high level industrial solutions, where stakes are wa-a-a-a-y higher and general consumer is a non-existent agent altogether.

Yet half of webpages struggle to maintain smooth 60fps scroll on the latest top-of-the-line MacBook Pro.

You care, I care, but most people don’t. Not because they’re stupid, they just don’t notice these things. And we can dream all day long that engineers finally wake up and realize the scale of the crap-monster we’ve been building for years, but I don’t think any significant change will happen until the general population starts to care.

I’ve seen this just a few weeks ago: a professional marketing specialist is using a high end laptop with some bloatware in the browser. Forget 60fps, his pages were doing 10-12 fps at best, and every time he moved the cursor to close a tab, an antivirus popup appeared on top of the button. He says “damn!”, moves the cursor away, then slowly moves it back again from a specific angle, carefully trying not to invoke that popup. He succeeds and carries on with his task. I asked him, it’s been months like that. Hundreds of times. Every day. He just doesn’t know better, this is what computers are.

When a car makes barely works, we think it needs to be fixed asap. When a computer barely works, we think “those damn computers!”.

The things that contributed to your depression are often minor annoyances to the people. Sorry. This whole statement is a good catalyst for a whole another mental issue, fuck.

This is why I stopped pleading to the developers and started pleading to the users. I want users to demand more and be angrier with the promise of computing.

But this is a wrong battle anyway, I think. You, me and our peers are in the 0.1% of the world population when it comes to opportunity, wealth and availability of technology. We care about browsers’ fps because we don’t think about whether or not our children will eat tomorrow. Not to dwell on the “first world problems”, this is how global economy works.

Today when we say “the population is growing” and “the internet is growing”, we actually mean “China, Asia and Africa are growing and connecting to the global economy and the internet”. Software industry is as global as globalization goes, and, like any other industry, it often adapts to the lowest denominator with the best margins.

Millions of people move from powerty to middle class, and they are “the big wave” of new users for technology. If your father died of hunger, but you suddenly found yourself having a job and buying a smartphone, I bet it’ll be a long time until you start caring about Android core size and Chrome’s render speed. Even if you know a thing or two about technology.

What I’m trying to say is:

Most users in the developed countries are used to bad software.
Most users in the developing countries are conditioned to bad software from the beginning.

This is weird! An average Western European family has very different notions of “enough food” and “a good job” and “nice life” to an average rural family from a developing country. But when it comes to, say, Android apps, both families have pretty similar experience and expectations, I guess.

Insert equality rhetoric.

Why software 20-40 years ago was actually faster, more stable and nicer to use (as long as you invest some time to RTFM)? Because it wasn’t built for the general public and wasn’t built by the general public, it was very limited in both audience and developers.

Today there is no good incentive to make good software unless it’s some highly specific professional product.

One can easily interpret this in a wrong way, I believe. Software becoming a mass product is ultimately good, but comes with some harsh transition periods.

Google Inbox, a web app written by Google, running in Chrome browser also by Google, takes 13 seconds to open moderately-sized emails.

Well, Google inbox is discontinued next Spring, so, that’s not a problem anymore. Especially for Google.

But, here comes my second point: even if you care about this stuff, you still use it, and as far as Google is concerned, that’s a success. I asked you recently why did you switch away from Firefox back to Chrome, even though so many Google’s design decisions are appalling for you and me. And I knew the answer: we don’t have much choice. There are just a handful of alternatives, and everything is bad.

Individual developers at Google will probably agree with you, they probably care a lot about all that stuff. But Google-the-company is not the collection of those developers, it’s another organism altogether. That organism, just like any other evolutionary being in a competitive global economy, tries to do the least to get the most. It’s a corporation, it wants to make… no, scratch that, it wants to have money, but not necessarily make it.

Windows 10 takes 30 minutes to update. What could it possibly be doing for that long?

Yeah, so? Is Microsoft having any difficulties because of that? Maybe, but I don’t think they believe that. What can you do? Switch to Linux, ha?

Yeah, if that update takes 24 hours, I bet not much changes, people will still use it, because for the majority of people Windows = Computers. It’s not “windows is slow”, it’s “this is what computers are”.

And organizations will develop special routines and systems to deal with that “intrinsic nature of computation”: have “windows update” weeks. As long as everybody in the world suffers, there is no competitive advantage of having faster updates. Or better software.

Android system with no apps takes almost 6 Gb. Just think for a second how obscenely HUGE that number is.

Look at plastic and garbage in general. The amount of packaging is staggering, but only a handful of activists care. It’s hidden and doesn’t really affect us. Yeah, you need larger garbage bins, whatever. Just buy them.

Just like you need a larger SD card or a new phone. Just buy them.

My third point is that software industry consists of amateurs, mostly (see https://rakhim.org/2018/07/software-shouldnt-fail/).

“The Web was the first global technological phenomena that was built and maintained by the amateurs. Computer hardware, software, and the internet itself were built by mathematicians and engineers. The Web was built by people like me.”

And that middle class thing comes into play again. Being an amateur web developer is a way to bring your family out of poverty for many people. See success stories of many boot camps and such. If I couldn’t provide for my family, but then I learned how to combine 10000 node modules into an electron app, and some company pays me money for that, I will happily make a lot of electron apps long before I start worrying about problems that most of the users don’t have.

Bad sofware design and bad UX are ethically acceptable.

The machine is self-supporting and recursive: the more amateurs build software, the more developers we need to support it, thus creating more demand for new people becoming developers ASAP. More amateurs building software creates more amateurs building software.

Keep in mind that this is a net positive result for the individual lives of people and communities in the short term, even though it might be a net negative for the civilization at large. Considering this, talking about this is very difficult.

A 3D game can fill whole screen with hundreds of thousands (!!!) of polygons in the same 16ms and also process input, recalculate the world and dynamically load/unload resources. How come?

Several things:

Gamers care. See millions of views and comments to videos about minor differences in gaming performance.
A game costs 60€. People can return games if they’re slow or bad (because they care). An iPhone/Android app costs cents. You can’t return them. Free apps cost nothing and have miniscule margins of profit for devs (ads).
Slow games are actully unusable. You can’t play at 15fps, it’s just physically uncomfortable. But reading web at 10fps is, well, like Kindle. It’s fine.

Another big idea about games I wanted to refer to is console games. This is the industry we can learn a lot from! Unlike PC games, console games seem to be much more stable. Because when Sony unveils a new PS, it says “this thing is THE console for the next 8 years. Have fun!”

Game devs know the schedule, and can take their time to tailor code to that particular, immutable and stable system. This is why every year new console games look better and better, even though the underlying hardware doesn’t change at all. Devs squeeze the shit out of the resources.

Compare first PS3 games and last PS3 games. It’s crazy. Same hardware!

Web developers don’t care that much, next year their product will probably work faster (given it doesn’t break due to browser update or API deprecation), because next year the average smartphone CPU will be faster.

A simple text chat is notorious for its load speed and memory consumption. Yes, you really have to count Slack in as a resource-heavy application. I mean, chatroom and barebones text editor, those are supposed to be two of the less demanding apps in the whole world. Welcome to 2018.

Notice how the biggest and probably one of the most complex software projects in the history is being developed without Slack. Those Linux kernel devs, how can they work like that?! Without real time chat! Without Slack integration with CI and github?! Without notifications?! They still communicate via email, those weirdos!!!

I often hear this: “Slack is great because of integrations, we see errors and status updates and CI live in our chat!”.

When the whole world is updating like crazy because everyone else is doing this, not necessarily because it’s intrinsically good for the users, and the teams have to grow big to cope with the speed, and the technology has to be fragile and complex because Lean and Agile… you have no choice but to monitor and react to the system updates like a team of military doctors.

This doesn’t make Slack an intrinsically good product. But it’s necessery given the state of things.

“This complex portable surgeon robot is great, it allows us to move fast every time we shoot ourselves in the foot!”

So, you can’t make truly good apps because you’re a team of amateurs in the world full of similar competition, and to be able to move you NEED slack. If slack is 2x slower tomorrow, you take it, you NEED it.

Nobody understands anything at this point. Neither they want to. We just throw barely baked shit out there, hope for the best and call it “startup wisdom”.

Yup. 100% this.

It seems, individual users don’t matter anymore. As long as the final majority of users end up in the “okay, I guess it works” state, we’re golden.

I’m all for your “Better world manifesto”, sign me up. But I think that developers are not the bottleneck, the users are. We do have the responsibility, but this is an industry change we’re talking about, and only markets seem to be able to effectively change industries.

Until we all live in some perfect society, there will be huge markets full of users with “other problems than your stupid app”. And as long as it makes economical sense to produce cheap crap, it will be produced.

Dumb Down the Context Until the Problem Goes Away

Fri, 14 Sep 2018 21:10:00 +0300

At work we use SCSS and HAML, so I rarely write pure HTML and CSS there. But for small side projects and my personal blog I tend to stick with the simplest (and dumbest) possible tools. This week I was working on a refreshed look for this blog. Being a good modern man, I tried to stick with em or rem for sizing and typography.

Using em means adding state to your specs, and I don’t like this. Looking at a particular element, it can be impossible to understand what em means. So rem it is.

The value rem is “equal to the computed value of font-size on the root element”, so starting with this:

html {
font-size: 21px;
}

we suppose to get a universal and stable variable. 10rem now means 210px. Cool? Not so much.

I wrote a simple media query to make headers smaller on narrow screens:

@media (max-width: 34rem) {
h1 {
font-size: 2.369rem;
}
}

But it doesn’t work at the specified break point of 34rem = 714px. Turns out that in media queries rem means “initial value of font-size”, as per spec¹. It’s 16px in most browsers.

You have two lines of code near one another, and the same symbol means different things. Check out this demonstration. And you dare to complain about mutations in your imperative programming language!

Using em’s in media queries brings problems as well. So, in the end, pixels are the only units that behave consistently across all browsers and don’t add hidden qualities to your styles.

I then thought okay, I can get around this problem by using calc, which seems to be supported in all browsers nowadays. Nope, it doesn’t work in media queries.

The first thought that came after that is almost a reflex for many web developers alike: just use some tools on top of this ugly and inconsistent language!

A pre-compiler like SCSS provides variables and calculations and other sweet features. It can seamlessly generate final CSS if you enable a watcher, or even better, set up something like Gulp or Webpack (oh, god). But then it’ll be kind of difficult to use the web inspector in the browser, since it shows the final CSS, but I never work with it directly.

Oh, no worries, you can generate source maps for SASS/SCSS. Magic²!

But wait… While this solves my problem, it adds a tremendous amount of complexity. Is it worth it? Clearly, not in my case, but for a huges project like Hexlet at my main job it clearly does. Where is the threshold? How does one know when it’s worth to invest into a set of new abstractions that comes with their own quirks and problems?

It’s a difficult question, but for me and my small projects I found it important to remind myself: resist complexity at all costs, resist adding new things into the system. If my problem asks for a solution that involves additional tools or systems, first and foremost consider dumbing the whole thing down so that the problem goes away. By regressing to pixels, which are so “not modern”, I managed to avoid a whole bag of cruft being put on top of this primitive project. The system became dumber. It’s a win for me.

This is a weak example, I agree, so let me provide another one. Few years ago I needed to launch a small wiki site. Many popular wiki engines (like MediaWiki) are way too complex and feature-rich, so I looked for simpler alternatives. I found a nice Ruby library³ and spend few hours setting it up, providing custom templates and styles. I was happy with the result, but then I found myself daunted by the worst part: deploy and maintenance.

Of course, setting up a server by hand is a no-no, so I had to write an Ansible recipe for Ubuntu Rails environment. Accidental complexity involved in this problem became so large I started forgetting what I was trying to achieve.

It took me some time to realize that the primary audience for this wiki will actually be much more comfortable editing text directly via Git rather than fiddle with a web interface. And if it’s hosted on GitHub, I don’t have to worry about authorization and accounts. I still needed it to run on my domain with some specific HTML, so I just made a simple Jekyll site and provided links to quickly edit and send pull requests via GitHub.

I had problems associated with deployment and maintenance, and instead of adding tools as solutions, I dumbed the whole context so that the problems went away.

Note that these problems are often of accidental complexity type. Intrinsic, real problems don’t surrender this easily.

If playing with lots of inter operating tools is fun, by all means go for it. As long as you remember and realize what is going on. Complexity is not inherently bad, it’s just sort of cunning when you’re not mindful.

A Simple Introduction to Proof by Contradiction

Wed, 12 Sep 2018 17:49:00 +0300

In mathematics, a theorem is a true statement, but the mathematician is expected to be able to prove it rather than take it on faith. The proof is a sequence of mathematical statements, a path from some basic truth to the desired outcome. An impeccable argument, if you will.

One of the basic techniques is proof by contradiction. Here is the idea:

Assume the statement is false.
Derive a contradiction, a paradox, something that doesn’t make sense. This will mean that the statement cannot possibly be false, therefore it’s true.

When I first saw this formal technique, it puzzled me. It didn’t seem to be valid: alright, assuming something is false leads to a paradox, so what? We haven’t proven that assuming it’s true doesn’t lead to another paradox! Or even the same paradox, for that matter. What I failed to understand conceptually is that a statement is a binary thing: it’s either true or untrue. Nothing in between. So, if one can definitely declare “X is not false”, then no other options are left: “X must be true”.

Direct proof

To demonstrate this, let’s first use another technique of a direct proof so that we have something to work with.

Theorem 1. If $n$ is an odd positive integer, then $n^2$ is odd.

A direct proof just goes head in, trying to see what the statement means if we kinda play with it.

Proof. An odd positive integer can be written as $ n = 2k + 1 $, since something like $ 2k $ is even and adding 1 makes it definitely odd. We’re interested in what odd squared looks like, so let’s square this definition:

\[ n^2 = (2k + 1)^2 = \]
\[4k^2 + 4k + 1 = \]
\[ 2(2k^2 + 2k) + 1 \]

So, we have this final formula $ 2(2k^2 + 2k) + 1 $ and it follows the pattern of $ 2k + 1 $. This means it’s odd! We have a proof. ■

This theorem is based on an idea that numbers described as $ 2k + 1 $ are definitely odd. This might be another theorem that requires another proof, and that proof might be based on some other theorems. The general idea of mathematics is that if you follow any theorem to the very beginning, you’ll meet the fundamental axioms, the basis of everything.

Now that we have this proven theorem in our arsenal, let’s take a look at another theorem and prove it by contradiction.

Proof by contradiction

Theorem 2. $n$ is a positive integer. If $ n^2 $ is even, then $n$ is even.

We may try to construct another direct proof, but creating paradoxes is much more fun!

Proof. Let’s assume that $n^2$ is even, but $n$ is odd. This is the opposite of what we want, and we will show that this scenario is impossible.

$n$ is odd, and from Theorem 1 we know that $n^2$ must be odd. This doesn’t make sense! Our assumption and our conclusion are the opposite. This is a paradox, so the assumption was wrong. Meaning, the idea “$n^2$ is even, but $n$ is odd” is false. Therefore, the idea “$n^2$ is even, $n$ is even” is true.■

Famous irrational $ \sqrt{2} $

Theorem 3. $ \sqrt{2} $ is irrational.

Woah, this is… different. In the first two theorems we had formulas, something to play with, something physical. This now is just an idea, so how would we even start?

Let’s start with a definition.

In mathematics, the irrational numbers are all the real numbers which are not rational numbers.¹

Doesn’t seem helpful, but let’s continue. What are rational numbers then? Are they some reasonable beings who make optimal decisions all the time?

A rational number is any number that can be expressed as the fraction $\frac{p}{q}$ of two integers.²

Oh! They are rational because they are ratios!

Just to make things super clear, let’s dig one more step and make sure we understand integers.

An integer (from the Latin integer meaning “whole”) is a number that can be written without a fractional component. For example, 21, 4, 0, and −2048 are integers, while $9.75$, $ 5\frac{1}{2} $ and $ \sqrt{2} $ are not.³

Combining these things, we can construct a comprehensive definition of an irrational number: it’s a number that cannot be expressed as the fraction of two whole numbers.

Now, let’s apply this to Theorem 3 so that it has some meat:

Theorem 3. $ \sqrt{2} $ cannot be expressed as $ \frac{p}{q} $, where $p$ and $q$ are integers.

Alright, now there is something to play with!

Proof. Start by assuming the opposite – $ \sqrt{2} $ is rational. This means it can be written as a fraction of two integers:

\[ \sqrt{2} = \frac{p}{q}\ \]

We can assume that $p$ and $q$ are not both even, because if they are, we can reduce them by dividing both by a common factor (like, for example, $ \frac{8}{10}\ $ should be reduced to $ \frac{4}{5}\ $). In other words, if they are both even, reduce them until at least one is odd and no further reductions are possible.

Now, let’s square the square root:

\[ (\sqrt{2})^2 = \frac{p^2}{q^2}\ \]

\[ 2 = \frac{p^2}{q^2}\ \]

\[ p^2 = 2q^2 \]

Remember, something like $2k + 1$ is odd, and $2k$ is even. Here we see this pattern: $p^2 = 2q^2$, which means that $p^2$ is even (it consists of two things).

Then, using Theorem 2, we can say that $p$ is even as well, which means we can write $p$ as $p = 2k$. So:

\[ 2q^2 = p^2 = (2k)^2 \]

\[ 2q^2 = 4k^2 \]

Divide both by two:

\[ q^2 = 2k^2 \]

So, $q^2$ is even. By the same Theorem 2 it follows that $q$ is even.

Let’s summarize the two conclusions:

$p$ is even.
$q$ is even.

Wait… We made sure that not both $p$ and $q$ are even before starting this whole thing! We made sure to reduce them until at least one is odd, but then, by applying Theorem 2, we ended up with two even numbers. This is impossible, so the idea that “$\sqrt{2}$ is rational” is not true.

Therefore, $\sqrt{2}$ is irrational.■

P.S. I often use proof by contradiction in real life by arguing that, for example, not eating the whole bucket of ice cream at once will lead to a paradox that endangers the whole fabric of space-time. It works for me, but your mileage my vary.

Keyboard fanaticism

Mon, 10 Sep 2018 16:54:00 +0300

I’ve been reading an article about Emacs, and this paragraph had nailed me right into the soul:

IDE users spend most of their time fumbling around with the mouse. They wouldn’t dream of doing it any other way, but they don’t realize how inefficient their motions are.

…

Whenever you need to jump the cursor backward or forward more than about 5 lines, and you can see the target location, you should be using i-search.
…

Let your eye defocus slightly and take in the whole paragraph or region around the target point, and choose a word that looks reasonably unique or easy to type. Then i-search for it to navigate to it. You may need to hit Ctrl-r or Ctrl-s repeatedly if your anchor word turns out not to be unique.

This is a common rhetoric: use keyboard only, don’t you dare to use the mouse — it’s so inefficient!

The scenario in question is simple: you have to move the cursor to some position you see on the screen. Instead of moving your hand to the mouse to move the pointer, the author suggests the following algorithm:

Determine if the place you need to go to is before or after current position. This is non-zero mental work.
Take a look around that point and “choose a word that looks reasonably unique”. Perform more mental work of determining which word is unique enough.
If the target is before the current position, use Ctrl+s. If it’s after, use Ctrl+r. This is more or less automatic, but still required mental work of maintaining the mapping between direction and binding.
If your judgement of the uniqueness wasn’t good enough, you’ll end up somewhere else. Possibly, in a completely different section of the document. Additional mental work — you have to realize what happened, disoriented. Keep hitting Ctrl+s or Ctrl+r. And you have to keep scanning the surroundings every time you jump until you get where you want.
Okay, you’re there! But remember, you’ve been jumping to a place near the target, so now you have to move a bit more — by word or by character.

Mastering it simply requires that you do it repeatedly until your fingers do it “automatically”. Emacs eventually becomes like an extension of your body, and you’ll be performing hundreds of different keystrokes and mini-techniques like this one without thinking about them.

While I understand the premise completely, and I occasionally use the same technique, I can’t help but think an advice like that rarely takes into account the trade-off. Yes, moving your hand to the mouse takes time, but it’s not uncommon that the time required is actually less than multi key multi step keybinding. Instead of spending a second, two motions and a single click the user is advised to analyze text, make several decisions and hit multiple keys, which might or might not be enough. But hey, you didn’t leave the home row, so, win, I guess?..

I’m not defending the mouse here, but I do think there are occasions where using the mouse is just better for me. Too often these articles are trying to make you feel like an unintelligent cave man for daring to use the “device of IDE users”.

Also, Emacs packages like avy or evil-snipe make jumping to visible text much simpler and cost less mentally.

The vast sea of discussions and advice about programming tools and especially text editing is full of opinions, approaches and cult-like repeated revelations. Often, the loud sounds of the echo chamber make it difficult to stop for a moment and evaluate something yourself. But please do try.

It’s easy to be indoctrinated.

Personal finances and controlled anarchy

Fri, 07 Sep 2018 12:31:00 +0300

Most of my life, I didn’t have much money. I was born in the 90s in a tiny Kazakh town, and nobody had a job there, it seemed. USSR just collapsed and my parents were trying to make ends meet in a constant hustle.

That life defined my relationship with money and wealth for years to come. Every time I had to buy coffee, I was thinking “is it worth it?.. Maybe I shouldn’t”. In restaurants and cafes I was looking at prices first, then at meals. “Hmm, this cheap pasta looks so attractive! Ooh, I bet this expensive steak is not that good”.

“You know what? I’ll just buy this coffee and not buy that iPhone game I wanted to buy. Yeah, perfect! Now I’m calm and safe!”

After graduating and starting working full time I decided to follow a popular advice: budget everything. I started using an excellent app called YNAB — You Need a Budget. Not only it allows you to track all your expenses and plan ahead, it also comes with a philosophy, a set of rules and ideas to help you navigate your personal finance world.

First things first: save one month worth of expenses and never let your account get dry. The idea is to be spending money that is at least 30 days old. So, if you got your salary on February 1st, you will spend this money in March or later, but not in February. This way you never get into «I need some money until my next salary».

Next, give every dollar a job. This means that each dollar you get — you decide what it’s for. I have regular expenses like rent, phone fees, groceries etc. Some amount of money MUST go there. But I also have other categories, like “Books” or “Electronics” or “Travel”. And if I want to buy a book or go for a vacation, I have to have enough money saved in that category.

Putting money into savings account is another type of a job.

YNAB allows you to assign every dollar a particular job. It actually encourages you to keep exactly ZERO cents unassigned! You feel like a finance director of a small enterprise. Serious business!

This way you know exactly whether you can afford something. And you never have to guess “hmm, if I buy this laptop now, will I be okay with the rent?..”.

YNAB classic app screenshot (not mine)

Another YNAB rule is to budget in detail and ahead. “Make your money boring” is their slogan for it. For every bill to come or an unexpected expense to surprise you, you’ll have money waiting.

For example, I was putting some money into “Car repair” category each month, even though for the most part my car didn’t require any repair. But when the AC compressor suddenly died in the middle of the hot German summer vacation, I knew I don’t have to worry.

Basically, save money for Christmas all year long, not just in December.

One more YNAB rule is to “Roll With The Punches”. When you overspend in a budget category, just adjust. No guilt necessary! It was easy for me to justify another gadget when I under-spent in some other categories.

I was an everyday user of YNAB for 7 years. The app itself is 14 years old and it has a great following and a nice community around it. It helped me tremendously! A huge amount of stress just went away, I was on top of my finances, I knew exactly what’s happening and how much money I’m getting and spending. When my girlfriend moved in with me and we started sharing our budgets, YNAB was able to accommodate it. I just added another bank account in the settings. In total, I was controlling multiple bank accounts (including “cash” account) and cards, several sources of income and tens of budgeting categories.

It was great.

So, why did I stop?

Don’t get me wrong: an app and a method like that makes a HUGE difference. I will never go back to having no control and no knowledge over my finances, but I still had lots of stress points.

First, it took a lot of time and energy to maintain the system. I had to put all the expenses precisely, every purchase, every fee, including cash purchases. The system makes sense only if you’re precise and 100% accurate.

Card purchases overseas were especially painful. They often change over time, like, you buy something off Amazon, and they charge you with currency conversion, and after a week or so an “adjustment” charge is made silently (since the exchange rate changed a bit). You have to track it all and “consolidate” your accounts every month.

Or you just forget what that $0.99 supermarket purchase was three weeks ago. Was it chocolate, so, groceries category? Or a LAN cable, so, electronics category? Does it really matter? It’s just 99 cents, so… whatever, let it be groceries.

Another problem was — I still had some stress over money. Less than before, but still. This “roll with the punches” rule is nice and liberating, but sometimes it seemed like I was just abusing the system. I want a new gadget, so, I’ll just transfer $100 from “car repair” and compensate next month by spending less in every category. It’ll be just fine!

It takes lots of energy not only to maintain the system, but to keep disciplined. I’m not that good at it.

A year ago I decided to deliberately simplify my life. Automate everything I can, ignore more stuff, eliminate pain points and minimize the mental energy requirements on everything except first-order things.

First-order things are the actual things I want to spend time on, the things that are intrinsically important for me. Money is a tool, so, it’s at most second-order. It allows for the first-order things, but it doesn’t have intrinsic value itself.

So I ditched YNAB and budgeting in general.

This was the most liberating moment I had in some time!

I call my new system “controlled anarchy”, and it’s pretty simple.

Every time our family gets salary payment or other income, I distribute it between three bank accounts:

Monthly bills. This account pays all the bills, from rent to Netflix. It has its own debit card, so I don’t really see the purchases very often. I know exactly how much money is spent, though, since all the expenses here are static. Like with YNAB philosophy, this account has 2 months of expenses all the time, so it never gets dry. (I am actually increasing this account to 6 months of expenses so it will act as the emergency fund. Bad things happen — we have 6 months to figure things out).
Savings.. Yup, just savings. At least 33% of all the income is saved. A portion of it is invested in mutual funds for the long term.
Everyday spending. The rest is free! This is the key — I don’t have to plan or to calculate or track anything. This account is the free money we can spend however we want! (Some of it goes to groceries, but the rest is truly free).

The Everyday spending account rarely gets to zero, and we never move money away from it. So, it actually grows gradually, and if we don’t spend it all one month, we get even more free money next month!

The goal is to eliminate guilt and uncertainty about purchases. You want that new thing? Just buy it if there’s enough money. Not enough in Everyday spending? Well, sorry, you can’t buy it. But hey — feel free to buy whatever — spend it all away!

Oh, man, this made our lives so much easier.

The “controlled anarchy” system lacks the precision of the previous one, but requires no time and energy to maintain. All the payments and transfers are automatic, it’s like we’re kids and a wise parent manages our spending money :)

This is what I call simplification: less decision-making means more energy for the truly important things.

Now, if you don’t do any sort of budgeting and don’t really control your money, I’m not sure going into “controlled anarchy” right away is a good idea. It seems like it’s alright, but maybe you should try real detailed budgeting first, maybe for a year or so, just so that you understand what’s going on, where money goes to.

Agile Food Truck

Fri, 24 Aug 2018 23:16:00 +0300

Reactive solution

Fri, 24 Aug 2018 21:16:00 +0300

Spaceship Problems

Sun, 19 Aug 2018 23:19:00 +0300

Podcasting is not walled (yet)

Tue, 07 Aug 2018 00:00:00 +0000

I’ve launched a podcast recently (EmacsCast) and received lots of feedback. One of the perplexing things people said was “that’s great, but how do I subscribe, it’s not on iTunes/Google Podcasts?”

I had a similar experience 10+ years ago when I started podcasting, but today it’s much worse. It worries me a lot.

Podcasts are simply RSS feeds with links to media files (usually mp3s). A podcast is basically a URL. And podcast clients are special browsers. They check that URL regularly and download new episodes if the content of the URL changes (new link added). That’s it, no magic, no special membership or anything else required. The technology is pretty “stupid” in a good way.

Video podcasts were a thing! It was basically distributed, un-censorable YouTube where both viewers and creators had full control over their things.

Ever since tech companies started waging war against RSS, podcast distribution became visually RSS-free. What do you do to subscribe? Easy, just search in the app! For the majority of iOS users that app is Apple Podcasts, and recently Google made their own “default client” for Android — Google Podcasts.

It looks like podcast clients are similar to web browsers and just provide a way to consume content, but the underlying listings make them very different. Corresponding services are actually isolated catalogs. When you perform a search on Apple Podcasts, you aren’t searching for podcasts. You are searching for Apple-approved podcasts. And if the thing you’re looking for is not there, then… well, you get nothing.

Imagine web browsers worked that way. You want to visit my blog, but you don’t know what URLs are, you’ve never seen or heard of things like https://rakhim.org. There’s no address bar in your browser, just a “search catalog” field.

I tell you my name and you type it into Safari. If my blog was already added to Apple Websites Catalogue, then great, you can visit my site.

But if I haven’t added my blog to their catalog, or, even worse, I’ve tried, but it wasn’t approved for some reason, then I’ll be just sitting here, shouting “dude, just go to https://rakhim.org, it’s there!”, but you’d have no idea what to do with that information. For you, my site doesn’t exist.

Your browser is capable of reaching my blog, but the feature is hidden or about to disappear. I can teach you to find some obscure menu item, but I can’t do it for all potential visitors.

Most Podcast clients still accept RSS. Apple Podcasts, iTunes, PocketCasts, OverCast, PodcastAddict.

Google Play Music doesn’t say anything explicitly, but you can just put RSS URL into the search field and it works. For now. I won’t be surprised if these apps gradually and silently remove this feature.

Lately, there are lots of discussions about Apple, Facebook, YouTube, and Spotify banning Alex Jones’s Infowars podcast. And most of the debate comes down to either “they have a right” or “they shouldn’t censor”.

The thing is, this shouldn’t matter. If some company decides not to include an URL in their catalog, this shouldn’t really matter. An URL is an URL, the content is there. Both Apple and Google are pretty much hiding the feature that makes podcasting as free and un-censorable as websites.

And it works! For most people, RSS for podcasts don’t exist, and a corporation is in charge of what they can listen to.

Last time I was talking about programmers’ responsibilities when it comes to software reliability. Today I want to add: we, developers and software engineers, also have the moral responsibility to educate the public about the way the internet was supposed to be. Open, un-censorable, with underlying protocols put in place to ensure it’s a forest, not a walled garden.

We shouldn't let people get used to the idea that software fails

Wed, 11 Jul 2018 00:00:00 +0000

Society has certain expectations when it comes to engineering and technology. We expect buildings and bridges to basically never fail. We expect cars to be extremely safe and fool-proof, as much as explosions-based metal boxes moving at hundred miles per hour can be safe and fool-proof. Home appliances are supposed to work for years without any serious maintenance. Electricity is just there, always. And when it goes out, we are incredibly frustrated and surprised.

The high expectation of engineering runs deep into relevant industries and educational institutions. Engineers take oaths!

The Obligation of The Engineer

I am an Engineer.

In my profession I take deep pride. To it I owe solemn obligations.

As an engineer, I, (full name), pledge to practice Integrity and Fair Dealing, Tolerance, and Respect, and to uphold devotion to the standards and dignity of my profession, conscious always that my skill carries with it the obligation to serve humanity by making best use of the Earth’s precious wealth.

As an engineer, I shall participate in none but honest enterprises. When needed, my skill and knowledge shall be given without reservation for the public good. In the performance of duty, and in fidelity to my profession, I shall give the utmost.

It might seem like I’m about to start a rant about buggy OS updates and how software engineering must be held accountable as much as industrial engineering. I’m not. Just to be clear: yes, I think the state of modern software development is ridiculous. Yes, seems like it’ll take quite some time and quite a few technological catastrophes until we take software more seriously.

What worries me more than the current state of the things is how it affects the mindset of society. These high expectations of engineering are reversely mirrored in software. We have learned to expect software to fail.

Just a few decades ago it was different. Software developers were the only ones who sometimes expected software to fail. The general public had high hopes, and journalists helped build it.

The rise of information technology was akin to the rise of mechanization of the early XX century. We could see the analogy clearly: in the 20s and 30s everyone in the developed world started having and using cars, washing machines, air conditioners and other wonderful devices. They were new, yet incredibly reliable. Every year they became better and cheaper.

In the 80s and 90s, everyone in the developed world started having and using computers and mobile phones. And at first, the process looked very similar. The first personal computers were clunky and weird, but pretty reliable. I’d say, incredibly stable and relatively fast compared to modern computers.

Of course, they were million times less complex and had fewer features. But hey, here is something to consider: I’m typing this in a macOS app on a very powerful 5K iMac computer. It’s just text, and I’m going to publish it on the web. The whole experience is mediocre at best. The software is not blazing fast, even though it’s a native app for text editing (it has very high app store rating and it’s one of the apps selected by the app store editors). The typing experience is laggy on a Bluetooth keyboard. To publish this, I’m going to interact with a browser, which is at this point the whole other operating system with another layer of delays. The only thing that really feels “20 years better” is the connection speed itself.

Almost slipped into a rant there.

Personal computers felt like the new “cars for everyone”. We all get computers! They will get better, faster and cheaper every year! They are becoming the backbone of the world, as much as motor vehicles did.

It all started to change with the Web. The Web was the first global technological phenomena that was built and maintained by the amateurs. Computer hardware, software, and the internet itself were built by mathematicians and engineers. The Web was built by people like me.

Mathematicians and engineers are tightly connected to the academia, and they operate with ideas like “proof”, “peer review” and “oath”. While many software developers operate with ideas like “ship early, ship often”, “move fast and break things” and “If you are not embarrassed by the first version of your product, you’ve launched too late”.

This isn’t bad in the long run. I think the society will go through The Age of the Amateur and get better in the end.

Amateurs were making things all the time. Up until the recent years, we just couldn’t make our amateur inventions ship to millions of people that easily. When amateurs invented something potentially useful, it took quite a few iterations and stages to get it to the public. By that time it became more refined and tested and stable, thanks to regulations, selection and just time.

Today I can make a bad piece of software that does something interesting, and potentially millions of people will get frustrated or harmed.

If you asked a member of the general population in the 90s or early 2000s about computers and the internet, I bet most of them would sound optimistic. That’s what everyone is talking about, right? Soon, we will do everything with computers! Computers are super smart!

While this is anecdotal evidence, today most of the people I know are frustrated with technology. Apps are buggy, the web is filled with ads and intrusive useless notifications (would you like some cookies?), touch screens everywhere suddenly made simple things like washing machines and car control panels barely usable.

We just got used to that. Electronics is something that’s wonky and buggy. That’s what we expect.

And this is scary.

When a society expects certain industries or institutions to fail or do harmful things due to lack of quality control or unprofessionalism, we think that society is in bad shape. When we expect politicians to fail, we revolt. When we expect medicine to fail, we protest. When we expect infrastructure to fail, we at least write angry letters to our representatives and try to make some difference through political means.

We can’t do much when software fails. Unlike science, there’s no public accountability. Even in those shrinking areas where there’s still competition, there isn’t much choice: all software fails. I, for one, can only name a couple of software products that are rock solid. But they don’t exist on their own, they operate on top of multiple other layers (hardware, OS, frameworks, web, etc). And it’s unlikely that all the layers are as good.

I’m in no position to propose a better way for the world to evolve. But I believe that computer scientists and software engineers have the moral responsibility to educate the public about the way software should be. We shouldn’t let people get used to the idea that software inevitably fails.

(Discussion on Lobste.rs)

What is binary?

Fri, 02 Dec 2016 00:00:00 +0000

My attempt to explain binary numbers.

Backups

Tue, 29 Nov 2016 00:00:00 +0000

I’m not going to try to convince you to backup data. I didn’t do backups for the most of my life, except for some photos and videos here and there. And those weren’t really backups, more like archives on external HDD’s.

I use computers every day since, gosh, I guess 5th grade, and never ever did any hard drive fail on me. This is remarkable. Screens, keyboards, mice, even fans have died over the years, but not hard drives. Good old magnetic, noisy, spinning monsters.

Backup evangelists love to say how “your HDD will fail, it’s inevitable”. Well, yes, it will fail inevitably as you use it for years, but the truth is — you’ll more likely switch computers before your drive fails.

But the truth is, it’s extremely unlikely that you’ll ever need a backup. You’re probably good. I prefer to think about this stuff as an insurance. This is how you buy peace of mind. There are so few aspects of life where you can actually do that — pay some money (not much, too, which is great), and get some peace of mind. Medical insurance, for example, is sort of like that, but not really that good. With data backup, I can be pretty sure I will get everything back as it used to be, effectively travel back in time. Recovered data is precisely the same as lost data, so it’s not really lost anymore. While my body after medical treatment is not the same anymore.

I’m not gonna go all “3-2-1 backup” on you. That’s the idea that in the perfect world you need at least 3 total copies of your data, 2 of which are local but on different devices, and at least 1 copy offsite. So, if all of your files are on your computer, then you need two external hard drives and a remote hard drive (maybe one at work, in another house or in the cloud). I don’t do that yet. For now my backup strategy looks like this:

Offsite backups with Arq (Dropbox and Amazon Cloud Drive)
Local backups with Time Machine and Carbon Copy Cloner

Most of the things I work on daily are on Github (personal and work), Dropbox (personal) and Google Drive (work).

Let me first explain why I said no to Backblaze and Crashplan. Long story short: I don’t trust them.

Backblaze

Backblaze is a beautiful, sleek guy who says “don’t worry about it bro”. Mac client is minimal and cool, and it “just works”.

There are few issues with Backblaze:

It’s not really a backup solution. If you delete a file from your computer, then in one month it will also be deleted from Backblaze’s servers. It syncs stuff just like Dropbox does. This is why Dropbox in and of itself is not a proper backup solution.
If you backup an external drive and disconnect it from your computer, then Backblaze will delete that backup from their servers.
Backblaze doesn’t backup 100% of files. You can remove some exceptions manually, but some of them are built in. So, I can’t really have a complete copy of my boot disk, for example.
If you need to restore files from Backblaze, you’re gonna have a bad time. Your options are: download a Zip-archive (if your file is 10 levels deep, then you’ll get all the upper level folder structure in the archive) or get a flash drive or a HDD via mail. You can’t restore files in place.
Some users say it’s very fast, some say it’s very slow. I can’t figure out the reasons, but for me it was dead slow. It took almost 120 hours (5 days) to upload less than 200G of data.
Android app seems to be made by a very intelligent puppy.

Crashplan

Crashplan is a douchey-looking guy in an expensive suit who says “the synergy is just overwhelming in this merger”. It took me a while to understand how it really works. It’s called CrashPlan Online Backup and it can backup, among other places, to your external hard drive. You know, because online.

But once you get it, it’s pretty great in theory. With Crashplan you can backup to any local hard drive and offsite machine (like friend’s computer or any other machine in your network) for free, and with additional fees you can also backup to Crashplan’s cloud.

If you use their cloud, then it’s really a backup solution. All your files are stored in the cloud forever^*. Unlike Backblaze.
You can restore any file to its original location or any other location. No need to download Zip-files from a cumbersome web interface.

(* not forever)

Crashplan Mac app is… well… ugly like hell.

Look at this Java shit.

I was happy with Crashplan for the first few days, and possibilities of adding more backup destinations if I decide to go all “3-2-1” was reassuring. But it turned out I can’t trust it.

Crashplan, like any other software of that type, is supposed to work in the background, doing its thing while I do mine. I was restructuring files in my photo archives, moving files from folder to folder, renaming stuff. Nothing extreme. But in few days when Crashplan said it backed everything up, I tried to restore some files to check how it works. And it just lost the whole photos folder, tens gigabytes of photos. That was the folder I was fiddling with.

I understand this is unfair. Trying to make a reliable copy of the file system while it changes is hard. But:

Should it fail, it must fail gracefully. Losing all the files is unacceptable.
The end user should never worry about this stuff. I wasn’t doing some crazy hacking, I just moved files around.

Oh, and Crashplan’s Android app seems to be made by a very intelligent puppy as well. Is there a software company ran by puppies I know nothing about?!

Arq

Arq is only an app for your Mac or PC. It doesn’t offer any cloud backup storage itself. It’s not even a guy like Backblaze or Crashplan. It’s a faceless, soulless robot who says nothing. This is what backup software should be like.

Arq can backup to Amazon Cloud Drive, AWS, Amazon Glacier, Google Drive, Google Cloud Drive, Dropbox, OneDrive, your SFTP server or NAS. You can set multiple sources and destinations. For example, I have the following setup:

Home folder → Dropbox
Photo archive → Dropbox and Amazon Cloud Drive
Podcasting archive → Dropbox
Current video projects → Dropbox and Amazon Cloud Drive
Work-related podcasting archive → Google Drive

And at any point I can add other destinations, including a network attached storage.

My Mac’s SSD is just 120G, but my Dropbox is 1TB. It’s great to finally make all that space useful by setting Dropbox as a destination for Arq. And Amazon Cloud Drive is just a great deal — unlimited storage for $60 per year. Of course, you have to remember, that it’s not really unlimited, and if you go crazy, Amazon is allowed to kick you out.

Some cool features of Arq include:

Local encryption. Arq encrypts files before sending it, and sends it with SSL.
Backups are stored in open documented format. Even if Arq dies and completely disappears, your encrypted data is safe and accessible.
A very nice and straight-forward native app with some advanced features like CPU usage, upload rate, scheduling, data validation, budget restrictions (relevant for AWS, for example). You can also make Arq run shell scripts before and after backup sessions.
Arq truly backups everything. All the files, any format, any size (even crazy tens-of-gigs files) without restrictions.
Fixed price and no recurring fees.

Arq also can archive: backup a folder, click “Detach…” and Arq will stop backing it up, but will store the previous backups indefinitely. This is great if you want to backup some external drives that rarely update.

Local backups: Time Machine and Carbon Copy Cloner

My photos and work-related audio- and video-projects live on an external drive and are backed up to the cloud along with the home folder. The whole internal SSD gets only local backups, because it’s not that important. There are two things I need from this system:

Restore the system to a previous state. І never needed this with Macs, but I just feel safer this way when I upgrade the OS.
Boot from USB drive if internal drive fails. As I said, I didn’t have failing drives (neither HDD’s nor SSD’s) in my life, but SATA cables do fail sometimes. Should that happen, I can just boot from an external drive and continue working until the problem is solved.

Time Machine is good enough for quick restoration. I use Carbon Copy Cloner to make an external bootable copy. Alternatively, SuperDuper! is also nice. I like CCC more because it also copies the recovery partition, which is used to reinstall macOS. You don’t really need it, because all modern Macs come with a Network Recovery option.

That’s it folks.

Home on Rakhim's homepage

The Yearly Run (sci-fi short story)

Blogging vs. blog setups

The wrong good solution

Eventual Consistency (sci-fi short story)

Small battery productivity

Fundamental Attribution of Ability Error

It's all just fashion shows

Weird Finnish elative case

Summary of Truth about Types, a talk by Bartosz Milewski

Intro

Truth

Category theory

Composability

Identity arrows

Set

Universal constructions in category theory

Initial object

Terminal object

Product

Function object

Negation

Cartesian Closed Category

Curry-Howard-Lambek

Logical universes

Intuitionistic logic

Goedel Gentzen

Conclusions

Summary of Transducers, a talk by Rich Hickey

Intro

Creating and using transducers

How do we get there?

Transducers are fast

Transducer types

Early termination

State

Completion

Init

Summary

Fastmail setup with Emacs, mu4e and mbsync on macOS

Installing and configuring mbsync

Installing and configuring mu and mu4e

Caveat 1: deletion vs. expunge

Caveat 2: Spam reporting

Sending mail

Links

Summary of Concurrency Is Not Parallellism, a talk by Rob Pike

Intro

Concurrency vs parallelism

Analogy

Concurrent composition

Different design

Finer-grained concurrency

Yet another design

Summary

Real world example

Goroutines

Channels

Select

Closures

Examples

Launching daemons

Simple load balancer

Another load balancer

Lessons

One more example: query a replicated DB

Conclusion

More information

The price of complexity

User Is Dead

Be Wary of Self Described Benefits

Process of Learning

Bicycles and Love

80-characters limit for text is wrong

Rethinking Twitter

Good old days

Outsourced schizophrenia

Change

Easy slides (for me)

How to podcast