22:17

Had a look at other distributed fs options. Might give Ceph a go based on this article but I’m always a bit wary I don’t have enough resources for Ceph, haha.

It seems so Enterprise and I don’t have Enterprise budget. I’ll see how it handles anyway.

19:41

swarmfs

I’m using docker swarm via coolify. I want to make a sorts-itself replicated file system to store persistent data on.

Handling multiple read writes and stuff like that is beyond the scope of this for now. Just don’t be silly and it won’t break.

Garage

Distributed object store exposed as an s3 API.

Note to self: XFS formatted (scroll to like the third best practise)

We recommend using XFS for the data partition, as it has the best performance. EXT4 is not recommended as it has more strict limitations on the number of inodes, which might cause issues with Garage when large numbers of objects are stored.

Rgr.

JuiceFS

Turn the s3 object storage into a POSIX (is that right?) file system and mount it for use on the parent node and from there services and containers

Two options here, give JuiceFS a mount like /swarmfs:/swarmfs and bosh it’s available on the host.

Option two: as a volume driver

Either way needs a redis server. Ideally clustered but I’ll just do the one nodule for now to get a proof of concept working.

Goal

I want it so that when I docker swarm join a new worker node to the cluster, give it a moment, and it’ll be running the garage+juicefs services ready to boot any service on the swarm.

Crazy wild day for traffic? Just start pumping out Fly Machines or something like docker swarm is going out of style.