Home on A blog

Running Postgres on Cloudflare Containers

Mon, 30 Jun 2025 23:06:23 -0500

Disclaimer: I work for Cloudflare. The views and opinions expressed in this post are my own and do not necessarily reflect the official policy or position of Cloudflare.

Cloudflare recently launched Cloudflare Containers allowing running containerized applications with workers. To use this, an user would need to define an application in a dockerfile, define a worker, and bind the container to that worker. Naturally, I was curious if we can run Postgres on the new container platform.

Postgres Protocol and a Side Quest

Thu, 10 Apr 2025 22:24:49 -0500

I was watching Heikki Linnakangas’s talk on the Postgres protocol. I wanted to follow along with and see the protocol in action for myself. I ran into a problem here: my ostgres server rejected all non secure connections. And with TLS enabled, I could not get wireshark to decode postgres protocol messages because of encryption.

Now, wireshark supports decoding TLS connections using a key log file. Looking through documentation I realized that there is no way for a client to log the TLS keys using psql. So the natural next step was to add that support and try to get that accepted upstream in postgres. Once I understood how this could be implemented, this became my side quest.

TIL: Looking at RocksDB Events

Sun, 22 Dec 2024 21:47:47 -0600

RocksDB has an eventing system that clients can use to listen on specific events within the DB. These events are written to the log file when RocksDB runs. The name of the log file is LOG by default and is located in the database directory.

The whole log file is not in json format, so we need to grep to convert the output to json. Here is a bash one-liner that continuously tails the log file and prints out the events.

pg_power: initialization and basic setup

Thu, 19 Dec 2024 20:46:06 -0600

I have been playing around with the powercap framework. I wrote a postgres extension that shows the energy usage of a query. Postgres has a hook mechanism that allows an extension to override the default executor. This implementation is very simple: the extension records the current energy reading when a query starts and then calls the actual executor that runs the query. When the query finishes, a second hook records the current energy reading. The overall energy usage of this query is the difference between the two values.

TIL: Read sysfs without root access in Linux

Sun, 10 Nov 2024 14:48:48 -0600

Typically an user needs root access to be able to read sysfs data. But such root access might not be possible in all case; an example being Postgres which cannot be run as root. I recently learned that it is possible to set an extra capability to executables such that it can read sysfs while not being root.

I wrote a simple C program to read a specific sysfs entry

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>

#define SYSFS_PATH "/sys/devices/virtual/powercap/intel-rapl/intel-rapl:0/energy_uj"
#define BUFFER_SIZE 32

int main() {
    int fd;
    char buffer[BUFFER_SIZE];
    ssize_t bytes_read;
    unsigned long long energy_uj;

    // Open the sysfs file
    fd = open(SYSFS_PATH, O_RDONLY);
    if (fd == -1) {
        fprintf(stderr, "Error opening file: %s\n", strerror(errno));
        return 1;
    }

    // Read the contents of the file
    bytes_read = read(fd, buffer, BUFFER_SIZE - 1);
    if (bytes_read == -1) {
        fprintf(stderr, "Error reading file: %s\n", strerror(errno));
        close(fd);
        return 1;
    }

    // Null-terminate the buffer
    buffer[bytes_read] = '\0';

    // Close the file descriptor
    close(fd);

    // Convert the string to an unsigned long long
    energy_uj = strtoull(buffer, NULL, 10);

    // Print the result
    printf("Energy (µJ): %llu\n", energy_uj);

    return 0;
}

When I compile and run it as my regular non-root user, I get this

Mapping Between Pgbouncer and Postgres Connection States

Sun, 27 Oct 2024 11:21:48 -0500

Pgbouncer is a popular connection pooler for Postgres. Connections from the end user to pgbouncer are called client connections from it’s point of view. Connections from pgbouncer to Postgres are called server connections. In a typical deployment, end users connect to pgbouncer, which maintains a pool of persistent server connections. The pooler assigns a server connection to a client connection based on the given pooling mode (that is beyond the scope of this discussion). Thus, from Postgres’ point of view, the client is actually pgbouncer which gets it’s own backend. Hence there is always a 1:1 correspondence between pgbouncer’s server connection and it’s backend in postgres. A mental model for their relationship is useful for debugging pgbouncer related issues. We will start with some definitions to establish that mapping.

TIL: Getting the current capacity of a named pipe in Linux

Fri, 17 May 2024 21:43:58 -0500

A named pipe (also known as FIFO) is a IPC channel, it allows one process to write and another to read. Like all things in Linux, it is implemented as a file like entity using either mkfifo or pipe system calls.

Pipes on Linux have a limited size, typically set to 65536 bytes on modern systems. I recently came across a situation where I needed to know the current remaining capacity of a given pipe without consuming data from it, while one process is writing to it and another is reading from it. I found that the ioctl system call takes in a flag FIONREAD that makes it read the count of unread bytes in a given pipe, identified by the file descriptor of either ends. The count is placed in an int buffer that ioctl takes in.

Measuring energy usage of a function

Tue, 07 May 2024 16:43:04 -0500

I recently needed a way to measure how much energy a piece of code consumes. This is easy to do in Linux using the perf utility assuming the underlying kernel has been built with appropriate flags. This looks something like this

$ sudo perf stat -a -e power/energy-pkg/ ls -la /usr
total 116
drwxr-xr-x  14 root root  4096 Mar 28 02:08 .
drwxr-xr-x  19 root root  4096 Apr 16 16:02 ..
drwxr-xr-x   2 root root 36864 May  7 15:05 bin
drwxr-xr-x   2 root root  4096 Apr 18  2022 games
drwxr-xr-x  37 root root 16384 May  7 15:51 include
drwxr-xr-x  88 root root  4096 May  7 15:05 lib
drwxr-xr-x   2 root root  4096 Mar 28 02:08 lib32
drwxr-xr-x   2 root root  4096 Apr 19 06:14 lib64
drwxr-xr-x   9 root root  4096 Apr 16 18:02 libexec
drwxr-xr-x   2 root root  4096 Mar 28 02:08 libx32
drwxr-xr-x  10 root root  4096 Mar 28 02:08 local
drwxr-xr-x   2 root root 20480 May  7 15:20 sbin
drwxr-xr-x 119 root root  4096 May  7 15:05 share
drwxr-xr-x   7 root root  4096 Apr 20 06:56 src

 Performance counter stats for 'system wide':

              0.03 Joules power/energy-pkg/

       0.000777146 seconds time elapsed

This tells me that the ls command consumed 0.03 Joules of energy and ran for 0.00078 seconds. I needed to get similar data for a C function that might be surrounded by code that I do not need to measure. Something like this: