<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="https://mattrighetti.com/feed.xml" rel="self" type="application/atom+xml" /><link href="https://mattrighetti.com/" rel="alternate" type="text/html" /><updated>2026-04-01T23:53:52+00:00</updated><id>https://mattrighetti.com/feed.xml</id><title type="html">mattrighetti</title><subtitle>Just another software engineering blog</subtitle><author><name>Mattia Righetti</name></author><entry><title type="html">TTY and Buffering</title><link href="https://mattrighetti.com/2026/01/12/tty-and-buffering.html" rel="alternate" type="text/html" title="TTY and Buffering" /><published>2026-01-12T00:00:00+00:00</published><updated>2026-01-12T00:00:00+00:00</updated><id>https://mattrighetti.com/2026/01/12/tty-and-buffering</id><content type="html" xml:base="https://mattrighetti.com/2026/01/12/tty-and-buffering.html"><![CDATA[<div class="paragraph">
<p>Every developer has at least once in their career stumbled upon the scenario
where a program would not print something as they initially thought. A few
years ago, you would search for an answer on StackOverflow and find a page
stating the dreaded</p>
</div>
<div class="quoteblock">
<blockquote>
<div class="paragraph">
<p>you have to flush the buffer</p>
</div>
</blockquote>
</div>
<div class="paragraph">
<p>I&#8217;d never been curious enough to dig into the 'why' until recently. Let&#8217;s take
a look at a quick example:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-c" data-lang="c">#include &lt;stdio.h&gt;
#include &lt;unistd.h&gt;

int main() {
    for (int i = 1; i &lt;= 5; i++) {
        printf("line %d\n", i);
        sleep(1);
    }
    return 0;
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>If we compile and run this in our terminal, we&#8217;ll get the following:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-console" data-lang="console">$ gcc -o cprint cprint.c
$ ./cprint
line 1       ← appears at t=1
line 2       ← appears at t=2
line 3       ← appears at t=3
line 4       ← appears at t=4
line 5       ← appears at t=5</code></pre>
</div>
</div>
<div class="paragraph">
<p>Something different happens if you pipe the result:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-console" data-lang="console">$ ./cprint | cat
             ← nothing for 5 seconds...
line 1       ← appears at t=5
line 2
line 3
line 4
line 5</code></pre>
</div>
</div>
<div class="paragraph">
<p>The answer lies in how buffering is handled in TTY and in non-TTY environments.
A <code>libc</code> implementation will use line buffering if it detects that we&#8217;re in a
TTY; otherwise, it will use full buffering if we&#8217;re in a non-TTY. Line
buffering flushes data as soon as a <code>\n</code> character is encountered. On the other
hand, full buffering typically accumulates data until the buffer is full, often
around 4KB to 8KB.</p>
</div>
<div class="openblock note">
<div class="content">
<div class="paragraph">
<p>It is worth noting that <code>stderr</code> is an exception to this rule. While <code>stdout</code>
changes behavior based on the environment, <code>stderr</code> is typically unbuffered or
line buffered even when piped. This ensures that if a program crashes, the
error message reaches the screen immediately rather than being stuck in a full
buffer.</p>
</div>
</div>
</div>
<div class="paragraph">
<p>Our previous example doesn&#8217;t demonstrate the effect of line buffering clearly
because we used newlines. Let&#8217;s look at a Rust example that highlights this:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">use std::io::{self, Write};
use std::thread::sleep;
use std::time::Duration;

fn main() {
    print!("Hello");
    sleep(Duration::from_secs(3));
    println!(" World!");
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-console" data-lang="console">$ cargo run
                ← nothing for 3 seconds...
Hello World!</code></pre>
</div>
</div>
<div class="paragraph">
<p>Rust&#8217;s <code>print!</code> implementation also performs line buffering, which causes the
first print to only show up along with the second one after a <code>\n</code> character is
encountered. You won&#8217;t notice any difference if you run <code>cargo run | cat</code> in
this case as both behave the same: output is printed in a single shot after 3
seconds.</p>
</div>
<div class="paragraph">
<p>However, we can force a <code>flush</code> to override this behavior:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">use std::io::{self, Write};
use std::thread::sleep;
use std::time::Duration;

fn main() {
    print!("Hello");
    io::stdout().flush().unwrap();
    sleep(Duration::from_secs(3));
    println!(" World!");
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-console" data-lang="console">$ cargo run
Hello           ← t=0
Hello World!    ← t=3</code></pre>
</div>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>Note that the above example does not actually appear on two different lines but it shows the single line at two different points in time</p>
        </div>
<div class="paragraph">
<p>We&#8217;re forcing a <code>flush</code> this time, and the first print appears immediately upon
program execution. Then the final print writes its content after 3 seconds.
What happens if we pipe this flushed output?</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-console" data-lang="console">$ cargo run | cat
Hello           ← t=0
Hello World!    ← t=3</code></pre>
</div>
</div>
<div class="paragraph">
<p>If you thought that <code>| cat</code> would change the result to print everything after
3s, you&#8217;re wrong. A manual flush will override the full buffering behavior and
push the buffered data through regardless of the environment.</p>
</div>
<div class="paragraph">
<p>Now we know that different libraries can behave differently when TTY/non-TTY is
detected, but what is a TTY?</p>
</div>
<div class="paragraph">
<p>In short, TTYs are interactive sessions opened by terminals. By contrast,
non-TTYs include data streams like pipes and redirects..</p>
</div>
<div class="paragraph">
<p>How do we detect a TTY programmatically? Different languages provide their own
methods. Rust has a simple <code>is_terminal()</code> function as part of the
<a href="https://doc.rust-lang.org/std/io/trait.IsTerminal.html"><code>std::io::IsTerminal</code></a> trait.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">use std::io::{self, IsTerminal, Write};

fn main() {
    let is_tty = io::stdout().is_terminal();
    if is_tty {
        write!(&amp;mut io::stdout(), "TTY!").unwrap();
    } else {
        write!(&amp;mut io::stdout(), "NOT TTY!").unwrap();
    }
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-console" data-lang="console">$ cargo run
TTY!

$ cargo run | cat
NOT TTY!

$ cargo run &gt; output &amp;&amp; cat output
NOT TTY!</code></pre>
</div>
</div>
<div class="paragraph">
<p>This output confirms the underlying logic: when we pipe to <code>cat</code> or redirect to
a file, the program detects a non-TTY environment.</p>
</div>
<div class="paragraph">
<p>By now, you may have guessed why this distinction matters. Far from being a
niche technical detail, TTY detection is the base for a wide range of
optimizations and DX decisions that depend on knowing exactly who or what is on
the other end of the line.</p>
</div>
<div class="paragraph">
<p>Let&#8217;s take a real example from the <a href="https://github.com/search?q=repo%3ABurntSushi%2Fripgrep%20is_terminal&amp;type=code">ripgrep</a> CLI
tool.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">let mut printer = StandardBuilder::new()
    .color_specs(ColorSpecs::default_with_color())
    .build(cli::stdout(if std::io::stdout().is_terminal() {
        ColorChoice::Auto
    } else {
        ColorChoice::Never
    }));</code></pre>
</div>
</div>
<div class="paragraph">
<p>This is a classic use case: colored text is essential for human readability in
a terminal, but it becomes a nuisance in a non-TTY environment. If you’ve ever
tried to grep through a file only to find it littered with messy ANSI escape
codes like <code>^[[31m</code>, you know exactly why ripgrep makes this choice.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">/// Returns a possibly buffered writer to stdout for the given color choice.
///
/// The writer returned is either line buffered or block buffered. The decision
/// between these two is made automatically based on whether a tty is attached
/// to stdout or not. If a tty is attached, then line buffering is used.
/// Otherwise, block buffering is used. In general, block buffering is more
/// efficient, but may increase the time it takes for the end user to see the
/// first bits of output.
///
/// If you need more fine grained control over the buffering mode, then use one
/// of `stdout_buffered_line` or `stdout_buffered_block`.
///
/// The color choice given is passed along to the underlying writer. To
/// completely disable colors in all cases, use `ColorChoice::Never`.
pub fn stdout(color_choice: termcolor::ColorChoice) -&gt; StandardStream {
    if std::io::stdout().is_terminal() {
        stdout_buffered_line(color_choice)
    } else {
        stdout_buffered_block(color_choice)
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>This implementation brings us full circle. By checking <code>is_terminal()</code>, ripgrep
chooses the best of both worlds: line buffering to output line by line when a
human is watching the terminal, and block buffering for max performance when
the output is being sent to another file or process.</p>
</div>
<div class="paragraph">
<p>You may have noticed that I’ve used Rust for every example except the first
one. There is a very specific and interesting reason for that.</p>
</div>
<div class="paragraph">
<p>Our first example&#8217;s aim is to show full buffering when the program is run in
non-TTYs. Let&#8217;s revisit that C example:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-c" data-lang="c">#include &lt;stdio.h&gt;
#include &lt;unistd.h&gt;

int main() {
    for (int i = 1; i &lt;= 5; i++) {
        printf("line %d\n", i);
        sleep(1);
    }
    return 0;
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Now here&#8217;s the equivalent Rust code:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">use std::thread::sleep;
use std::time::Duration;

fn main() {
    for i in 1..=5 {
        println!("line {}", i);
        sleep(Duration::from_secs(1));
    }
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-console" data-lang="console">$ cargo run
line 1       ← appears at t=1
line 2       ← appears at t=2
line 3       ← appears at t=3
line 4       ← appears at t=4
line 5       ← appears at t=5

$ cargo run | cat
line 1       ← appears at t=1
line 2       ← appears at t=2
line 3       ← appears at t=3
line 4       ← appears at t=4
line 5       ← appears at t=5</code></pre>
</div>
</div>
<div class="paragraph">
<p>Unlike the C version, Rust produces identical output in both TTY and non-TTY
environments and line buffering is used in both cases. How&#8217;s that even
possible?</p>
</div>
<div class="paragraph">
<p>Well, I previously wrote that this behavior is an implementation detail and not
something that happens in every language or library. Surprisingly, Rust, as of
now, uses line buffering for both TTYs and non-TTYs. We can see this by looking
at the <a href="https://github.com/rust-lang/rust/blob/main/library/std/src/io/stdio.rs"><code>Stdout</code></a> implementation:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">#[stable(feature = "rust1", since = "1.0.0")]
pub struct Stdout {
    // FIXME: this should be LineWriter or BufWriter depending on the state of
    //        stdout (tty or not). Note that if this is not line buffered it
    //        should also flush-on-panic or some form of flush-on-abort.
    inner: &amp;'static ReentrantLock&lt;RefCell&lt;LineWriter&lt;StdoutRaw&gt;&gt;&gt;,
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>As you can see, <code>LineWriter</code> is the default struct used by <code>Stdout</code>. The
<code>FIXME</code> comment shows the Rust team acknowledges that ideally they should check
if something is executed in TTYs or not and use <code>LineWriter</code> or <code>BufWriter</code>
accordingly, but I guess this was not on their priority list.</p>
</div>
<div class="paragraph">
<p>Next time your output isn&#8217;t appearing when you expect it to, you&#8217;ll know
exactly where to look. And it will be interesting to see if the Rust team
eventually addresses that <code>FIXME</code> and implements the best buffering strategy
for different environments. Until then, at least Rust&#8217;s consistent behavior
means one less thing to debug.</p>
</div>
<div class="paragraph">
<p>If you want to read more about TTYs, here are some really cool resources:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://computer.rip/2024-02-25-a-history-of-the-tty.html">A History of the TTY</a></p>
</li>
<li>
<p><a href="https://www.sobyte.net/post/2022-05/tty/">What Is a TTY</a></p>
</li>
<li>
<p><a href="https://unix.stackexchange.com/questions/4126/what-is-the-exact-difference-between-a-terminal-a-shell-a-tty-and-a-con">What is the exact difference between a 'terminal', a 'shell', a 'tty' and a 'console'?</a></p>
</li>
</ul>
</div>]]></content><author><name>Mattia Righetti</name></author><summary type="html"><![CDATA[Every developer has at least once in their career stumbled upon the scenario where a program would not print something as they initially thought. A few years ago, you would search for an answer on StackOverflow and find a page stating the dreaded]]></summary></entry><entry><title type="html">Too Many Open Files</title><link href="https://mattrighetti.com/2025/06/04/too-many-files-open.html" rel="alternate" type="text/html" title="Too Many Open Files" /><published>2025-06-04T00:00:00+00:00</published><updated>2025-06-04T00:00:00+00:00</updated><id>https://mattrighetti.com/2025/06/04/too-many-files-open</id><content type="html" xml:base="https://mattrighetti.com/2025/06/04/too-many-files-open.html"><![CDATA[<div class="paragraph">
<p>Recently I&#8217;ve been working on a pretty big rust project and to my surprise I
couldn&#8217;t get tests to work properly.</p>
</div>
<div class="paragraph">
<p>Running <code>cargo test</code> would start running all the tests in the repo and after a
couple of milliseconds every single test would start to fail because of an error
that I&#8217;m not very familiar with</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">Io(Os { code: 24, kind: Other, message: "Too many open files" })</code></pre>
</div>
</div>
<div class="paragraph">
<p>Fortunately, the error is pretty explicit and straightfoward so I was able to
understand what was going on in a reasonable time. I&#8217;ve started digging a bit
and learned some stuff along the way.</p>
</div>
<div class="paragraph">
<p>Ever wondered how your programs juggle multiple tasks - reading files, sending
data over the network, or even just displaying text on your screen - all at
once? File descriptors are what make this all possible (in Unix systems).</p>
</div>
<div class="paragraph">
<p>At its core, a file descriptor (often abbreviated as fd) is simply a positive
integer used by the operating system kernel to identify an open file. In Unix,
<a href="https://en.wikipedia.org/wiki/Everything_is_a_file">"everything is a file."</a> Contrary to what the word says,
a file descriptor doesn&#8217;t just refer to regular files on your disk. It can
represent:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><strong>Regular files</strong>: The documents, images, and code files you interact with daily.</p>
</li>
<li>
<p><strong>Directories</strong>: Yes, even directories are treated like files to some extent,
allowing programs to list their contents.</p>
</li>
<li>
<p><strong>Pipes</strong>: Used for inter-process communication, allowing one program&#8217;s output to
become another&#8217;s input.</p>
</li>
<li>
<p><strong>Sockets</strong>: The endpoints for network communication, whether it&#8217;s talking to a
web server or another application on your local machine.</p>
</li>
<li>
<p><strong>Devices</strong>: Hardware devices like your keyboard, mouse, and printer are also
accessed via file descriptors.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>When a program wants to interact with any of these resources, it first asks the
kernel to "open" it. If successful, the kernel returns a file descriptor, which
the program then uses for all subsequent operations (reading, writing, closing,
etc.).</p>
</div>
<div class="paragraph">
<p>By convention, every Unix process starts with at least three standard file
descriptors automatically opened:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>0</code>: Standard Input (stdin) - Typically connected to your keyboard for user
input.</p>
</li>
<li>
<p><code>1</code>: Standard Output (stdout) - Usually connected to your terminal for
displaying normal program output.</p>
</li>
<li>
<p><code>2</code>: Standard Error (stderr) - Also usually connected to your terminal, but
specifically for displaying error messages.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>On macOS we can quickly check this, open your favorite terminal and run <code>ls
/dev/fd</code>.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-console" data-lang="console">$ ls -lah /dev/fd
Permissions Size User         Date Modified Name
crw--w----  16,2 mattrighetti  4 Jun 00:44  0
crw--w----  16,2 mattrighetti  4 Jun 00:44  1
crw--w----  16,2 mattrighetti  4 Jun 00:44  2
dr--r--r--     - root         24 May 08:23  3</code></pre>
</div>
</div>
<div class="paragraph">
<p>On Linux we can do something similar but the repository is different and usually
follows the current pattern <code>/proc/&lt;pid&gt;/fd</code>. Running the same on Linux gives me this:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-console" data-lang="console">$ echo $$ // prints the current process id
2806524

$ sudo ls -lah /proc/2806524/fd
total 0
dr-x------ 2 root root 11 Jun  4 00:40 .
dr-xr-xr-x 9 pi   pi    0 Jun  4 00:39 ..
lrwx------ 1 root root 64 Jun  4 00:40 0 -&gt; /dev/null
lrwx------ 1 root root 64 Jun  4 00:40 1 -&gt; /dev/null
lrwx------ 1 root root 64 Jun  4 00:40 10 -&gt; /dev/ptmx
lrwx------ 1 root root 64 Jun  4 00:40 11 -&gt; /dev/ptmx
lrwx------ 1 root root 64 Jun  4 00:40 2 -&gt; /dev/null
lrwx------ 1 root root 64 Jun  4 00:40 3 -&gt; 'socket:[14023056]'
lrwx------ 1 root root 64 Jun  4 00:40 4 -&gt; 'socket:[14023019]'
lrwx------ 1 root root 64 Jun  4 00:40 5 -&gt; 'socket:[14022300]'
lrwx------ 1 root root 64 Jun  4 00:40 6 -&gt; 'socket:[14023037]'
lrwx------ 1 root root 64 Jun  4 00:40 7 -&gt; /dev/ptmx
l-wx------ 1 root root 64 Jun  4 00:40 8 -&gt; /run/systemd/sessions/1501.ref</code></pre>
</div>
</div>
<div class="paragraph">
<p>As you can see, we have <code>0</code>, <code>1</code> and <code>2</code> as expected, but we also have a bunch of
other file descriptors.</p>
</div>
<div class="paragraph">
<p>Another useful command to check for open file descriptors is <code>lsof</code>, which
stands for "list open files".</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-console" data-lang="console">$ lsof -p $(echo $$)
COMMAND   PID         USER   FD   TYPE DEVICE SIZE/OFF                NODE NAME
zsh     39367 mattrighetti  cwd    DIR   1,17     2496              250127 /Users/mattrighetti
zsh     39367 mattrighetti  txt    REG   1,17  1361200 1152921500312522433 /bin/zsh
zsh     39367 mattrighetti  txt    REG   1,17    81288 1152921500312535786 /usr/share/locale/en_US.UTF-8/LC_COLLATE
zsh     39367 mattrighetti  txt    REG   1,17   170960 1152921500312525313 /usr/lib/zsh/5.9/zsh/zutil.so
zsh     39367 mattrighetti  txt    REG   1,17   118896 1152921500312525297 /usr/lib/zsh/5.9/zsh/terminfo.so
zsh     39367 mattrighetti  txt    REG   1,17   171344 1152921500312525281 /usr/lib/zsh/5.9/zsh/parameter.so
zsh     39367 mattrighetti  txt    REG   1,17   135696 1152921500312525255 /usr/lib/zsh/5.9/zsh/datetime.so
zsh     39367 mattrighetti  txt    REG   1,17   135568 1152921500312525291 /usr/lib/zsh/5.9/zsh/stat.so
zsh     39367 mattrighetti  txt    REG   1,17   338592 1152921500312525247 /usr/lib/zsh/5.9/zsh/complete.so
zsh     39367 mattrighetti  txt    REG   1,17   136880 1152921500312525293 /usr/lib/zsh/5.9/zsh/system.so
zsh     39367 mattrighetti  txt    REG   1,17   593088 1152921500312525303 /usr/lib/zsh/5.9/zsh/zle.so
zsh     39367 mattrighetti  txt    REG   1,17   134928 1152921500312525287 /usr/lib/zsh/5.9/zsh/rlimits.so
zsh     39367 mattrighetti  txt    REG   1,17   117920 1152921500312525263 /usr/lib/zsh/5.9/zsh/langinfo.so
zsh     39367 mattrighetti  txt    REG   1,17  2289328 1152921500312524246 /usr/lib/dyld
zsh     39367 mattrighetti  txt    REG   1,17   208128 1152921500312525249 /usr/lib/zsh/5.9/zsh/complist.so
zsh     39367 mattrighetti  txt    REG   1,17   118688 1152921500312525285 /usr/lib/zsh/5.9/zsh/regex.so
zsh     39367 mattrighetti  txt    REG   1,17   118288 1152921500312525305 /usr/lib/zsh/5.9/zsh/zleparameter.so
zsh     39367 mattrighetti    0u   CHR   16,1  0t17672                1643 /dev/ttys001
zsh     39367 mattrighetti    1u   CHR   16,1  0t17672                1643 /dev/ttys001
zsh     39367 mattrighetti    2u   CHR   16,1  0t17672                1643 /dev/ttys001
zsh     39367 mattrighetti   10u   CHR   16,1   0t5549                1643 /dev/ttys001</code></pre>
</div>
</div>
<div class="paragraph">
<p>According to the lsof documentation:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>cwd</code>: The current working directory of the process.</p>
</li>
<li>
<p><code>txt</code>: Executable files or shared libraries loaded into memory (e.g.,
/bin/zsh, modules like zutil.so, or system libraries like /usr/lib/dyld).</p>
</li>
<li>
<p><code>0u</code>, <code>1u</code>, <code>2u</code>: Standard input (0), output (1), and error (2) streams,
respectively. The <code>u</code> means the descriptor is open for both reading and writing.
These are tied to <code>/dev/ttys001</code> (my current terminal device).</p>
</li>
<li>
<p><code>10u</code>: Another file descriptor (also tied to <code>/dev/ttys001`</code>), likely used for
additional terminal interactions.</p>
</li>
</ul>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>We now know that file descriptors are a way for the operating system to keep track of open files and other resources, nice!</p>
        </div>
<div class="paragraph">
<p>Have you ever wondered how many file descriptors can be open at the same time?
The most common answer in software engineering applies here too: It depends.</p>
</div>
<div class="paragraph">
<p>Each operating system has its own limits on the number of file descriptors a
process can open simultaneously. These limits are in place to prevent a single
misbehaving program from hogging all available resources and crashing the
system.</p>
</div>
<div class="paragraph">
<p>On macOS, we can easily inspect these limits using the <code>sysctl</code> and
<a href="https://linuxcommand.org/lc3_man_pages/ulimith.html"><code>ulimit</code></a> commands in your terminal.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-console" data-lang="console">$ sysctl kern.maxfiles
kern.maxfiles: 245760

$ sysctl kern.maxfilesperproc
kern.maxfilesperproc: 122880

$ ulimit -n
256</code></pre>
</div>
</div>
<div class="ulist">
<ul>
<li>
<p><code>kern.maxfiles</code> represents the absolute maximum number of file descriptors that
can be open across the entire macOS system at any given moment. It&#8217;s a global
governor, preventing the system from running out of file descriptor resources,
even if many different applications are running.</p>
</li>
<li>
<p><code>kern.maxfilesperproc</code> is the hard limit on the number of file descriptors
that a single process can have open. Think of it as the ultimate ceiling for an
individual application. No matter what, a process cannot open more files than
this hard limit set by the kernel.</p>
</li>
<li>
<p><code>ulimit -n</code> is your shell&#8217;s "soft" limit for the number of open file
descriptors. If a process tries to open more files than its soft limit, the
operating system will typically return an error (e.g., "Too many open files").
The good news is that a process can raise its own soft limit, but only up to its
hard limit.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Enough with the theory, let&#8217;s get back to the problem I was having with my rust
tests.  My assumption was that since <code>cargo test</code> gets executed in my terminal,
it inevitably reaches a point where it tries to open more files than the soft
limit set by my shell, which is 256 in this case. When that happens, the
operating system screams at <code>cargo</code> and tells it that it can&#8217;t open any more
files, <code>cargo</code> then propagates that error to the tests and they all fail.</p>
</div>
<div class="paragraph">
<p>I wanted to confirm this hypothesis, so I created this monitoring script that
watches for <code>cargo test</code> PID and prints the number of open file descriptors at
different intervals.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">#!/bin/bash

# This function exits the script gracefully
function cleanup() {
    echo -e "\nstopping."
    exit 0
}

# This function encapsulates the logic for formatting and printing the monitoring output.
# Arguments:
#   $1: Initial PID
#   $2: Total number of open files
print_status() {
    local initial_pid="$1"
    local total_open_files="$2"
    echo "$(date '+%H:%M:%S') - Main PID ($initial_pid) - open: ${total_open_files}"
}

PROCESS_NAME="cargo"
COMMAND_ARGS="test"

echo "press ctrl+c to stop."

# Find the Process ID (PID) of the initial command.
INITIAL_PID=$(pgrep -f "$PROCESS_NAME.*$COMMAND_ARGS" | head -n 1)

if [ -z "$INITIAL_PID" ]; then
    echo "waiting for '$PROCESS_NAME $COMMAND_ARGS' to start..."
    # If the process isn't found immediately, loop and wait for it.
    sleep 0.01
    while [ -z "$INITIAL_PID" ]; do
        INITIAL_PID=$(pgrep -f "$PROCESS_NAME.*$COMMAND_ARGS" | head -n 1)
    done
fi

echo "Found '$PROCESS_NAME $COMMAND_ARGS' with PID: $INITIAL_PID"

# trap command catches the INT signal (triggered by Ctrl+C)
# and calls the cleanup function to exit gracefully.
trap cleanup INT

while true; do
    # check if the main process (INITIAL_PID) is still running.
    if ! ps -p "$INITIAL_PID" &gt; /dev/null; then
        echo "PID $INITIAL_PID no longer running. bye!"
        break
    fi

    # `sudo lsof -p "$INITIAL_PID"` lists all open files for this specific PID.
    # `2&gt;/dev/null` redirects stderr (errors like "process not found") to null.
    # `grep -v " txt "` filters out loaded executable code and libraries for a more relevant count.
    # `wc -l` counts the lines, effectively the number of open files.
    # `tr -d ' '` removes any leading/trailing spaces for clean arithmetic.
    OPEN_FILES_COUNT=$(sudo lsof -p "$INITIAL_PID" 2&gt;/dev/null | grep -v " txt " | wc -l | tr -d ' ')

    # Ensure COUNT is not empty (it might be if lsof returns nothing)
    if [ -z "$OPEN_FILES_COUNT" ]; then
        OPEN_FILES_COUNT=0
    fi

    print_status "$INITIAL_PID" "$TOTAL_OPEN_FILES"
done</code></pre>
</div>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>Note that usually to get an accurate count of open file descriptors you would also need to consider the entire process tree, not just the main process. This is because child processes can also open files, and their file descriptors contribute to the total count.  In my case, there was only one process (`cargo test`) running, so I didn't.</p>
        </div>
<div class="paragraph">
<p>I can now run this script in one terminal and run <code>cargo test</code> in another
terminal. I actually had to do this a couple of times to get a good sample of
data, rust runs pretty fast once the code is compiled and this monitor script
does not run fast enough to catch all the open file descriptors changes.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-console" data-lang="console">$ sudo ./monitor.sh
press ctrl+c to stop.
waiting for 'cargo test' to start...
Found 'cargo test' with PID: 44152
01:46:21 - Main PID (44152) - open: 14
01:46:21 - Main PID (44152) - open: 32
01:46:21 - Main PID (44152) - open: 78
01:46:21 - Main PID (44152) - open: 155
01:46:21 - Main PID (44152) - open: 201
01:46:21 - Main PID (44152) - open: 228
01:46:21 - Main PID (44152) - open: 231
01:46:21 - Main PID (44152) - open: 237 # errors started happening here
01:46:21 - Main PID (44152) - open: 219
01:46:21 - Main PID (44152) - open: 205
01:46:21 - Main PID (44152) - open: 180
01:46:21 - Main PID (44152) - open: 110
01:46:21 - Main PID (44152) - open: 55
01:46:21 - Main PID (44152) - open: 28
01:46:21 - Main PID (44152) - open: 15
01:46:21 - Main PID (44152) - open: 0
PID 44152 no longer running. bye!</code></pre>
</div>
</div>
<div class="paragraph">
<p>I couldn&#8217;t get the script to catch the exact moment the process reached the soft
limits, but I can clearly see that the tests starts failing when the number of
open file descriptors reaches 237, which is pretty close to the soft limit of
256.</p>
</div>
<div class="paragraph">
<p>Time to fix this! This is a bit underwhelming, but the solution is to just bump
the soft limit of open file descriptors in my shell. I can do this by using the
<code>ulimit</code> command again.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-console" data-lang="console">$ ulimit -n 8192
$ ulimit -n
8192</code></pre>
</div>
</div>
<div class="paragraph">
<p>Running <code>cargo test</code> now works as expected and no "Too many open files" error is thrown.</p>
</div>
<div id="chart">
    <h4 class="text-xl pb-4 font-bold">Open File Descriptors Over Time</h4>
    <div class="ct-chart" id="lineChart1"></div>
</div>
<script>
    var chartLabels = [
        '01:58:28', '01:58:30', '01:58:32', '01:58:34', '01:58:37', '01:58:40',
        '01:58:43', '01:58:45', '01:58:48', '01:58:50', '01:58:53', '01:58:56',
        '01:58:59', '01:59:02', '01:59:04', '01:59:07', '01:59:10', '01:59:13',
        '01:59:16', '01:59:19', '01:59:22', '01:59:25', '01:59:28', '01:59:31',
        '01:59:34', '01:59:37', '01:59:40', '01:59:43', '01:59:46', '02:00:34'
    ];

    var chartSeries = [
        10, 1615, 1534, 1455, 1289, 1127,
        735, 803, 772, 987, 739, 552,
        667, 1368, 941, 898, 799, 933,
        880, 807, 502, 915, 1051, 1092,
        1064, 994, 1434, 856, 1175, 6
    ];

    // Combine into Chartist.js data format
    var openFilesData = {
        labels: chartLabels,
        series: [chartSeries] // Wrap the series in an array for a single line
    };

    // Options for our line chart
    var openFilesOptions = {
        height: 300,
        axisX: {
            showGrid: true,
            showLabel: true,
            // Only show labels for every 5th point to avoid clutter
            labelInterpolationFnc: function(value, index) {
                return index % 7 === 0 ? value : null;
            }
        },
        axisY: {
            showGrid: true,
            showLabel: true,
            low: 0,
        },
        fullWidth: true,
        chartPadding: {
            right: 40
        },
    };

    // Initialize the line chart
    new Chartist.Line('#lineChart1', openFilesData, openFilesOptions);
</script>
<div class="paragraph">
<p>The above chart shows the number of open file descriptors with the new soft
limit. As you can see the max value reached is around 1600, which is way above
the previous limit of 256.</p>
</div>
<div class="paragraph">
<p>All in all, this was a fun exercise that taught me a lot about file descriptors
and how they work in Unix-like systems. Now you know how to troubleshoot this
error that might pop up in your own projects!</p>
</div>]]></content><author><name>Mattia Righetti</name></author><summary type="html"><![CDATA[Recently I&#8217;ve been working on a pretty big rust project and to my surprise I couldn&#8217;t get tests to work properly.]]></summary></entry><entry><title type="html">Authentication with Axum</title><link href="https://mattrighetti.com/2025/05/03/authentication-with-axum.html" rel="alternate" type="text/html" title="Authentication with Axum" /><published>2025-05-03T00:00:00+00:00</published><updated>2025-05-03T00:00:00+00:00</updated><id>https://mattrighetti.com/2025/05/03/authentication-with-axum</id><content type="html" xml:base="https://mattrighetti.com/2025/05/03/authentication-with-axum.html"><![CDATA[<div class="paragraph">
<p>Consider this scenario: you&#8217;re building a website that has a classic navbar at
the top, this navbar has a button that reflects the user authentication status,
showing a "Profile" button if the user is authenticated and showing a
"Login" button in case the user is unauthenticated.</p>
</div>
<div class="paragraph">
<p>This is a very common scenario, let&#8217;s sketch something quick using axum and
askama.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-html" data-lang="html">&lt;!DOCTYPE html&gt;
&lt;html lang="en"&gt;

&lt;head&gt;
    &lt;title&gt;{% block title %}{% endblock %}&lt;/title&gt;
    &lt;meta charset="UTF-8" /&gt;
    &lt;meta name="viewport" content="width=device-width, initial-scale=1.0" /&gt;
    {% block head %}{% endblock %}
&lt;/head&gt;
&lt;body&gt;
    &lt;nav&gt;
        &lt;div&gt;
            &lt;div&gt;
                {% if ctx.authed %}
                    &lt;a href="/profile"&gt;Profile&lt;/a&gt;
                {% else %}
                    &lt;a href="/login"&gt;Login&lt;/a&gt;
                {% endif %}
            &lt;/div&gt;
        &lt;/div&gt;
    &lt;/nav&gt;
    {% block content %}
    {% endblock %}
&lt;/body&gt;
&lt;/html&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>This will be our <code>layout.jinja</code> file that we can build upon. The template above
would be served by an endpoint that looks like the following</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">#[derive(Debug, Default)]
pub struct Context {
    authed: bool,
}

#[derive(Template)]
#[templage = "layout.jinja"]
struct HomeTemplate {
    ctx: Context
};

pub async fn home() -&gt; impl IntoResponse {
    HtmlTemplate(
        HomeTemplate { ctx: Context::default() }
    ).into_response()
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>We have something to work with, all is missing is a way derive a
<code>Context</code> from a user&#8217;s HTTP request.</p>
</div>
<div class="paragraph">
<p>I&#8217;d argue the simplest way to handle user authentication if you&#8217;re doing SSR is
using cookies. Cookies are a cornerstone of backend authentication because
they’re reliable, browser-managed, and can be hardened with specific attributes
to mitigate common security risks. Here’s why:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>HttpOnly</code> Attribute: Prevents client-side JavaScript from accessing the
cookie, neutralizing XSS attacks. If an attacker injects malicious scripts,
they can’t steal your session cookie.</p>
</li>
<li>
<p><code>Secure</code> Attribute: Ensures the cookie is only sent over HTTPS, protecting it
from interception on insecure networks (e.g., public Wi-Fi).</p>
</li>
<li>
<p><code>SameSite</code> Attribute: Mitigates CSRF (Cross-Site Request Forgery) by
controlling when cookies are sent in cross-origin requests. SameSite=Strict
blocks cookies in requests from external sites, while SameSite=Lax allows
safe methods like GET.</p>
</li>
<li>
<p>Expiration and Domain/Path Scoping: Cookies can be set to expire after a
session or a fixed time, reducing the window for misuse. Scoping to specific
domains and paths (e.g., <code>domain=api.example.com</code>, <code>path=/auth</code>) limits their
exposure.</p>
</li>
<li>
<p>Signed Cookies: Frameworks often support signing cookies with a secret key,
ensuring they haven’t been tampered with on the client side.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>This is a typical reponse that uses the <code>Set-Cookies</code> header to instruct the
browser to set those cookies</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-http" data-lang="http">HTTP/1.1 200 OK
Content-Type: application/json
Set-Cookie: session=xyz123; HttpOnly; Secure; SameSite=Strict; Max-Age=86400; Path=/; Domain=api.example.com</code></pre>
</div>
</div>
<div class="paragraph">
<p>When configured correctly, cookies are a fortress for storing session IDs,
JWTs, or other authentication tokens in SSR apps. They’re automatically
sent by the browser with every request, simplifying server-side validation
compared to managing tokens in <code>localStorage</code> or HTTP headers.</p>
</div>
<div class="paragraph">
<p>Axum provides a cool <a href="https://docs.rs/axum-extra/0.10.1/axum_extra"><code>axum-extra</code></a> crate that makes it easy to
work with them. That crate contains a very useful extractor called <code>CookieJar</code>
that exposes a very minimal interface to <code>.add</code> and <code>.remove</code> cookies for a
user. This is the utility function I use to generate a default cookie</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">pub(crate) fn default_cookie&lt;'a&gt;(
    key: &amp;str,
    token: String,
    duration_hrs: i64
) -&gt; Cookie&lt;'a&gt; {
    Cookie::build((key.to_string(), token))
        .path("/")
        .http_only(true)
        .max_age(Duration::hours(duration_hrs))
        .secure(if cfg!(debug_assertions) {
            // Safari won't allow secure cookies
            // coming from localhost in debug mode
            false
        } else {
            // Secure cookies in release mode
            true
        })
        .build()
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>I won’t get sucked into the session ID vs. JWT argument, but honestly, using
JWTs in cookies is a win because you don’t have to fuss with storing session
data on the server.</p>
</div>
<div class="paragraph">
<p>Jwt are usually very short-lived, they shouldn&#8217;t last for long periods of time
and they must be renewed frequently for security purposes. For that reason you
usually issue two different cookies:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>jwt: short-lived token containing information about a user in json format,
signed with a secret key so you know you were the one who issued it</p>
</li>
<li>
<p>refresh token: a longer-lived token with which you can request new jwts</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Now that we&#8217;ve covered the cookies and jwt basics, let&#8217;s start by implementing
a standard login endpoint with which users can be given these two cookies.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">#[derive(Debug)]
struct LoginData {
    username: String,
    password: String
}

pub async fn login(
    State(app): State&lt;AppState&gt;,
    jar: CookieJar, // CookieJar is available in axum_extras
    Form(LoginData { username, password }): Form&lt;LoginData&gt;
) -&gt; impl IntoResponse {
    // dummy function to get a user
    let user = match db::user::get(&amp;app.pg_pool, &amp;username, &amp;password).await {
        None =&gt; return Redirect::to("/signup").into_response()
        Some(user) =&gt; user
    };

    // get/create a refresh token for the user
    let refresh_token = match db::refresh_tokens::create(user.id).await {
        Ok(token) =&gt; token,
        Err(_) =&gt; {
            return (
                StatusCode::INTERNAL_SERVER_ERROR,
                "Somethign bad happened, try again later"
            ).into_response();
        }
    };

    let claims = Claims::with(user.email, user.id);
    match jwt::generate_jwt(app.jwt_signing_key.as_bytes(), claims) {
        Ok(token) =&gt; (
            [("hx-redirect", "/")],
            jar.add(default_cookie("jwt", token, 1)).add(default_cookie(
                "refresh",
                refresh_token,
                30 * 24,
            ),
        )
            .into_response()),
        Err(_) =&gt; {
            return (
                StatusCode::INTERNAL_SERVER_ERROR,
                "Somethign bad happened, try again later"
            ).into_response();
        }
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>This login endpoint will receive a request with some form data that contain a
<code>username</code> and a <code>password</code>. First thing you usually have to do is check if the
user exists in your database, otherwise you&#8217;ll kindly <code>302</code> to a signup page
where he/she has to register, returning a message to show in the login form
sometimes works as well - whatever suits you.</p>
</div>
<div class="paragraph">
<p>Once you know the user exists you need to create a refresh token. It usually
makes sense to implement <code>refresh_token::create</code> so that it returns a valid
non-expired refresh token stored in your database associated with the user
before creating a new one. This is because users can delete cookies and/or
users can authenticate with different devices and you don&#8217;t want to create a
refresh token each time.</p>
</div>
<div class="paragraph">
<p>When you get your refresh token back you&#8217;re ready to move on and handle the
last part of the process, which is generating the jwt and returning a valid
response to the user that will set those cookies.</p>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>Ignore `hx-redirect` header for now, this was a snippet of code that I had laying around on github. Also, note that the responses I return in case of errors are not very exhaustive for most scenarios, I'm conciously leaving out the details because it's not the focus of this blog post.</p>
        </div>
<div class="paragraph">
<p>If <code>login</code> is successful the user will be redirected to the homepage at <code>/</code> and
will trigger the <code>home</code> endpoint again but his navbar will still show the login
button because we&#8217;re using <code>Context::default()</code>. Let&#8217;s change that with our
first approach using Axum extractors.</p>
</div>
<div class="paragraph">
<p>When I first started using Axum I really liked the idea of <code>Extractors</code>, if
you&#8217;ve used the framework you&#8217;re probably familiar with them (i.e <code>Json</code>,
<code>Form</code> etc.). Everything that implements <code>FromRequest</code> or
<code>FromRequestParts</code> (and the <code>Option</code> alternative since Axum 0.8!) can be
considered an extractor and can be used in the function signature to get
something out of a request.</p>
</div>
<div class="paragraph">
<p>In our case, we would like to get some user data out of a request (cookies are
always sent with an HTTP request), in particular we can create a custom
extractor that tries to extract our user data from the jwt token in the user&#8217;s
request, if present. Let&#8217;s implement <code>CookieJwt&lt;T&gt;</code> which we&#8217;re going to use to
get that information out of requests that reaches our endpoints.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">/// Basic claims that a classic jwt contain
#[derive(Debug, Serialize, Deserialize)]
pub struct Claims {
    pub sub: String,
    pub exp: usize,
    pub user_id: uuid::Uuid,
}

/// A flexible extractor that tries
/// to get a type `T` from a request cookie
#[derive(Debug, PartialEq, Eq, Clone)]
pub struct CookieJwt&lt;T: DeserializeOwned&gt;(pub T);

// since axum 0.8 you can implement extractors meant to be Option&lt;T&gt;
// this is very useful, expecially for scenarios where endpoint can be accessed
// both by authed users and non-authed users
impl&lt;S, T&gt; OptionalFromRequestParts&lt;S&gt; for CookieJwt&lt;T&gt;
where
    AppState: FromRef&lt;S&gt;,
    S: Send + Sync,
    T: DeserializeOwned,
{
    type Rejection = Redirect;

    async fn from_request_parts(
        req: &amp;mut Parts,
        state: &amp;S,
    ) -&gt; Result&lt;Option&lt;Self&gt;, Self::Rejection&gt; {
        let jar = CookieJar::from_headers(&amp;req.headers);
        if let Some(jwt) = jar.get("jwt").map(|c| c.value()) {
            return match validate_jwt::&lt;T&gt;(JWT_SIGNING_KEY, jwt) {
                Ok(data) =&gt; return Ok(Some(CookieJwt(data))),
                // user tampered with cookie here, we want to delete that cookie
                // returning None here would have been okay too if you're okay with
                // manufactured cookies :)
                Err(_) =&gt; Err(Redirect::to("/logout")),
            };
        }

        // if refresh token is present, try and get a new jwt
        // by redirecting user to /refresh_token endpoint
        if jar.get("refresh").is_some() {
            return Err(Redirect::to(
                format!("/refresh_token?next={}", req.uri).as_str(),
            ));
        }

        // at this point, user has no jwt and no refresh token
        Ok(None)
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>And we can use our brand new extractor in our <code>home</code> function</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">pub async fn home(jwt: Option&lt;CookieJwt&lt;Claims&gt;&gt;) -&gt; impl IntoResponse {
    HtmlTemplate(
        HomeTemplate { ctx: Context { authed: jwt.is_some() } }
    ).into_response()
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Now each time try and navigate to <code>/</code> the extractor is going to peek into your
request and look for <code>jwt</code> and <code>refresh</code> cookies, if <code>jwt</code> is found and can be
decoded to <code>Claims</code> then <code>jwt: Option&lt;_&gt;</code> is going to contain <code>Some(jwt)</code> data
and our user will see the "Profile" button in his navbar, indicating he&#8217;s
correctly logged in. If neither of the cookies is found then the user will be
returned the classic navbar with the option to "Login". If however <code>jwt</code> can&#8217;t
be found but a <code>refresh</code> cookie is present we can still do something for the user
and get him a proper <code>jwt</code>.</p>
</div>
<div class="paragraph">
<p>Indeed, the logic implemented above will redirect the user to <code>/refresh_token</code>
along with a query parameter indicating where the user was previously
navigating to. This way we&#8217;re not distrupting the original user&#8217;s intent and
everything is going to happen in a quick succession of requests. How does our
<code>/refresh_token</code> endpoint look like?</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">#[derive(Debug, Deserialize)]
pub struct RefreshTokenQuery {
    next: Option&lt;String&gt;,
}

pub async fn refresh_token(
    State(app): State&lt;AppState&gt;,
    jar: CookieJar,
    Query(RefreshTokenQuery { next }): Query&lt;RefreshTokenQuery&gt;,
) -&gt; impl IntoResponse {
    let token = match jar.get("refresh") {
        Some(token) =&gt; token,
        None =&gt; {
            // if there's no token then the user goes back to /login
            return Redirect::to("/login").into_response();
        }
    };

    // if something goes wrong here we remove the token, otherwise the user could end up
    // in a loop where he's constantly being redirected here and this function fails every time
    let user = match db::refresh_tokens::get_user(&amp;app.pg_pool, token.value()).await {
        Ok(Some(user)) =&gt; user,
        _ =&gt; {
            return (jar.remove(Cookie::from("refresh")), Redirect::to("/login")).into_response();
        }
    };

    // set new jwt
    let claims = Claims::with(user.email, user.id);
    match jwt::generate_jwt(app.jwt_signing_key.as_bytes(), claims) {
        Ok(token) =&gt; (
            jar.add(default_cookie("jwt", token, 1)),
            Redirect::to(&amp;next.unwrap_or("/".to_owned())),
        )
        .into_response(),
        Err(_) =&gt; {
            (jar.remove(Cookie::from("refresh")), Redirect::to("/login")).into_response()
        }
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Nothing surprising here, our refresh token endpoint is going to check that the
<code>refresh</code> cookie is part of the request and if it is it will try to ask the db to
return the user information associated to that refresh token. Once the
information is retrieved it re-generates the valid <code>jwt</code> and redirect the user
to his previous url <code>next</code>, if present.</p>
</div>
<div class="paragraph">
<p>Even though it is a great starting point and has worked very well for me, it&#8217;s
not flawless:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>It doesn&#8217;t feel right: handling authentication logic in an extractor doesn&#8217;t
feel quite right.</p>
</li>
<li>
<p>It&#8217;s not flexible for more complex authentication scenarios i.e restricting
some endpoints to users with a specific role.</p>
</li>
<li>
<p>If a user sends a POST request that has a body attached to it the
<code>/refresh_token</code> redirect will break that flow because almost every browser
won&#8217;t expect a <code>302</code> redirect to have a body.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>If you’re a backend developer, authentication screams middleware. To level up
the cookie-based authentication we’ve discussed, authentication middleware
offers a cleaner, reusable way to validate cookies and secure routes.</p>
</div>
<div class="paragraph">
<p>Axum gives you quite a few options when you want to implement a
<a href="https://docs.rs/axum/latest/axum/middleware/index.html">middleware</a>. You don&#8217;t have to give up on the granularity
that extractor provided because axum middleware can be applied at the app level
down to the individual route level.</p>
</div>
<div class="quoteblock">
<blockquote>
<div class="paragraph">
<p>Axum allows you to add middleware just about anywhere</p>
</div>
</blockquote>
</div>
<div class="paragraph">
<p>The easiest way to implement an Axum middleware is to create a function that
matches the <a href="https://docs.rs/axum/latest/axum/middleware/fn.from_fn.html"><code>axum::middleware::from_fn</code></a> (or
<a href="https://docs.rs/axum/latest/axum/middleware/fn.from_fn_with_state.html"><code>axum::middleware::from_fn_with_sate</code></a> if you need
<code>State</code>) function. The requirements are pretty straightfoward:</p>
</div>
<div class="quoteblock">
<blockquote>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Be an async <code>fn</code>.</p>
</li>
<li>
<p>Take zero or more <code>FromRequestParts</code> extractors.</p>
</li>
<li>
<p>Take exactly one <code>FromRequest</code> extractor as the second to last argument.</p>
</li>
<li>
<p>Take <code>Next</code> as the last argument.</p>
</li>
<li>
<p>Return something that implements <code>IntoResponse</code>.</p>
</li>
</ol>
</div>
</blockquote>
</div>
<div class="paragraph">
<p>With that in mind, let&#8217;s try and create our authentication middleware.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">/// Middleware that handles both authenticated and unauthenticated requests.
///
/// This middleware performs JWT-based authentication by checking for `jwt` and `refresh` cookies.
/// It establishes a [`UserContext`] that flows through the request chain and manages cookie updates.
///
/// # Behavior
/// - **JWT Present**: Validates the JWT and extracts user claims if successful.
///   - Invalid JWT: Clears auth cookies (potential tampering)
/// - **No JWT but Refresh Token Present**:
///   - Attempts to refresh the token and issue a new JWT
///   - On success: Sets new cookies and establishes authenticated context
/// - **No Auth Cookies**: Proceeds with default unauthenticated context
///
/// # Cookie Management
/// - Automatically removes suspicious/invalid auth cookies
/// - Adds new JWT cookies when refresh is successful
/// - Propagates all cookie changes in the response
pub async fn base(
    State(app): State&lt;AppState&gt;,
    mut request: Request,
    next: Next,
) -&gt; impl IntoResponse {
    let mut jar = CookieJar::from_headers(request.headers());
    let jwt = jar.get("jwt");
    let refresh = jar.get("refresh");

    // Default context for unauthenticated requests
    let mut context = UserContext {
        user_id: None,
        is_admin: false,
    };

    // JWT takes precedence if present
    if let Some(jwt) = jwt {
        match validate_jwt::&lt;Claims&gt;(JWT_SIGNING_KEY, jwt.value()) {
            Ok(claims) =&gt; {
                context.user_id = Some(claims.user_id);
            }
            Err(_) =&gt; {
                // Clear potentially compromised cookies
                jar = jar.remove("jwt").remove("refresh");
            }
        }
    }
    // Fall back to refresh token if JWT is absent/invalid
    else if let Some(refresh) = refresh {
        if let Ok(Some(user)) = db::refresh_tokens::get_user(&amp;app.pg_pool, refresh.value()).await {
            let claims = Claims::with(user.email, user.uuid);
            if let Ok(jwt) = generate_jwt(app.jwt_signing_key.as_bytes(), claims) {
                context.user_id = Some(user.uuid);
                jar = jar.add(default_cookie("jwt", jwt, 1));
            }
            // Note: JWT generation errors are intentionally swallowed here
            // to prevent refresh token from being invalidated due to
            // temporary JWT generation issues
        }
    }

    // Inject the resolved context into request extensions
    request.extensions_mut().insert(context);

    let response = next.run(request).await;

    // Merge cookie updates with the response
    (jar, response).into_response()
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>The middleware we&#8217;ve built takes the same core idea as our initial extractor
but makes it far more powerful. Unlike a simple extractor, middleware can
intercept and modify responses and modify it as needed. This enables us to do
much more interesting things.</p>
</div>
<div class="paragraph">
<p>First of all, the middleware initializes the <code>UserContext</code> with some default
values that an un-authenticated users will reflect. After that, it goes through
the first authentication step which tries to look for a valid <code>jwt</code> token in
the request&#8217;s cookies, if it finds one it updates the <code>UserContext</code> accordingly
with the data decoded from the <code>jwt</code>. If a <code>jwt</code> token is not found, it falls
back to the <code>refresh</code> token and uses it to generate a valid <code>jwt</code> for the user
and updates the <code>UserContext</code> accordingly. The authentication part of
middleware is now complete and the <code>UserContext</code> is passed along with the
request so that handlers can make use of it. But we&#8217;re not done yet! The
middleware will wait for the <code>response</code> returned by whatever we have running in
<code>next.run(_)</code> and does something pretty cool: in case the request wasn&#8217;t
originally authenticated (did not have a <code>jwt</code> attached) it sends back with the
response new a new cookie containing the generated <code>jwt</code>.</p>
</div>
<div class="paragraph">
<p>This last part is very cool for different reasons:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p><strong>Silent Authentication</strong>: Requests that come in un-authenticated will be
treated as authenticated (if <code>refresh</code> is present!) because we do the heavy
lifting of generating the <code>jwt</code> in the middleware.</p>
</li>
<li>
<p><strong>Works with all request types</strong>: whatever the request was (POST, PUT, GET
etc.), the middleware won&#8217;t distrupt the flow and the user will get back a
fresh <code>jwt</code> if he&#8217;s missing one.</p>
</li>
<li>
<p><strong>Simplified Architecture</strong>: with this middleware we don&#8217;t need extra round trips
to authenticate the user, therefore we can also get rid of the
<code>/refresh_token</code> endpoint.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Our new <code>home</code> endpoint now would end up looking like this</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">pub async fn home(Extension(usr_ctx): Extension&lt;UserContext&gt;) -&gt; impl IntoResponse {
    HtmlTemplate(
        HomeTemplate { ctx: Context { authed: usr_ctx.user_id.is_some() } }
    ).into_response()
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Nothing is stopping us from generating a <code>Context</code> directly in the middleware,
that would actually be a better approach here so that we can pop it in
directly in the template</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">pub async fn home(Extension(ctx): Extension&lt;Context&gt;) -&gt; impl IntoResponse {
    HtmlTemplate(HomeTemplate { ctx }).into_response()
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>We&#8217;re not done yet - one last cool thing that you can do with
middlewares is stack them on top of each other and have multiple layers of
logic to protect different parts of your backend, just like an onion.</p>
</div>
<div class="paragraph">
<p>Let&#8217;s consider the scenario where you want some parts of your application to be
public, some others to be for authenticated only users and then you have a very
special dashboard that only super admins can access. You can leverage middlewares
to implement all of those protection layers</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">/// middleware that requires the user to be authenticated
pub async fn required_auth(
    Extension(context): Extension&lt;UserContext&gt;,
    request: Request,
    next: Next
) -&gt; impl IntoResponse {
    if context.user_id.is_none() {
        return Redirect::to("/login").into_response();
    }

    next.run(request).await
}

/// middleware that requires the user to be authenticated
pub async fn required_admin(
    Extension(context): Extension&lt;UserContext&gt;,
    request: Request,
    next: Next,
) -&gt; impl IntoResponse {
    if !context.is_admin {
        return Redirect::to("/").into_response();
    }

    next.run(request).await
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>If you remember correctly, middlewares can have zero or more <code>FromRequestParts</code>
in its signature, which means you can use as many extractors as you want to and
<code>Extension</code> is an extractor too! This means you can re-use something that the
previous middleware computed in the following middlewares. The only thing you
have to pay attention to in this case is to apply the middlewares in the
correct order. This is a pretty good look of how middlewares work in Axum</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-Graph" data-lang="Graph">+-----------------------+
|       Requests        |
+-----------------------+
           |
           v
+-----------------------+
|      Layer Three      |
|  +-----------------+  |
|  |    Layer Two    |  |
|  |  +-----------+  |  |
|  |  | Layer One |  |  |
|  |  |   +---+   |  |  |
|  |  |   | H |   |  |  |
|  |  |   | a |   |  |  |
|  |  |   | n |   |  |  |
|  |  |   | d |   |  |  |
|  |  |   | l |   |  |  |
|  |  |   | e |   |  |  |
|  |  |   | r |   |  |  |
|  |  |   +---+   |  |  |
|  |  +-----------+  |  |
|  +-----------------+  |
+-----------------------+
           |
           v
+-----------------------+
|      Responses        |
+-----------------------+</code></pre>
</div>
</div>
<div class="paragraph">
<p>It really is like an onion after all</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">let app = Router::new()
    .route("/admin", get(admin::get))
    // only admins can access the routes above
    .layer(
        middleware::from_fn_with_state(state.clone(), required_admin)
    )
    .route("/profile", get(profile::get))
    // routes above will need to be authenticated
    .layer(
        middleware::from_fn_with_state(state.clone(), required_auth)
    )
    .route("/", get(home))
    // most external layer, will provide
    // `Extension&lt;UserContext&gt;` to all the routes above
    .layer(
        middleware::from_fn_with_state(state.clone(), base)
    );</code></pre>
</div>
</div>
<div class="paragraph">
<p>For better readability you can also create different routers with different
layers and finally merge them together into the main app&#8217;s router.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">let admin = Router::new()
    .route("/admin", get(handler))
    .layer(
        middleware::from_fn_with_state(state.clone(), required_admin)
    );

let protected = Router::new()
    .route("/profile")
    .layer(
        middleware::from_fn_with_state(state.clone(), required_auth)
    );

let public = Router::new()
    .route("/");

let app = Router::new()
    .merge(public)
    .merge(protected)
    .merge(admin);
    .layer(
        middleware::from_fn_with_state(state.clone(), base)
    )</code></pre>
</div>
</div>
<div class="paragraph">
<p>I&#8217;ve been playing with middlewares for a while now in Axum and I feel they
provide a much better option for this scenario than creative alternatives like
the one I&#8217;ve talked about initially. Axum provides much more powerful features
for middlewares if you need it, but I still haven&#8217;t delved into those that much
because there was no need for me to do it, I almost always can get stuff done
with the simple <code>middleware::from_fn_with_sate</code> function. You should give them
a try!</p>
</div>]]></content><author><name>Mattia Righetti</name></author><summary type="html"><![CDATA[Consider this scenario: you&#8217;re building a website that has a classic navbar at the top, this navbar has a button that reflects the user authentication status, showing a "Profile" button if the user is authenticated and showing a "Login" button in case the user is unauthenticated.]]></summary></entry><entry><title type="html">Reverse Engineering Playtomic</title><link href="https://mattrighetti.com/2025/03/03/reverse-engineering-playtomic.html" rel="alternate" type="text/html" title="Reverse Engineering Playtomic" /><published>2025-03-03T00:00:00+00:00</published><updated>2025-03-03T00:00:00+00:00</updated><id>https://mattrighetti.com/2025/03/03/reverse-engineering-playtomic</id><content type="html" xml:base="https://mattrighetti.com/2025/03/03/reverse-engineering-playtomic.html"><![CDATA[<div class="paragraph">
<p>Playtomic is probably the biggest application for booking tennis/padel courts in
Europe. Every club that I go to is on Playtomic and that makes it very easy to
book a court or look for upcoming matches. The app is great, which I guess is
why it&#8217;s so popular.</p>
</div>
<div class="paragraph">
<p>All this popularity comes at a price though: thousands and thousands of users
competing to get the empty courts as soon they become available. This means that
if you&#8217;re not fast enough to book a court for the next sunny Saturday, you&#8217;re
out of luck.</p>
</div>
<div class="paragraph">
<p>Playtomic offers a premium subscription that gives you special powers, like
getting notifications in advance when courts become available but the price does
not seem to be worth it. Especially if we can get the same by being a little
smart, right?</p>
</div>
<div class="paragraph">
<p>What I wanted to end up with was some script that notifies me in some kind of
way whenever a specific court becomes available.</p>
</div>
<div class="paragraph">
<p>That&#8217;s when I started to get curious about reverse engineering the iOS app that
Playtomic offers</p>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>Don't ask me why but when I was doing this yesterday I was not aware that Playtomic also had a website at https://www.playtomic.io, so there could be an even easier solution here.</p>
        </div>
<div class="paragraph">
<p>The best solution here is to try and find the APIs necessary to query the
available courts.  Once you have that, you can just poll the API every few
minutes and notify yourself when a court becomes available or a schedule
changes.</p>
</div>
<div class="paragraph">
<p>Where do you look for those APIs when you only have your iPhone&#8217;s app? In the
app itself of course!</p>
</div>
<div class="paragraph">
<p>We can try and intercept the network requests that the app makes to its servers
and see if we can find the API that returns us court information.</p>
</div>
<div class="paragraph">
<p>You can intercept network requests in multiple different ways but the easiest
one is probably by sitting in the middle of the connection between your phone
and the internet. This is usually achieved with a proxy server.</p>
</div>
<div class="paragraph">
<p>I&#8217;ve used Burp Suite Community Edition for this purpose. While many alternative
tools exist, Burp Suite is widely recognized as a leading penetration testing
platform with extensive documentation. I won&#8217;t detail the Burp Suite setup
process here, but an excellent guide is available in the references
<a href="https://portswigger.net/burp/documentation/desktop/getting-started/intercepting-http-traffic">[0]</a>. Once configured with your iPhone using Burp
as a proxy, all iPhone network traffic will be routed through and intercepted by
Burp Suite for analysis.</p>
</div>
<div class="paragraph">
<p>What&#8217;s left to do at this point is opening the app and navigate to the screen
where the court information is displayed, that&#8217;s where the API call to get the
courts availability is probably made.</p>
</div>
<div class="paragraph">
<p>The process ended up being a lot easier than I thought, and after fiddling a
couple of minutes with the app and frenetically popping up courts availability
pages I was able to find the API call that I was looking for:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-http" data-lang="http">GET /v1/availability?sport_id=PADEL&amp;start_max=2025-03-06T23:59:59&amp;start_min=2025-03-06T00:00:00&amp;tenant_id=2ab75436-9bb0-4e9c-9a6f-b12931a9ca4a
Host: api.playtomic.io
Content-Type: application/json
X-Datadog-Parent-Id: ********
Tracestate: ********
X-Datadog-Sampling-Priority: 0
Authorization: Bearer *******
Accept: */*
X-Requested-With: com.playtomic.app 6.13.0
X-Datadog-Trace-Id: ********
Accept-Language: en-US,en;q=0.9
Accept-Encoding: gzip, deflate, br
User-Agent: iOS 18.3.1
X-Datadog-Tags: _dd.p.tid=67c4561000000000
X-Datadog-Origin: rum
Traceparent: ********
Connection: keep-alive</code></pre>
</div>
</div>
<div class="paragraph">
<p>This looks like a classic authenticated request to the endpoint
<code>/v1/availability</code> with some query parameters: <code>sport_id=PADEL</code>,
<code>start_max=2025-03-06T23:59:59</code>, <code>start_min=2025-03-06T00:00:00</code> and <code>tenant_id</code>
which corresponds to the courts that I usually go to.</p>
</div>
<div class="paragraph">
<p>In this case we&#8217;ve been quite lucky and Burp was able to intercept the traffic
without a single issue.  Nowadays, a lot of applications will use SSL pinning to
prevent this kind of interception. This is a topic for another day, but it&#8217;s
worth mentioning that there are ways to bypass SSL pinning and still intercept
the traffic.</p>
</div>
<div class="paragraph">
<p>Let&#8217;s go on and play with the API! I bet we don&#8217;t need most of the request
headers displayed above. Let&#8217;s try and get rid of all the analytics and tracking
stuff and see if we can still get a <code>200</code> from the Playtomic APIs.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-http" data-lang="http">GET /v1/availability?sport_id=PADEL&amp;start_max=2025-03-06T23:59:59&amp;start_min=2025-03-06T00:00:00&amp;tenant_id=2ab75436-9bb0-4e9c-9a6f-b12931a9ca4a
Host: api.playtomic.io
Content-Type: application/json
Authorization: Bearer *******
X-Requested-With: com.playtomic.app 6.13.0
User-Agent: iOS 18.3.1

&gt;&gt;&gt;

HTTP/1.1 200 OK
Content-Type: application/json
Transfer-Encoding: chunked
Connection: close
Date: Mon, 03 Mar 2025 21:38:46 GMT
Vary: Origin, Access-Control-Request-Method, Access-Control-Request-Headers
X-Content-Type-Options: nosniff
Expires: 0
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
X-XSS-Protection: 1; mode=block
Pragma: no-cache
Timing-Allow-Origin: *
X-Frame-Options: DENY
X-Cache: Miss from cloudfront
Via: 1.1 2eadda0e57cd7e495ec3550f05424d3e.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: LHR50-P5
X-Amz-Cf-Id: h5udBjkJjJJyPBavztnnhDXbsS1E-7VesGxqmH7riMWhv225ShTZqg==

[
  {
    "resource_id": "021872eb-b49b-47f4-a66e-ad19173a7a75",
    "start_date": "2025-03-06",
    "slots": [
      {
        "start_time": "09:30:00",
        "duration": 90,
        "price": "72 GBP"
      },
      {
        "start_time": "16:30:00",
        "duration": 60,
        "price": "52 GBP"
      }
    ]
  },
  {
    "resource_id": "b983547f-93c7-489d-9475-52f269b1c39a",
    "start_date": "2025-03-06",
    "slots": [
      {
        "start_time": "09:30:00",
        "duration": 90,
        "price": "72 GBP"
      },
      {
        "start_time": "16:30:00",
        "duration": 60,
        "price": "52 GBP"
      }
    ]
  },
  {
    "resource_id": "3df036e3-7dba-4c39-b966-ab088edaade4",
    "start_date": "2025-03-06",
    "slots": [
      {
        "start_time": "09:30:00",
        "duration": 60,
        "price": "48 GBP"
      },
      {
        "start_time": "10:30:00",
        "duration": 60,
        "price": "48 GBP"
      },
      {
        "start_time": "13:30:00",
        "duration": 90,
        "price": "72 GBP"
      }
    ]
  }
]</code></pre>
</div>
</div>
<div class="paragraph">
<p>Nice! The server seems to be okay if we just provide the bare minimum, but what if
we also remove the <code>Authorization</code> header?</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-http" data-lang="http">GET /v1/availability?sport_id=PADEL&amp;start_max=2025-03-06T23:59:59&amp;start_min=2025-03-06T00:00:00&amp;tenant_id=2ab75436-9bb0-4e9c-9a6f-b12931a9ca4a
Host: api.playtomic.io
Content-Type: application/json
X-Requested-With: com.playtomic.app 6.13.0
User-Agent: iOS 18.3.1

&gt;&gt;&gt;

HTTP/1.1 200 OK
Content-Type: application/json
...</code></pre>
</div>
</div>
<div class="paragraph">
<p>Can&#8217;t get any better than this, we don&#8217;t even need to be authenticated to call
the API! This saved us a bunch of time and effort that we would have spent
trying to reverse engineer the authentication mechanism.</p>
</div>
<div class="paragraph">
<p>It would be cool if we could ask for an entire month of availability, maybe we can tweak the <code>start_max</code> and <code>start_min</code> parameters to achieve that:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-http" data-lang="http">GET /v1/availability?sport_id=PADEL&amp;start_max=2025-04-06T23:59:59&amp;start_min=2025-03-06T00:00:00&amp;tenant_id=2ab75436-9bb0-4e9c-9a6f-b12931a9ca4a
Host: api.playtomic.io
Content-Type: application/json
X-Requested-With: com.playtomic.app 6.13.0
User-Agent: iOS 18.3.1

&gt;&gt;&gt;

HTTP/1.1 400 Bad Request
...
{
  "localized_message": "Not allowed to request more than 25h",
  "status": "INCORRECT_PARAMETERS"
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>The server doesn&#8217;t seem to like that, indeed we can only request a maximum of 25
hours of availability at a time. That&#8217;s a bummer, but we can work with that.</p>
</div>
<div class="paragraph">
<p>Let&#8217;s start building a script that queries the API and returns us with the
availability of the courts.  I&#8217;m only playing during weekends so it makes sense
for me to have a lookahead window of 2 weeks, only asking specifically for the
weekends.</p>
</div>
<div class="paragraph">
<p>I haven&#8217;t been posting Go in a while on this blog, I feel like that would be the best language for this task.
Let&#8217;s define our structs first:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-go" data-lang="go">// Slot represents a single availability slot
type Slot struct {
	StartTime string `json:"start_time"` // Start time of the slot (e.g., "09:30:00")
	Duration  int    `json:"duration"`   // Duration of the slot in minutes
	Price     string `json:"price"`      // Price of the slot (e.g., "48 GBP")
}

// AvailabilityResponse represents the entire JSON response
type AvailabilityResponse struct {
	ResourceID string `json:"resource_id"` // Resource ID (e.g., "3df036e3-7dba-4c39-b966-ab088edaade4")
	StartDate  string `json:"start_date"`  // Start date of the availability (e.g., "2025-03-06")
	Slots      []Slot `json:"slots"`       // List of available slots
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>I&#8217;d then need a function to conveniently get the days of the weekend for the next 2 weeks:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-go" data-lang="go">// getNextWeekends returns the next two Saturdays and Sundays from today
func getNextWeekends() []time.Time {
	today := time.Now()
	var weekends []time.Time

	// Find the next Saturday and Sunday
	daysUntilSaturday := (time.Saturday - today.Weekday() + 7) % 7
	nextSaturday := today.AddDate(0, 0, int(daysUntilSaturday))
	nextSunday := nextSaturday.AddDate(0, 0, 1)

	// Add the next two weekends
	weekends = append(weekends, nextSaturday, nextSunday)
	weekends = append(weekends, nextSaturday.AddDate(0, 0, 7), nextSunday.AddDate(0, 0, 7))

	return weekends
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>And a function to build the request that is going to be issued for a single date:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-go" data-lang="go">// buildGetRequest builds a GET request for the given target date
func buildGetRequest(targetDate time.Time) (*http.Request, error) {
	// Base URL
	baseURL := "https://api.playtomic.io/v1/availability"

	// Derive start and end times from the target date
	startDate := time.Date(targetDate.Year(), targetDate.Month(), targetDate.Day(), 0, 0, 0, 0, targetDate.Location())
	endDate := time.Date(targetDate.Year(), targetDate.Month(), targetDate.Day(), 23, 59, 59, 0, targetDate.Location())

	// Query parameters
	params := url.Values{}
	params.Add("sport_id", "PADEL")
	params.Add("start_min", startDate.Format("2006-01-02T15:04:05"))
	params.Add("start_max", endDate.Format("2006-01-02T15:04:05"))
	params.Add("tenant_id", "2ab75436-9bb0-4e9c-9a6f-b12931a9ca4a")

	// Construct the full URL with query parameters
	fullURL := fmt.Sprintf("%s?%s", baseURL, params.Encode())

	// Create the GET request
	req, err := http.NewRequest("GET", fullURL, nil)
	if err != nil {
		return nil, fmt.Errorf("failed to create request: %w", err)
	}

	return req, nil
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Finally, we can issue the request and parse the response with this other function:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-go" data-lang="go">// fetchAvailability fetches the availability for the given target date
func fetchAvailability(targetDate time.Time) ([]AvailabilityResponse, error) {
	// Build the GET request
	req, err := buildGetRequest(targetDate)
	if err != nil {
		return nil, fmt.Errorf("error building request: %w", err)
	}

	// Make the HTTP request
	client := &amp;http.Client{}
	resp, err := client.Do(req)
	if err != nil {
		return nil, fmt.Errorf("error making request: %w", err)
	}
	defer resp.Body.Close()

	// Read the response body
	body, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, fmt.Errorf("error reading response body: %w", err)
	}

	// Parse the JSON response into a slice of AvailabilityResponse
	var availability []AvailabilityResponse
	err = json.Unmarshal(body, &amp;availability)
	if err != nil {
		return nil, fmt.Errorf("error unmarshalling JSON: %w", err)
	}

	return availability, nil
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>This is a good start, we could now call <code>fetchAvailability</code> and with a bit of pretty printing logic we can get a table
with the available slots for the next two weekends.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-go" data-lang="go">func main() {
	// Get the next two Saturdays and Sundays
	weekends := getNextWeekends()

	// Fetch availability for each date
	availabilityByDate := make(map[string][]Slot)
	for _, targetDate := range weekends {
		fmt.Printf("Fetching availability for date: %s\n", targetDate.Format("2006-01-02"))

		// Fetch availability
		availabilities, err := fetchAvailability(targetDate)
		if err != nil {
			fmt.Printf("Error fetching availability: %v\n", err)
			continue
		}

		// Process each availability response
		for _, availability := range availabilities {
			// Append slots to the date key
			availabilityByDate[availability.StartDate] = append(availabilityByDate[availability.StartDate], availability.Slots...)
		}
	}

    fmt.Println(prettyPrintTable(availabilityByDate))
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-console" data-lang="console">$ go run main.go
DATE         START TIME  DURATION
2025-03-08   19:00:00       60
2025-03-09   20:00:00       60</code></pre>
</div>
</div>
<div class="paragraph">
<p>This is just what I wanted, now we need to think of a way to get this delivered
somewhere to us so that we can rush to the app and book the court whenever our
favorite slot becomes available.</p>
</div>
<div class="paragraph">
<p>Running this script in a cron job every 10 minutes would do it, but I don&#8217;t want to be notified every single time
a request is made. Ideally, for a first iteration I would like to get notified only when a new slot becomes available or
another slot is removed. Basically, I want to receive a notification whenever the availability changes in any way.</p>
</div>
<div class="paragraph">
<p>We can detect a change in availability by comparing the hash of the new and previous table of slots. Whenever the hash changes,
the script would need to store a file containing the hash of the current table of slots so that it can be compared the next time
a request is issued.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-go" data-lang="go">// computeHash computes a SHA-256 hash of the availability data
func computeHash(data map[string][]Slot) string {
	jsonData, err := json.Marshal(data)
	if err != nil {
		return ""
	}
	hash := sha256.Sum256(jsonData)
	return hex.EncodeToString(hash[:])
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>We have a couple of options when it comes to notifications, the two most common
ones that I always use are emails and Slack. For this particular case I&#8217;m going
to use email, I can send the notification to all of my friends without actually
creating a Slack channel just for that, which seems a bit too much work.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-go" data-lang="go">// notify sends an email notification using Gmail using App passwords
func notify(message string) {
	// Gmail account credentials
	from := "my-supersecret-email@youwillneverknow.com"
	password := os.Getenv("SMTP_PWD")

	// List of recipients
	subscribers := os.Getenv("SUBSCRIBERS")
	var to []string

	err := json.Unmarshal([]byte(subscribers), &amp;to)
	if err != nil {
		panic("cannot parse emails")
	}

	// SMTP server configuration
	smtpHost := "smtp.gmail.com"
	smtpPort := "587"

	// Email content
	subject := "Playtomic Update"
	body := fmt.Sprintf("The availability table has changed:\n\n%s", message)

	// Construct the email message
	msg := fmt.Sprintf("From: %s\nTo: %s\nSubject: %s\n\n%s",
		from, strings.Join(to, ","), subject, body)

	// Authenticate and send the email
	auth := smtp.PlainAuth("", from, password, smtpHost)
	err = smtp.SendMail(smtpHost+":"+smtpPort, auth, from, to, []byte(msg))
	if err != nil {
		fmt.Printf("Error sending email: %v\n", err)
	} else {
		fmt.Println("Email notification sent successfully!")
	}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>We can make use of these two new functions now</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-go" data-lang="go">func main() {
	// Get the next two Saturdays and Sundays
	weekends := getNextWeekends()

	// Fetch availability for each date
	availabilityByDate := make(map[string][]Slot)
	for _, targetDate := range weekends {
		fmt.Printf("Fetching availability for date: %s\n", targetDate.Format("2006-01-02"))

		// Fetch availability
		availabilities, err := fetchAvailability(targetDate)
		if err != nil {
			fmt.Printf("Error fetching availability: %v\n", err)
			continue
		}

		// Process each availability response
		for _, availability := range availabilities {
			// Append slots to the date key
			availabilityByDate[availability.StartDate] = append(availabilityByDate[availability.StartDate], availability.Slots...)
		}
	}

	// Compute the hash of the current availability data
	currentHash := computeHash(availabilityByDate)

	// Read the previous hash from a file (if it exists)
	var previousHash string
	hashFile := "availability_hash.txt"
	if _, err := os.Stat(hashFile); err == nil {
		hashBytes, err := os.ReadFile(hashFile)
		if err != nil {
			fmt.Printf("Error reading hash file: %v\n", err)
		} else {
			previousHash = string(hashBytes)
		}
	}

	// Compare hashes to detect changes
	if currentHash != previousHash {
		// Write the new hash to the file
		err := os.WriteFile(hashFile, []byte(currentHash), 0644)
		if err != nil {
			fmt.Printf("Error writing hash file: %v\n", err)
		}

		// Write the table to a file
		outputFile := "availability_table.txt"
		err = writeTableToFile(outputFile, availabilityByDate)
		if err != nil {
			fmt.Printf("Error writing table to file: %v\n", err)
		} else {
			fmt.Printf("Table written to %s\n", outputFile)
		}

		// Send a notification
		notify(prettyPrintTable(availabilityByDate))
	} else {
		fmt.Println("No changes detected in the availability table.")
	}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>If I run this now, I&#8217;m going to get an email to my inbox, a self-sent email
because I&#8217;m currently using my own email address to send the notifications.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-console" data-lang="console">$ SUBSCRIBERS="[\"my-supersecret-email@youwillneverknow.com\"]" SMTP_PWD="secret" go run main.go</code></pre>
</div>
</div>
<div class="paragraph">
<p>With that, we have our script that notifies us whenever the availability of the
courts changes. What&#8217;s left to do is write a cron job that runs this script periodically.</p>
</div>
<div class="paragraph">
<p>I could have written my cron job so that it runs every 10 minutes on one of my
raspis at home, but I have a better and more reliable solution: GitHub Actions.</p>
</div>
<div class="paragraph">
<p>Did you know you can run cron jobs on GitHub Actions? It&#8217;s a great way to run
periodic tasks without having to worry about the infrastructure. Plus, if
something goes wrong, you can always check the logs on GitHub on the go, which
is not always super convenient if the cron job is running in your LAN.</p>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>Don't trust me on this one, but another advantage here is that Github Actions runners will change the IP address every time they run, so you can't be easily blocked by the Playtomic servers and those will look like legitimate requests.</p>
        </div>
<div class="paragraph">
<p>Here&#8217;s the CI/CD pipeline configuration file</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-yaml" data-lang="yaml">name: Run Availability Checker

on:
  schedule:
    # Run every 10 minutes
    - cron: '*/10 * * * *'
  workflow_dispatch: # Allow manual triggering

jobs:
  check-availability:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Set up Go
        uses: actions/setup-go@v4
        with:
          go-version: '1.24'

      # We can leverage the runner's cache to store the hash of the availability table
      - name: Restore hash cache
        id: cache-hash
        uses: actions/cache@v3
        with:
          path: availability_hash.txt
          key: availability-hash

      # Run the Go program
      - name: Run Go program
        env:
          SUBSCRIBERS: ${{ secrets.SUBSCRIBERS }}
          SMTP_PWD: ${{ secrets.SMTP_PWD }}
        run: |
          echo ${{ env.SUBSCRIBERS }}
          go run main.go

      # Save the hash file to cache
      - name: Save hash cache
        if: steps.cache-hash.outputs.cache-hit != 'true'
        uses: actions/cache@v3
        with:
          path: availability_hash.txt
          key: availability-hash</code></pre>
</div>
</div>
<div class="paragraph">
<p>That is it! I can now sit back and relax, knowing that I will be notified
whenever a court becomes available and I&#8217;ll be able to book it, or at least have
a chance to do so.</p>
</div>]]></content><author><name>Mattia Righetti</name></author><summary type="html"><![CDATA[Playtomic is probably the biggest application for booking tennis/padel courts in Europe. Every club that I go to is on Playtomic and that makes it very easy to book a court or look for upcoming matches. The app is great, which I guess is why it&#8217;s so popular.]]></summary></entry><entry><title type="html">Database Tests for the Lazy</title><link href="https://mattrighetti.com/2025/02/17/rust-testing-sqlx-lazy-people.html" rel="alternate" type="text/html" title="Database Tests for the Lazy" /><published>2025-02-17T00:00:00+00:00</published><updated>2025-02-17T00:00:00+00:00</updated><id>https://mattrighetti.com/2025/02/17/rust-testing-sqlx-lazy-people</id><content type="html" xml:base="https://mattrighetti.com/2025/02/17/rust-testing-sqlx-lazy-people.html"><![CDATA[<div class="paragraph">
<p>Anyone who has developed with Rust using sqlx for database operations has likely
needed to write tests for their database interactions.</p>
</div>
<div class="paragraph">
<p>What I tend to do is setup a utility function that is called at the very
beginning of each test which basically spins up the database connection and
returns that to the caller, ready to execute queries.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">#[cfg(test)]
pub async fn test_db() -&gt; EnvelopeDb {
    let pool = sqlx::sqlite::SqlitePoolOptions::new()
        .connect(":memory:")
        .await
        .expect("cannot connect to db");

    // Run migrations
    sqlx::migrate!("./migrations")
        .run(&amp;pool)
        .await?;

    // Connection wrapper
    EnvelopeDb::with(pool)
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>If you&#8217;re project uses sqlite then you can consider yourself lucky - sqlite can
create an in-memory database with the <code>:memory:</code> uri.  That turns out to be
super nice to work with because tests are super fast and you don&#8217;t have to take
care of anything, really. Each time you use <code>test_db()</code> a brand new connection
to an in-memory database is established and each connection is isolated to the
caller.</p>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>There's a reason why everyone loves sqlite, right?</p>
        </div>
<div class="paragraph">
<p>Things are a little bit different though in pretty much every other scenario.</p>
</div>
<div class="paragraph">
<p>Most of the databases out there need their own dedicated server to which you can
then connect to and communicate over with different network protocols. I mostly use Postgres
so I&#8217;ll use that as an example for the rest of this post.</p>
</div>
<div class="paragraph">
<p>This is the classic docker-compose file that I use to spin up a local postgres instance
with a very bad password</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-yaml" data-lang="yaml">version: "3.8"

services:
  postgres:
    image: postgres:latest
    restart: always
    ports:
      - "5432:5432"
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: 1234
      POSTGRES_DB: postgres</code></pre>
</div>
</div>
<div class="paragraph">
<p>It&#8217;s not much more work, but it definitely is not as convenient as the sqlite in-memory setup.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">#[cfg(test)]
mod tests {
    use sqlx::postgres::PgPoolOptions;
    use sqlx::PgPool;

    use super::model::UserRow;

    pub async fn test_db() -&gt; Result&lt;(PgPool, i64), sqlx::Error&gt; {
        let db_url = "postgres://postgres:1234@localhost:5432/postgres";

        let pool = PgPoolOptions::new()
            .max_connections(1)
            .acquire_timeout(Duration::from_secs(3))
            .connect(&amp;db_url)
            .await?;

        sqlx::migrate!("./migrations")
            .run(&amp;pool)
            .await?;

        let id: i64 = sqlx::query_as(
            r"INSERT INTO users(id, name, lastname, email)
            VALUES (
                0,
                'testname',
                'testlastname',
                'test@example.com'
            )
            RETURNING id",
        )
        .fetch_one(&amp;pool)
        .await?;

        Ok((pool, id))
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>This is another classic function that I always have around in my projects. It
connects to a local postgres instance and inserts a user that you can use in your tests.
The macro <code>#[cfg(test)]</code> is used to compile this function only when running tests.</p>
</div>
<div class="paragraph">
<p>You can then use this function in your tests like the following:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">#[cfg(test)]
mod tests {
    use super::*;

    #[tokio::test]
    async fn test_user_exists() {
        let (pool, _) = test_db().await.unwrap();
        assert!(
            user_exists(&amp;pool, "test@example.com").await.unwrap(),
            "user should exist"
        );
        assert!(
            !user_exists(&amp;pool, "another@example.com").await.unwrap(),
            "this user should not exist"
        );
    }
}</code></pre>
</div>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>user_exists is not defined above, I'll let your imagination run free</p>
        </div>
<div class="paragraph">
<p>Great, we now have a very lazy and good enough setup to test our functions
against our local database.  I say lazy because it&#8217;s not going to work for long
as you will see. If you try to add more tests you will soon realize that a good
chunk of your tests will inevitably fail because, unlike sqlite in-memory,
the postgres database is shared among all the tests.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">#[cfg(test)]
mod tests {
    use super::*;

    #[tokio::test]
    async fn test_user_exists() {
        let (pool, _) = test_db().await.unwrap();
        assert!(
            user_exists(&amp;pool, "test@example.com").await.unwrap(),
            "user should exist"
        );
        assert!(
            !user_exists(&amp;pool, "another@example.com").await.unwrap(),
            "this user should not exist"
        );
    }

    #[tokio::test]
    async fn test_get_user_id() {
        let (pool, _) = test_db().await.unwrap();
        assert_eq!(
            get_user_id(&amp;pool, "test@example.com").await.unwrap(),
            0,
            "test user should have id = 0"
        );
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>If you run the tests above, one of the two will fail. The issue lies in the fact
that the first <code>test_db()</code> will insert the test user just fine, the second call
to <code>test_db()</code> won&#8217;t because the user already exists in the database and the
insert will fail, but <code>fetch_one</code> expects a row to be returned and your test
will fail because that <code>?</code> ruins the party for everyone by returning the error
to the caller.</p>
</div>
<div class="paragraph">
<p>There are many solutions to this specific problem, the first one that comes to
mind is using an <code>ON CONFLICT</code> clause when you&#8217;re trying to insert the user.
This is not a good solution though, imagine how many "hacks" like this you will
have to take care of when you have hundreds of different tables with hundreds of
rows. Do you really want to craft an sql statement that is going to make every
single test of yours run okay (if that is even possible)?</p>
</div>
<div class="paragraph">
<p>No, you don&#8217;t of course. Ideally, each test should have a brand new database to
work with, or at least something that resembles one.  With that said, I&#8217;m ready
to give you another lazy solution: empty your database at the end of each test
and run your Rust tests sequentially.</p>
</div>
<div class="paragraph">
<p>You may already know that Rust tests are run in parallel by default, but you can
change that by setting the <code>RUST_TEST_THREADS</code> environment variable to 1.</p>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>After waiting 10 minutes for cargo to compile, who wouldn't want to spend even more time watching tests run one after another?</p>
        </div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">#[cfg(test)]
mod tests {
    use sqlx::postgres::PgPoolOptions;
    use sqlx::PgPool;

    use super::model::UserRow;

    pub async fn test_db() -&gt; Result&lt;(PgPool, i64), sqlx::Error&gt; {
        let db_url = "postgres://postgres:1234@localhost:5432/postgres";

        let pool = PgPoolOptions::new()
            .max_connections(1)
            .acquire_timeout(Duration::from_secs(3))
            .connect(&amp;db_url)
            .await?;

        sqlx::migrate!("./migrations")
            .run(&amp;pool)
            .await?;

        // truncate everything that's left in the database
        sqlx::query("TRUNCATE TABLE users")
            .execute(&amp;pool)
            .await?;

        let id: i64 = sqlx::query_as(
            r"INSERT INTO users(id, name, lastname, email)
            VALUES (
                0,
                'testname',
                'testlastname',
                'test@example.com',
            )
            RETURNING id",
        )
        .fetch_one(&amp;pool)
        .await?;

        Ok((pool, id))
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>You can then run <code>RUST_TEST_THREADS=1 cargo test</code>, wait a couple of minutes and
your tests will happily pass just fine.</p>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>Just make sure to never run your tests on the production database, okay? :)</p>
        </div>
<div class="paragraph">
<p>Enough with the lazy solutions, let&#8217;s talk about how we could actually have this
sorted out in a more elegant way.</p>
</div>
<div class="paragraph">
<p>Previously I&#8217;ve talked about how each test would ideally have its own database,
and that&#8217;s exactly what we&#8217;re going to do. We&#8217;re going to create a new database
for each test.</p>
</div>
<div class="paragraph">
<p>I remember I first learned this while I was reading "Zero to Production in Rust"
by Luca Palmieri. So I highly suggest you to read the chapter on test isolation
<a href="https://lpalmieri.com/posts/2020-08-31-zero-to-production-3-5-html-forms-databases-integration-tests/#3-7-1-test-isolation">[1]</a> where he goes into the nitty gritty details of what I am
about to explain you more briefly.</p>
</div>
<div class="paragraph">
<p>The idea is pretty simple: <code>test_db()</code> will create a new <em>logical</em> database with
a random name (a uuid works fine) and return a connection to it.  This way each
test will have its own database to work with and no test will be able to access
and interfere with the others.</p>
</div>
<div class="paragraph">
<p>Here&#8217;s a simple implementation of <code>test_db()</code> that does exactly that (again,
credits to Luca for this!):</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">#[cfg(test)]
mod tests {
    use sqlx::postgres::{PgPoolOptions, PgConnection};
    use sqlx::{Connection, Executor, PgPool};
    use uuid::Uuid;

    pub async fn test_db() -&gt; Result&lt;(PgPool, i64), sqlx::Error&gt; {
        // Generate a unique database name
        let db_base = "postgres://postgres:1234@localhost:5432";
        let db_name = Uuid::new_v4().to_string();
        let connection_string = format!("{db_base}/{db_name}");

        // Connect to the default `postgres` database to create a new database
        let mut connection = PgConnection::connect(db_base)
            .await?;

        // create unique logical database
        connection
            .execute(format!(r#"CREATE DATABASE "{}";"#, db_name).as_str())
            .await?;

        // Connect to the new database and run migrations
        let pool = PgPool::connect(&amp;connection_string).await?;

        sqlx::migrate!("./migrations")
            .run(&amp;pool)
            .await?;

        // Insert a test user and return the pool and database name
        let id: i64 = sqlx::query_as(
            r"INSERT INTO users(id, name, lastname, email)
            VALUES (
                0,
                'testname',
                'testlastname',
                'test@example.com'
            )
            RETURNING id",
        )
        .fetch_one(&amp;pool)
        .await?;

        Ok((pool, Uuid::parse_str(&amp;db_name).unwrap()))
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>You can now remove the <code>RUN_TEST_THREADS=1</code> environment variable and run your
tests in parallel again.</p>
</div>
        <div class="dialog" data-character="professor" aria-hidden="true">
          <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 80 80"><path d="M 22.912109 9 C 19.960756 9 16.358303 9.7221658 13.375 11.421875 C 10.391697 13.121584 8 15.945797 8 19.863281 C 8 23.912396 10.891498 29.481134 13.701172 34.283203 C 16.296271 38.718537 18.669822 42.066616 19.050781 42.601562 C 17.846834 43.88996 17.980826 45.617638 18.001953 46.048828 C 18.091373 47.889143 19.090293 49.025828 20.119141 50.009766 C 21.147988 50.993703 22.29396 52.003168 23.417969 54.146484 C 23.539513 54.378162 24.004424 55.269066 24.628906 56.189453 C 25.24048 57.090816 25.918145 58.062135 27 58.542969 L 27 62 A 1.0001 1.0001 0 1 0 29 62 L 29 58.896484 C 29.354145 58.891575 29.868383 58.864096 30.429688 58.5625 C 31.787375 57.836032 31.951153 56.396403 31.988281 56.148438 A 1.0001 1.0001 0 1 0 30.011719 55.851562 C 29.992849 55.977598 29.808687 56.625296 29.484375 56.798828 A 1.0001 1.0001 0 0 0 29.484375 56.800781 C 29.32398 56.886961 29.109495 56.908203 28.824219 56.908203 C 28.42168 56.908203 28.114506 56.845583 27.839844 56.730469 A 1.0001 1.0001 0 0 0 27.837891 56.730469 C 27.590274 56.626904 26.829471 55.871519 26.283203 55.066406 C 25.736935 54.261294 25.312909 53.452119 25.189453 53.216797 C 23.919462 50.795113 22.489106 49.508516 21.501953 48.564453 C 20.5148 47.62039 20.058624 47.197857 19.998047 45.951172 C 19.976167 45.504675 19.999847 44.424134 20.576172 43.884766 C 21.028522 43.465344 21.714425 43.416172 22.232422 43.408203 C 23.766212 43.383873 24.938922 44.373548 25.3125 44.726562 A 1.0002743 1.0002743 0 1 0 26.6875 43.273438 C 26.275119 42.883757 25.089926 41.848234 23.351562 41.519531 C 23.684054 41.310892 24.051927 41.094628 24.453125 40.890625 C 25.797547 40.207705 29.03107 39.122361 33.191406 38.230469 C 33.362903 39.669244 33.848762 40.995803 34.662109 41.990234 C 35.637853 43.183182 36.971947 44 38.431641 44 L 47 44 L 47 43.970703 C 47.165836 43.983009 47.331031 44 47.5 44 C 51.713438 44 55 40.110516 55 35.5 C 55 35.333552 54.986972 35.170279 54.978516 35.005859 C 55.073803 35.01009 56.315831 35.06789 58.087891 35.404297 C 59.928007 35.753621 62.199459 36.410722 63.869141 37.503906 C 65.629152 38.655646 66.920293 40.814446 67.544922 42.654297 C 67.845362 43.539246 67.971865 44.326947 67.982422 44.830078 C 67.8245 44.776948 67.775568 44.787958 67.476562 44.662109 C 66.877167 44.409819 66.00089 44.047905 64.669922 43.699219 C 63.396201 43.366066 62.05944 43.286417 60.980469 43.095703 C 59.901498 42.904989 59.211917 42.607238 58.945312 42.208984 C 58.739458 41.900705 58.727505 41.54692 58.730469 41.353516 L 58.730469 41.351562 C 58.747339 40.563466 59.456845 39.915621 59.625 39.78125 A 1.0004882 1.0004882 0 1 0 58.375 38.21875 C 58.105295 38.434268 56.766737 39.493484 56.730469 41.3125 A 1.0001 1.0001 0 0 0 56.730469 41.316406 C 56.729579 41.371576 56.669939 42.401916 57.283203 43.320312 A 1.0001 1.0001 0 0 0 57.283203 43.322266 C 58.065599 44.491012 59.390783 44.844917 60.632812 45.064453 C 61.206024 45.165771 61.701344 45.250974 62.261719 45.326172 A 1.0001 1.0001 0 0 0 62.365234 45.669922 C 62.365234 45.669922 64.86532 50.149397 70.380859 52.326172 C 70.066178 52.387382 69.842456 52.467172 69.421875 52.509766 C 67.775834 52.676464 65.347141 52.681641 61.869141 52.681641 C 60.410926 52.681641 59.025939 52.816151 57.824219 53.234375 C 56.622499 53.652599 55.548945 54.41984 55.070312 55.630859 A 1.0001 1.0001 0 1 0 56.929688 56.367188 C 57.147056 55.817206 57.616939 55.425526 58.480469 55.125 C 59.343999 54.824474 60.544355 54.681641 61.869141 54.681641 C 62.686115 54.681641 63.272154 54.67395 63.980469 54.671875 C 63.991735 54.744254 64 54.705608 64 55 C 64 55.437053 63.877671 55.664997 63.5625 55.982422 C 63.247329 56.299847 62.713826 56.63574 62.078125 56.990234 C 60.806723 57.699222 59.073341 58.493869 58.189453 60.285156 C 56.857013 62.98452 55.733424 64.124354 53.070312 65.472656 C 51.996454 66.016788 51.078649 66.994555 50.269531 68.115234 C 49.460414 69.235914 48.788791 70.501858 48.398438 71.6875 A 1.0001 1.0001 0 1 0 50.296875 72.3125 C 50.602522 71.384142 51.198696 70.24818 51.892578 69.287109 C 52.58646 68.326039 53.408468 67.542727 53.974609 67.255859 C 56.877498 65.786161 58.534815 64.106558 59.984375 61.169922 C 60.491487 60.142209 61.715387 59.48209 63.052734 58.736328 C 63.721408 58.363447 64.404296 57.972872 64.980469 57.392578 C 65.556642 56.812284 66 55.971947 66 55 C 66 54.863078 65.984562 54.780633 65.980469 54.662109 C 67.402198 54.642097 68.628908 54.600673 69.623047 54.5 C 71.455256 54.314448 72.78119 53.916388 73.710938 52.974609 A 1.0001 1.0001 0 0 0 73.199219 51.291016 C 68.046363 50.247259 65.904396 47.324169 65.021484 45.927734 C 65.635572 46.129085 66.311463 46.339874 66.701172 46.503906 C 67.290277 46.751872 67.803666 47 68.490234 47 C 69.069103 47 69.600788 46.524089 69.783203 46.121094 C 69.965618 45.718098 70 45.334232 70 44.916016 C 70 44.079583 69.804185 43.091793 69.4375 42.011719 C 68.93179 40.522147 68.087632 38.874396 66.859375 37.494141 C 67.586499 36.16648 68 34.528283 68 32.751953 C 68 30.707785 67.458226 28.840877 66.513672 27.423828 C 65.569118 26.00678 64.148 25 62.5 25 C 62.198677 25 61.907673 25.043753 61.623047 25.107422 C 60.608562 23.557292 58.772228 22.865658 57.222656 22.474609 C 55.724809 22.096614 54.696278 22.068044 54.404297 22.054688 C 53.369258 20.624763 51.713906 19.85094 50.222656 19.474609 C 48.853601 19.129115 48.072381 19.115957 47.646484 19.091797 C 46.698895 17.029339 44.859694 15.934026 43.265625 15.457031 C 41.714286 14.992823 40.640372 15.046519 40.339844 15.052734 C 40.200011 14.970291 37.831103 13.568401 34.46875 12.085938 C 31.009066 10.56056 26.67809 9 22.912109 9 z M 22.912109 11 C 26.103129 11 30.317543 12.43944 33.662109 13.914062 C 36.591489 15.205629 38.559962 16.332428 39.078125 16.632812 C 39.031274 16.749234 39 16.874603 39 17 C 39 17.26 39.109063 17.520937 39.289062 17.710938 L 39.439453 17.830078 C 39.499453 17.870078 39.559141 17.899922 39.619141 17.919922 C 39.679141 17.949922 39.740781 17.970469 39.800781 17.980469 C 39.870781 17.990469 39.94 18 40 18 C 40.26 18 40.520938 17.890937 40.710938 17.710938 C 40.879411 17.533105 40.977202 17.29191 40.990234 17.048828 C 41.462183 17.094999 42.05333 17.182117 42.691406 17.373047 C 44.090367 17.791659 45.481694 18.583147 46.050781 20.3125 A 1.0001 1.0001 0 0 0 46.136719 20.503906 C 46.115766 20.542254 46.092861 20.580793 46.080078 20.619141 C 46.050078 20.679141 46.029531 20.740547 46.019531 20.810547 C 46.009531 20.870547 46 20.94 46 21 C 46 21.26 46.109063 21.520937 46.289062 21.710938 C 46.479062 21.890938 46.74 22 47 22 C 47.13 22 47.260859 21.979922 47.380859 21.919922 C 47.510859 21.869922 47.620937 21.800937 47.710938 21.710938 C 47.86877 21.544337 47.957243 21.320799 47.980469 21.09375 C 48.470812 21.153682 49.078329 21.246548 49.734375 21.412109 C 51.191603 21.779855 52.640369 22.505079 53.103516 23.443359 A 1.0001 1.0001 0 0 0 53.966797 24 C 53.966797 24 55.277147 24.04436 56.734375 24.412109 C 57.726571 24.6625 58.706905 25.080934 59.382812 25.617188 C 58.824296 25.86258 58.539744 25.98812 57.689453 26.361328 C 56.959362 26.681778 56.226486 27.003639 55.669922 27.248047 C 55.113358 27.492455 54.829216 27.614987 54.699219 27.673828 L 54.695312 27.675781 L 54.691406 27.677734 C 53.992541 28.001013 53.204998 28.546857 52.652344 29.335938 C 51.318755 27.899257 49.518306 27 47.5 27 C 46.088192 27 44.784831 27.445042 43.669922 28.199219 L 43.619141 28.074219 C 43.619141 28.074219 41.839903 28.806549 40.052734 29.541016 C 39.15915 29.908249 38.263012 30.277439 37.587891 30.554688 C 37.25033 30.69331 36.967179 30.808775 36.767578 30.890625 C 36.667778 30.931555 36.589701 30.964028 36.535156 30.986328 C 36.507886 30.997478 36.486666 31.006019 36.472656 31.011719 L 36.46875 31.013672 C 34.671404 31.696821 33.47822 33.590501 33.173828 36.177734 C 28.67056 37.106859 25.2554 38.241503 23.546875 39.109375 C 21.798376 39.998459 20.656347 40.983978 20.472656 41.144531 C 19.868962 40.289449 17.808667 37.342736 15.427734 33.273438 C 12.672408 28.564256 10 22.859167 10 19.863281 C 10 16.735766 11.783287 14.629244 14.365234 13.158203 C 16.947182 11.687162 20.298462 11 22.912109 11 z M 19 16 A 2 2 0 0 0 19 20 A 2 2 0 0 0 19 16 z M 35.697266 16.384766 C 35.567734 16.369922 35.433281 16.379922 35.300781 16.419922 C 34.770781 16.579922 34.470859 17.139922 34.630859 17.669922 C 34.760859 18.099922 35.159844 18.380859 35.589844 18.380859 C 35.679844 18.380859 35.779141 18.369844 35.869141 18.339844 C 36.399141 18.179844 36.700781 17.619609 36.550781 17.099609 C 36.430781 16.702109 36.085859 16.429297 35.697266 16.384766 z M 31.787109 18.501953 C 31.532109 18.481953 31.270547 18.560234 31.060547 18.740234 C 30.640547 19.100234 30.589219 19.740156 30.949219 20.160156 C 31.149219 20.390156 31.430937 20.5 31.710938 20.5 C 31.940937 20.5 32.169375 20.419766 32.359375 20.259766 C 32.779375 19.899766 32.830703 19.269609 32.470703 18.849609 C 32.290703 18.639609 32.042109 18.521953 31.787109 18.501953 z M 43.460938 20.029297 C 42.900937 20.079297 42.489297 20.549609 42.529297 21.099609 C 42.569297 21.629609 42.999297 22.029297 43.529297 22.029297 L 43.599609 22.029297 C 44.149609 21.989297 44.559531 21.510938 44.519531 20.960938 C 44.479531 20.410937 44.010937 19.999297 43.460938 20.029297 z M 40.207031 20.570312 C 40.0775 20.557344 39.943047 20.569375 39.810547 20.609375 C 39.280547 20.779375 38.990391 21.339375 39.150391 21.859375 C 39.280391 22.289375 39.679375 22.570312 40.109375 22.570312 C 40.199375 22.570312 40.300391 22.549531 40.400391 22.519531 C 40.930391 22.359531 41.220547 21.799531 41.060547 21.269531 C 40.940547 20.872031 40.595625 20.609219 40.207031 20.570312 z M 28.90625 22.005859 C 28.808594 22.015547 28.711641 22.040078 28.619141 22.080078 C 28.499141 22.130078 28.389063 22.199063 28.289062 22.289062 C 28.109062 22.479062 28 22.74 28 23 C 28 23.13 28.030078 23.260859 28.080078 23.380859 C 28.130078 23.510859 28.199063 23.610937 28.289062 23.710938 C 28.389062 23.800938 28.499141 23.869922 28.619141 23.919922 C 28.739141 23.969922 28.87 24 29 24 C 29.26 24 29.520938 23.890938 29.710938 23.710938 C 29.800938 23.610938 29.869922 23.510859 29.919922 23.380859 C 29.979922 23.260859 30 23.13 30 23 C 30 22.74 29.890937 22.479063 29.710938 22.289062 C 29.500937 22.079062 29.199219 21.976797 28.90625 22.005859 z M 37.169922 22.267578 C 36.914922 22.252578 36.654219 22.334531 36.449219 22.519531 C 36.039219 22.889531 35.999141 23.519453 36.369141 23.939453 C 36.569141 24.159453 36.839141 24.269531 37.119141 24.269531 C 37.349141 24.269531 37.589297 24.179766 37.779297 24.009766 C 38.189297 23.639766 38.229375 23.009609 37.859375 22.599609 C 37.674375 22.394609 37.424922 22.282578 37.169922 22.267578 z M 18.5 24 A 1.5 1.5 0 0 0 18.5 27 A 1.5 1.5 0 0 0 18.5 24 z M 34.800781 25.019531 C 34.740781 25.029531 34.679141 25.050078 34.619141 25.080078 C 34.559141 25.100078 34.499453 25.129922 34.439453 25.169922 C 34.389453 25.199922 34.339062 25.249063 34.289062 25.289062 C 34.109062 25.479062 34 25.74 34 26 C 34 26.26 34.109063 26.520937 34.289062 26.710938 C 34.339063 26.750938 34.389453 26.800078 34.439453 26.830078 C 34.499453 26.870078 34.559141 26.899922 34.619141 26.919922 C 34.679141 26.949922 34.740781 26.970469 34.800781 26.980469 C 34.870781 26.990469 34.94 27 35 27 C 35.06 27 35.129219 26.990469 35.199219 26.980469 C 35.259219 26.970469 35.320859 26.949922 35.380859 26.919922 C 35.440859 26.899922 35.500547 26.870078 35.560547 26.830078 C 35.610547 26.800078 35.660938 26.750937 35.710938 26.710938 C 35.750938 26.660938 35.800078 26.610547 35.830078 26.560547 C 35.870078 26.500547 35.899922 26.440859 35.919922 26.380859 C 35.949922 26.320859 35.970469 26.259219 35.980469 26.199219 C 35.990469 26.129219 36 26.06 36 26 C 36 25.94 35.990469 25.870781 35.980469 25.800781 C 35.970469 25.740781 35.949922 25.679141 35.919922 25.619141 C 35.899922 25.559141 35.870078 25.499453 35.830078 25.439453 C 35.800078 25.389453 35.750937 25.339063 35.710938 25.289062 C 35.660937 25.249062 35.610547 25.199922 35.560547 25.169922 C 35.500547 25.129922 35.440859 25.100078 35.380859 25.080078 C 35.320859 25.050078 35.259219 25.029531 35.199219 25.019531 C 35.069219 24.989531 34.930781 24.989531 34.800781 25.019531 z M 62.5 27 C 63.337 27 64.165413 27.504798 64.849609 28.53125 C 65.533805 29.557702 66 31.068121 66 32.751953 C 66 34.00733 65.737519 35.160765 65.320312 36.09375 C 65.200872 36.006694 65.089403 35.911589 64.964844 35.830078 C 63.115425 34.619215 60.926832 33.96549 59.054688 33.570312 C 59.029888 33.300341 59 33.032794 59 32.751953 C 59 31.068121 59.466195 29.557702 60.150391 28.53125 C 60.834587 27.504798 61.663 27 62.5 27 z M 57.916016 28.445312 C 57.328962 29.690383 57 31.166369 57 32.751953 C 57 32.906001 57.009389 33.055718 57.015625 33.207031 C 55.772365 33.037808 54.913216 33 54.912109 33 A 1.0001 1.0001 0 0 0 54.673828 33.027344 C 54.498232 32.374737 54.249591 31.758461 53.947266 31.179688 L 53.982422 31.185547 C 54.0947 30.58994 54.973824 29.750039 55.53125 29.492188 C 55.45871 29.525108 55.922938 29.320384 56.474609 29.078125 C 56.871683 28.903755 57.404225 28.670009 57.916016 28.445312 z M 47.5 29 C 50.466269 29 53 31.826245 53 35.5 C 53 39.173755 50.466269 42 47.5 42 C 44.533731 42 42 39.173755 42 35.5 C 42 31.826245 44.533731 29 47.5 29 z M 40.976562 31.322266 C 40.355891 32.56516 40 33.991941 40 35.5 C 40 38.077287 41.028118 40.426794 42.669922 42 L 38.431641 42 C 37.756334 42 36.895938 41.562119 36.210938 40.724609 C 35.525937 39.8871 35.042969 38.697103 35.042969 37.390625 C 35.042969 34.679516 36.061342 33.310422 37.177734 32.884766 C 37.177734 32.884766 37.179688 32.882812 37.179688 32.882812 C 37.226627 32.865252 37.212823 32.868871 37.226562 32.863281 C 37.242672 32.856681 37.265109 32.849281 37.292969 32.837891 C 37.348679 32.815111 37.427234 32.781294 37.527344 32.740234 C 37.727564 32.658134 38.009936 32.542986 38.347656 32.404297 C 39.023097 32.126919 39.918835 31.757892 40.8125 31.390625 C 40.894854 31.35678 40.894409 31.356021 40.976562 31.322266 z M 52.027344 45.986328 A 1.0001 1.0001 0 0 0 51.001953 47.056641 C 51.12024 49.123838 50.810157 50.953649 50.085938 52.595703 A 1.0001 1.0001 0 0 0 50.083984 52.597656 C 49.747402 53.362036 49.483444 53.693798 49.085938 54.595703 C 48.224157 56.549649 47.86824 58.719838 48.001953 61.056641 A 1.0001 1.0001 0 1 0 49.998047 60.943359 C 49.87976 58.876162 50.189842 57.046351 50.914062 55.404297 A 1.0001 1.0001 0 0 0 50.916016 55.402344 C 51.249338 54.645539 51.513827 54.312598 51.914062 53.404297 C 52.775843 51.450351 53.13176 49.280162 52.998047 46.943359 A 1.0001 1.0001 0 0 0 52.027344 45.986328 z M 27.988281 47.988281 A 1.0001 1.0001 0 0 0 27.167969 49.554688 C 27.834713 50.554304 28.500713 51.554304 29.167969 52.554688 A 1.0001 1.0001 0 1 0 30.832031 51.445312 C 30.165287 50.445696 29.499287 49.445696 28.832031 48.445312 A 1.0001 1.0001 0 0 0 27.988281 47.988281 z M 28 64.330078 C 27.45 64.330078 27 64.780078 27 65.330078 C 27 65.890078 27.45 66.330078 28 66.330078 C 28.55 66.330078 29 65.890078 29 65.330078 C 29 64.780078 28.55 64.330078 28 64.330078 z M 28 67.669922 C 27.45 67.669922 27 68.119922 27 68.669922 C 27 69.219922 27.45 69.669922 28 69.669922 C 28.55 69.669922 29 69.219922 29 68.669922 C 29 68.119922 28.55 67.669922 28 67.669922 z M 28.09375 71.005859 C 27.800781 70.976797 27.499063 71.079062 27.289062 71.289062 L 27.169922 71.439453 C 27.129922 71.499453 27.100078 71.559141 27.080078 71.619141 C 27.050078 71.679141 27.029531 71.740781 27.019531 71.800781 C 27.009531 71.870781 27 71.93 27 72 C 27 72.13 27.030078 72.260859 27.080078 72.380859 C 27.130078 72.500859 27.199063 72.610938 27.289062 72.710938 C 27.479062 72.890937 27.74 73 28 73 C 28.13 73 28.260859 72.969922 28.380859 72.919922 C 28.500859 72.869922 28.610937 72.800938 28.710938 72.710938 C 28.800938 72.610937 28.869922 72.500859 28.919922 72.380859 C 28.969922 72.260859 29 72.13 29 72 C 29 71.93 28.990469 71.870781 28.980469 71.800781 C 28.970469 71.740781 28.949922 71.679141 28.919922 71.619141 C 28.899922 71.559141 28.870078 71.499453 28.830078 71.439453 L 28.710938 71.289062 C 28.610938 71.199063 28.500859 71.130078 28.380859 71.080078 C 28.288359 71.040078 28.191406 71.015547 28.09375 71.005859 z"/></svg>
          <p>But the title of the post said "for lazy people", and this is not lazy at all!</p>
        </div>
<div class="paragraph">
<p>I promised in the title that this post would be for lazy people, and although it
surely started that way, those were not good and valid solutions. But don&#8217;t
despair, there is a lazy solution after all!</p>
</div>
<div class="paragraph">
<p>Lately I&#8217;ve been doing some work in the <code>sqlx</code> crate, and I stumbled upon a
really useful macro that will return a connection to an isolated database that
your tests can use: <a href="https://docs.rs/sqlx/latest/sqlx/attr.test.html"><code>#[sqlx::test</a></code>].</p>
</div>
<div class="quoteblock">
<blockquote>
<div class="paragraph">
<p>#[sqlx::test] can automatically create test databases for you and provide live
connections to your test.</p>
</div>
<div class="paragraph">
<p>For every annotated function, a new test database is created so tests can run
against a live database but are isolated from each other.</p>
</div>
</blockquote>
</div>
<div class="paragraph">
<p>That looks just perfect, the macro will automatically behave as a classic
<code>#[tokio::test]</code> but it will also inject a <code>PgPool</code> into our test function.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">#[cfg(test)]
mod tests {
    use super::*;

    // note that the function now takes a PgPool as an argument
    #[sqlx::test] // by default this will also apply the migrations!
    async fn test_user_exists(pool: PgPool) {
        assert!(user_exists(&amp;pool, "test@example.com").await.unwrap());
        assert!(!user_exists(&amp;pool, "another@example.com").await.unwrap())
    }

    // if you want you can specify a different migrations directory
    #[sqlx::test(migrations = "./someothermigrations")]
    async fn test_get_user_id(pool: PgPool) {
        assert_eq!(
            get_user_id(&amp;pool, "test@example.com").await.unwrap(),
            0,
            "test user should have id = 0"
        );
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>We&#8217;ve got rid of our setup logic, everything is given to us for free by the
macro and we have our test isolation - is that lazy enough?</p>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>This is yet another wake-up call for me to roam around the documentation more</p>
        </div>]]></content><author><name>Mattia Righetti</name></author><summary type="html"><![CDATA[Anyone who has developed with Rust using sqlx for database operations has likely needed to write tests for their database interactions.]]></summary></entry><entry><title type="html">You Probably Don’t Need Query Builders</title><link href="https://mattrighetti.com/2025/01/20/you-dont-need-sql-builders.html" rel="alternate" type="text/html" title="You Probably Don’t Need Query Builders" /><published>2025-01-20T00:00:00+00:00</published><updated>2025-01-20T00:00:00+00:00</updated><id>https://mattrighetti.com/2025/01/20/you-dont-need-sql-builders</id><content type="html" xml:base="https://mattrighetti.com/2025/01/20/you-dont-need-sql-builders.html"><![CDATA[<div class="paragraph">
<p>In a previous <a href="https://mattrighetti.com/2025/01/14/ditching-sea-query.html">[0]</a> post I&#8217;ve discussed why I ditched sea-query and why sql is almost
always the best way to not re-learn something new from the beginning that will
inevitably end up slowing you down or simply not working at all in the long run.</p>
</div>
<div class="paragraph">
<p>From time to time, I still stumble upon stackoverflow questions like this one
<a href="https://stackoverflow.com/questions/74956100/how-to-build-safe-dynamic-query-with-sqlx-in-rust">[1]</a>.</p>
</div>
<div class="paragraph">
<p>OP is basically asking for a way to dynamically build a query based on a
search filter that is declared like the following</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">struct Search {
    id: i64,
    username: Option&lt;String&gt;,
    min_age: Option&lt;i8&gt;,
    max_age: Option&lt;i8&gt;,
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>You may agree that this is a very common scenario in every single API that&#8217;s out
there. You have an endpoint that returns some items and you want to expose a way
to filter those items so that the client can ask only for things it&#8217;s interested in.</p>
</div>
<div class="paragraph">
<p>What&#8217;s even more interesting is that all the answers to OP involve some kind of
<em>complex</em> way to build the query and <code>if</code> and <code>else</code> that, at least in that
case, are most likely unnecessary.</p>
</div>
<div class="paragraph">
<p>One of the answers suggests the following implementation</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">fn search_query(search: Search) -&gt; String {
    let mut query = QueryBuilder::new("SELECT * from users where id = ");
    query.push_bind(search.id);

    if let Some(username) = search.username {
        query.push(" AND username = ");
        query.push_bind(username);
    }

    if let Some(min_age) = search.min_age {
        query.push(" AND age &gt; ");
        query.push_bind(min_age);
    }

    if let Some(max_age) = search.max_age {
        query.push(" AND age &lt; ");
        query.push_bind(max_age);
    }

    query.build().sql().into()
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>The above solution works just fine, sure - the fact that the field <code>id</code> is
always present in the <code>Search</code> filter simplifies stuff a bit. What if <code>id</code> was
an <code>Option&lt;i64&gt;</code>? If you want to keep using the query builder you&#8217;d need to
introduce another <code>if</code> statement checking if the <code>id</code> is present or not</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">fn search_query(search: Search) -&gt; String {
    let mut query = QueryBuilder::new("SELECT * from users");

    if let Some(id) = search.id {
        query.push(" AND id = ");
        query.push_bind(search.id);
    }

    if let Some(username) = search.username {
        query.push(" AND username = ");
        query.push_bind(username);
    }

    if let Some(min_age) = search.min_age {
        query.push(" AND age &gt; ");
        query.push_bind(min_age);
    }

    if let Some(max_age) = search.max_age {
        query.push(" AND age &lt; ");
        query.push_bind(max_age);
    }

    query.build().sql().into()
}</code></pre>
</div>
</div>
        <div class="dialog" data-character="professor" aria-hidden="true">
          <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 80 80"><path d="M 22.912109 9 C 19.960756 9 16.358303 9.7221658 13.375 11.421875 C 10.391697 13.121584 8 15.945797 8 19.863281 C 8 23.912396 10.891498 29.481134 13.701172 34.283203 C 16.296271 38.718537 18.669822 42.066616 19.050781 42.601562 C 17.846834 43.88996 17.980826 45.617638 18.001953 46.048828 C 18.091373 47.889143 19.090293 49.025828 20.119141 50.009766 C 21.147988 50.993703 22.29396 52.003168 23.417969 54.146484 C 23.539513 54.378162 24.004424 55.269066 24.628906 56.189453 C 25.24048 57.090816 25.918145 58.062135 27 58.542969 L 27 62 A 1.0001 1.0001 0 1 0 29 62 L 29 58.896484 C 29.354145 58.891575 29.868383 58.864096 30.429688 58.5625 C 31.787375 57.836032 31.951153 56.396403 31.988281 56.148438 A 1.0001 1.0001 0 1 0 30.011719 55.851562 C 29.992849 55.977598 29.808687 56.625296 29.484375 56.798828 A 1.0001 1.0001 0 0 0 29.484375 56.800781 C 29.32398 56.886961 29.109495 56.908203 28.824219 56.908203 C 28.42168 56.908203 28.114506 56.845583 27.839844 56.730469 A 1.0001 1.0001 0 0 0 27.837891 56.730469 C 27.590274 56.626904 26.829471 55.871519 26.283203 55.066406 C 25.736935 54.261294 25.312909 53.452119 25.189453 53.216797 C 23.919462 50.795113 22.489106 49.508516 21.501953 48.564453 C 20.5148 47.62039 20.058624 47.197857 19.998047 45.951172 C 19.976167 45.504675 19.999847 44.424134 20.576172 43.884766 C 21.028522 43.465344 21.714425 43.416172 22.232422 43.408203 C 23.766212 43.383873 24.938922 44.373548 25.3125 44.726562 A 1.0002743 1.0002743 0 1 0 26.6875 43.273438 C 26.275119 42.883757 25.089926 41.848234 23.351562 41.519531 C 23.684054 41.310892 24.051927 41.094628 24.453125 40.890625 C 25.797547 40.207705 29.03107 39.122361 33.191406 38.230469 C 33.362903 39.669244 33.848762 40.995803 34.662109 41.990234 C 35.637853 43.183182 36.971947 44 38.431641 44 L 47 44 L 47 43.970703 C 47.165836 43.983009 47.331031 44 47.5 44 C 51.713438 44 55 40.110516 55 35.5 C 55 35.333552 54.986972 35.170279 54.978516 35.005859 C 55.073803 35.01009 56.315831 35.06789 58.087891 35.404297 C 59.928007 35.753621 62.199459 36.410722 63.869141 37.503906 C 65.629152 38.655646 66.920293 40.814446 67.544922 42.654297 C 67.845362 43.539246 67.971865 44.326947 67.982422 44.830078 C 67.8245 44.776948 67.775568 44.787958 67.476562 44.662109 C 66.877167 44.409819 66.00089 44.047905 64.669922 43.699219 C 63.396201 43.366066 62.05944 43.286417 60.980469 43.095703 C 59.901498 42.904989 59.211917 42.607238 58.945312 42.208984 C 58.739458 41.900705 58.727505 41.54692 58.730469 41.353516 L 58.730469 41.351562 C 58.747339 40.563466 59.456845 39.915621 59.625 39.78125 A 1.0004882 1.0004882 0 1 0 58.375 38.21875 C 58.105295 38.434268 56.766737 39.493484 56.730469 41.3125 A 1.0001 1.0001 0 0 0 56.730469 41.316406 C 56.729579 41.371576 56.669939 42.401916 57.283203 43.320312 A 1.0001 1.0001 0 0 0 57.283203 43.322266 C 58.065599 44.491012 59.390783 44.844917 60.632812 45.064453 C 61.206024 45.165771 61.701344 45.250974 62.261719 45.326172 A 1.0001 1.0001 0 0 0 62.365234 45.669922 C 62.365234 45.669922 64.86532 50.149397 70.380859 52.326172 C 70.066178 52.387382 69.842456 52.467172 69.421875 52.509766 C 67.775834 52.676464 65.347141 52.681641 61.869141 52.681641 C 60.410926 52.681641 59.025939 52.816151 57.824219 53.234375 C 56.622499 53.652599 55.548945 54.41984 55.070312 55.630859 A 1.0001 1.0001 0 1 0 56.929688 56.367188 C 57.147056 55.817206 57.616939 55.425526 58.480469 55.125 C 59.343999 54.824474 60.544355 54.681641 61.869141 54.681641 C 62.686115 54.681641 63.272154 54.67395 63.980469 54.671875 C 63.991735 54.744254 64 54.705608 64 55 C 64 55.437053 63.877671 55.664997 63.5625 55.982422 C 63.247329 56.299847 62.713826 56.63574 62.078125 56.990234 C 60.806723 57.699222 59.073341 58.493869 58.189453 60.285156 C 56.857013 62.98452 55.733424 64.124354 53.070312 65.472656 C 51.996454 66.016788 51.078649 66.994555 50.269531 68.115234 C 49.460414 69.235914 48.788791 70.501858 48.398438 71.6875 A 1.0001 1.0001 0 1 0 50.296875 72.3125 C 50.602522 71.384142 51.198696 70.24818 51.892578 69.287109 C 52.58646 68.326039 53.408468 67.542727 53.974609 67.255859 C 56.877498 65.786161 58.534815 64.106558 59.984375 61.169922 C 60.491487 60.142209 61.715387 59.48209 63.052734 58.736328 C 63.721408 58.363447 64.404296 57.972872 64.980469 57.392578 C 65.556642 56.812284 66 55.971947 66 55 C 66 54.863078 65.984562 54.780633 65.980469 54.662109 C 67.402198 54.642097 68.628908 54.600673 69.623047 54.5 C 71.455256 54.314448 72.78119 53.916388 73.710938 52.974609 A 1.0001 1.0001 0 0 0 73.199219 51.291016 C 68.046363 50.247259 65.904396 47.324169 65.021484 45.927734 C 65.635572 46.129085 66.311463 46.339874 66.701172 46.503906 C 67.290277 46.751872 67.803666 47 68.490234 47 C 69.069103 47 69.600788 46.524089 69.783203 46.121094 C 69.965618 45.718098 70 45.334232 70 44.916016 C 70 44.079583 69.804185 43.091793 69.4375 42.011719 C 68.93179 40.522147 68.087632 38.874396 66.859375 37.494141 C 67.586499 36.16648 68 34.528283 68 32.751953 C 68 30.707785 67.458226 28.840877 66.513672 27.423828 C 65.569118 26.00678 64.148 25 62.5 25 C 62.198677 25 61.907673 25.043753 61.623047 25.107422 C 60.608562 23.557292 58.772228 22.865658 57.222656 22.474609 C 55.724809 22.096614 54.696278 22.068044 54.404297 22.054688 C 53.369258 20.624763 51.713906 19.85094 50.222656 19.474609 C 48.853601 19.129115 48.072381 19.115957 47.646484 19.091797 C 46.698895 17.029339 44.859694 15.934026 43.265625 15.457031 C 41.714286 14.992823 40.640372 15.046519 40.339844 15.052734 C 40.200011 14.970291 37.831103 13.568401 34.46875 12.085938 C 31.009066 10.56056 26.67809 9 22.912109 9 z M 22.912109 11 C 26.103129 11 30.317543 12.43944 33.662109 13.914062 C 36.591489 15.205629 38.559962 16.332428 39.078125 16.632812 C 39.031274 16.749234 39 16.874603 39 17 C 39 17.26 39.109063 17.520937 39.289062 17.710938 L 39.439453 17.830078 C 39.499453 17.870078 39.559141 17.899922 39.619141 17.919922 C 39.679141 17.949922 39.740781 17.970469 39.800781 17.980469 C 39.870781 17.990469 39.94 18 40 18 C 40.26 18 40.520938 17.890937 40.710938 17.710938 C 40.879411 17.533105 40.977202 17.29191 40.990234 17.048828 C 41.462183 17.094999 42.05333 17.182117 42.691406 17.373047 C 44.090367 17.791659 45.481694 18.583147 46.050781 20.3125 A 1.0001 1.0001 0 0 0 46.136719 20.503906 C 46.115766 20.542254 46.092861 20.580793 46.080078 20.619141 C 46.050078 20.679141 46.029531 20.740547 46.019531 20.810547 C 46.009531 20.870547 46 20.94 46 21 C 46 21.26 46.109063 21.520937 46.289062 21.710938 C 46.479062 21.890938 46.74 22 47 22 C 47.13 22 47.260859 21.979922 47.380859 21.919922 C 47.510859 21.869922 47.620937 21.800937 47.710938 21.710938 C 47.86877 21.544337 47.957243 21.320799 47.980469 21.09375 C 48.470812 21.153682 49.078329 21.246548 49.734375 21.412109 C 51.191603 21.779855 52.640369 22.505079 53.103516 23.443359 A 1.0001 1.0001 0 0 0 53.966797 24 C 53.966797 24 55.277147 24.04436 56.734375 24.412109 C 57.726571 24.6625 58.706905 25.080934 59.382812 25.617188 C 58.824296 25.86258 58.539744 25.98812 57.689453 26.361328 C 56.959362 26.681778 56.226486 27.003639 55.669922 27.248047 C 55.113358 27.492455 54.829216 27.614987 54.699219 27.673828 L 54.695312 27.675781 L 54.691406 27.677734 C 53.992541 28.001013 53.204998 28.546857 52.652344 29.335938 C 51.318755 27.899257 49.518306 27 47.5 27 C 46.088192 27 44.784831 27.445042 43.669922 28.199219 L 43.619141 28.074219 C 43.619141 28.074219 41.839903 28.806549 40.052734 29.541016 C 39.15915 29.908249 38.263012 30.277439 37.587891 30.554688 C 37.25033 30.69331 36.967179 30.808775 36.767578 30.890625 C 36.667778 30.931555 36.589701 30.964028 36.535156 30.986328 C 36.507886 30.997478 36.486666 31.006019 36.472656 31.011719 L 36.46875 31.013672 C 34.671404 31.696821 33.47822 33.590501 33.173828 36.177734 C 28.67056 37.106859 25.2554 38.241503 23.546875 39.109375 C 21.798376 39.998459 20.656347 40.983978 20.472656 41.144531 C 19.868962 40.289449 17.808667 37.342736 15.427734 33.273438 C 12.672408 28.564256 10 22.859167 10 19.863281 C 10 16.735766 11.783287 14.629244 14.365234 13.158203 C 16.947182 11.687162 20.298462 11 22.912109 11 z M 19 16 A 2 2 0 0 0 19 20 A 2 2 0 0 0 19 16 z M 35.697266 16.384766 C 35.567734 16.369922 35.433281 16.379922 35.300781 16.419922 C 34.770781 16.579922 34.470859 17.139922 34.630859 17.669922 C 34.760859 18.099922 35.159844 18.380859 35.589844 18.380859 C 35.679844 18.380859 35.779141 18.369844 35.869141 18.339844 C 36.399141 18.179844 36.700781 17.619609 36.550781 17.099609 C 36.430781 16.702109 36.085859 16.429297 35.697266 16.384766 z M 31.787109 18.501953 C 31.532109 18.481953 31.270547 18.560234 31.060547 18.740234 C 30.640547 19.100234 30.589219 19.740156 30.949219 20.160156 C 31.149219 20.390156 31.430937 20.5 31.710938 20.5 C 31.940937 20.5 32.169375 20.419766 32.359375 20.259766 C 32.779375 19.899766 32.830703 19.269609 32.470703 18.849609 C 32.290703 18.639609 32.042109 18.521953 31.787109 18.501953 z M 43.460938 20.029297 C 42.900937 20.079297 42.489297 20.549609 42.529297 21.099609 C 42.569297 21.629609 42.999297 22.029297 43.529297 22.029297 L 43.599609 22.029297 C 44.149609 21.989297 44.559531 21.510938 44.519531 20.960938 C 44.479531 20.410937 44.010937 19.999297 43.460938 20.029297 z M 40.207031 20.570312 C 40.0775 20.557344 39.943047 20.569375 39.810547 20.609375 C 39.280547 20.779375 38.990391 21.339375 39.150391 21.859375 C 39.280391 22.289375 39.679375 22.570312 40.109375 22.570312 C 40.199375 22.570312 40.300391 22.549531 40.400391 22.519531 C 40.930391 22.359531 41.220547 21.799531 41.060547 21.269531 C 40.940547 20.872031 40.595625 20.609219 40.207031 20.570312 z M 28.90625 22.005859 C 28.808594 22.015547 28.711641 22.040078 28.619141 22.080078 C 28.499141 22.130078 28.389063 22.199063 28.289062 22.289062 C 28.109062 22.479062 28 22.74 28 23 C 28 23.13 28.030078 23.260859 28.080078 23.380859 C 28.130078 23.510859 28.199063 23.610937 28.289062 23.710938 C 28.389062 23.800938 28.499141 23.869922 28.619141 23.919922 C 28.739141 23.969922 28.87 24 29 24 C 29.26 24 29.520938 23.890938 29.710938 23.710938 C 29.800938 23.610938 29.869922 23.510859 29.919922 23.380859 C 29.979922 23.260859 30 23.13 30 23 C 30 22.74 29.890937 22.479063 29.710938 22.289062 C 29.500937 22.079062 29.199219 21.976797 28.90625 22.005859 z M 37.169922 22.267578 C 36.914922 22.252578 36.654219 22.334531 36.449219 22.519531 C 36.039219 22.889531 35.999141 23.519453 36.369141 23.939453 C 36.569141 24.159453 36.839141 24.269531 37.119141 24.269531 C 37.349141 24.269531 37.589297 24.179766 37.779297 24.009766 C 38.189297 23.639766 38.229375 23.009609 37.859375 22.599609 C 37.674375 22.394609 37.424922 22.282578 37.169922 22.267578 z M 18.5 24 A 1.5 1.5 0 0 0 18.5 27 A 1.5 1.5 0 0 0 18.5 24 z M 34.800781 25.019531 C 34.740781 25.029531 34.679141 25.050078 34.619141 25.080078 C 34.559141 25.100078 34.499453 25.129922 34.439453 25.169922 C 34.389453 25.199922 34.339062 25.249063 34.289062 25.289062 C 34.109062 25.479062 34 25.74 34 26 C 34 26.26 34.109063 26.520937 34.289062 26.710938 C 34.339063 26.750938 34.389453 26.800078 34.439453 26.830078 C 34.499453 26.870078 34.559141 26.899922 34.619141 26.919922 C 34.679141 26.949922 34.740781 26.970469 34.800781 26.980469 C 34.870781 26.990469 34.94 27 35 27 C 35.06 27 35.129219 26.990469 35.199219 26.980469 C 35.259219 26.970469 35.320859 26.949922 35.380859 26.919922 C 35.440859 26.899922 35.500547 26.870078 35.560547 26.830078 C 35.610547 26.800078 35.660938 26.750937 35.710938 26.710938 C 35.750938 26.660938 35.800078 26.610547 35.830078 26.560547 C 35.870078 26.500547 35.899922 26.440859 35.919922 26.380859 C 35.949922 26.320859 35.970469 26.259219 35.980469 26.199219 C 35.990469 26.129219 36 26.06 36 26 C 36 25.94 35.990469 25.870781 35.980469 25.800781 C 35.970469 25.740781 35.949922 25.679141 35.919922 25.619141 C 35.899922 25.559141 35.870078 25.499453 35.830078 25.439453 C 35.800078 25.389453 35.750937 25.339063 35.710938 25.289062 C 35.660937 25.249062 35.610547 25.199922 35.560547 25.169922 C 35.500547 25.129922 35.440859 25.100078 35.380859 25.080078 C 35.320859 25.050078 35.259219 25.029531 35.199219 25.019531 C 35.069219 24.989531 34.930781 24.989531 34.800781 25.019531 z M 62.5 27 C 63.337 27 64.165413 27.504798 64.849609 28.53125 C 65.533805 29.557702 66 31.068121 66 32.751953 C 66 34.00733 65.737519 35.160765 65.320312 36.09375 C 65.200872 36.006694 65.089403 35.911589 64.964844 35.830078 C 63.115425 34.619215 60.926832 33.96549 59.054688 33.570312 C 59.029888 33.300341 59 33.032794 59 32.751953 C 59 31.068121 59.466195 29.557702 60.150391 28.53125 C 60.834587 27.504798 61.663 27 62.5 27 z M 57.916016 28.445312 C 57.328962 29.690383 57 31.166369 57 32.751953 C 57 32.906001 57.009389 33.055718 57.015625 33.207031 C 55.772365 33.037808 54.913216 33 54.912109 33 A 1.0001 1.0001 0 0 0 54.673828 33.027344 C 54.498232 32.374737 54.249591 31.758461 53.947266 31.179688 L 53.982422 31.185547 C 54.0947 30.58994 54.973824 29.750039 55.53125 29.492188 C 55.45871 29.525108 55.922938 29.320384 56.474609 29.078125 C 56.871683 28.903755 57.404225 28.670009 57.916016 28.445312 z M 47.5 29 C 50.466269 29 53 31.826245 53 35.5 C 53 39.173755 50.466269 42 47.5 42 C 44.533731 42 42 39.173755 42 35.5 C 42 31.826245 44.533731 29 47.5 29 z M 40.976562 31.322266 C 40.355891 32.56516 40 33.991941 40 35.5 C 40 38.077287 41.028118 40.426794 42.669922 42 L 38.431641 42 C 37.756334 42 36.895938 41.562119 36.210938 40.724609 C 35.525937 39.8871 35.042969 38.697103 35.042969 37.390625 C 35.042969 34.679516 36.061342 33.310422 37.177734 32.884766 C 37.177734 32.884766 37.179688 32.882812 37.179688 32.882812 C 37.226627 32.865252 37.212823 32.868871 37.226562 32.863281 C 37.242672 32.856681 37.265109 32.849281 37.292969 32.837891 C 37.348679 32.815111 37.427234 32.781294 37.527344 32.740234 C 37.727564 32.658134 38.009936 32.542986 38.347656 32.404297 C 39.023097 32.126919 39.918835 31.757892 40.8125 31.390625 C 40.894854 31.35678 40.894409 31.356021 40.976562 31.322266 z M 52.027344 45.986328 A 1.0001 1.0001 0 0 0 51.001953 47.056641 C 51.12024 49.123838 50.810157 50.953649 50.085938 52.595703 A 1.0001 1.0001 0 0 0 50.083984 52.597656 C 49.747402 53.362036 49.483444 53.693798 49.085938 54.595703 C 48.224157 56.549649 47.86824 58.719838 48.001953 61.056641 A 1.0001 1.0001 0 1 0 49.998047 60.943359 C 49.87976 58.876162 50.189842 57.046351 50.914062 55.404297 A 1.0001 1.0001 0 0 0 50.916016 55.402344 C 51.249338 54.645539 51.513827 54.312598 51.914062 53.404297 C 52.775843 51.450351 53.13176 49.280162 52.998047 46.943359 A 1.0001 1.0001 0 0 0 52.027344 45.986328 z M 27.988281 47.988281 A 1.0001 1.0001 0 0 0 27.167969 49.554688 C 27.834713 50.554304 28.500713 51.554304 29.167969 52.554688 A 1.0001 1.0001 0 1 0 30.832031 51.445312 C 30.165287 50.445696 29.499287 49.445696 28.832031 48.445312 A 1.0001 1.0001 0 0 0 27.988281 47.988281 z M 28 64.330078 C 27.45 64.330078 27 64.780078 27 65.330078 C 27 65.890078 27.45 66.330078 28 66.330078 C 28.55 66.330078 29 65.890078 29 65.330078 C 29 64.780078 28.55 64.330078 28 64.330078 z M 28 67.669922 C 27.45 67.669922 27 68.119922 27 68.669922 C 27 69.219922 27.45 69.669922 28 69.669922 C 28.55 69.669922 29 69.219922 29 68.669922 C 29 68.119922 28.55 67.669922 28 67.669922 z M 28.09375 71.005859 C 27.800781 70.976797 27.499063 71.079062 27.289062 71.289062 L 27.169922 71.439453 C 27.129922 71.499453 27.100078 71.559141 27.080078 71.619141 C 27.050078 71.679141 27.029531 71.740781 27.019531 71.800781 C 27.009531 71.870781 27 71.93 27 72 C 27 72.13 27.030078 72.260859 27.080078 72.380859 C 27.130078 72.500859 27.199063 72.610938 27.289062 72.710938 C 27.479062 72.890937 27.74 73 28 73 C 28.13 73 28.260859 72.969922 28.380859 72.919922 C 28.500859 72.869922 28.610937 72.800938 28.710938 72.710938 C 28.800938 72.610937 28.869922 72.500859 28.919922 72.380859 C 28.969922 72.260859 29 72.13 29 72 C 29 71.93 28.990469 71.870781 28.980469 71.800781 C 28.970469 71.740781 28.949922 71.679141 28.919922 71.619141 C 28.899922 71.559141 28.870078 71.499453 28.830078 71.439453 L 28.710938 71.289062 C 28.610938 71.199063 28.500859 71.130078 28.380859 71.080078 C 28.288359 71.040078 28.191406 71.015547 28.09375 71.005859 z"/></svg>
          <p>Mmmh, the query seems to be broken now!</p>
        </div>
<div class="paragraph">
<p>If each field is <code>None</code> the query will work just fine, if one of them is at
least <code>Some(_)</code> the query won&#8217;t work because that will translate to <code>SELECT
* from users AND username = 'testname'</code>. A quick fix would be adding a <code>WHERE 1 =
1</code> clause at the beginning of the query</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">fn search_query(search: Search) -&gt; String {
    let mut query = QueryBuilder::new("SELECT * from users WHERE 1 = 1");

    if let Some(id) = search.id {
        query.push(" AND id = ");
        query.push_bind(search.id);
    }

    if let Some(username) = search.username {
        query.push(" AND username = ");
        query.push_bind(username);
    }

    if let Some(min_age) = search.min_age {
        query.push(" AND age &gt; ");
        query.push_bind(min_age);
    }

    if let Some(max_age) = search.max_age {
        query.push(" AND age &lt; ");
        query.push_bind(max_age);
    }

    query.build().sql().into()
}</code></pre>
</div>
</div>
        <div class="dialog" data-character="professor" aria-hidden="true">
          <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 80 80"><path d="M 22.912109 9 C 19.960756 9 16.358303 9.7221658 13.375 11.421875 C 10.391697 13.121584 8 15.945797 8 19.863281 C 8 23.912396 10.891498 29.481134 13.701172 34.283203 C 16.296271 38.718537 18.669822 42.066616 19.050781 42.601562 C 17.846834 43.88996 17.980826 45.617638 18.001953 46.048828 C 18.091373 47.889143 19.090293 49.025828 20.119141 50.009766 C 21.147988 50.993703 22.29396 52.003168 23.417969 54.146484 C 23.539513 54.378162 24.004424 55.269066 24.628906 56.189453 C 25.24048 57.090816 25.918145 58.062135 27 58.542969 L 27 62 A 1.0001 1.0001 0 1 0 29 62 L 29 58.896484 C 29.354145 58.891575 29.868383 58.864096 30.429688 58.5625 C 31.787375 57.836032 31.951153 56.396403 31.988281 56.148438 A 1.0001 1.0001 0 1 0 30.011719 55.851562 C 29.992849 55.977598 29.808687 56.625296 29.484375 56.798828 A 1.0001 1.0001 0 0 0 29.484375 56.800781 C 29.32398 56.886961 29.109495 56.908203 28.824219 56.908203 C 28.42168 56.908203 28.114506 56.845583 27.839844 56.730469 A 1.0001 1.0001 0 0 0 27.837891 56.730469 C 27.590274 56.626904 26.829471 55.871519 26.283203 55.066406 C 25.736935 54.261294 25.312909 53.452119 25.189453 53.216797 C 23.919462 50.795113 22.489106 49.508516 21.501953 48.564453 C 20.5148 47.62039 20.058624 47.197857 19.998047 45.951172 C 19.976167 45.504675 19.999847 44.424134 20.576172 43.884766 C 21.028522 43.465344 21.714425 43.416172 22.232422 43.408203 C 23.766212 43.383873 24.938922 44.373548 25.3125 44.726562 A 1.0002743 1.0002743 0 1 0 26.6875 43.273438 C 26.275119 42.883757 25.089926 41.848234 23.351562 41.519531 C 23.684054 41.310892 24.051927 41.094628 24.453125 40.890625 C 25.797547 40.207705 29.03107 39.122361 33.191406 38.230469 C 33.362903 39.669244 33.848762 40.995803 34.662109 41.990234 C 35.637853 43.183182 36.971947 44 38.431641 44 L 47 44 L 47 43.970703 C 47.165836 43.983009 47.331031 44 47.5 44 C 51.713438 44 55 40.110516 55 35.5 C 55 35.333552 54.986972 35.170279 54.978516 35.005859 C 55.073803 35.01009 56.315831 35.06789 58.087891 35.404297 C 59.928007 35.753621 62.199459 36.410722 63.869141 37.503906 C 65.629152 38.655646 66.920293 40.814446 67.544922 42.654297 C 67.845362 43.539246 67.971865 44.326947 67.982422 44.830078 C 67.8245 44.776948 67.775568 44.787958 67.476562 44.662109 C 66.877167 44.409819 66.00089 44.047905 64.669922 43.699219 C 63.396201 43.366066 62.05944 43.286417 60.980469 43.095703 C 59.901498 42.904989 59.211917 42.607238 58.945312 42.208984 C 58.739458 41.900705 58.727505 41.54692 58.730469 41.353516 L 58.730469 41.351562 C 58.747339 40.563466 59.456845 39.915621 59.625 39.78125 A 1.0004882 1.0004882 0 1 0 58.375 38.21875 C 58.105295 38.434268 56.766737 39.493484 56.730469 41.3125 A 1.0001 1.0001 0 0 0 56.730469 41.316406 C 56.729579 41.371576 56.669939 42.401916 57.283203 43.320312 A 1.0001 1.0001 0 0 0 57.283203 43.322266 C 58.065599 44.491012 59.390783 44.844917 60.632812 45.064453 C 61.206024 45.165771 61.701344 45.250974 62.261719 45.326172 A 1.0001 1.0001 0 0 0 62.365234 45.669922 C 62.365234 45.669922 64.86532 50.149397 70.380859 52.326172 C 70.066178 52.387382 69.842456 52.467172 69.421875 52.509766 C 67.775834 52.676464 65.347141 52.681641 61.869141 52.681641 C 60.410926 52.681641 59.025939 52.816151 57.824219 53.234375 C 56.622499 53.652599 55.548945 54.41984 55.070312 55.630859 A 1.0001 1.0001 0 1 0 56.929688 56.367188 C 57.147056 55.817206 57.616939 55.425526 58.480469 55.125 C 59.343999 54.824474 60.544355 54.681641 61.869141 54.681641 C 62.686115 54.681641 63.272154 54.67395 63.980469 54.671875 C 63.991735 54.744254 64 54.705608 64 55 C 64 55.437053 63.877671 55.664997 63.5625 55.982422 C 63.247329 56.299847 62.713826 56.63574 62.078125 56.990234 C 60.806723 57.699222 59.073341 58.493869 58.189453 60.285156 C 56.857013 62.98452 55.733424 64.124354 53.070312 65.472656 C 51.996454 66.016788 51.078649 66.994555 50.269531 68.115234 C 49.460414 69.235914 48.788791 70.501858 48.398438 71.6875 A 1.0001 1.0001 0 1 0 50.296875 72.3125 C 50.602522 71.384142 51.198696 70.24818 51.892578 69.287109 C 52.58646 68.326039 53.408468 67.542727 53.974609 67.255859 C 56.877498 65.786161 58.534815 64.106558 59.984375 61.169922 C 60.491487 60.142209 61.715387 59.48209 63.052734 58.736328 C 63.721408 58.363447 64.404296 57.972872 64.980469 57.392578 C 65.556642 56.812284 66 55.971947 66 55 C 66 54.863078 65.984562 54.780633 65.980469 54.662109 C 67.402198 54.642097 68.628908 54.600673 69.623047 54.5 C 71.455256 54.314448 72.78119 53.916388 73.710938 52.974609 A 1.0001 1.0001 0 0 0 73.199219 51.291016 C 68.046363 50.247259 65.904396 47.324169 65.021484 45.927734 C 65.635572 46.129085 66.311463 46.339874 66.701172 46.503906 C 67.290277 46.751872 67.803666 47 68.490234 47 C 69.069103 47 69.600788 46.524089 69.783203 46.121094 C 69.965618 45.718098 70 45.334232 70 44.916016 C 70 44.079583 69.804185 43.091793 69.4375 42.011719 C 68.93179 40.522147 68.087632 38.874396 66.859375 37.494141 C 67.586499 36.16648 68 34.528283 68 32.751953 C 68 30.707785 67.458226 28.840877 66.513672 27.423828 C 65.569118 26.00678 64.148 25 62.5 25 C 62.198677 25 61.907673 25.043753 61.623047 25.107422 C 60.608562 23.557292 58.772228 22.865658 57.222656 22.474609 C 55.724809 22.096614 54.696278 22.068044 54.404297 22.054688 C 53.369258 20.624763 51.713906 19.85094 50.222656 19.474609 C 48.853601 19.129115 48.072381 19.115957 47.646484 19.091797 C 46.698895 17.029339 44.859694 15.934026 43.265625 15.457031 C 41.714286 14.992823 40.640372 15.046519 40.339844 15.052734 C 40.200011 14.970291 37.831103 13.568401 34.46875 12.085938 C 31.009066 10.56056 26.67809 9 22.912109 9 z M 22.912109 11 C 26.103129 11 30.317543 12.43944 33.662109 13.914062 C 36.591489 15.205629 38.559962 16.332428 39.078125 16.632812 C 39.031274 16.749234 39 16.874603 39 17 C 39 17.26 39.109063 17.520937 39.289062 17.710938 L 39.439453 17.830078 C 39.499453 17.870078 39.559141 17.899922 39.619141 17.919922 C 39.679141 17.949922 39.740781 17.970469 39.800781 17.980469 C 39.870781 17.990469 39.94 18 40 18 C 40.26 18 40.520938 17.890937 40.710938 17.710938 C 40.879411 17.533105 40.977202 17.29191 40.990234 17.048828 C 41.462183 17.094999 42.05333 17.182117 42.691406 17.373047 C 44.090367 17.791659 45.481694 18.583147 46.050781 20.3125 A 1.0001 1.0001 0 0 0 46.136719 20.503906 C 46.115766 20.542254 46.092861 20.580793 46.080078 20.619141 C 46.050078 20.679141 46.029531 20.740547 46.019531 20.810547 C 46.009531 20.870547 46 20.94 46 21 C 46 21.26 46.109063 21.520937 46.289062 21.710938 C 46.479062 21.890938 46.74 22 47 22 C 47.13 22 47.260859 21.979922 47.380859 21.919922 C 47.510859 21.869922 47.620937 21.800937 47.710938 21.710938 C 47.86877 21.544337 47.957243 21.320799 47.980469 21.09375 C 48.470812 21.153682 49.078329 21.246548 49.734375 21.412109 C 51.191603 21.779855 52.640369 22.505079 53.103516 23.443359 A 1.0001 1.0001 0 0 0 53.966797 24 C 53.966797 24 55.277147 24.04436 56.734375 24.412109 C 57.726571 24.6625 58.706905 25.080934 59.382812 25.617188 C 58.824296 25.86258 58.539744 25.98812 57.689453 26.361328 C 56.959362 26.681778 56.226486 27.003639 55.669922 27.248047 C 55.113358 27.492455 54.829216 27.614987 54.699219 27.673828 L 54.695312 27.675781 L 54.691406 27.677734 C 53.992541 28.001013 53.204998 28.546857 52.652344 29.335938 C 51.318755 27.899257 49.518306 27 47.5 27 C 46.088192 27 44.784831 27.445042 43.669922 28.199219 L 43.619141 28.074219 C 43.619141 28.074219 41.839903 28.806549 40.052734 29.541016 C 39.15915 29.908249 38.263012 30.277439 37.587891 30.554688 C 37.25033 30.69331 36.967179 30.808775 36.767578 30.890625 C 36.667778 30.931555 36.589701 30.964028 36.535156 30.986328 C 36.507886 30.997478 36.486666 31.006019 36.472656 31.011719 L 36.46875 31.013672 C 34.671404 31.696821 33.47822 33.590501 33.173828 36.177734 C 28.67056 37.106859 25.2554 38.241503 23.546875 39.109375 C 21.798376 39.998459 20.656347 40.983978 20.472656 41.144531 C 19.868962 40.289449 17.808667 37.342736 15.427734 33.273438 C 12.672408 28.564256 10 22.859167 10 19.863281 C 10 16.735766 11.783287 14.629244 14.365234 13.158203 C 16.947182 11.687162 20.298462 11 22.912109 11 z M 19 16 A 2 2 0 0 0 19 20 A 2 2 0 0 0 19 16 z M 35.697266 16.384766 C 35.567734 16.369922 35.433281 16.379922 35.300781 16.419922 C 34.770781 16.579922 34.470859 17.139922 34.630859 17.669922 C 34.760859 18.099922 35.159844 18.380859 35.589844 18.380859 C 35.679844 18.380859 35.779141 18.369844 35.869141 18.339844 C 36.399141 18.179844 36.700781 17.619609 36.550781 17.099609 C 36.430781 16.702109 36.085859 16.429297 35.697266 16.384766 z M 31.787109 18.501953 C 31.532109 18.481953 31.270547 18.560234 31.060547 18.740234 C 30.640547 19.100234 30.589219 19.740156 30.949219 20.160156 C 31.149219 20.390156 31.430937 20.5 31.710938 20.5 C 31.940937 20.5 32.169375 20.419766 32.359375 20.259766 C 32.779375 19.899766 32.830703 19.269609 32.470703 18.849609 C 32.290703 18.639609 32.042109 18.521953 31.787109 18.501953 z M 43.460938 20.029297 C 42.900937 20.079297 42.489297 20.549609 42.529297 21.099609 C 42.569297 21.629609 42.999297 22.029297 43.529297 22.029297 L 43.599609 22.029297 C 44.149609 21.989297 44.559531 21.510938 44.519531 20.960938 C 44.479531 20.410937 44.010937 19.999297 43.460938 20.029297 z M 40.207031 20.570312 C 40.0775 20.557344 39.943047 20.569375 39.810547 20.609375 C 39.280547 20.779375 38.990391 21.339375 39.150391 21.859375 C 39.280391 22.289375 39.679375 22.570312 40.109375 22.570312 C 40.199375 22.570312 40.300391 22.549531 40.400391 22.519531 C 40.930391 22.359531 41.220547 21.799531 41.060547 21.269531 C 40.940547 20.872031 40.595625 20.609219 40.207031 20.570312 z M 28.90625 22.005859 C 28.808594 22.015547 28.711641 22.040078 28.619141 22.080078 C 28.499141 22.130078 28.389063 22.199063 28.289062 22.289062 C 28.109062 22.479062 28 22.74 28 23 C 28 23.13 28.030078 23.260859 28.080078 23.380859 C 28.130078 23.510859 28.199063 23.610937 28.289062 23.710938 C 28.389062 23.800938 28.499141 23.869922 28.619141 23.919922 C 28.739141 23.969922 28.87 24 29 24 C 29.26 24 29.520938 23.890938 29.710938 23.710938 C 29.800938 23.610938 29.869922 23.510859 29.919922 23.380859 C 29.979922 23.260859 30 23.13 30 23 C 30 22.74 29.890937 22.479063 29.710938 22.289062 C 29.500937 22.079062 29.199219 21.976797 28.90625 22.005859 z M 37.169922 22.267578 C 36.914922 22.252578 36.654219 22.334531 36.449219 22.519531 C 36.039219 22.889531 35.999141 23.519453 36.369141 23.939453 C 36.569141 24.159453 36.839141 24.269531 37.119141 24.269531 C 37.349141 24.269531 37.589297 24.179766 37.779297 24.009766 C 38.189297 23.639766 38.229375 23.009609 37.859375 22.599609 C 37.674375 22.394609 37.424922 22.282578 37.169922 22.267578 z M 18.5 24 A 1.5 1.5 0 0 0 18.5 27 A 1.5 1.5 0 0 0 18.5 24 z M 34.800781 25.019531 C 34.740781 25.029531 34.679141 25.050078 34.619141 25.080078 C 34.559141 25.100078 34.499453 25.129922 34.439453 25.169922 C 34.389453 25.199922 34.339062 25.249063 34.289062 25.289062 C 34.109062 25.479062 34 25.74 34 26 C 34 26.26 34.109063 26.520937 34.289062 26.710938 C 34.339063 26.750938 34.389453 26.800078 34.439453 26.830078 C 34.499453 26.870078 34.559141 26.899922 34.619141 26.919922 C 34.679141 26.949922 34.740781 26.970469 34.800781 26.980469 C 34.870781 26.990469 34.94 27 35 27 C 35.06 27 35.129219 26.990469 35.199219 26.980469 C 35.259219 26.970469 35.320859 26.949922 35.380859 26.919922 C 35.440859 26.899922 35.500547 26.870078 35.560547 26.830078 C 35.610547 26.800078 35.660938 26.750937 35.710938 26.710938 C 35.750938 26.660938 35.800078 26.610547 35.830078 26.560547 C 35.870078 26.500547 35.899922 26.440859 35.919922 26.380859 C 35.949922 26.320859 35.970469 26.259219 35.980469 26.199219 C 35.990469 26.129219 36 26.06 36 26 C 36 25.94 35.990469 25.870781 35.980469 25.800781 C 35.970469 25.740781 35.949922 25.679141 35.919922 25.619141 C 35.899922 25.559141 35.870078 25.499453 35.830078 25.439453 C 35.800078 25.389453 35.750937 25.339063 35.710938 25.289062 C 35.660937 25.249062 35.610547 25.199922 35.560547 25.169922 C 35.500547 25.129922 35.440859 25.100078 35.380859 25.080078 C 35.320859 25.050078 35.259219 25.029531 35.199219 25.019531 C 35.069219 24.989531 34.930781 24.989531 34.800781 25.019531 z M 62.5 27 C 63.337 27 64.165413 27.504798 64.849609 28.53125 C 65.533805 29.557702 66 31.068121 66 32.751953 C 66 34.00733 65.737519 35.160765 65.320312 36.09375 C 65.200872 36.006694 65.089403 35.911589 64.964844 35.830078 C 63.115425 34.619215 60.926832 33.96549 59.054688 33.570312 C 59.029888 33.300341 59 33.032794 59 32.751953 C 59 31.068121 59.466195 29.557702 60.150391 28.53125 C 60.834587 27.504798 61.663 27 62.5 27 z M 57.916016 28.445312 C 57.328962 29.690383 57 31.166369 57 32.751953 C 57 32.906001 57.009389 33.055718 57.015625 33.207031 C 55.772365 33.037808 54.913216 33 54.912109 33 A 1.0001 1.0001 0 0 0 54.673828 33.027344 C 54.498232 32.374737 54.249591 31.758461 53.947266 31.179688 L 53.982422 31.185547 C 54.0947 30.58994 54.973824 29.750039 55.53125 29.492188 C 55.45871 29.525108 55.922938 29.320384 56.474609 29.078125 C 56.871683 28.903755 57.404225 28.670009 57.916016 28.445312 z M 47.5 29 C 50.466269 29 53 31.826245 53 35.5 C 53 39.173755 50.466269 42 47.5 42 C 44.533731 42 42 39.173755 42 35.5 C 42 31.826245 44.533731 29 47.5 29 z M 40.976562 31.322266 C 40.355891 32.56516 40 33.991941 40 35.5 C 40 38.077287 41.028118 40.426794 42.669922 42 L 38.431641 42 C 37.756334 42 36.895938 41.562119 36.210938 40.724609 C 35.525937 39.8871 35.042969 38.697103 35.042969 37.390625 C 35.042969 34.679516 36.061342 33.310422 37.177734 32.884766 C 37.177734 32.884766 37.179688 32.882812 37.179688 32.882812 C 37.226627 32.865252 37.212823 32.868871 37.226562 32.863281 C 37.242672 32.856681 37.265109 32.849281 37.292969 32.837891 C 37.348679 32.815111 37.427234 32.781294 37.527344 32.740234 C 37.727564 32.658134 38.009936 32.542986 38.347656 32.404297 C 39.023097 32.126919 39.918835 31.757892 40.8125 31.390625 C 40.894854 31.35678 40.894409 31.356021 40.976562 31.322266 z M 52.027344 45.986328 A 1.0001 1.0001 0 0 0 51.001953 47.056641 C 51.12024 49.123838 50.810157 50.953649 50.085938 52.595703 A 1.0001 1.0001 0 0 0 50.083984 52.597656 C 49.747402 53.362036 49.483444 53.693798 49.085938 54.595703 C 48.224157 56.549649 47.86824 58.719838 48.001953 61.056641 A 1.0001 1.0001 0 1 0 49.998047 60.943359 C 49.87976 58.876162 50.189842 57.046351 50.914062 55.404297 A 1.0001 1.0001 0 0 0 50.916016 55.402344 C 51.249338 54.645539 51.513827 54.312598 51.914062 53.404297 C 52.775843 51.450351 53.13176 49.280162 52.998047 46.943359 A 1.0001 1.0001 0 0 0 52.027344 45.986328 z M 27.988281 47.988281 A 1.0001 1.0001 0 0 0 27.167969 49.554688 C 27.834713 50.554304 28.500713 51.554304 29.167969 52.554688 A 1.0001 1.0001 0 1 0 30.832031 51.445312 C 30.165287 50.445696 29.499287 49.445696 28.832031 48.445312 A 1.0001 1.0001 0 0 0 27.988281 47.988281 z M 28 64.330078 C 27.45 64.330078 27 64.780078 27 65.330078 C 27 65.890078 27.45 66.330078 28 66.330078 C 28.55 66.330078 29 65.890078 29 65.330078 C 29 64.780078 28.55 64.330078 28 64.330078 z M 28 67.669922 C 27.45 67.669922 27 68.119922 27 68.669922 C 27 69.219922 27.45 69.669922 28 69.669922 C 28.55 69.669922 29 69.219922 29 68.669922 C 29 68.119922 28.55 67.669922 28 67.669922 z M 28.09375 71.005859 C 27.800781 70.976797 27.499063 71.079062 27.289062 71.289062 L 27.169922 71.439453 C 27.129922 71.499453 27.100078 71.559141 27.080078 71.619141 C 27.050078 71.679141 27.029531 71.740781 27.019531 71.800781 C 27.009531 71.870781 27 71.93 27 72 C 27 72.13 27.030078 72.260859 27.080078 72.380859 C 27.130078 72.500859 27.199063 72.610938 27.289062 72.710938 C 27.479062 72.890937 27.74 73 28 73 C 28.13 73 28.260859 72.969922 28.380859 72.919922 C 28.500859 72.869922 28.610937 72.800938 28.710938 72.710938 C 28.800938 72.610937 28.869922 72.500859 28.919922 72.380859 C 28.969922 72.260859 29 72.13 29 72 C 29 71.93 28.990469 71.870781 28.980469 71.800781 C 28.970469 71.740781 28.949922 71.679141 28.919922 71.619141 C 28.899922 71.559141 28.870078 71.499453 28.830078 71.439453 L 28.710938 71.289062 C 28.610938 71.199063 28.500859 71.130078 28.380859 71.080078 C 28.288359 71.040078 28.191406 71.015547 28.09375 71.005859 z"/></svg>
          <p>Ahhh it's working again now!</p>
        </div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>Yep, that was an easy fix, but we can do even better</p>
        </div>
<div class="paragraph">
<p>The <code>sqlx</code> crate is capable of handling <code>Option&lt;T&gt;</code> types easily: if the value
is <code>Some(_)</code> that will be used in the binding, otherwise sqlx will bind <code>NULL</code>
for <code>None</code> values. With that in mind we can use SQL to only apply those <code>WHERE</code>
clauses if the value <code>IS NOT NULL</code>.</p>
</div>
<div class="paragraph">
<p>The function above becomes</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">async fn search_query(search: Search) -&gt; String {
    let query = r"
    SELECT * from users
    WHERE id = $1
        AND ($2 IS NULL OR username = $2)
        AND ($3 IS NULL or age &gt; $3)
        AND ($4 IS NULL or age &lt; $4)
    ".into()
}</code></pre>
</div>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>Note that you need to use check for IS NULL to avoid running the filtering when the value is None</p>
        </div>
<div class="paragraph">
<p>This approach does not let you push bindings one by one as the previous method,
but you don&#8217;t actually need it here, you can bind values all at once later.
Let&#8217;s query the data directly</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">async fn search_query(search: Search, pg: &amp;PgPool) {
    let query = sqlx::query(r"
    SELECT * FROM users
    WHERE id = $1
        AND ($2 IS NULL OR username = $2)
        AND ($3 IS NULL OR age &gt; $3)
        AND ($4 IS NULL OR age &lt; $4)
    ")
    .bind(search.id)
    .bind(search.username)
    .bind(search.min_age)
    .bind(search.max_age)
    .fetch_all(pg)
    .await
    .expect("failed querying users");
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>This is what I usually prefer, it looks nicer and I don&#8217;t have to write more
Rust logic that I&#8217;d have to test later. The dynamically built query shown before
can end up being 16 different queries, on the other hand you only have one query
if you exclusively use sql.</p>
</div>
<div class="paragraph">
<p>Another reason why I prefer to do queries this way is that I can copy and paste
the statement in Datagrip and test it directly in the database, mimicking what
<code>sqlx</code> will end up doing.</p>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>In the following examples I am using Postgres syntax, it may be different for other engines like Sqlite or MySql</p>
        </div>
<div class="paragraph">
<p>By now you should have a better idea of how you can work with sql to reduce the
Rust logic that&#8217;s involved in your queries, but I&#8217;d like to give some other
common examples and functions you can work with.</p>
</div>
<div class="paragraph">
<p>A common type that I encounter pretty frequently is a <code>Vec&lt;T&gt;</code>. Most of the
times I do not want to filter at all if <code>vec.is_empty()</code>. To make this a bit
more complicated let&#8217;s consider the scenario where I have an <code>Option&lt;Vec&lt;T&gt;&gt;</code>
and I only want to apply the filter if <code>!vec.is_empty()</code>.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">async fn filter(nicknames: Option&lt;Vec&lt;i64&gt;&gt;, pg: &amp;PgPool) {
    sqlx::query(
        r"SELECT *
        FROM users
        WHERE 1 = 1
            AND ($1 IS NULL
                 OR CARDINALITY($1::integer[]) = 0
                 OR nickname = $1
            )"
    )
    .bind(nicknames)
    .fetch_all(pg)
    .await
    .unwrap();
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Let&#8217;s break it down:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>$1 IS NULL</code> is satisfied if <code>nicknames.is_none()</code> and won&#8217;t apply the filter</p>
</li>
<li>
<p><code>CARDINALITY($1::integer[])</code> stops the filtering if <code>nickname.is_some() &amp;&amp;
nickname.unwrap().len() = 0</code></p>
</li>
<li>
<p>Finally, if the vector is not <code>None</code> and its length is greater than one then
<code>nickname = $1</code> will filter all the users that have <code>nickname</code> as nickname</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Let&#8217;s move on to another similar scenario, this time you have a vector
represented as string with comma separated values: you may have an endpoint that
accepts a query parameter with multiple values separated by a comma (e.g
<code>?ids=11,22,33,44</code>). My naive-self in the past used to create a fancy
custom deserializer function that transformed <code>11,22,33,44</code> from a <code>String</code> into
a <code>Vec&lt;i64&gt;</code> and that is useless work that could have easily been handled by the
database.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">async fn filter(ids: String, pg: &amp;PgPool) {
    sqlx::query(
        r"SELECT *
        FROM users
        WHERE
            id IN (ARRAY_REMOVE(STRING_TO_ARRAY($1, ','), ''))"
    )
    .bind(ids)
    .fetch_all(pg)
    .await
    .unwrap();
}</code></pre>
</div>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>The syntax above works for Postgres but may be different for other database servers. I'm sure such a basic feature is available on most of them and you can use it.</p>
        </div>
<div class="paragraph">
<p><code>(ARRAY_REMOVE(STRING_TO_ARRAY($1, ','), '')</code> creates an array of
ids by splitting comma separated values and also removes empty values in
case someone decides to mess with your backend and tries to pass
<code>?ids=11,,,</code>.</p>
</div>
<div class="paragraph">
<p>The next feature I&#8217;d like to explore is probably the de-facto API must-have:
pagination. Pagination basically lets your client say "give me page 2 with a
maximum of 10 items in it". You can model that filter with the following struct</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">struct Filter {
    /// current page
    pub page: Option&lt;i64&gt;,
    /// number of items per page
    pub limit_per_page: Option&lt;i64&gt;
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>I&#8217;ve marked them as <code>Option&lt;i64&gt;</code> because clients are rude and want you to do
the guessing. Most of them won&#8217;t ask for a specific page but they really want
the fist one. Also, if the client is lazy and does not provide a
<code>limit_per_page</code> you should fallback to a good default value to not overload
your Postgres instance.</p>
</div>
<div class="paragraph">
<p>Pagination is tricky because a user could pass a <code>?page=-100</code> and negative pages
do not exist. You could have avoided that by using an <code>Option&lt;u64&gt;</code> but then
you&#8217;d have to cast that value to an <code>i64</code> whenever you want to bind that to a
query (in Postgres at least). To make things worse, an hacker could request a
<code>limit_per_page=1000000000</code> to make your server crash and wake you up while
you&#8217;re sleeping because your app is dead.</p>
</div>
<div class="paragraph">
<p>My less sql-centric self would have implemented a filter validation to check for its correctness</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">impl Filter {
    fn validate(&amp;self) -&gt; Result&lt;(), String&gt; {
        match self.page {
            Some(page) if page &lt; 0 =&gt; {
                return Err("page can't be negative")
            }
            _ =&gt; {}
        }

        match self.limit_per_page {
            Some(limit) if limit &lt; 10 &amp;&amp; limit &gt; 100 {
                return Err("limit must be between 10 and 100")
            }
            _ =&gt; {}
        }

        Ok(())
    }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>But guess what&#8230;&#8203; SQL has the solution for you</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">async fn filter(filter: Filter, pg: &amp;PgPool) {
    sqlx::query(
    r"SELECT *
    FROM users
    LIMIT
        CASE WHEN $2 &gt; 0 THEN $2 END
    OFFSET
        CASE
            WHEN $2 BETWEEN 0 AND 100 AND $1 &gt; 0
                THEN (($1 - 1) * $2)
            ELSE
                50
        END"
    )
    // if page is not provided, fallback to 0
    .bind(filter.page.unwrap_or(0))
    // if limit is not provided, fallback to 50
    .bind(filter.limit_per_page.unwrap_or(50))
    .fetch_all(pg)
    .await
    .unwrap();
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>The <code>LIMIT</code> statement is only applied if the passed value is greater than zero,
on the other hand the <code>OFFSET</code> is applied only if the passed
<code>limit_per_page</code> is between zero and a hundred and the <code>page</code> is not
negative, all other cases are defaulted to <code>OFFSET 50</code>.</p>
</div>
<div class="paragraph">
<p>Lastly, I&#8217;d like to cover an <code>UPDATE</code> statement which I feel is also quite
common in APIs out there. Let&#8217;s say we have an <code>UpdateForm</code> struct that models
our HTML form.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">struct UpdateForm {
    id: Uuid
    name: Option&lt;String&gt;
    surname: Option&lt;String&gt;
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>This form takes the user unique id and, optionally, its name and surname. I want
to provide a way for my client to only update the value that it passes to my
backend with a <code>Some(_)</code> value. Seems like we can do just that with the use of <code>COALESCE</code></p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">async fn update_user(UpdateForm { id, name, surname }: UpdateForm, pg: &amp;PgPool) {
    sqlx::query(
        r"UPDATE user
        SET name = COALESCE($2, name),
            surname = COALESCE($3, surname)
        WHERE id = $1"
    )
    .bind(id)
    .bind(name)
    .bind(surname)
    .fetch_one(pg)
    .await
    .unwrap();
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>The query above uses <code>COALESCE</code> to only update those values that are
<code>.is_some()</code>, the others will remain the same and no update will take place.</p>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>Neat! be mindful, `COALESCE` will return the first non-null value in its statement, so order matters!</p>
        </div>
<div class="paragraph">
<p>As you can see we&#8217;ve got a lot done by employing sql alone, you can get smart
with it and avoid writing over-complicated Rust logic in your queries, isn&#8217;t
that a neater and better solution overall?</p>
</div>]]></content><author><name>Mattia Righetti</name></author><summary type="html"><![CDATA[In a previous [0] post I&#8217;ve discussed why I ditched sea-query and why sql is almost always the best way to not re-learn something new from the beginning that will inevitably end up slowing you down or simply not working at all in the long run.]]></summary></entry><entry><title type="html">Back to Good Old SQL</title><link href="https://mattrighetti.com/2025/01/14/ditching-sea-query.html" rel="alternate" type="text/html" title="Back to Good Old SQL" /><published>2025-01-14T00:00:00+00:00</published><updated>2025-01-14T00:00:00+00:00</updated><id>https://mattrighetti.com/2025/01/14/ditching-sea-query</id><content type="html" xml:base="https://mattrighetti.com/2025/01/14/ditching-sea-query.html"><![CDATA[<div class="paragraph">
<p>Query builders and ORMs have been around for as long as I can remember, each
serving different purposes. ORMs, for example, allow you to focus on application
logic without worrying about SQL or database intricacies. However, this
convenience comes at a cost—using an ORM almost always means losing control over
what’s happening in your database. That’s a trade-off I’ve never been willing to
make in any of my projects.</p>
</div>
<div class="paragraph">
<p>Query builders on the other hand just&#8230;&#8203; build queries.</p>
</div>
<div class="paragraph">
<p>At the time it seemed like the best of both worlds because you&#8217;re still in
control of the query and you&#8217;re getting a little hand by the framework to
generate your query.</p>
</div>
<div class="paragraph">
<p>During GSoC 2023 I got really interested in <code>sea-query</code>, which is a well known
query builder crate that also integrates well in their <code>sea-orm</code> crate. I really
liked the idea of having strongly typed database queries and having a database
model representation, so I gave it a shot.</p>
</div>
<div class="paragraph">
<p>With sea-query you can have all your database tables living alongside your code
like the following</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">/// This is a strongly typed version of your `environments` database table
/// with its columns
#[derive(Debug, sea_query::Iden)]
pub enum Environments {
    Table,
    Env,
    Key,
    Value,
    CreatedAt,
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>with that you can then create your queries like the following</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">/// inserts `key` and `value` to environment `env`
pub async fn insert(&amp;self, env: &amp;str, key: &amp;str, var: &amp;str) -&gt; io::Result&lt;()&gt; {
    let (sql, values) = Query::insert()
        .into_table(Environments::Table)
        .columns([Environments::Env, Environments::Key, Environments::Value])
        .values([env.into(), Func::upper(key).into(), var.into()])
        .unwrap()
        .build_sqlx(SqliteQueryBuilder);

    sqlx::query_with(&amp;sql, values)
        .execute(&amp;self.db)
        .await
        .map_err(|e| std_err!("db error: {}", e))?;

    Ok(())
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>A year later I am now going back writing goold old SQL queries and I&#8217;m surprised
it took me that long. "Why?" you may wonder - well, mostly for the following
reasons:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>You don&#8217;t get 100% of the SQL expressiveness</p>
</li>
<li>
<p>It just does not work for complex queries, or looses its purpose</p>
</li>
<li>
<p>Documentation covers basic examples and could be better</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Let&#8217;s say that you need an uncommon sql statement that only Postgres offers,
like <code>txid_current_snapshot()</code> - you simply won&#8217;t find it in <code>sea-query</code>.</p>
</div>
<div class="paragraph">
<p>Even though mine could seem like a very uncommon SQL statement that you&#8217;ll
probably never need (you could argue that that is also the reason why it&#8217;s not
being developed), you will for certain end up in a scenario where
something is not available in the crate.</p>
</div>
<div class="paragraph">
<p>Not too bad, as <code>sea-query</code> provides a solution for this—you can still write raw
SQL using <code>Expr::cust("txid_current_snapshot()")</code> when needed. While this is
useful for cases where the query builder lacks a specific feature, it comes at
the cost of losing strong typing—the very reason you chose to use the query
builder in the first place.</p>
</div>
<div class="paragraph">
<p>What if you&#8217;re not sure how to do something with <code>sea-query</code>? I’m no genius, but
I&#8217;ve lost count of the times I&#8217;ve searched for a solution, only to come up
empty—or worse, discovered that it&#8217;s simply not possible and had to resort to
hacks to make it work.</p>
</div>
<div class="paragraph">
<p>Documentation does not help at all here, it basically covers the simplest
examples and it won&#8217;t get you that far when you have to write complicated
queries and you need to understand how to glue all the piecies together.</p>
</div>
<div class="paragraph">
<p>Eventually, this starts to feel like learning a new SQL dialect on top of
the real SQL: you think of the query you need, check the documentation, and
come up empty-handed. Then you turn to Google, searching Stack Overflow or
GitHub Issues, hoping someone has asked the same question. If you&#8217;re lucky,
you&#8217;ll find the answer—after wasting countless minutes looking for it.</p>
</div>
<div class="paragraph">
<p>This is why I&#8217;ve ditched the idea of query builders entirely and started writing
raw SQL queries again. It&#8217;s not perfect, but it&#8217;s probably the best we can do.</p>
</div>
<div class="paragraph">
<p>I&#8217;ll remind myself not to fall into that trap again when the next shiny,
promising SQL alternative comes along.</p>
</div>]]></content><author><name>Mattia Righetti</name></author><summary type="html"><![CDATA[Query builders and ORMs have been around for as long as I can remember, each serving different purposes. ORMs, for example, allow you to focus on application logic without worrying about SQL or database intricacies. However, this convenience comes at a cost—using an ORM almost always means losing control over what’s happening in your database. That’s a trade-off I’ve never been willing to make in any of my projects.]]></summary></entry><entry><title type="html">Why Askama?</title><link href="https://mattrighetti.com/2024/12/06/why-askama.html" rel="alternate" type="text/html" title="Why Askama?" /><published>2024-12-06T00:00:00+00:00</published><updated>2024-12-06T00:00:00+00:00</updated><id>https://mattrighetti.com/2024/12/06/why-askama</id><content type="html" xml:base="https://mattrighetti.com/2024/12/06/why-askama.html"><![CDATA[<div class="paragraph">
<p>Askama implements a template rendering engine based on Jinja. It generates Rust
code from your templates at compile time based on a user-defined struct to hold
the template&#8217;s context.</p>
</div>
<div class="paragraph">
<p>This was very appealing from the very first moment. You can keep all your
templates in your root folder, for example <code>templates/home.jinja</code>.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-jinja" data-lang="jinja">{% block content %}
    &lt;h1&gt;Hello, {{email}}&lt;/h1&gt;
    {% if let Some(name) = name %}
        &lt;h1&gt;Hello, {{name}}&lt;/h1&gt;
    {% else %}
    {% endif %}
{% endblock %}</code></pre>
</div>
</div>
<div class="paragraph">
<p>And then you can use your template with a struct in your project</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">use askama::Template;

#[derive(Template)]
#[template(path="home.jinja")]
pub struct HomeTemplate {
    // some users might not have one upon sign-up
    name: Option&lt;String&gt;,
    // email is required
    email: String,
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>What&#8217;s really cool is that you can even use rust syntax inside of templates</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-jinja" data-lang="jinja">{% block content %}
    {% if let Some(name) = name %}
        &lt;h1&gt;Hello, {{name}}&lt;/h1&gt;
    {% else %}
        &lt;h1&gt;Hello, {{email}}&lt;/h1&gt;
    {% endif %}
{% endblock %}</code></pre>
</div>
</div>
<div class="paragraph">
<p>I love rust <code>match</code> arms, you can also use those in Askama templates. In Ulry I
have to render different components depending on the value of a certain enum</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">#[derive(Debug, PartialEq, Eq)]
pub enum LinkNote {
    /// note is a valid HN link
    HN(String),
    /// note is a valid link
    Link(String),
    /// general note
    General(String),
}</code></pre>
</div>
</div>
<div class="paragraph">
<p><code>LinkNote</code> models three different kind of notes that you can attach to a link,
here&#8217;s how I can use this in my template.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-jinja" data-lang="jinja">{% if let Some(note) = link.note %}
    {% match note %}
        {% when LinkNote::HN with (url) %}
            &lt;a href="{{url}}"&gt;&lt;p&gt;hn&lt;/p&gt;&lt;/a&gt;
        {% when LinkNote::Link with (url) %}
            &lt;a href="{{url}}"&gt;&lt;p&gt;link&lt;/p&gt;&lt;/a&gt;
        {% else %}
            &lt;p&gt;note&lt;/p&gt;
    {% endmatch %}
{% endif %}</code></pre>
</div>
</div>
<div class="paragraph">
<p>These are just some simple examples of what you can do with Askama templates, if
you want to find out more they have a pretty good <a href="https://rinja-rs.github.io/askama/template_syntax.html">book</a> that you can
take a look at.</p>
</div>]]></content><author><name>Mattia Righetti</name></author><summary type="html"><![CDATA[Askama implements a template rendering engine based on Jinja. It generates Rust code from your templates at compile time based on a user-defined struct to hold the template&#8217;s context.]]></summary></entry><entry><title type="html">Axum, Askama, HTMX</title><link href="https://mattrighetti.com/2024/12/05/axum-askama-htmx.html" rel="alternate" type="text/html" title="Axum, Askama, HTMX" /><published>2024-12-05T00:00:00+00:00</published><updated>2024-12-05T00:00:00+00:00</updated><id>https://mattrighetti.com/2024/12/05/axum-askama-htmx</id><content type="html" xml:base="https://mattrighetti.com/2024/12/05/axum-askama-htmx.html"><![CDATA[<div class="paragraph">
<p>Lately I&#8217;ve been working on <a href="https://ulry.app">Ulry</a>, after tiressly trying to make Next.js
work for me, I realised it wasn&#8217;t the right fit (<a href="https://blog.erodriguez.de/dependency-management-fatigue-or-why-i-forever-ditched-react-for-go-htmx-templ/">I&#8217;m not alone</a>).  I decided
to make a U-turn and go all in with Rust and SSR.</p>
</div>
<div class="paragraph">
<p>I already have my APIs running on <code>axum</code>, I know and like HTMX a bit as I&#8217;ve
used it in some other minor project using Go [<a href="https://mettag.ulry.app">1</a>]. Time to try something in
new with Rust.</p>
</div>
<div class="paragraph">
<p>I only need something to render the pages for the webiste, axum bare
<code>Html&lt;&amp;str&gt;</code> is not going to cut it.</p>
</div>
<div class="paragraph">
<p>I stumbled upon <a href="https://github.com/Keats/tera">Tera</a> and <a href="https://github.com/djc/askama">Askama</a>. The two are very similar templating
frameworks - the only difference is that Askama compiles templates at compile
time, Tera does not. This, in theory, makes Askama a lot faster. But I did not
chose if for that reason alone, I&#8217;d argue that the selling point for me was that
with Askama you don&#8217;t have to have templates to be parsed at runtime, you just
have your binary file and everything is packaged in there.</p>
</div>
<div class="paragraph">
<p>Off to a good start.</p>
</div>]]></content><author><name>Mattia Righetti</name></author><summary type="html"><![CDATA[Lately I&#8217;ve been working on Ulry, after tiressly trying to make Next.js work for me, I realised it wasn&#8217;t the right fit (I&#8217;m not alone). I decided to make a U-turn and go all in with Rust and SSR.]]></summary></entry><entry><title type="html">Files and the OS</title><link href="https://mattrighetti.com/2024/06/01/os-and-files.html" rel="alternate" type="text/html" title="Files and the OS" /><published>2024-06-01T00:00:00+00:00</published><updated>2024-06-01T00:00:00+00:00</updated><id>https://mattrighetti.com/2024/06/01/os-and-files</id><content type="html" xml:base="https://mattrighetti.com/2024/06/01/os-and-files.html"><![CDATA[<div class="paragraph">
<p>The other day I working on a side project of mine and I stumbled upon a bug in
my code that I couldn&#8217;t immediately figure out how to solve.</p>
</div>
<div class="paragraph">
<p>The task is pretty simple:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Create a file with some text in it</p>
</li>
<li>
<p>Spawn whatever editor you want to work with the file</p>
</li>
<li>
<p>Read all the file and return its content</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>This is a very similar process that <code>git</code> runs whenever you want to commit
something</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">pub fn spawn_with(data: &amp;[u8]) -&gt; Result&lt;Vec&lt;u8&gt;&gt; {
    let editor = editor_cmd();
    let pb = env::current_dir()?.join(".ENVELOPE_EDITMSG");

    let mut file = OpenOptions::new()
        .write(true)
        .read(true)
        .create(true)
        .truncate(true)
        .open(&amp;pb)?;

    file.write_all(data)
	    .map_err(|e| std_err!("error writing data to file: {}", e))?;
    file.write(b"\n\n# Comment variables to remove them")?;

    let args = &amp;[pb.to_str().unwrap()];
    let cmd = ChildProcess::new(&amp;editor, args, &amp;[]);

    // this command is blocking, so it's going to wait
    // for the child process to exit before proceeding
    cmd.run_shell_command()
        .map_err(|e| std_err!("error running child process: {}", e))?;

    let mut buf = Vec::new();
    file.read_to_end(&amp;mut buf).unwrap();

    std::fs::remove_file(pb)?;
    Ok(buf)
}</code></pre>
</div>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>Can you spot the issue? Well, I couldn't at first and the lack of tests sure didn't help.</p>
        </div>
<div class="paragraph">
<p>Let&#8217;s print some stuff and see where we end up</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">pub fn spawn_with(data: &amp;[u8]) -&gt; Result&lt;Vec&lt;u8&gt;&gt; {
    // ...

    let mut file = OpenOptions::new()
        .write(true)
        .read(true)
        .create(true)
        .truncate(true)
        .open(&amp;pb)?;

    println!("{:?}", file);

    // ...

    Ok(buf)
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-output" data-lang="output">File { fd: 7, path: "/home/matt/Developer/envelope/.ENVELOPE_EDITMSG", read: true, write: true }</code></pre>
</div>
</div>
<div class="paragraph">
<p>My editor opens the file correctly and I can see all the data that I wanted in
it. That part of the process looks okay so far. I am still getting an empty
buffer out of this function though, why is that?</p>
</div>
<div class="paragraph">
<p>The answer lies in how Unix systems (in my case) handle files.</p>
</div>
<div class="paragraph">
<p>Unix systems have a system-wide <strong>open file table</strong> that keeps track of numerous
info about files opened by processes, like: current offset, underlying inode, if
the file is either readable, writeable or both etc.</p>
</div>
<div class="paragraph">
<p>Whenever a process opens a file, the operating system will return a <strong>file
descriptor</strong> to the process, which is going to later be used to refer to a
specific file in the <strong>open file table</strong> of the OS. The <code>fd</code> (file descriptor)
field in the <a href="https://doc.rust-lang.org/std/fs/struct.File.html"><code>File</code></a>
struct above refers to exactly that.</p>
</div>
<div class="paragraph">
<p>The <a href="https://linux.die.net/man/2/open"><code>open</code></a> syscall is the one responsible
for creating a new file descriptor in the system-wide OFT and returning its
values to the process so that operations can be made on it. Here is its
signature</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-c" data-lang="c">int open(const char *pathname, int flags, mode_t mode);</code></pre>
</div>
</div>
<div class="paragraph">
<p>When the process obtains the file descriptor, it can read and write to it
(granted it has permissions to do so), and guess what, <code>read</code> and <code>write</code>
syscalls are made just for that. This are their signatures</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-c" data-lang="c">ssize_t read(int fd, void *buf, size_t count);

ssize_t write(int fd, const void *buf, size_t count);</code></pre>
</div>
</div>
<div class="paragraph">
<p>Both of them take a file descriptor as first argument, that&#8217;s the <code>int</code> value
returned by the <code>open</code> syscall. They both take a <code>size_t count</code> as last argument
which tells the size in bytes that the function should read/write. The middle
argument is in both cases a pointer to a buffer that tells where bytes should be
taken from/placed in.</p>
</div>
<div class="paragraph">
<p>We&#8217;re now missing a piece of the puzzle to completely understand how this all
comes together: the file&#8217;s <em>current offset</em>.</p>
</div>
<div class="paragraph">
<p>The <em>current offset</em> is used by the OS to keep track of where the next read or
write will begin reading from or writing to within the file.</p>
</div>
<div class="paragraph">
<p>We humans do pretty much the same thing when we are reading or writing
something. Think about it, when you are reading a 800 pages long book you may
start reading the first 60 pages in a single day and then you bookmark that page
so that when you come back to it later you know exactly where you previously
left off. Something similar happens when you are writing an essay, you continue
on the same line, word after word, character after character.</p>
</div>
<div class="paragraph">
<p>Operating Systems work very similarly in this case and they use the file&#8217;s
current offset to do just that. Each time a file gets created in the OFT with
the <code>open</code> syscall, its current offset is set to 0. Each <code>write</code> and <code>read</code>
operation is going to increment that current offset value by <code>size_t count</code>
implicitly.</p>
</div>
<div class="paragraph">
<p>We can see this in action by using the <code>strace</code> tool to inspect all the syscalls
invoked by the process:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-shell session" data-lang="shell session">$ strace -e trace=open,openat,write,read,close,lseek target/debug/run_function</code></pre>
</div>
</div>
<div class="paragraph">
<p>And this is the output that I get</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-shell session" data-lang="shell session">...
openat(AT_FDCWD, "/tmp/ENVELOPE_EDITMSG", O_RDWR|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 7
write(7, "\n\n# Comment variables to remove "..., 36) = 36
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=28814, si_uid=1000, si_status=0, si_utime=1, si_stime=0} ---
lseek(7, 0, SEEK_CUR)                   = 36
read(7, "\n", 8)                        = 1
read(7, "", 7)                          = 0
close(7)                                = 0
...</code></pre>
</div>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>For simplicity, you can interpret `openat` as `open` in this case, they are very similar.</p>
        </div>
<div class="paragraph">
<p>This is exactly what we expected to get, right? The <code>openat</code> syscall created the
file with the directives <code>O_RDWR</code> (read and write), <code>O_CREAT</code> (create),
<code>O_TRUNC</code> (trucate), <code>O_CLOEXEC</code> (close, eventually) and it returns the
file descriptor value of 7. The function then writes the string (buffer)
which is 36 bytes in size to the file descriptor 7. Finally, it calls <code>read</code>
twice, initially it tries to read 8 bytes from file descriptor 7 and <code>"\n"</code> is
what&#8217;s been able read, the second one tries to read up to 7 bytes but
there&#8217;s nothing left to read, so the buffer returned is empty.</p>
</div>
<div class="paragraph">
<p>See that <code>lseek</code> function before the <code>read</code>? That is the syscall to <strong>explicitly</strong>
position the current offset of the file descriptor. Here&#8217;s the signature</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-c" data-lang="c">off_t lseek(int fd, off_t offset, int whence);</code></pre>
</div>
</div>
<div class="paragraph">
<p>As usual, the first arg is the file descriptor. The second arg is the file
offset, which positions the current offset to a particular location within the
file. The last arg determines how the seek is performed and it has 3 different
possible values:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p><code>SEEK_CUR</code>: the offset is set to its current location + offset bytes.</p>
</li>
<li>
<p><code>SEEK_SET</code>: the offset is set to offset bytes.</p>
</li>
<li>
<p><code>SEEK_END</code>: the offset is set to the file size + offset bytes.</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>Therefore, <code>lseek(7, 0, SEEK_CUR) = 36</code> positions the offset to current location
+ 0 bytes (which was 36 bytes after the first write).</p>
</div>
<div class="paragraph">
<p>It is clear now why the original function that I wrote returned an empty buffer,
   here&#8217;s what happens:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>The file is opened with current offset set to 0</p>
</li>
<li>
<p>I write a bunch of data to the file itself and the current offset is set to
the number of bytes that I write to the file</p>
</li>
<li>
<p>Lastly, when I call <code>read_to_end</code>, nothing gets read because there is
nothing to read since the current offset already is set to the end of the file</p>
</li>
</ol>
</div>
<div class="paragraph">
<p>I was mislead by the <code>read_to_end</code> documentation which tells us that</p>
</div>
<div class="quoteblock">
<blockquote>
<div class="paragraph">
<p>Read all bytes until EOF in this source, placing them into <code>buf</code>.</p>
</div>
</blockquote>
</div>
<div class="paragraph">
<p>Now we know that it&#8217;s going to read all bytes <strong>starting from the current offset</strong>
until EOF.</p>
</div>
<div class="paragraph">
<p>Let&#8217;s move on to the solution, which is trivial at this point. We have two
different options:</p>
</div>
<div class="paragraph">
<p>Since <code>File</code> implements the
<a href="https://doc.rust-lang.org/std/io/trait.Seek.html"><code>Seek</code></a> trait, we can
explicitly reposition the current offset of the file descriptor to the beginning
of the file just before calling <code>read_to_end</code>, the following would be the same
as calling <code>lseek(7, 0, SEEK_SET)</code>.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">pub fn spawn_with(data: &amp;[u8]) -&gt; Result&lt;Vec&lt;u8&gt;&gt; {
    // ...

    cmd.run_shell_command()
        .map_err(|e| std_err!("error running child process: {}", e))?;

    // Reposition the offset at the start of the file
    file.seek(SeekFrom::Start(0)).unwrap();

    let mut buf = Vec::new();
    file.read_to_end(&amp;mut buf).unwrap();

    // ...
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>We know that each time a file is opened, its current offset is set to 0 by
default, so the other option would be to re-open the file before calling
<code>read_to_end</code></p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-rust" data-lang="rust">pub fn spawn_with(data: &amp;[u8]) -&gt; Result&lt;Vec&lt;u8&gt;&gt; {
    // ...

    cmd.run_shell_command()
        .map_err(|e| std_err!("error running child process: {}", e))?;

    let mut file = OpenOptions::new()
        .read(true)
        .open(&amp;pb)?;

    let mut buf = Vec::new();
    file.read_to_end(&amp;mut buf).unwrap();

    // ...
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>This time, since we just need to read the file, we can just open the file with
read-only permissions.</p>
</div>
<div class="paragraph">
<p>These solutions both solve my initial problem.</p>
</div>
<div class="paragraph">
<p>If I was in a performance critical environment and I&#8217;d have to choose between
the two I would go the first one because <code>lseek</code> is a much cheaper syscall than
<code>open</code> for obvious reasons.</p>
</div>
        <div class="dialog" data-character="matt" aria-hidden="true">
          <svg version="1.0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 476 440"><g transform="translate(0.000000,440.000000) scale(0.100000,-0.100000)"> <path d="M2003 3812 c-182 -62 -283 -171 -354 -382 -41 -124 -50 -259 -24 -376 19 -89 37 -124 63 -124 15 0 146 171 268 348 1 2 32 -20 68 -47 213 -161 482 -274 721 -302 171 -20 212 -5 186 67 -18 51 -74 143 -116 194 -20 24 -34 44 -32 47 2 2 35 -4 72 -12 62 -15 70 -15 84 -1 13 14 13 22 3 62 -47 174 -163 285 -333 317 -32 6 -59 14 -59 17 0 3 26 19 58 35 45 24 58 36 60 57 5 41 -44 53 -213 53 -140 0 -164 -5 -267 -50 -16 -6 -17 -3 -11 26 4 19 7 48 8 64 0 29 -2 30 -50 33 -34 1 -76 -7 -132 -26z"/> <path d="M3347 3243 c-18 -17 -1 -44 68 -113 l75 -75 -80 -80 c-59 -59 -79 -86 -74 -98 3 -9 13 -17 21 -17 17 0 178 152 195 184 9 16 -4 33 -83 112 -89 90 -107 102 -122 87z"/> <path d="M3653 2914 c-13 -35 23 -44 182 -44 159 0 193 8 179 43 -5 15 -25 17 -180 17 -153 0 -175 -2 -181 -16z"/> <path d="M1573 2853 c-56 -86 23 -297 119 -318 30 -7 33 -14 42 -102 4 -43 25 -107 62 -189 3 -5 -98 -64 -223 -130 -252 -132 -301 -166 -355 -245 -43 -62 -311 -763 -328 -858 -26 -146 46 -285 204 -390 l40 -26 1121 0 c1085 0 1122 1 1150 19 74 47 143 118 175 178 29 54 35 75 38 148 l4 85 -132 355 c-150 405 -166 444 -216 511 -45 62 -122 112 -367 239 -156 80 -198 106 -194 119 3 9 17 44 32 78 14 35 28 92 32 128 6 65 6 65 47 84 23 10 54 32 69 50 66 78 78 271 17 271 -22 0 -22 0 -20 -87 1 -67 -21 -134 -57 -165 -10 -10 -37 -21 -59 -26 l-39 -8 -11 -86 c-16 -117 -37 -177 -90 -248 -104 -142 -275 -214 -448 -189 -208 29 -372 197 -398 407 -16 125 -15 122 -48 122 -85 0 -162 166 -116 249 28 51 -19 73 -51 24z m412 -791 c198 -119 458 -89 635 74 l55 51 230 -121 c261 -137 289 -157 339 -229 38 -56 309 -767 321 -844 9 -60 -11 -145 -48 -200 -18 -26 -60 -68 -95 -95 l-63 -48 -95 0 -94 0 0 198 0 197 78 45 c74 44 95 71 73 93 -7 7 -29 -1 -71 -27 -34 -20 -66 -36 -71 -36 -5 0 -9 115 -9 280 0 311 -2 323 -63 360 -31 19 -56 20 -852 20 -805 0 -821 0 -853 -20 -58 -36 -62 -58 -62 -369 l0 -279 -69 40 c-71 40 -91 42 -91 8 0 -12 26 -34 80 -67 l79 -48 1 -197 0 -198 -92 0 c-73 0 -99 4 -123 19 -63 39 -125 103 -152 159 -23 48 -28 69 -27 132 0 71 7 93 137 438 75 199 146 381 158 404 43 83 100 126 354 260 l240 127 50 -48 c28 -27 73 -63 100 -79z m1084 -347 c16 -8 34 -28 40 -46 7 -22 11 -190 11 -526 l0 -493 -862 2 -863 3 -3 485 c-2 326 1 497 8 522 20 71 -24 67 853 68 674 0 792 -2 816 -15z"/> <path d="M1773 2854 c-3 -9 7 -57 23 -106 39 -124 45 -128 196 -128 158 0 170 6 198 103 12 39 23 78 25 85 3 6 20 12 39 12 34 0 35 -1 61 -80 38 -115 47 -120 201 -120 155 0 163 5 200 134 14 49 23 95 20 102 -4 12 -86 14 -481 14 -429 0 -476 -2 -482 -16z"/> <path d="M3105 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3294 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 33 -17 60 -18 92 -3 42 19 57 59 57 153 0 80 -2 88 -27 116 -30 32 -75 45 -109 31z m64 -63 c7 -7 12 -42 12 -89 0 -83 -11 -103 -52 -97 -22 3 -23 8 -26 86 -3 87 5 112 37 112 9 0 22 -5 29 -12z"/> <path d="M3515 2658 c-3 -8 -4 -79 -3 -158 3 -137 4 -145 23 -145 19 0 20 7 20 155 0 140 -2 155 -18 158 -9 2 -19 -3 -22 -10z"/> <path d="M3704 2661 c-48 -20 -64 -58 -64 -147 0 -94 12 -129 51 -150 52 -27 115 -8 137 41 7 14 12 64 12 110 0 78 -2 87 -27 115 -30 32 -75 45 -109 31z m65 -65 c8 -9 11 -45 9 -98 l-3 -83 -29 -3 c-16 -2 -32 2 -37 10 -13 20 -11 162 3 176 16 16 43 15 57 -2z"/> <path d="M3342 2218 c-14 -14 -17 -275 -3 -296 5 -8 17 -12 27 -10 17 3 19 15 22 147 2 97 -1 148 -9 157 -14 17 -21 18 -37 2z"/> <path d="M3514 2220 c-12 -4 -31 -21 -43 -37 -21 -25 -22 -37 -19 -126 3 -94 4 -100 31 -123 32 -27 75 -31 116 -9 41 21 54 61 50 159 -4 76 -7 88 -29 110 -28 27 -75 39 -106 26z m64 -62 c7 -7 12 -44 12 -95 0 -78 -2 -84 -22 -92 -46 -17 -58 2 -58 92 0 84 9 107 40 107 9 0 21 -5 28 -12z"/> <path d="M3774 2220 c-12 -4 -31 -21 -43 -36 -18 -23 -21 -40 -21 -121 0 -91 1 -96 29 -124 47 -47 132 -33 159 26 14 31 16 163 2 199 -16 44 -84 74 -126 56z m64 -62 c13 -13 17 -156 4 -174 -15 -22 -42 -24 -62 -4 -26 26 -28 134 -4 168 17 24 43 29 62 10z"/> <path d="M3981 2221 c-8 -5 -11 -50 -9 -157 3 -141 4 -149 23 -149 19 0 20 8 23 151 2 121 0 153 -11 157 -8 3 -19 2 -26 -2z"/> </g> </svg>
          <p>EDIT: Repositioning the current offset does not behave as expected on macOS, maybe I'll try and investigate why on a later post, but for the moment it seems like I'm stuck with re-opening the file if I want this to work as expected across Linux and macOS.</p>
        </div>]]></content><author><name>Mattia Righetti</name></author><summary type="html"><![CDATA[The other day I working on a side project of mine and I stumbled upon a bug in my code that I couldn&#8217;t immediately figure out how to solve.]]></summary></entry></feed>