The state of SIMD in Rust in 2026

2026-09-25T00:00:00+00:00
A lot of progress was made since last year, and I made some of it!
After last year's survey</a> I started contributing to the SIMD library that seemed the most promising. One thing led to another, and now I'm a maintainer of Fearless SIMD.
To avoid a conflict of interest, I invited authors of other libraries (std::simd</code>, wide</code>, pulp</code>, macerator</code>) to review and provide feedback on a draft of this article. However, I retained editorial control, and all mistakes are my own.
This year's survey is more in-depth than my previous one. So buckle up, and let's take it... from the top!
What’s SIMD? Why SIMD?
</i></a> 
</h2>
Hardware that does arithmetic is cheap, so any CPU made this century has plenty of it. But you still only have one instruction decoding block and it is hard to get it to go fast, so the arithmetic hardware is vastly underutilized.</p>
To get around the instruction decoding bottleneck, you can feed the CPU a batch of numbers all at once for a single arithmetic operation like addition. Hence the name: “single instruction, multiple data,” or SIMD.</p>
Instead of adding two numbers together, you can add two batches or “vectors” of numbers and it takes about the same amount of time as doing just one addition.</p>
On recent x86 chips these batches can be up to 512 bits in size, so in theory you can get an 8x speedup for math on f64</code> or a 64x speedup on u8</code>. In practice it can run both slower</a> and faster</a>.</p>
Instruction sets</i></a> 
</h2>
Historically, SIMD instructions were added after the CPU architecture was already designed, so SIMD is an extension with its own marketing name on each architecture.</p>
ARM calls theirs “NEON”, and all 64-bit ARM CPUs have it.</p>
WebAssembly doesn’t have a marketing department, so they just call theirs “WebAssembly 128-bit packed SIMD extension”.</p>
64-bit x86 shipped with one called “SSE2” which has basic instructions for 128-bit vectors, but later</em> they added a whole menagerie of extensions on top of that, with SSE 4.2 adding more operations, AVX and AVX2 adding 256-bit vectors and AVX-512 adding 512-bit vectors and even more operations.</p>
The word “later” in the above paragraph creates a problem.</p>
Does this CPU have that instruction?</i></a> 
</h3>
If you’re running a program on an x86_64 CPU, it’s not a given that the CPU has any particular SIMD extension. So by default the compiler isn’t allowed to use instructions beyond SSE2 because that won’t work on all x86_64 CPUs.</p>
There are two ways around this problem.</p>
If you work for a company that only ever runs their binaries on their own servers or on a public cloud, you can just assert that they’re all recent enough to at least have AVX2 that was introduced over 10 years ago, and have the program crash or misbehave if it ever runs on anything without AVX2:</p>
RUSTFLAGS='-C target-cpu=x86-64-v3' cargo build --release</span></span></code></pre>
However, if you are distributing the binaries for other people to run, that’s not really an option.</p>
Instead you can do something called function multiversioning:</strong> compile the same function multiple times for different SIMD extensions, and when the program actually runs, check what features the CPU supports and select the appropriate version based on that.</p>
Fortunately, this problem only exists on x86.</p>
ARM made NEON mandatory on its 64-bit CPUs and hasn't really added useful SIMD extensions after that (more on that later).</p>
WebAssembly makes you compile two different binaries, one with SIMD and one without, and use JavaScript to check if the browser supports SIMD.</p>
How do I SIMD?</i></a> 
</h2>
There are three ways to leverage SIMD:</p>

Automatic vectorization: &[i32].sum()</code></li>
Portable SIMD abstractions: i32x4 + i32x4</code></li>
Platform-specific intrinsics - hang on, we're gonna need a bigger code block:</li>
</ol>
#</span>[</span>cfg</span>(</span>all</span>(</span>any</span>(</span>target_arch </span>=</span> "x86"</span>,</span> target_arch </span>=</span> "x86_64"</span>),</span>target_feature </span>=</span> "sse2"</span>))]</span></span>
_mm_add_epi32</span>(</span>__m128i</span> ,</span> __m128i</span>)</span></span>
#</span>[</span>cfg</span>(</span>all</span>(</span>target_arch </span>=</span> "aarch64"</span>,</span> target_feature </span>=</span> "neon"</span>))]</span></span>
vaddq_u32</span>(</span>int32x4_t</span>,</span> int32x4_t</span>)</span></span></code></pre>
Let's look at what each one entails and what the state of each programming model is.</p>
Automatic vectorization</i></a> 
</h2>
Just write plain Rust and let the compiler heuristics do the work!</p>
You can get it to work quite well, if you are careful to write code in a way that the compiler can reliably(ish) vectorize. This usually involves iterating over &[i32].as_chunks()</code> instead of &[i32]</code> and benchmarking or staring at the assembly to verify it worked. See Can You Trust a Compiler to Optimize Your Code?</a></strong> for details.</p>
This is the easiest option to use, requires no dependencies, and automatically supports all instruction sets the compiler supports, no matter how obscure.</p>
The downside is that this method is not very reliable. The larger and more complex your function is, the greater is the chance that the compiler will not be able to vectorize it. Performance can also swing wildly depending on the compiler version or due to changes to the surrounding code.</p>
Floating-point types also need special care.</p>

Floats are weird.</strong> Even something as trivial as summing an array of floats with reasonable precision gets surprisingly involved, see Taming Floating-Point Sums</a></strong>.</p>
</blockquote>
Previously automatic vectorization didn't work with floating-point types because it would change the precision of the result (often for the better, but the compiler is not permitted to change any observable results).</p>
This changed in Rust 1.98</a> which stabilized algebraic ops such as algebraic_add()</code></a> that let the compiler change the observable result, like a less dangerous -ffast-math</code></a>. You still have to rewrite your code to use them for it to be eligible for vectorization in most cases.</p>
And you still need to get multiversioning somehow. So while we're at it...</p>
The 'multiversion' crate</i></a> 
</h3>
The all-in-one SIMD crates discussed below also provide multiversioning, but let's take a look at multiversion</code></a> real quick since it's most useful for automatic vectorization.</p>
It's very easy to use: you add the #[multiversion(targets = "simd")]</code> annotation to your function and that's it.</p>
But that ease hides an undocumented pitfall: calling a function annotated with #[multiversion]</code> has a little bit of overhead. It is very small - under a dozen instructions, but it shows up as significant overhead if the function you put it on is itself tiny.</p>
As a rule of thumb, if your function has a loop in it, add #[multiversion]</code>; if it processes a handful of values add #[inline(always)]</code>, so long as there is #[multiversion]</code> somewhere up the call chain.</p>
The other crates listed below don't have this pitfall and don't make you think about the sizes of functions, at the cost of more boilerplate.</p>
multiversion</code> is the only crate that allows you to list the exact CPU extensions you require, as opposed to opting in to a predefined SIMD level. So if your code happens to benefit from some very recent instruction, you can opt in to it. But in my experience this hardly ever comes up for autovectorized code.</p>
For AVX-512 multiversion</code> checks if it's present, not whether it's actually fast, which may hurt performance in practice (more on that below). You can work around that at the cost of boilerplate - you have to put this on every function:</p>
#</span>[</span>multiversion</span>::</span>multiversion</span>(</span>targets</span>(</span></span>
    "x86_64+cmpxchg16b+popcnt+sse3+sse4.1+sse4.2+ssse3"</span>,</span> // x86_64-v2</span></span>
    "x86_64+avx+avx2+bmi1+bmi2+cmpxchg16b+f16c+fma+lzcnt+movbe+popcnt+sse3+sse4.1+sse4.2+ssse3+xsave"</span>,</span> // x86_64-v3</span></span>
    "x86_64+fxsr,adx,avx512bitalg,avx512bw,avx512cd,avx512dq,avx512f,avx512ifma,avx512vbmi,avx512vbmi2,avx512vl,avx512vnni,avx512vpopcntdq,bmi1,bmi2,cmpxchg16b,fma,gfni,lzcnt,movbe,pclmulqdq,popcnt,vpclmulqdq,xsave,xsavec,xsaveopt,xsaves"</span>,</span> // Ice Lake and later</span></span>
)]]</span></span></code></pre>Portable SIMD abstractions</i></a> 
</h2>
There are several production-ready ones. The desirable features are:</p>

Fixed-width vectors:</strong> write code in terms of f32x4</code>, u8x16</code>, etc (known size)</li>
Hardware-width vectors:</strong> use the largest vector size the hardware supports, without knowing it in advance</li>
Generic over element type:</strong> write code that works on both f32x4</code> and f64x2</code></li>
Generic over vector width:</strong> write code that works on all of f32x4</code>, f32x8</code>, f32x16</code></li>
</ul>
The TL;DR table:</p>
</th> std::simd (nightly)</th> fearless simd</th> wide</th> pulp</th> macerator</th></tr></thead>

multiversioning</td> 📦/🛠️</td> ✅</td> ❌</td> ✅</td> ✅</td></tr>
fixed-width vectors</td> ✅</td> ✅</td> ✅</td> ☑️</td> ❌</td></tr>
hardware-width vectors</td> 📦/🛠️</td> ✅</td> 🛠️</td> ✅</td> ✅</td></tr>
generic over element type</td> ✅</td> ✅</td> 🛠️</td> 🛠️</td> ✅</td></tr>
generic over vector width</td> ✅</td> ✅</td> 🛠️</td> ✅</td> ✅</td></tr>
safe access to intrinsics</td> 🛠️</td> ✅</td> ☑️</td> ✅</td> 🛠️</td></tr>
trigonometry</td> 📦/🛠️</td> 🛠️</td> ☑️</td> 🛠️</td> 🛠️</td></tr>
</tbody></table>

✅ Yes</li>
☑️ Yes, with caveats</li>
📦 Yes, with a third-party crate</li>
🛠️ Build it yourself</li>
❌ Absolutely not</li>
</ul>
And the instruction set support:</p>
</th> std::simd</th> fearless simd</th> wide</th> pulp</th> macerator</th></tr></thead>

SSE2</td> ✅</td> ✅</td> ✅</td> ☑️</td> 🐌</td></tr>
SSE4.x</td> ✅</td> ✅</td> ✅</td> ☑️</td> ✅</td></tr>
AVX2</td> ✅</td> ✅</td> ✅</td> ✅</td> ✅</td></tr>
AVX-512</td> ✅</td> ✅</td> ✅</td> ✅</td> ✅</td></tr>
NEON</td> ✅</td> ✅</td> ✅</td> ✅</td> ✅</td></tr>
WASM</td> ✅</td> ✅</td> ✅</td> ✅</td> ✅</td></tr>
All the rest</td> ✅</td> 🐌</td> 🐌</td> 🐌</td> 🐌*</td></tr>
</tbody></table>

✅ Has optimized routines</li>
☑️ Implemented but not used. Requires writing a custom dispatch to opt in.</li>
🐌 Reliant on autovectorization, often slow</li>
</ul>
* macerator also supports LoongArch because the author was, and I quote, "bored".</p>
std::simd</i></a> 
</h3>
std::simd</a> is not a complete solution for SIMD. It's more of a set of building blocks that absolutely has to be in the standard library, while everything else is left up to the ecosystem crates.</p>
The largest drawback is that it's nightly-only, and still undergoes infrequent breaking API changes. So one day you update the compiler and your code stops compiling, and you have to go and fix it. But so long as you're OK with that, and only need fixed-width vectors and maybe multiversioning, it's pretty great!</p>
std::simd</code>'s raison d'être</em> is that it sits directly on top of LLVM and can target any platform LLVM can target, including weird CPUs that only large banks use or that only the Chinese government uses. On the flip side, if LLVM doesn't have a perfectly matching operation inside it for std::simd</code> to make use of, there is no plan B and no SIMD is actually used</a>.</p>
This happens disturbingly often. Its sin()</code> could not be more apt: shipping scalar implementations in a SIMD guise is the cardinal sin. And reduce_sum()</code> is somehow the worst case for both</em> performance and accuracy. So don't bother using any non-trivial functions on floats.</p>
The closest thing we have to proper trigonometry is the sleef</a> crate, a partial port of SLEEF</a> to std::simd</code> that's only a little</a> buggy</a>. And that's the best trigonometry I have in this whole article!</p>
std::simd</code> is uniquely flexible when it comes to multiversioning. You can use the multiversion</a> crate or the multiversioning from any other SIMD crate in this section. All the other crates work with their own built-in multiversioning only.</p>
Its Simd<T, N></code> API looks like it would be very elegant and work great if you could just do math on N</code>, but you cannot</a>. That feature is very incomplete even on nightly. Without it using Simd<T, N></code> to get hardware-sized vectors is doable, but a lot uglier</a>.</p>
While you can use std::simd</code> directly in many cases and have it perform okay, disparately tacking on features through third-party crates only gets you so far. As an example, the sleef</code> crate</a> doesn't work with the multiversion</code> crate</a>, you have to fork sleef</code> and mate them yourself. Third-party extensions work in isolation but don't compose.</p>
What you need is an all-in-one solution where all the parts work together. Speaking of which...</p>
fearless_simd</i></a> 
</h3>
Fearless SIMD</a> is an all-in-one solution where all the parts work together.</p>
Just look at that beautiful column of green check boxes that makes in the tables!</p>
Beyond the tables, the features unique to fearless_simd</code> are:</p>

Orders of magnitude less unsafe</code> code under the hood than other crates thanks to a clever design</a>.</li>
Multiversioning that Just Works, even for tiny functions. Just slap #[simd]</code> on a function and you're done. A manual mode is available if you hate procedural macros.</li>
Multiversioning is controlled by whoever builds the final binary. You can configure it</a> without patching the libraries.</li>
</ol>
The main drawback is boilerplate: instead of</p>
fn</span> my_func</span>(</span>a</span>:</span> A</span>,</span> b</span>:</span> B</span>) {</span></span></code></pre>
you have to write</p>
#</span>[</span>simd</span>]</span></span>
fn</span> my_func</span><</span>S</span>:</span> Simd</span>></span>(</span>simd</span>:</span> S</span>,</span> a</span>:</span> A</span>,</span> b</span>:</span> B</span>) {</span></span></code></pre>
which is a mouthful.</p>
AVX-512 is only used on recent-ish CPUs where it doesn't hurt performance (see the hardware section below). You can manually configure multiversion</code> to behave like this, but it's not the default and requires a lot of boilerplate (see above). All the other SIMD abstraction crates just check if AVX-512 is present or not.</p>
It recently shipped v1.0, with a security policy</a> and everything.</p>
The biggest gap is trigonometry. There just isn't a port of anything like SLEEF to fearless_simd</code> machinery yet.</p>
wide</i></a> 
</h3>
wide</code> has a lot going for it: good platform coverage, lots of implemented operations, and it's v1.0 already. It even has trigonometric functions, although their precision is explicitly left unspecified</a>.</p>
The biggest downside is that it's fundamentally incompatible with multiversioning. This is fine if you're not targeting x86, or if you always build with -C target-cpu=</code> for known hardware, but cripples performance otherwise. The only workaround is cargo multivers</code></a>, but it only works for long-running programs, otherwise its startup costs dwarf the performane gains from SIMD.</p>
The other downside is not supporting any kind of generics, either over element types or vector widths. However, you can work around that using macros. Instead of making a function generic, wrap it in macro_rules!</code> and write $type::from_slice</code> instead of T::from_slice</code>. It adds a bit of boilerplate, but removes the boilerplate for generic bounds, so win some lose some. I've done it, it's not too bad, especially if you pull in something like the paste</a> crate.</p>
If you're only targeting a handful of types, e.g. f32</code> and f64</code>, you might want to use the macro approach regardless, even in libraries with generics, because it also allows you to have arrays of "generic" sizes. Actual generic array sizes are a nightly-only and incomplete feature. But you can also work around that with the generic-array</a> crate or just by making an array of the largest possible SIMD size.</p>
pulp</i></a> 
</h3>
pulp</a> was built to power the faer</a> linear algebra library. This informs its priorities: the implemented operations are mostly math (e.g. no swizzles), and the API is geared towards native-width vectors.</p>
Fixed-width vectors are technically possible, but completely undocumented and quite awkward to use. I've contributed some fixes</a> for them while I was researching them, including for a soundness bug</a>.</p>
There is no native support for being generic over the element type, but the macro trick I described for wide</code> should work fine here too.</p>
Its multiversioning is the most verbose I've ever seen</a>. There's a macro to reduce boilerplate</a> but even that is rather verbose compared to the alternatives.</p>
macerator</i></a> 
</h3>
macerator</a> is a relative of pulp</code> with a similar design. It was built to power the CPU backend for burn</a>.</p>
Compared to pulp</code> it adds support for code generic over element type, but removes safe access to intrinsics and most of the documentation. There is no attempt at fixed-width vectors.</p>
It also enables SSE4.2 by default and adds optimized codepaths for LoongArch.</p>
This is the only library other than std::simd</code> with some portable operations on f16</code> data, albeit the list of supported operations is very limited. Using it with AVX-512 requires a nightly compiler, while NEON works on stable. It is still rather awkward because the standard library's f16</code> is nightly-only and this crate has to get by without it.</p>
It isn't used by anything on crates.io other than burn</code>.</p>
Others</i></a> 
</h3>
I'm excluding SIMD crates made for a single specific project (e.g. jxl_simd</a>, pathfinder_simd</a>) since they are not intended for a general audience. I'm also excluding crates whose development is primarily AI-driven (e.g. magetypes</a>, simdeez</a>, thermite</a>) because I cannot recommend them for production use, especially since the latter two are disconcertingly</a> buggy</a>.</p>
Safe access to intrinsics</i></a> 
</h2>
Portable SIMD is good, but sometimes you want a very specific instruction that only a certain instruction set has. In that case you have to use intrinsics directly.</p>
Rust v1.87+ allows safely calling platform-specific intrinsics:</p>
#</span>[</span>target_feature</span>(</span>enable </span>=</span> "avx2"</span>)]</span></span>
fn</span> add_avx2</span>(</span>a</span>:</span> __m256</span>,</span> b</span>:</span> __m256</span>)</span> -></span> __m256</span> {</span></span>
    _mm256_add_ps</span>(</span>a</span>,</span> b</span>)</span> // this is an avx2 intrinsic</span></span>
}</span></span></code></pre>
There are two caveats:</p>

Intrinsics to load data from memory or store it are still unsafe</code> because they operate on raw pointers</li>
The function we defined, add_avx2</code>, still requires an unsafe</code> block to call from a function not annotated with #[target_feature(enable = "avx2")]</code></li>
</ol>
But there are established solutions for both:</p>

Use safe wrappers for loads/stores that add bounds checks, which the optimizer then trivially removes</a> from machine code so no performance is lost.</li>
Check if a CPU feature is available at runtime and encode it in a type-level token</a>, then use that to call functions requiring those features safely.</li>
</ol>
Everything on this list is various implementations of these two ideas.</p>
archmage</i></a> 
</h3>
archmage</a> provides the CPU feature tokens and uses the safe_unaligned_simd</a> crate for safe load/store wrappers.</p>
Its centerpiece is the #[arcane]</code> procedural macro</a>.</p>
You get a selection of predefined SIMD levels: the usual suspects of SSE2/SSE4.2/AVX2, and there are two different levels of AVX-512: the early slow implementations, and Ice Lake and later which is actually useful, at your option. On ARM there's baseline NEON plus a couple of extension levels.</p>
No support for 32-bit x86 (you always get the scalar fallback), but that's not a big deal in 2026.</p>
fearless_simd</i></a> 
</h3>
fearless_simd</a> gives you basically the same tools as archmage</code> via its kernel!</code> macro</a>.</p>
This is a declarative macro, not a procedural one. This improves build times, but unlike archmage</code> it doesn't support annotating generic or const-generic functions with it. Intrinsics and generics don't gel anyway, so it's usually not a big deal.</p>
It doesn't bundle safe_unaligned_simd</code> since safe loads can be done through its portable SIMD abstraction, but you can pull it yourself if you really want to spell loads as _mm256_loadu_epi64()</code>, usually for porting existing code written like that.</p>
The SIMD levels are the same as for the portable SIMD abstraction. So you don't get to opt in to early, slow AVX-512 if you really know what you're doing, or access NEON's non-baseline extensions like aes</code> or bf16</code>.</p>
On the upside, you can easily mix and match portable SIMD and intrinsics</a>. This lets you write most of the algorithm in portable SIMD, and use a handful of intrinsics only where they are really needed.</p>
pulp</i></a> 
</h3>
pulp</a> is deceptively powerful in this regard.</p>
If you want to use intrinsics that aren't part of any SIMD level, such as _mm_aesenc_si128</code> from the aes</code> feature, this is the best (and only) way to do it safely without rolling your own SIMD feature tokens.</p>
Unfortunately it does not document how to do that.</strong></p>
If you look up the docs, you'll find structs named after various CPU features, e.g. Avx512ifma</a>, with a way to construct it and with the intrinsics corresponding to that CPU feature on it. So you'd think you just construct it and call the function, right? Wrong.</strong></p>
You can do that, and it works, but performance is awful. You are calling an intrinsic that requires extra CPU features from a function that isn't guaranteed to have them (remember, the check for the CPU feature can fail), so the intrinsic has to be in its own separate function. And now you are paying function call overhead - several instructions - to call a single instruction. "Several instructions" is a lot more than one, so the function call overhead dominates and performance plummets.</p>
What you have to do instead is create a context with all required features enabled in it, and then call a bunch of intrinsics from that context. Like this:</p>
pulp</span>::</span>simd_type!</span> {</span></span>
    pub struct</span> Ifma</span> {</span></span>
        pub</span> ifma</span>:</span> "avx512ifma"</span>,</span></span>
    }</span></span>
}</span></span>
</span>
if let</span> Some</span>(</span>isa</span>)</span> =</span> Ifma</span>::</span>try_new</span>() {</span></span>
    isa</span>.</span>vectorize</span>(</span></span>
        #</span>[</span>inline</span>(</span>always</span>)]</span></span>
        ||</span> {</span></span>
            // Put the entire hot loop here.</span></span>
            // isa.ifma._mm512_madd52lo_epu64(...)</span></span>
        },</span></span>
    );</span></span>
}</span></span></code></pre>
See here</a> for a more complete example you can actually run.</p>
For completeness, I should mention that a similar feature was proposed for the fearless_simd</code> repository as a separate, independent crate. It was fully implemented</a>, but nobody stepped up to actually maintain it, so it was never merged. If something irks you about pulp</code>, try that instead and see if you're willing to take it over.</p>
The state of intrinsics</i></a> 
</h2>
SIMD intrinsics underpin all SIMD code except for std::simd</code> and autovectorization. They're quite straightforward, too: intrinsics are supposed to clearly map to specific CPU instructions. Given how simple and important they are, you'd expect them to work really well.</p>
They don't. Not in Rust, not in C++, not in C.</p>
There is a fundamental tension between "give me this exact instruction" and compiler optimizations. If you have the compiler treat SIMD intrinsics as pure black boxes, you end up with inefficiencies elsewhere.</p>
For example, a real bug I've run into on ARM is that u32x4::from([1,2,3,4])</code> was slow. This is literally loading a constant, and u32x4</code> has the exact same memory layout as an array of four u32</code>, so it should be really</em> cheap - just a single load.</p>
It turns out that the underlying ARM load intrinsic, vld1_u32_x4</code>, was implemented as a black-box operation in the compiler, so all LLVM saw was a black-box operation on some on-stack value. The generated assembly first loaded the constant into registers, then placed it onto the stack, and then loaded it back into registers through the black-box vld1_u32_x4</code>.</p>
The fix was to drop the black-box implementation for vld1_u32_x4</code></a> and make it into a compatibility wrapper for regular loads that the compiler can properly optimize. Many thanks to Folkert de Vries</a>, a Rust stdarch maintainer, for helping investigate this and implementing the fix.</p>
So let's just turn all intrinsics into wrappers for regular compiler ops, right? I wish.</p>
That vld1_u32_x4</code> isn't really a black box. It's a hardware operation that nobody has written optimization passes for yet. And if you want to make some exotic operation into basic blocks comprehensible to the compiler (or just abstract over common behavior of slightly different hardware instructions</a>), the operation needs to be made up of several</em> basic blocks. This is really attractive for Rust because it makes supporting backends other than LLVM easier, but Clang has also been moving in this direction.</p>
But then to emit the desired operation from several building blocks, you need the optimizer to recombine them into a single instruction. This can be easily messed up by unrelated optimization patterns that e.g. reorder these blocks and break the pattern-matching. So these optimizations sometimes work on simple test cases but break in real-world code</a>.</p>
So SIMD intrinsics are stuck in an endless tug of war</strong> between lowering into the expected instructions and working with the expected compiler optimizations.</p>
And on top of the fundamental limitations, there are also compiler instruction selection bugs. I've run into LLVM seeing a 512-bit vector shuffle operation with constant indices and going "oh, I know, I can optimize this!" except its "optimization" uses SSE4.2-era operations and is far, far slower</a> than just running the actual shuffle instruction I asked for. (That one's fixed in LLVM 23, following my report). Or lowering an intrinsic whose sole purpose is efficient encoding into a less efficient encoding</a>. I literally have a list of such compiler bugs</a> I've found. The Rust-specific ones got fixed after I reported them, but there is a bunch of LLVM bugs affecting all of Rust, C++ and C still unfixed.</p>
And in case you're wondering - no, this isn't just an LLVM problem. GCC has similar issues, and MSVC is noticeably worse at this than either of the major open-source compilers.</p>
Bonus fun fact:</strong> Intel forgot to include some AVX-512 instructions</a> into their searchable Intrinsics Guide</a>, so most compilers didn't implement them, and now you can't reach those instructions from high-level languages. I've contributed them to rustc</a> but they haven't shipped on stable yet.</p>
Inline assembly</i></a> 
</h3>
You'd think you could outsmart the compiler that way and bypass all the issues with intrinsics. And you kinda sorta can, except now your inline assembly block is a real honest-to-goodness black box.</p>
Not only are all the optimization issues back with a vengeance, but entering and exiting the inline assembly block has some overhead.</p>
This works okay if you want to write a large-ish function in it and are willing to sacrifice compiler optimizations, but isn't profitable if you just want to use an instruction or two.</p>
Compiler feature wishlist</i></a> 
</h2>
I'll keep this brief, in the order of importance:</p>

min_generic_const_args</code> would allow using arrays in conjunction with hardware-width SIMD vectors. There are workarounds (just use a huge array, use generic-array</code> crate</a>, or use macros instead of generics</a>) but all are partial and/or ugly.</li>
With the Struct Target Features RFC</a>, fearless_simd</code>/pulp</code>/macerator</code> would no longer need #[simd]</code> annotations on functions. It's less boilerplate, but most importantly you can no longer accidentally forget to put them there and cause performance to drop.</li>
We need a way to make iterators not conflict with multiversioning</a>. The Struct Target Features RFC</a> would solve this too. Alternatively an equivalent of GCC's __attribute__((flatten))</code></a> should do the trick, but that requires either even more boilerplate or proc macros.</li>
generic_const_args</code> (not min</code>) would significantly improve build times</a> when wrapping certain intrinsics into portable abstractions, and make using std::simd</code> much nicer.</li>
</ol>
In the standard library I'd love to see crater-like verification of changes to intrinsics</a>, and std::simd</code> available on stable so that ecosystem crates would delete most of their code and gain support for all the obscure platforms.</p>
Conclusion</i></a> 
</h2>
Support for SIMD in the Rust ecosystem has matured a great deal.</p>
While some things could still be improved (notably trigonometry), recent advances in compiler features and the library ecosystem made Rust attractive for SIMD code even when memory safety is not a hard requirement.</p>
Bonus round: The state of hardware</i></a> 
</h2>
After writing portable SIMD code for 5 different instruction sets, I have opinions.</em></p>
x86</i></a> 
</h3>
It's a mess, and we have Intel to thank for it.</p>
AVX2 is simultaneously the most common and the most cursed instruction set I've ever had to work with.</p>
Whenever I try to implement a simple, straightforward SIMD operation, half the time AVX-512 and NEON have it natively, but AVX2 needs complex and slow emulation. The fact that instead of a proper 256-bit ISA it's more like two 128-bit execution units smushed together really doesn't help.</p>
But it gets worse. Despite CPUs with AVX2 launching in 2013, the most recent Intel CPU without AVX2 launched in 2021</a>! So you can't even count on having AVX2, good luck making do with SSE4.2 from... checks notes...</em> 2008!</p>
That's how you get 15% of x86 CPUs in the Firefox hardware survey</a> still not having AVX2 in 2026. Have fun writing and maintaining SSE4.2 codepaths just for them! What year is this?!</em></p>
Intel launched AVX-512 in 2015, which fixed much of the insanity of AVX2, and then... just didn't put it into any CPUs? It was only really present on the server, everyone else was stuck with AVX2 or even just SSE4.2. So Intel has three completely different SIMD extensions</strong> all existing at the same time!</p>
But wait, it gets even worse!</p>
Running AVX-512 instructions on early Intel CPUs with AVX-512, even on a single core, reduces CPU frequency of all</em> cores</a>. An AVX-512 workload anywhere hurts performance of the entire rest of the chip! Ironically, AVX-512 only appeared in high-end CPUs with lots of cores where this kind of fallout is especially</em> bad!</p>
You'd think you could still benefit from AVX-512 on these CPUs if you run it on all cores at once for a long time, but then you end up bottlenecked on memory anyway, and whatever the CPU is doing becomes irrelevant. So on those CPUs AVX-512 doesn't actually give you any performance and often hurts it, except in artificial microbenchmarks that don't touch memory.</p>
This is such a shame, because AVX-512 is such a big improvement on AVX2 otherwise. Forget the 512-bit width, just give me the sane set of supported operations!</p>
This downclocking behavior was only fixed in 2019, in the Ice Lake architecture. Not fully, but enough to make AVX-512 profitable overall. This is why fearless_simd</code> only supports AVX-512 on Ice Lake and later, and on AMD which never had downclocking issues to begin with.</p>
AMD showed how badly Intel messed this up by releasing Zen 4, which didn't even have hardware 512-bit operations. It mapped most 512-bit operations to 256-bit execution units, and still smoked</a> Intel's native 512-bit hardware in benchmarks. Zen 5 with its native 512-bit hardware sealed the deal. No wonder Intel is struggling recently.</p>
Not that AMD is blameless. They're the reason we can't use scatter/gather instructions because in Zen they're not implemented natively in hardware, and end up being slower than issuing lots of small loads. Intel made scatter/gather slower than scalar loads in early AVX2 CPUs too, but they got their act together eventually, sort of; AMD didn't even try.</p>
LLVM sometimes emits scatter/gather instructions for AVX-512 when autovectorizing code, which hurts performance by 1.75x on Intel</a> and by 4x on AMD</a>, so I'm not even sure why LLVM even bothers. I believe you need to pass -C target-cpu=</code> to hit this, but I haven't extensively tested it.</p>
Steam hardware survey</a> shows that 23.9% of systems have AVX-512. This is skewed towards high-end/gaming systems; for example, the 15% of systems with only SSE4.2 from the Firefox graphics survey</a> are at only 2% here. It also shows a breakdown by AVX-512 optional features, and from them we can infer that 23.95% (avx512vnni</code>) minus 23.90% (baseline avx512f</code>) equals -0.05% of systems with awfully slow AVX-512. Your guess on why this percentage is negative is as good as mine.</p>
So at least on desktop, the broken AVX-512 is nonexistent, which means you don't have to worry about it. Therefore setting Ice Lake as a requirement for AVX-512 loses you nothing and gains some useful instructions, but using the baseline AVX-512 isn't awful either.</p>
There is no public data on the prevalence of Skylake servers, where AVX-512 is present but degrades performance. Using Ice Lake as a requirement for AVX-512 so that Skylake uses AVX2 should prevent that degradation.</p>
Intel was this</em> close to messing things up again by replacing AVX-512 with AVX10, which is AVX-512 but with either 256-bit or 512-bit vectors, and you don't know which ones. But AMD's clearly superior design that just maps 512-bit vectors onto 256-bit hardware averted this disaster and forced Intel back into a sane programming model. Whew. Thanks, AMD.</p>
This year, at long last, AVX-512 is becoming mandatory in upcoming Intel CPUs via its rebranding into AVX 10.2. Which is what we wanted all along.</p>
Well, not the rebranding.</p>
Also, doing math on floating-point values very close to zero makes performance plummet</a>. AMD is about 2x slower on those, but on Intel you get a 30x slowdown.</p>
Somehow, every time I learn something horrifying about SIMD, it's always Intel's fault.</p>
ARM</i></a> 
</h3>
Everything that AVX2 got wrong, 64-bit NEON gets right.</p>
With only 128-bit vectors you'd think it is an equivalent of SSE4.2, but it's actually closer to AVX2.</p>
NEON has the same register space as AVX2, which is often the limiting factor in practice. And instead of making you deal with two 128-bit execution units side by side explicitly, beefy ARM cores transparently run 128-bit operations in parallel via instruction-level parallelism</a>, while cheap power-constrained cores can still execute them one by one.</p>
NEON also adds just enough instructions larger than 128 bits to make common operations Just Work. In my tests with SIMD base64 decoding, the same algorithm runs 1.5x to 2x faster on NEON than on AVX2 (but still 2x slower than AVX-512).</p>
And all of this in a single, simple programming model instead of several different ones. And it's mandatory in 64-bit ARM chips, with no need for multiversioning!</p>
The only criticism I can level at Aarch64 NEON is that a single chip has two kinds of cores ("performance" and "efficiency") with completely different execution characteristics, so an instruction sequence that is fast on performance cores is slow on efficiency cores, and vice versa. So even if you know a specific CPU you're targeting, you can't really select an optimal implementation, it's all trade-offs! And when you consider the diversity of ARM CPUs out there, it only gets worse. Fortunately, NEON has enough operations implemented directly as hardware instructions with reasonable performance to prevent this from turning into a total nightmare.</p>
Meanwhile SVE is pretty much useless. SVE2 is now mandatory in ARM CPUs, but it's implemented at 128-bit width even in high-end server chips, so it's just an awkward NEON with extra steps. Technically there was 256-bit SVE in a single generation of server ARM chips for the cloud, but that's not SVE2, and in the cloud you just use AVX-512 instead anyway. Maybe we need to wait another decade or so to see its genius, when 256-bit SVE2 hardware becomes widespread, but for now - don't bother.</p>
ARM doesn't have an answer to AVX-512, with its 4x larger register space and four 512-bit execution units for crunching through 2048 bits at once. But considering their target markets, and that good AVX-512 ends up bottlenecked by memory bandwidth anyway</a>, I'm not convinced ARM needs one. The main benefit of AVX-512 is a much wider range of supported operations, which NEON already has.</p>
RISC-V</i></a> 
</h3>
RISC-V vectors (RVV) are completely irrelevant because vector-capable RISC-V hardware is completely irrelevant. RISC-V is dominating in cheap microcontrollers, but the performance/price ratio for vector-capable RISC-V hardware in 2026 is abysmal. Maybe Tenstorrent will change that in 2028 or so when they actually tape out some silicon, but you definitely don't have to worry about it in 2026.</p>
Even if decent hardware existed, the way the spec is written makes certain crucial instructions unusable to compilers</a>. This alone degrades performance to ridiculous levels unless you mess with obscure compiler flags.</p>
Others</i></a> 
</h3>
The remaining SIMD-capable architectures are so obscure that you shouldn't bother thinking about them unless someone is paying you to work on them - IBM and/or banks in case of POWER and s390x, or the Chinese government in the case of LoongArch. std::simd</code> still runs CI on 64-bit SPARC, but nobody's going to pay you to support that.</p>


Implementing FMA and finding bugs in C and Rust standard libraries
2026-08-20T00:00:00+00:00
This is a story of how I tried to compute a * b + c</code>, and found that Rust and musl libc get it subtly wrong.</p>
</span>
Fused multiply-add</strong> (FMA) computes a * b + c</code> with only one rounding error instead of two. It's an important building block for e.g. trigonometric functions like sin(x)</code> and cos(x)</code> if you want to implement them accurately.</p>
It's a basic primitive that is usually implemented in hardware, but there is still some hardware out there that doesn't have it. You'd think it would be something like cheap phones, but no, it's Intel. Cheap ARM phones have it and it's been required in 64-bit ARM since the very beginning, but Intel has been launching new parts without AVX2 or fused multiply-add as recently as 2021</a>.</p>
(I've come to learn that whenever something is holding SIMD back, it's usually Intel.)</p>
15% of machines in the Firefox hardware survey</a> don't have AVX2 and hardware FMA that comes with it, so it has to be emulated for precise algorithms built on top of it to work correctly.</p>
On machines without hardware FMA, Rust's std::simd</code> gives up and runs scalar FMA on each f32</code> in [f32; 4]</code> individually, which is slow. I wanted to do better in fearless_simd</a> and provide an actually vectorized implementation.</p>
Emulating FMA with SIMD</i></a> 
</h2>
Since FMA works on three f32</code> values, the total number of possible inputs is 2 to the 96th power. It would take the world's largest supercomputer only 500 years to try them all. We've come a long way! But I need working FMA later this year, so exhaustive verification isn't really on the cards. The best I could do is some known values plus some random tests.</p>
I followed the 2008 paper</a> "Emulation of FMA and correctly-rounded sums: proved algorithms using rounding to odd" by Sylvie Boldo and Guillaume Melquiond, which has a formal proof of correctness in Coq. That way I don't have to worry about trying to verify the algorithm myself.</p>
For f32</code>, their algorithm is refreshingly simple: compute a * b + c</code> in f64</code>, then round it to f32</code>. The only caveat is special handling of rounding errors in the conversion, in case the result falls exactly between two representable values.</p>
Translating the algorithm to SIMD was also straightforward: just do all that basic math per-lane. The special handling of rounding is needed very rarely - you hit it less than once in a million when processing values in the [-1, 1) range, so just put it under an if</code> and it'll be fine. Done!</a></p>
Benchmarks look great: it's 5x faster than std::simd</code>, even bigger than the expected 4x speedup because Rust standard library has to check if FMA is available on the system in every call, and also has to worry about setting floating-point exception flags, both of which add overhead.</p>
Follow the White Rabbit</i></a> 
</h2>
Just a few hours later, a wild @awxkee</strong></a> appeared and posted some inputs</a> on which my implementation diverged from the hardware results.</p>
Moments like these are why I love open source. I have no idea where he came from or how he even found this PR, he's never contributed code to Fearless SIMD before. But there it was, a counter-example that broke my translation of a formally verified algorithm.</p>
Turns out I translated the paper into code incorrectly. I forgot to add special handling for values very close to zero, called subnormal</strong> values, which have a slightly different representation, and many of the usual floating-point "tricks" don't work on them. The check that would apply the rounding fixup handled them incorrectly.</p>

Aside:</strong> Subnormals are fascinating! Learn all about them here</a>.</p>
</blockquote>
So I looked at the paper more carefully and fixed my implementation to match it more closely. Then I added tests covering the problematic inputs, random tests that try a million subnormals, and random tests that try a million values that should require fixup, for good measure.</p>
Wait... Why are tests still failing, but only</em> on old systems?</p>
Down the rabbit hole</i></a> 
</h2>
Fearless SIMD has CI configured to run tests in an emulator for every supported SIMD level, on top of running them on the host. This is the only way to test AVX-512 codepaths on CI. It would also fail if we tried to use instructions not available in hardware, although the Rust compiler already verifies this for us</a>. It's just good practice to test the codepaths on the hardware where they'd actually run.</p>
The built-in f32::mul_add</code> in the Rust standard library has the exact same bug!</strong></p>
It fails to properly handle rounding for subnormals. As does std::simd</code>. But the software implementation is only invoked for systems that don't have FMA in hardware, so without the emulator we wouldn't have noticed.</p>
So I report the bug</a> to the Rust standard library, then transcribe the formally verified algorithm from the paper</a> again (hopefully correctly) to insulate the scalar fallback inside fearless_simd</code> from it. Tests pass.</p>
Here's the algorithm, it's not that scary:</p>
fn</span> scalar_mul_add_precise_f32</span>(</span>a</span>:</span> f32</span>,</span> b</span>:</span> f32</span>,</span> c</span>:</span> f32</span>)</span> -></span> f32</span> {</span></span>
    let</span> product</span> =</span> (</span>a</span> as f64</span>)</span> *</span> (</span>b</span> as f64</span>);</span></span>
    let</span> c</span> =</span> c</span> as f64</span>;</span></span>
    let mut</span> sum</span> =</span> product</span> +</span> c</span>;</span></span>
    if</span> sum</span>.</span>is_finite</span>() {</span></span>
        let</span> virtual_sum</span> =</span> sum</span> -</span> product</span>;</span></span>
        let</span> rounding_error</span> =</span> (</span>product</span> -</span> (</span>sum</span> -</span> virtual_sum</span>))</span> +</span> (</span>c</span> -</span> virtual_sum</span>);</span></span>
        let</span> sum_bits</span> =</span> sum</span>.</span>to_bits</span>();</span></span>
        if</span> rounding_error</span> !=</span> 0</span>.</span>0</span> &&</span> sum_bits</span> &</span> 1</span> ==</span> 0</span> {</span></span>
            let</span> corrected_bits</span> =</span> if</span> sum</span>.</span>is_sign_negative</span>()</span> ==</span> rounding_error</span>.</span>is_sign_negative</span>() {</span></span>
                sum_bits</span>.</span>wrapping_add</span>(</span>1</span>)</span></span>
            }</span> else</span> {</span></span>
                sum_bits</span>.</span>wrapping_sub</span>(</span>1</span>)</span></span>
            };</span></span>
            sum</span> =</span> f64</span>::</span>from_bits</span>(</span>corrected_bits</span>);</span></span>
        }</span></span>
    }</span></span>
    sum</span> as f32</span></span>
}</span></span></code></pre>
Just the Rust standard library left to fix.</p>
How deep does this go?</i></a> 
</h2>
Fixing the standard library is trickier because it not only computes the result but also sets the floating-point status flags</a>. It also has this codepath not just for f32</code> using f64</code> but also for f64</code> using f128</code> on hardware where that's available.</p>
So I write tests, for f32</code> and f64</code> both, then fix the code</a> to the best of my understanding.</p>
Wait, what's this at the top of the file?</p>
/* origin: musl src/math/fmaf.c Ported to generic Rust algorithm in 2025, TG. */</span></span></code></pre>
I wonder if...</p>
if</span> ((</span>u.i </span>&</span> 0x</span>1fffffff</span>)</span> !=</span> 0x</span>10000000</span>)</span> /* not a halfway case */</span></span></code></pre>
Yep, that's the same bug: this check ignores subnormals.</p>
So it's not just Rust std that's buggy, musl libc is buggy too</strong>. And they credit this implementation to FreeBSD</a>, with copyright years 2005-2011, so who knows where else this buggy code was copy-pasted over the past two decades.</p>
Musl has no bug tracker, so let's report it</a> on the mailing list, and pray that it isn't an intentional "optimization" (the mailing list has no search function so I can't check) and that it won't just be forgotten forever once it stops showing up on the recent messages page. Boy do I love development processes from 30 years ago!</p>
Where is the bottom?!</i></a> 
</h2>
Oh hey, CI checks for my standard library PR are complete!</p>
Wait, why are they failing, but only on 32-bit ARM? With completely garbage values?!</p>
Somehow the experimental, nightly-only f128</code> type gets enabled on 32-bit ARM platforms, and... goes completely haywire? I sure am not going to read that much ARM assembly, but an LLM can make quick work of it</a>, and... it's a rustc ABI bug. On this type, on this architecture, localized entirely to my pull request.</p>

Aside:</strong> In case you're wondering, using LLMs for analysis is permitted</a> by the Rust LLM policy, under certain conditions.</p>
</blockquote>
So there's now a bug report about this too</a> from a maintainer who could verify the LLM findings and understands this area way better than I do.</p>
Fixes</i></a> 
</h2>
fearless_simd</code> is easy. It's fixed. You can use that and get FMA without this bug.</p>
The nightly-only f128</code> bug in the Rust compiler is also fixed.</p>
My pull request for the Rust standard library</a> broke setting floating-point exception flags in some cases. It occurred to me to run an LLM to search for bugs I could have introduced, and it found a counter-example. So I've put some more work into making a patch for it that touched as little code as possible. It's still awaiting review.</p>
To my surprise, musl developers took my bug report seriously. Both a minimal fix and a substantial rewrite of the fmaf()</code> function were quickly proposed and iterated upon based on feedback from other developers. And after all the feedback was addressed... nothing happened. It's still not merged as I'm writing this.</p>
Looks like code review is a bottleneck regardless of the age of your development tools.</p>
Does this bug matter?</i></a> 
</h2>
I have no idea.</p>
On one hand, it's just an incorrect rounding. The result is off by one least significant bit in some rare cases. The original counter-example is off by about 15 parts per million of the correct result.</p>
On the other hand, the error might get amplified dramatically depending on what you do with the result. And those math routines with exact error bounds are used for something</em>, so whatever that is will get the wrong error bounds.</p>
Perhaps most noticeably, deterministic simulations running across different machines will diverge.</p>
Buggy FMA? In MY computer?</i></a> 
</h2>
On x86 this is only a problem on cheap and/or very old Intel without AVX2, and on Chinese Hygon x86 chips that don't have FMA at all. 64-bit ARM is unaffected.</p>
However, this will probably haunt 32-bit ARM and various embedded systems for years to come. I would not be surprised if the buggy FreeBSD implementation got copy-pasted into lots of different toolchains for various obscure platforms.</p>
If FMA accuracy matters to you, check that fmaf(a, b, c)</code> with these input bit patterns</p>
a</span> =</span> 0x97000800</span></span>
b</span> =</span> 0x1cfff001</span></span>
c</span> =</span> 0x00010002</span></span></code></pre>
evaluates to bit pattern 0x00010001</code> (correct) rather than 0x00010002</code> (buggy).</p>


Improving std::simd::swizzle_dyn
2026-06-20T00:00:00+00:00
Fearless SIMD</a> recently received a pull request</a> that started like this:</p>

Swizzles are a whole can of worm and I do not intend to figure out how we should implement this generically right now. 😄</p>
</blockquote>
</span>
The pull request author only wanted to convert RGBA image layouts, e.g. RGBA <-> BGRA, so we solved the immediate need by implementing a simpler version</a> that shuffles bytes within 128-bit blocks. That way we can trivially support this operation even on 512-bit vectors on 128-bit hardware, and 512-bit hardware still gets to utilize its full potential with native instructions.</p>
But that made me wonder: how does std::simd</code></a> implement arbitrary swizzles that aren't confined to 128-bit blocks? Turns out the answer is: poorly.</strong></p>
Over the course of this article we're going to fix some of that.</p>
What is a swizzle?</i></a> 
</h2>
A swizzle, or a shuffle, rearranges elements within an array.</p>
For example, if I have the input array [A,D,F,I,M,R,S,T]</code> and apply the swizzle mask [2,0,6,7,6,3,4,1]</code>, I get [F,A,S,T,S,I,M,D]</code>.</p>
The dyn</code> part indicates that the swizzle mask is dynamic (not hardcoded), so you could supply the swizzle mask [6,4,0,5,7,0,6,6]</code> at runtime to get a different word.</p>
This example uses only 8 bytes, or 64 bits. In practice hardware implements shuffles on vector sizes from 128 to 512 bits, and so does std::simd</code>.</p>
Understanding the implementation</i></a> 
</h2>
On a high level, the current implementation</a> of std::simd::swizzle_dyn</code> is very simple: use the native hardware operation if available, otherwise give up and move bytes one by one.</p>
Even when hardware shuffles for a given size are available, swizzle_dyn</code> doesn't always map to the hardware cleanly. It promises that the values for out-of-bounds indices will be set to 0</code>, while x86 hardware shuffle just lets them wrap, so swizzle_dyn</code> has to do extra work to uphold this guarantee.</p>
Wait a minute... what is this warning in its documentation?</p>

Note that the current implementation is selected during build-time of the standard library, so cargo build -Zbuild-std</code> may be necessary to unlock better performance, especially for larger vectors. A planned compiler improvement will enable using #[target_feature]</code> instead.</p>
</blockquote>
Uh oh.</p>
The black sheep of std::simd</i></a> 
</h2>
Rust uses LLVM</a> for optimizing your code and turning into something CPUs can execute. And LLVM doesn't like dealing with platform-specific operations. It's much easier to implement an operation like "add two SIMD vectors" generically, run all optimizations on this generic form, and then select the appropriate instructions for a given CPU later.</p>
Most std::simd</code> operations simply emit the generic form such as "add two SIMD vectors", which makes the implementation remarkably straightforward.</p>
But swizzle_dyn</a> does not. And the implementation</a> looks something like this:</p>
#</span>[</span>cfg</span>(</span>target_feature </span>=</span> "neon"</span>))]</span></span>
unsafe fn</span> armv7_neon_swizzle_u8x16</span>(</span>bytes</span>:</span> Simd</span><</span>u8</span>,</span> 16</span>></span>,</span> idxs</span>:</span> Simd</span><</span>u8</span>,</span> 16</span>></span>)</span> -></span> Simd</span><</span>u8</span>,</span> 16</span>></span> {</span></span>
    use</span> core</span>::</span>arch</span>::</span>arm</span>::</span>{</span>uint8x8x2_t</span>,</span> vcombine_</span>u8</span>,</span> vget_high_</span>u8</span>,</span> vget_low_</span>u8</span>,</span> vtbl2_</span>u8</span>};</span></span>
    unsafe</span> {</span></span>
        let</span> bytes</span> =</span> uint8x8x2_t</span>(</span>vget_low_u8</span>(</span>bytes</span>.</span>into</span>()),</span> vget_high_u8</span>(</span>bytes</span>.</span>into</span>()));</span></span>
        let</span> lo</span> =</span> vtbl2_u8</span>(</span>bytes</span>,</span> vget_low_u8</span>(</span>idxs</span>.</span>into</span>()));</span></span>
        let</span> hi</span> =</span> vtbl2_u8</span>(</span>bytes</span>,</span> vget_high_u8</span>(</span>idxs</span>.</span>into</span>()));</span></span>
        vcombine_u8</span>(</span>lo</span>,</span> hi</span>)</span>.</span>into</span>()</span></span>
    }</span></span>
}</span></span></code></pre>
I've seen that before! That's no LLVM magic, even if it's completely inscrutable. Those are plain old SIMD intrinsics!</p>
The problem is those platform-specific intrinsics are under #[cfg(target_feature = ...))]</code>. It's that cfg</code> that kills performance, because it gets resolved very early in the build process, before the context from which the function is called is considered. fearless_simd</code> and every other SIMD crate that isn't called wide</code></a> has a solution</a> for this, but std::simd</code> doesn't, and it can't just adopt the ecosystem solution without a large change to the API.</p>
The warning in documentation doesn't convey the full extent of the problem. Not only is -Z build-std</code> necessary, you also have to pass RUSTFLAGS=-C target-cpu=</code> to get decent performance. But if you do that and you run the program on a CPU that's older than the one you selected, the program will crash. The typical solution is selecting the best implementation at runtime via crates such as multiversion</code></a>, but that does not work for swizzle_dyn</code>, even with -Z build-std</code>.</p>
And let me be clear: I'm not a compiler engineer. I'm not the one who's going to build whatever compiler improvement is planned to fix this.</p>
But I am a fearless_simd</code> maintainer, and I already know more about SIMD intrinsics than ever wanted to. So there is plenty here that I can improve.</p>
No NEON?</i></a> 
</h2>
The first thing that stuck out to me when reading the swizzle_dyn code</a> is that it only implements 128-bit swizzles on ARM NEON, even though NEON does have hardware instructions for larger swizzles.</p>
It turned out that std::simd</code> isn't actually developed as part of the Rust standard library - it has its own repository</a> where development happens, and the result is periodically synced into the standard library. And the missing NEON implementations were already implemented there</a>.</p>
That wasn't at all apparent when browsing the code. I nearly wasted a lot of time making pull requests for an outdated version of the code in the wrong repository. Adding headers to synced-in files that specify how to contribute to them would save other people in my position a lot of time.</p>
Doubling the vector width</i></a> 
</h2>
Let's say we want to execute [A,D,F,I,M,R,S,T].swizzle_dyn([2,0,6,7,6,3,4,1])</code> but all we have is half-width vectors. swizzle_dyn</code> setting out-of-bounds elements to zero is going to help us here.</p>
To process the first half, [2,0,6,7]</code>, we run [A,D,F,I].swizzle_dyn([2,0,6,7])</code> and [M,R,S,T].swizzle_dyn([-2,-4,2,3])</code>, which gives us [F,A,0,0]</code> and [0,0,S,T]</code>. Then we combine these two intermediate results with a cheap bitwise OR, et voilà! We get [F,A,S,T]</code>! Now repeat the same for the second half, and we're done!</p>
It is rare that a portability guarantee actually helps optimize something instead of incurring additional work, but this time we're in luck!</p>
This takes four shuffles instead of one, but it's still much faster than moving bytes around one at a time. I've measured a 4x performance improvement from implementing this on AVX2 for 512-bit vectors inside fearless_simd</code>.</p>
This is now merged</a> into std::simd</code>, and will be available in the next release of fearless_simd</code> too.</p>
Using this approach for 4x the vector size would require 16 shuffles, so I don't think it's worth it, but it might still be worth benchmarking.</p>
Optimizing AVX2</i></a> 
</h2>
AVX2 is weird. It has 256-bit vectors, but most operations process 128-bit halves independently. That's fine for most operations, but problematic for a swizzle that may move values across those halves. So the AVX2 implementation needs two separate steps: shuffling within 128-bit blocks, and then using a trick similar to what we've just done.</p>
Another complication is that swizzle_dyn</code> promises that values for out-of-bounds indices will be set to 0</code>, which x86 hardware shuffles don't do, so we have to implement that in software.</p>
The current implementation</a> looks like this:</p>
use</span> x86</span>::</span>_mm256_permute2x128_si256 </span>as</span> avx2_cross_shuffle</span>;</span></span>
use</span> x86</span>::</span>_mm256_shuffle_epi8 </span>as</span> avx2_half_pshufb</span>;</span></span>
</span>
let</span> mid</span> =</span> Simd</span>::</span>splat</span>(</span>16</span>u8</span>);</span></span>
let</span> high</span> =</span> mid</span> +</span> mid</span>;</span></span>
</span>
// This is ordering sensitive, and LLVM will order these how you put them.</span></span>
// Most AVX2 impls use ~5 "ports", and only 1 or 2 are capable of permutes.</span></span>
// But the "compose" step will lower to ops that can also use at least 1 other port.</span></span>
// So this tries to break up permutes so composition flows through "open" ports.</span></span>
// Comparative benches should be done on multiple AVX2 CPUs before reordering this</span></span>
</span>
let</span> hihi</span> =</span> avx2_cross_shuffle</span>::</span><</span>0x11</span>></span>(</span>bytes</span>.</span>into</span>(),</span> bytes</span>.</span>into</span>());</span></span>
let</span> hi_shuf</span> =</span> Simd</span>::</span>from</span>(</span>avx2_half_pshufb</span>(</span></span>
    hihi</span>,</span>        // duplicate the vector's top half</span></span>
    idxs</span>.</span>into</span>(),</span> // so that using only 4 bits of an index still picks bytes 16-31</span></span>
));</span></span>
// A zero-fill during the compose step gives the "all-Neon-like" OOB-is-0 semantics</span></span>
let</span> compose</span> =</span> idxs</span>.</span>simd_lt</span>(</span>high</span>)</span>.</span>select</span>(</span>hi_shuf</span>,</span> Simd</span>::</span>splat</span>(</span>0</span>));</span></span>
let</span> lolo</span> =</span> avx2_cross_shuffle</span>::</span><</span>0x00</span>></span>(</span>bytes</span>.</span>into</span>(),</span> bytes</span>.</span>into</span>());</span></span>
let</span> lo_shuf</span> =</span> Simd</span>::</span>from</span>(</span>avx2_half_pshufb</span>(</span>lolo</span>,</span> idxs</span>.</span>into</span>()));</span></span>
// Repeat, then pick indices < 16, overwriting indices 0-15 from previous compose step</span></span>
let</span> compose</span> =</span> idxs</span>.</span>simd_lt</span>(</span>mid</span>)</span>.</span>select</span>(</span>lo_shuf</span>,</span> compose</span>);</span></span>
compose</span></span></code></pre>
That's pretty clever. The core insight is this:</p>
CPUs have a bunch of different hardware curcuits, and the circuit that handles shuffles is entirely separate from a circuit that handles arithmetic, so you can run them both at the same time. This is known as "instruction-level parallelism</a>".</p>
The above code is carefully tuned to spread the work across different circuits in such a way that it doesn't pile too much work on a single circuit (aka "port") and take advantage of this parallelism to speed things up.</p>
You can measure the effectiveness of this by viewing the resulting assembly with cargo-show-asm</a> and feeding the result to llvm-mca</a> (Machine Code Analyzer), which models the execution of the given assembly on a specific CPU model accounting for all of these effects.</p>
According to llvm-mca, spreading operations across different kinds of ports is important on the early Intel CPUs with AVX2, namely Haswell and Broadwell. On the other end of the spectrum, recent Intel Tiger Lake and all AMD Zen generations are so stuffed with all kinds of circuits that they favor doing less work overall, and you don't have to worry about overwhelming one circuit anymore. Intel Skylake is indifferent, dealing with all formulations in an equally mediocre fashion.</p>
It's easy to get distracted arguing over which platform should be prioritized and optimized for. On one hand, recent CPUs are much more common these days, so optimizing for them benefits more people. On the other, the older CPUs are underpowered by today's standards and need all the help they can get.</p>
We can sidestep that entirely by using a clever algorithm that eliminates the compare-select steps, which improves performance on recent CPUs without regressing it on antiquated ones:</p>
use</span> x86</span>::</span>_mm256_permute2x128_si256 </span>as</span> avx2_cross_shuffle</span>;</span></span>
use</span> x86</span>::</span>_mm256_shuffle_epi8 </span>as</span> avx2_half_pshufb</span>;</span></span>
</span>
let</span> lolo</span> =</span> avx2_cross_shuffle</span>::</span><</span>0x00</span>></span>(</span>bytes</span>.</span>into</span>(),</span> bytes</span>.</span>into</span>());</span></span>
let</span> hihi</span> =</span> avx2_cross_shuffle</span>::</span><</span>0x11</span>></span>(</span>bytes</span>.</span>into</span>(),</span> bytes</span>.</span>into</span>());</span></span>
</span>
// Adding 0x60 preserves the low nibble and bit 4 for valid</span></span>
// indices 0..=31. Larger indices get their high bit set, so</span></span>
// VPSHUFB supplies the required out-of-bounds zeroing.</span></span>
let</span> control</span> =</span> x86</span>::</span>_mm256_adds_epu8</span>(</span>idxs</span>.</span>into</span>(),</span> x86</span>::</span>_mm256_set1_epi8</span>(</span>0x60</span>));</span></span>
</span>
// Move index bit 4 into each byte's sign bit for VPBLENDVB.</span></span>
let</span> select_high</span> =</span> x86</span>::</span>_mm256_slli_epi16</span>::</span><</span>3</span>></span>(</span>control</span>);</span></span>
let</span> from_low</span> =</span> avx2_half_pshufb</span>(</span>lolo</span>,</span> control</span>);</span></span>
let</span> from_high</span> =</span> avx2_half_pshufb</span>(</span>hihi</span>,</span> control</span>);</span></span>
x86</span>::</span>_mm256_blendv_epi8</span>(</span>from_low</span>,</span> from_high</span>,</span> select_high</span>)</span>.</span>into</span>()</span></span></code></pre>
llvm-mca shows no change on Haswell and Broadwell, but Zen 1, Zen 2 and Tiger Lake show a 23% improvement, and a 7-20% improvement on Zen 3. Skylake is unchanged for the 256-bit shuffle, but the decomposition of a 512-bit shuffle into four 256-bit ones improves by 20%.</p>
I'm glossing over the throughput vs latency</a> distinction here, since this formulation improves both.</em></p>
This change is currently under review</a> in std::simd</code>, and fearless_simd</code> will ship with this formulation from the start.</p>
Was it worth it?</i></a> 
</h2>
Performance of a single SIMD operation in isolation does not necessarily translate to improvements in algorithms using it.</p>
For example, AVX2 only has 16 SIMD registers to hold data the CPU can directly operate on. If your operation runs faster but uses a lot more registers, the entire algorithm it's in may run out of registers and have to spill data onto the stack, which adds costly load/store instructions, which negates the gains or even slows down the entire algorithm!</p>
We need to measure performance of some plausible code that uses 512-bit swizzles. 512 bits is 64 bytes, so base64</a> encoding/decoding is a natural fit.</p>
I'm not aware of any shuffle-based Rust implementations of base64 that I could simply reuse. But it's easy enough to have an LLM conjure a prototype</a>, for educational purposes only. Here's how it performs with different shuffle implementations:</p>
scalar fallback:   1.6 GiB/s encode, 0.9 GiB/s decode</span></span>
AVX2 double-width: 8.2 GiB/s encode, 5.5 GiB/s decode</span></span>
AVX2 dw optimized: 9.4 GiB/s encode, 5.9 GiB/s decode</span></span></code></pre>
That's a ~6x improvement just from the work described in this article!</p>
Let's see how it stacks up against Sunny Young's base64 using std::simd</a>, which uses a lot of clever tricks and you should totally go read their article about it</a>:</p>
AVX2 vb64: 5.9 GiB/s encode, 5.7 GiB/s decode</span></span></code></pre>
Brute-forcing the work with emulated swizzles and landing in the same ballpark as an implementation that carefully avoids them using a clever perfect hash</a> is not too shabby! And ours have the added benefit of swapping out the base64 alphabet at will, while vb64</code>'s perfect hash is tied to the standard base64 alphabet.</p>
A production implementation without SIMD</a> runs at 2.7 GiB/s encode and 2.3 GiB/s decode, so SIMD is well worth it. Our implementation on an AVX-512 CPU runs at ~13 GiB/s, while vb64 doesn't benefit from AVX-512 nearly as much. A state-of-the art</a> AVX-512 implementation translated to safe Rust</a> runs at 35 GiB/s encode and 20 GiB/s decode, so our prototype is not optimal, but still lets us see improvements to individual operations benefit a larger algorithm.</p>
I've run these measurements on my Zen 4 workstation on fearless_simd</code>, because the contributing guide for std::simd</code> doesn't say how to benchmark your changes.</p>
Is fearless_simd better?</i></a> 
</h2>
For this one specific operation, swizzle_dyn</code>, yes. The implementations are now identical in both std::simd</code> and fearless_simd</code>, but the latter doesn't require -Z build-std</code> and can multiversion this function just fine.</p>
But when std::simd</code> is playing to its strengths, it is really hard to beat or even match.</p>
For example, std::simd::swizzle</code></a> that accepts indices known at compile time translates directly into an LLVM "shuffle vector" operation, and LLVM selects the best formulation for your CPU out of a long list of special cases.</p>
Replicating this on stable Rust would require a lot of work, and either a lot of const evaluation or a procedural macro, both of which would negatively impact build time. To the best of my knowledge, nobody has even attempted this yet.</p>
That said, LLVM is not perfect. For example, I've found that it destroys performance of certain AVX-512 shuffles by turning them into a lot of SSE operations instead of just keeping it as a native AVX-512 shuffle. I've reported this as an issue in Rust</a>, then a Rust standard library maintainer reported it to LLVM</a> and attempted to fix it</a>, and that led to a whole</a> bunch</a> of improvements</a> made by LLVM developers. This work will ship in LLVM 23.</p>
I've already uncovered another LLVM inefficiency</a>, but the PR to fix it</a> just sends LLVM into an infinite loop. So here's a fun debugging puzzle if you're looking for one!</p>
LLVM issues affect std::simd</code>, all other SIMD crates, and even other languages like C and C++, so fixing them is always a big win.</p>
Takeaways</i></a> 
</h2>
So here's what I've learned:</p>

std::simd::swizzle_dyn</code> is even less useful than it appears, since multiversioning doesn't work on it.</li>
std::simd</code> on nightly is outdated compared to the actual development state.</li>
The contributing documentation for std::simd</code> is poor:

It's easy to open PRs for an outdated version in the wrong repository.</li>
There's no documented way to disassemble or benchmark your changes.</li>
</ul>
</li>
LLVM can mangle perfectly optimal SIMD code, even if you use intrinsics that are supposed to lower to a specific instruction. Not just in Rust, in C and C++ too!</li>
</ul>
And, well, std::simd::swizzle_dyn</code> will be up to 6x faster once this work gets synced into the standard library. You're welcome!</p>
Addenum</i></a> 
</h2>
Since this article was originally written, I've also found a better formulation</a> of swizzle_dyn</code> for AVX-512 with VBMI, and Cristi Vîjdea</a> suggested a better formulation</a> for ssse3. And together with @dzaima</a> we've found an even better formulation</a> for AVX2. All of these are now merged into std::simd</code>.</p>
There is also an ongoing effort</a> to add a native LLVM operation for swizzle_dyn</code>, the timing of which coincided with my work. This would alleviate the miltiversioning issue in std::simd</code>, and allow natively targeting even more architectures.</p>
There's also a separate proposal</a> to add a Rust compiler feature that delays target feature checks, which could allow the current intrinsics-based implementation to be multiversioned.</p>
In the meantime fearless_simd</code> picked up the swizzle function matching std::simd</code> behavior, and a cheaper variant that doesn't return zero out-of-bounds indices. The cheaper variant produces arbitrary (but memory-safe) values instead. That way you don't have to pay for the zeroing if you don't use it.</p>


The unreasonable effectiveness of LLMs for auditing Rust code
2026-06-20T00:00:00+00:00
As a lead of the Rust Secure Code Working Group</a>, I got free access to GPT-5.5 via the Codex for Open Source</a>. Since then I’ve found and reported dozens of issues of varying severity in widely used Rust crates.</p>
</span>
Separately, the Rust Foundation security initiative</a> got access to Mythos</a> via Project Glasswing</a>, and their report should also be coming soon. I’ve coordinated with them so that our audit targets would not overlap.</p>
While I haven’t found any truly devastating vulnerabilities, I am very impressed with GPT-5.5 for auditing Rust source code, and I’ll absolutely be adding it to my toolkit alongside fuzzers.</p>
Note:</strong></em> All opinions expressed in this article are my own, not that of any organizations I am a part of.</em></p>
Methodology</i></a> 
</h2>
Since maintainers could already be dealing with a large amount of vulnerability reports, it is imperative</em> that I do not submit any invalid vulnerability reports and waste maintainers’ already limited time.</p>
So I’ve decided to look for unambiguously problematic class of vulnerability that’s easy to verify: memory safety bugs.</p>
Wait, isn’t Rust memory-safe?</i></a> 
</h3>
Yes, with an asterisk.</p>
Most code you’d write in Rust is memory-safe, but at some point you have to talk to the operating system or a C library or implement things like intrusive data structures, all of which involves raw pointers.</p>
Most languages implement these parts in C (e.g. CPython), provide unsafe interoperability with C, and have you write C for your own unsafe code, while Rust has its own unsafe subset where you can muck about with raw pointers.</p>
This puts Rust’s memory safety properties on par with Python’s, ahead of Go which violates safety on data races</a>, and behind browser-sandboxed JavaScript (but you can match that by compiling Rust to WebAssembly).</p>
As for the amount of safe vs unsafe code in the wild, my own scan from 2020</a> showed that 95% of the code on crates.io</a> is memory safe. The authors of the 2020 paper “How do programmers use unsafe Rust?</a>” independently arrived to the 95% number, although they didn’t put it into the final paper because they weren’t confident in their methodology for it. My own scan is also rather crude, but two completely different measurements arriving to the same number is encouraging.</p>
In practice memory safety vulnerability rate reduction compared to C++ is about 1000x</a>, which is more than you’d expect based on the above figures.</p>
Preventing false positives</i></a> 
</h3>
Rust has a tool called miri</a> that runs Rust code in an interpreter and tells you precisely whether it committed any crimes against the language rules or not. Safe Rust cannot violate them by construction, but unsafe Rust can, and a validator that immediate tells you whether you messed up or not instead of having to parse dozens of pages of dense prose is indispensable.</p>
It also completely eliminates false positives</strong> from LLM vulnerability findings.</p>
If the LLM can construct a unit test that causes miri to fail, I can report that to the maintainers and be certain that it’s a bug. I don’t ever have to argue if it’s a real issue or not, either — the proof is right there. And if miri says the execution is completely fine, then the LLM false positive gets discarded before anyone even sees it.</p>
To the best of my knowledge, no other language</strong> has a practical tool with this level of precision. Sanitizers</a> are very nice, but can’t catch everything, so verifying against them does not prove absence of issues.</p>
Sadly miri is not without limitations — execution with extra checks is slow, calling into C is not supported, and syscall support is limited. When miri is not applicable, you can fall back on the sanitizers</a> and get some</em> filtering.</p>
I also had to switch miri to the newer Tree Borrows</a> aliasing model (as opposed to the older Stacked Borrows</a>) to avoid false positives, but fortunately that’s just one flag, -Zmiri-tree-borrows.</p>
Harness</i></a> 
</h3>
My setup was very basic: just Codex and a prompt</a>, with GPT-5.5 set to xhigh reasoning effort. It is important for the model to be able to write and run unit tests to try and trigger the issue under miri, so I consider something like Codex essential.</p>
It would be interesting to try a more elaborate harness like metis</a>, but even this basic setup was enough to discover interesting bugs.</p>
Findings</i></a> 
</h2>
The most serious issue I’ve found is an out-of-bounds write in</a> jxl-grid</a> crate</a>. It is a part of jxl-oxide</a>, a JPEG XL decoder in Rust (not to be confused with jxl-rs</a>, which Firefox and Chromium are adopting for JPEG XL decoding; that one came up clean in my audit).</p>
I’ve already fuzzed this crate earlier, but the fuzzer didn’t catch this issue because it only happens on 32-bit platforms and requires very large image dimensions. I wasn’t running the fuzzer on 32-bit, and the fuzzer is limited to small image dimensions to avoid exhausting my computer’s RAM, so it never had a chance to trigger this condition at runtime.</p>
The initial demonstrator showed this as a jxl-grid issue, but it was not clear if it’s a theoretical problem or if it can be triggered by decoding a crafted image. GPT-5.5 helped analyze that too</a>, and it turned out to be reachable. This was very valuable information to correctly prioritize the bug.</p>
This still isn’t that big a deal in practice because old 32-bit devices and very recent and computationally intensive image formats rarely meet, but it does showcase the capability of the tool quite well.</p>
Here are some more samples of bugs GPT-5.5 has discovered, showing the breadth of the kinds issues it has found:</p>


Use-after-free</a>, data races</a> and panic safety issues</a> in intrusive-collections</p>
</li>

Out-of-bounds reads</a> on deserializing crafted archives in rkyv</p>
</li>

Exposing uninitialized memory in serialized data</a> in wincode</p>
</li>

Incorrect</a> Send</a>/</a>Sync</a> impls</a> in yoke</p>
</li>

Construction of invalid enum values</a> in zerovec</p>
</li>

Data races for types with interior mutability</a> in hashlink</p>
</li>

Soundness issue in Gecko FFI</a> in thin_vec (crossing languages!)</p>
</li>

Out-of-bounds reads</a> in webp and another one</a> inwebpx (C library wrappers)</p>
</li>

Turning a</a> &</a> into a</a> &mut</a> in miette</p>
</li>

Multiple</a> &mut</a> pointing to the same memory</a> in nihav-core</p>
</li>

Aliasing violation</a> in arraydeque</p>
</li>

Incorrect alignment handling</a> in v_frame</p>
</li>

An interesting type confusion issue that’s not yet public</p>
</li>
</ul>
This demonstrates not just understanding of generic issues like out-of-bounds accesses, but also the ability to reason about Rust-specific concepts such as panic safety, aliasing, and the Send/Sync traits that enforce thread safety.</p>
I also got some results out of Claude before I got GPT-5.5 access:</p>


Use-after-free</a> in one function on a zero-capacity SmallVec (Opus 4.6)</p>
</li>

Multiple out-of-bounds reads</a> in imageproc (Opus 4.7)</p>
</li>
</ul>
I haven’t used Claude enough to be able to compare the models. I also cannot compare GPT-5.5 to Mythos, as interesting as that would be, because I deliberately picked different targets to avoid duplicate vulnerability reports putting extra load on maintainers.</p>
Fixes</i></a> 
</h2>
Once the issue is identified and explained, the model usually can also fix it autonomously. I avoid “fix the bugs you found” style prompts and instead discuss possible solutions with the model first, then have the model implement one of them.</p>
Submitting a possible fix alongside the vulnerability report puts less pressure on the maintainers. If you look up my pull requests, the first commit usually adds proof-of-concept tests that cause miri to complain, and the subsequent commit fixes the issues and turns the proof-of-concept snippets into regression tests.</p>
GPT-5.5 has also assisted me in locating the version where the bug was introduced, which is essential for security advisories.</p>
Reflections</i></a> 
</h2>
In my experience Rust code is a lot easier to audit than C code. In C, if I look at a line like data[a + b], I have to trace through the entire codebase and find all the possible values it can be set to just to validate this one line and make sure it doesn't have out-of-bounds accesses.</p>
Rust, even unsafe Rust, still relies on local reasoning: if I see unsafe { data.get_unchecked(a + b) }, then the function it's in must either validate a and b to make sure their addition is in-bounds, or be itself unsafe to call. Either way, there is a clear point where verification must happen</a> - and if it's not there, it's a bug.</p>
I don’t have to chase the data flow through the entire JPEG XL decoder by hand, and neither does an LLM. This reduces the complexity of auditing the code from combinatorial (all possible combinations of call trees) to linear (each function in isolation).</p>
In that light, it’s not terribly surprising that LLMs are so good at auditing Rust code. And it’s also not terribly surprising that I haven’t found any devastating vulnerabilities after all.</p>
Limitations</i></a> 
</h2>
Just like fuzzers before them, LLMs surface numerous bugs that weren’t economical to discover previously. But at the end of the day, no heuristic tool can prove the absence of vulnerabilities.</p>
For example, a GPT-5.5 alone didn’t discover several bugs</a> that a combination of a simpler LLM with a fuzzer did.</p>
But we don’t have to rely on heuristics. Rust without unsafe does guarantee the absence of memory safety bugs.</p>
So I’ll keep</a> shrinking</a> the unsafe surface where I can, and I'm glad to have these tools for when I can't.</p>
This article was originally published on my Medium</a></em></p>


Safe SIMD in Rust, even on the inside
2026-06-19T00:00:00+00:00
Rust’s</em> SIMD abstractions</em></a> were not as safe as I’d like. Until now.</em></p>
</span>
It’s no secret that raw SIMD intrinsics are unpleasant to use.</p>
You want to write a + b</code>, not this monstrosity:</p>
unsafe</span> {</span></span>
    #</span>[</span>cfg</span>(</span>all</span>(</span>any</span>(</span>target_arch </span>=</span> "x86"</span>,</span> target_arch </span>=</span> "x86_64"</span>),</span> target_feature </span>=</span> "avx2"</span>))]</span></span>
    _mm256_add_ps</span>(</span>a</span>,</span> b</span>)</span></span>
    #</span>[</span>cfg</span>(</span>all</span>(</span>any</span>(</span>target_arch </span>=</span> "x86"</span>,</span> target_arch </span>=</span> "x86_64"</span>),</span> target_feature </span>=</span> "sse"</span>,</span> not</span>(</span>target_feature </span>=</span> "avx2"</span>)))]</span></span>
    _mm_add_ps</span>(</span>a</span>,</span> b</span>)</span></span>
    #</span>[</span>cfg</span>(</span>all</span>(</span>target_arch </span>=</span> "aarch64"</span>,</span> target_feature </span>=</span> "neon"</span>))]</span></span>
    vaddq_f32</span>(</span>a</span>,</span> b</span>)</span></span>
}</span></span></code></pre>
Look at it. It’s hideous. And the whole thing is wrapped in unsafe</code>!</p>
And that’s a simplified example. It still doesn’t handle:</p>

Other common platforms: AVX-512, 32-bit ARM, WebAssembly</li>
Platforms without SIMD or obscure platforms like RISC-V</li>
Actually loading data like &[f32]</code> into a form that each intrinsic accepts</li>
Selecting the best implementation for the CPU it’s running on</li>
</ul>
Luckily, Rust provides many SIMD abstractions</a> that handle all of that for you and let you simply write a + b</code>.</p>
There is just one wrinkle. Inside, they’re still full of unsafe</code>. It wasn’t gone, just hidden. Vast quantities of it lurking just beneath the surface, getting screwed</a> up</a> occasionally</a>.</p>
Or rather, they were. Until now.</p>
Why do we even need ‘unsafe’?</i></a> 
</h2>
For the longest time you couldn’t get around wrapping the call to each intrinsic function such as _mm256_add_ps</code> into unsafe</code> because it is illegal to call one when it’s not available on the CPU you’re running on.</p>
So you had</em> to have some mechanism for tracking which instructions are needed for each intrinsic, and which instructions you have access to, and cross-referencing them to decide if it’s safe to call a given function.</p>
It was either tedious if done by hand or complex if done by a code generator, always error-prone, and required unsafe</code> around every intrinsic.</p>
This changed in Rust 1.87 when the compiler started tracking the required instruction sets itself, so you could write this:</p>
#</span>[</span>target_feature</span>(</span>enable </span>=</span> "avx2"</span>)]</span></span>
fn</span> add_avx2</span>(</span>a</span>:</span> __m256</span>,</span> b</span>:</span> __m256</span>)</span> -></span> __m256</span> {</span></span>
    _mm256_add_ps</span>(</span>a</span>,</span> b</span>)</span> // this is an avx2 intrinsic</span></span>
}</span></span></code></pre>
Look ma, no unsafe</code>!</p>
…yet.</p>
You still cannot write a + b</code> with this. The best you can do is this:</p>
unsafe</span> {</span> add_avx2</span>(</span>a</span>,</span> b</span>) }</span></span></code></pre>
This only shifts the unsafe</code> up a layer. You can call intrinsics inside functions annotated with the correct #[target_feature]</code> now, but there still has to be unsafe</code> somewhere in the chain.</p>
The other problem is more fundamental. You cannot put #[target_feature]</code> on the implementation of +</code> for your type, because +</code> must be available always. So no a + b</code> for us using this mechanism.</p>
Lemma: CPU feature tokens</i></a> 
</h2>
To understand how the final solution works, you first need to understand how CPU feature detection works.</p>
Normally, checking for a CPU feature like AVX2 is done at runtime using is_x86_feature_detected!("avx2")</code>. But we definitely don’t want to run this check every single time we add two numbers together — that would completely tank performance. We want to check it once</em>, and then prove to the compiler that it’s safe to use AVX2 instructions from that point on.</p>
Instead we can encode this proof into the type system using an unforgeable token</em>: a zero-sized type with a private inner field. The only</em> way to obtain this token is to call a function that performs the CPU feature check. If the check passes, the function hands you the token:</p>
pub struct</span> Avx2</span>(());</span></span>
</span>
fn</span> detect_avx2</span>()</span> -></span> Option</span><</span>Avx2</span>></span> {</span></span>
    if</span> is_x86_feature_detected!</span>(</span>"avx2"</span>) {</span></span>
        Some</span>(</span>Avx2</span>(()))</span></span>
    }</span> else</span> {</span></span>
        None</span></span>
    }</span></span>
}</span></span></code></pre>
And because it’s a zero-sized type, passing this token around has no runtime overhead. It exists purely as a compile-time proof.</p>
The upshot is that as long as you have an instance of the Avx2</code> struct, you can be sure that AVX2 instructions are available on the system.</p>
The key insight</i></a> 
</h2>
The compiler doesn’t know it, but this function is safe to call:</p>
#</span>[</span>target_feature</span>(</span>enable </span>=</span> "avx2"</span>)]</span></span>
fn</span> add_avx2</span>(</span>token</span>:</span> Avx2</span>,</span> a</span>:</span> __m256</span>,</span> b</span>:</span> __m256</span>)</span> -></span> __m256</span> {</span></span>
    _mm256_add_ps</span>(</span>a</span>,</span> b</span>)</span></span>
}</span></span></code></pre>
You can only</em> call this function if you have an Avx2</code> token, which you can only get if AVX2 instructions are available on the system.</p>
If we can explain to the compiler that this is valid (using unsafe</code>), we can write that unsafe</code> only once and reuse it everywhere.</p>
What we need is a macro which is safe to invoke:</p>
with_avx2!</span>(</span></span>
    fn</span> add_avx2</span>(</span>token</span>:</span> Avx2</span>,</span> a</span>:</span> __m256</span>,</span> b</span>:</span> __m256</span>)</span> -></span> __m256</span> {</span></span>
        _mm256_add_ps</span>(</span>a</span>,</span> b</span>)</span></span>
    }</span></span>
)</span></span></code></pre>
but expands into this behind the scenes:</p>
fn</span> add_avx2</span>(</span>token</span>:</span> Avx2</span>,</span> a</span>:</span> __m256</span>,</span> b</span>:</span> __m256</span>)</span> -></span> __m256</span> {</span></span>
    // SAFETY: Avx2 is available according to the token,</span></span>
    // and we verified that the inner function is not an `unsafe fn`</span></span>
    unsafe</span> {</span> inner</span>(</span>token</span>,</span> a</span>,</span> b</span>) }</span></span>
</span>
    #</span>[</span>target_feature</span>(</span>enable </span>=</span> "avx2"</span>)]</span></span>
    fn</span> inner</span>(</span>token</span>:</span> Avx2</span>,</span> a</span>:</span> __m256</span>,</span> b</span>:</span> __m256</span>)</span> -></span> __m256</span> {</span></span>
        _mm256_add_ps</span>(</span>a</span>,</span> b</span>)</span></span>
    }</span></span>
}</span></span></code></pre>
Now if you use an intrinsic that isn’t in AVX2, the compiler will reject it!</strong></p>
We’ve just managed to provide a safe programming interface to SIMD intrinsics without any bespoke tracking of target features!</p>
Even though there is one unsafe</code> block still inside it, it’s encapsulated in a sound API, so you cannot misuse it to cause memory safety bugs. In that sense it is just like println!</code>, safely abstracting unsafe code.</p>
This way you only ever need to review and audit this one macro,</em> not hundreds upon hundreds of bespoke unsafe</code> blocks. And the only things we could possibly</em> screw up in the implementation are:</p>

Mapping the token to the wrong #[target_feature]</code></li>
Allowing an unsafe fn</code> to be called from a safe context</li>
</ol>
And both failure modes are quite easy to check for.</p>
So now we can call add_avx2(token, a, b)</code> without unsafe</code>, but that still doesn’t get us to a + b</code>. How do we solve that?</em></p>
Generics to the rescue</i></a> 
</h2>
We cannot annotate the implementation of a + b</code> with #[target_feature]</code> because it must be safe to call from anywhere. And we cannot pass a token into the function because it accepts a</code> and b</code> but not token</code>.</p>
But even if we could do that, it would make for a pretty ugly API. We want a + b</code> to always work and automatically use the best SIMD instructions without the user ever fussing with tokens.</p>
We can use generics to solve both problems at once: by defining an f32x8</code> type that’s generic over the available instruction sets, we can implement addition on it that both smuggles a token inside it and</em> creates a separate implementation for each SIMD instruction set!</p>
This is what it looks like:</p>
pub trait</span> Level</span> {}</span></span>
</span>
#</span>[</span>derive</span>(</span>Clone</span>,</span> Copy</span>)]</span></span>
pub struct</span> Avx2</span>(());</span></span>
impl</span> Level</span> for</span> Avx2</span> {}</span></span>
</span>
pub struct</span> f32x8</span><</span>L</span>:</span> Level</span>></span> {</span></span>
    // For simplicity we'll back this with an array in this example.</span></span>
    // In production code we use native SIMD types for the level.</span></span>
    data</span>:</span> [</span>f32</span>;</span> 8</span>],</span></span>
    // The smuggled token!</span></span>
    token</span>:</span> L</span>,</span></span>
}</span></span>
</span>
/// implementation of `a + b` for Avx2</span></span>
impl</span> std</span>::</span>ops</span>::</span>Add</span> for</span> f32x8</span><</span>Avx2</span>></span> {</span></span>
    type</span> Output</span> =</span> Self</span>;</span></span>
</span>
    fn</span> add</span>(</span>self</span>,</span> rhs</span>:</span> Self</span>)</span> -></span> Self</span>::</span>Output</span> {</span></span>
        // (type conversions abbreviated)</span></span>
        // Use the Avx2 token to call our safe wrapper</span></span>
        let</span> result</span> =</span> add_avx2</span>(</span>self</span>.</span>token</span>,</span> self</span>,</span> rhs</span>);</span></span>
        Self</span> {</span></span>
            data</span>:</span> store_m256</span>(</span>result</span>),</span></span>
            token</span>:</span> self</span>.</span>token</span>,</span></span>
        }</span></span>
    }</span></span>
}</span></span></code></pre>
And then we can just as easily make it work for any other instruction set, or when SIMD is not available at all:</p>
#</span>[</span>derive</span>(</span>Clone</span>,</span> Copy</span>)]</span></span>
pub struct</span> NoSimd</span>(());</span></span>
impl</span> Level</span> for</span> NoSimd</span> {}</span></span>
</span>
/// implementation of `a + b` when no SIMD is available</span></span>
impl</span> std</span>::</span>ops</span>::</span>Add</span> for</span> f32x8</span><</span>NoSimd</span>></span> {</span></span>
    type</span> Output</span> =</span> Self</span>;</span></span>
</span>
    fn</span> add</span>(</span>self</span>,</span> rhs</span>:</span> Self</span>)</span> -></span> Self</span>::</span>Output</span> {</span></span>
        let</span> result</span> =</span> std</span>::</span>array</span>::</span>from_fn</span>(</span>|</span>i</span>|</span> self</span>.</span>data</span>[</span>i</span>]</span> +</span> rhs</span>.</span>data</span>[</span>i</span>]);</span></span>
        Self</span> {</span></span>
            data</span>:</span> result</span>,</span></span>
            token</span>:</span> self</span>.</span>token</span>,</span></span>
        }</span></span>
    }</span></span>
}</span></span></code></pre>
We’ve just solved safety and</em> runtime instruction selection at once!</p>
Add a convenience function that gives you the best Level</code> available on the system, and you get pretty much the perfect API for SIMD!</p>
The ABI would like a word</i></a> 
</h2>
There is, unfortunately, a fundamental problem to writing a + b</code> and having it lower to SIMD instructions: function call overhead.</strong></p>
Calling a function is not free but pretty cheap — just a handful of CPU instructions. But a handful</em> of instructions is a lot more than one</em> instruction we’ve just used for implementing addition!</p>
So if there’s a function call in the way, addition performance will plummet. And performance is the entire point of using SIMD in the first place!</p>
The compiler is usually pretty good at erasing this overhead via inlining</a>. It basically copy-pastes the implementation of a function you called into the function calling it, so there’s no more function and no more overhead.</p>
But #[target_feature]</code> annotations throw a wrench in the works. The compiler cannot inline a function that has a #[target_feature]</code> annotation into one that doesn’t, because the required features are not available in it!</p>
And guess what cannot have a #[target_feature]</code> annotation? Yeah.</p>
So how do</em> we make a + b</code> work with SIMD?</p>
Inline it, inline it with fire!</i></a> 
</h2>
We cannot put #[target_feature]</code> on the function that implements a + b</code>, but we can</em> put it on the function that calls a + b</code>!</p>
Then we can use inlining</a> to get the implementation of a + b</code> copied into the function calling it, and it ends up in a #[target_feature]</code> context.</p>
So the call chain looks like this:</p>
#</span>[</span>target_feature</span>(</span>enable </span>=</span> "avx2"</span>)]</span></span>
fn</span> do_stuff</span>() {</span></span>
    // TODO: some computation</span></span>
    c</span> =</span> a</span> +</span> b</span>;</span></span>
    // TODO: some more computation</span></span>
}</span></span>
</span>
// which calls into...</span></span>
</span>
#</span>[</span>inline</span>(</span>always</span>)]</span> // Function body will be copied into the caller</span></span>
fn</span> add</span>(</span>self</span>,</span> rhs</span>:</span> Self</span>)</span> -></span> Self</span>::</span>Output</span> {</span></span>
    // Use the Avx2 token to call our safe wrapper</span></span>
    add_avx2</span>(</span>self</span>.</span>token</span>,</span> self</span>,</span> rhs</span>);</span></span>
    // return statement abbreviated</span></span>
}</span></span>
</span>
// which calls into...</span></span>
</span>
#</span>[</span>inline</span>]</span> // Function body will be copied into the caller if feasible</span></span>
#</span>[</span>target_feature</span>(</span>enable </span>=</span> "avx2"</span>)]</span></span>
fn</span> add_avx2</span>(</span>token</span>:</span> Avx2</span>,</span> a</span>:</span> __m256</span>,</span> b</span>:</span> __m256</span>)</span> -></span> __m256</span> {</span></span>
    _mm256_add_ps</span>(</span>a</span>,</span> b</span>)</span></span>
}</span></span></code></pre>
This works.</p>
You can add these annotations in the right places and abstract over the SIMD levels and you don’t even need macros</a>.</p>
The problem is that whenever you call a + b</code> on SIMD types, you have to do it from a function with either #[inline(always)]</code> or #[target_feature]</code> on it, otherwise the code still compiles but performance plummets.</p>
Wanna see for yourself? Open this example</a>, remove #[target_feature]</code> from it and watch the generated assembly turn into absolute horror show.</p>
I’m not sure what can be done about this. The limitation seems quite fundamental for any approach that implements a + b</code> with SIMD.</p>
The Struct Target Features RFC</a> solves this for add_avx2(token, a, b)</code> and add::<S: Simd>(token, a, b)</code> but I don’t see a path to a + b</code> just yet.</p>
Where does that leave us?</i></a> 
</h2>
Despite the inlining wrinkle inherent to all SIMD code, we’ve managed to provide a remarkably pleasant SIMD abstraction at a staggeringly low, never-before-seen level of unsafe code.</p>
You can find the production version of these ideas in fearless_simd</code></a> v0.5, available now</strong> in a package registry near you! And here’s a small example</a> to see how it all fits together in production.</p>
The macro used to implement it is also exposed publicly, so you can easily mix and match</a> high-level operations like a + b</code> and platform-specific intrinsics to take full advantage of the hardware.</p>
There is more than one unsafe</code> block in fearless_simd</code> because it also provides the functionality of safe_unaligned_simd</code></a> crate, but that too is done at a significantly lower</a> amount of unsafe code than the original.</p>
For me the barrier to using high-level SIMD abstractions was always the sheer amount of unsafe</code> they brought. It was scary and hard to justify.</p>
But now SIMD in Rust can be truly fearless.</p>
Acknowledgements</i></a> 
</h2>
I am shocked that I am the first to put this into production (as far as I can tell), because I’m certainly not the first person to think of this.</p>
CPU feature tokens are an old and common idea. The pulp</code></a> crate has been using them for years, but they relied on handwritten unsafe wrappers around intrinsics, and occasionally got them wrong</a>.</p>
Generating multiple implementations using generics is also an old idea. It is part of the original fearless_simd</code> concept</a> from 8 years ago. The simdeez</code></a> crate, which predates it, also seems to use something similar</a>.</p>
The key insight of combining tokens with a single safe wrapper that delegates to rustc</code> is not unique to me either. Just in the context of fearless_simd</code> crate, Raph Levien has experimented with it</a>, and Daniel McNab</a> created a more elaborate implementation</a> than mine months ago.</p>
Daniel’s approach allows fine-grained tracking of every individual CPU feature, as opposed to a handful of fixed CPU feature levels that fearless_simd</code> uses. It is more expressive, but came at the cost of complexity, and his approach never got merged because no other maintainer stepped up to review it. I still hope it will be published as a standalone crate someday.</p>
Thanks to Daniel and to Laurenz Stampfl</a> for reviewing all my PRs to fearless_simd</code>, they were big and the quick reviews are really appreciated!</p>
This article was originally published on my Medium</a>.</em></p>


My old articles are on Medium
2026-06-18T00:00:00+00:00
Find them at https://shnatsel.medium.com/</a></p>
Sergey "Shnatsel" Davidoff

The state of SIMD in Rust in 2026

Implementing FMA and finding bugs in C and Rust standard libraries

Improving std::simd::swizzle_dyn

Takeaways</i></a> </h2> So here's what I've learned:</p> std::simd::swizzle_dyn</code> is even less useful than it appears, since multiversioning doesn't work on it.</li> std::simd</code> on nightly is outdated compared to the actual development state.</li>

The unreasonable effectiveness of LLMs for auditing Rust code

Safe SIMD in Rust, even on the inside

My old articles are on Medium
