Today I am publishing plotly.repel, an R package that brings ggrepel-style label repulsion to interactive plotly charts.

If you have ever tried to annotate an interactive volcano plot, you know the moment you label more than a few points, everything overlaps and the plot turns into a block of text. With static ggplot2 charts, we can reach for ggrepel. Plotly users have been asking for an equivalent for a long time (see the plotly.js issue thread here): https://github.com/plotly/plotly.js/issues/4674

I recently joined BigOmics, and we rely heavily on plotly for interactive charts. Interactivity is a big part of how we give users context (hover tooltips, zooming into regions, selecting subsets of genes). But once the information density goes up, labels stop scaling. Manual annotation becomes a chore, and a repel layer becomes a must. The most egregious example we encountered at BigOmics was a UMAP of gene features: dozens of cluster labels stacking on top of each other the second you zoom in (Figure 1).

Figure 1. Standard plotly labels on a dense UMAP of gene features: overlapping text makes the plot unreadable at scale.

At the same time, I wanted a contained project to experiment with agentic coding workflows: take a clear spec, turn it into a tested package, and iterate quickly.

The First Instinct: Copy ggrepel

My first instinct was simple: take ggrepel and copy the idea into the plotly world. I cloned the repo, looked through the internals, and quickly realized that most of the hard logic lives in C++.

That was an excellent chance to test whether an agent could help me build an accurate mental model quickly. Since I am fluent in R, I asked for a high-level explanation in pseudo-R (made-up functions, but familiar syntax). I cannot claim it was a perfect reproduction, but it worked for its purpose.

The mental model I came away with: ggrepel is a force-directed solver. Each label is a body under two competing forces: repulsion from neighbouring labels and data points (proportional to 1/d², where d is the distance to a neighbour), and a restoring spring pulling it back toward its anchor point. Think of it as a small physics simulation where every label nudges every other, and the system relaxes toward equilibrium. ggrepel runs up to 10,000 of these iterations, with early passes handling large displacements and later passes fine-tuning positions.

With that base, I treated the expected behavior as a reference, added constraints and specs, and started iterating toward a plotly version. It worked. Then I hit the thing that makes plotly categorically different.

The Reality Check: Interactive Charts Change Under You

ggrepel solves an offline layout problem: the coordinate space is fixed, the plot will not change after render, and the solver can spend as long as it needs to find a good configuration.

Plotly turns that into an online, latency-constrained layout problem: the coordinate space is a function of viewport state. It shifts on every zoom, pan, and resize. The solver cannot run once; it is called continuously, and it has roughly 25 ms per call to feel instant to the user.

In plotly, the user changes the plot:

• zoom in: the coordinate system changes
• pan: the viewport changes
• resize: the pixel geometry changes
• animate: the whole scene changes per frame

This shifted what “good enough” meant. A solver that is perfect once and wrong after zoom is worthless. A solver that is “pretty good” but stable across reruns and fast enough to be transparent to the user is the right goal.

The naive port made this painfully concrete: at 50 labels on a volcano plot, the solver ran in 162 ms per call. On every zoom event. Effectively unusable.

The fix came from a direction I would not have reached on my own. Working with a search-and-retrieval agent that pulled in literature on spatial indexing, I landed on approximate nearest-neighbor collision detection, a technique from the world of vector databases and ML search that transfers cleanly to label geometry. Rather than checking every label against every other label on each iteration, the plot space is partitioned into a uniform grid and each label is indexed into a cell. Collision checks then touch only the 9 cells adjacent to a label, turning an operation that scales with the square of the label count into one that is effectively constant.

The result: 1.80 ms per solve at 50 labels. The solver now re-runs on every zoom, pan, and resize without the user noticing.

This is also where “agentic programming” paid off in a way I had not expected. It was not just about writing code faster. It was about having a research layer that could pull ideas from outside my usual reading and apply them to a concrete engineering problem.

What You Get

plotly.repel gives you:

• add_text_repel() for repelled text labels
• add_label_repel() for repelled labels with boxes
• optional rules like “label only these points” and “when space is tight, keep the most important labels”
• the ability to choose when relabeling happens (for example: on initial render and zoom)

I am deliberately not going into the implementation details here. The goal is to show what you get and how to use it, not to require you to care about the internals.

Figure 2 shows the same UMAP after plotly.repel — labels pushed apart, connected to their points with leader lines, and stable across zoom and pan.

Figure 2. The same plot with plotly.repel: labels are pushed apart and connected to their points with leader lines, remaining legible after zoom and pan.

Install

remotes::install_github("bigomics/plotly.repel")

A Small Example

library(plotly)
library(plotly.repel)

set.seed(1)
df <- data.frame(
  x = rnorm(40),
  y = rnorm(40),
  id = paste0("P", seq_len(40))
)

plot_ly(df, x = ~x, y = ~y) %>%
  add_markers() %>%
  add_text_repel(
    x = ~x, y = ~y, text = ~id,
    # Tuning knobs (start here when you need to tweak behavior):
    force = 1.6,          # repulsion strength (higher = push labels apart more)
    force_pull = 0.8,     # pull back toward the point (higher = stay closer)
    box_padding = 0.35,   # extra space around labels
    point_padding = 0.15, # extra space around points
    direction = "both",   # "both", "x", or "y"
    max_time_ms = 25,     # keep interaction responsive
    seed = 42,            # reproducible initial positions
    on = c("render", "zoom", "resize")
  )

Even in this toy plot, the “feel” matters: labels shouldn’t fight you when you zoom in. If the layout becomes unreadable every time you interact, you stop interacting.

A Pattern I Use Constantly: “Label the Outliers, Not Everything”

The right UX for most scientific plots is:

• show all points
• label only the ones worth discussing

Here is a common pattern where visibility selects which points are eligible for labeling, and priority decides which labels win when space is limited:

set.seed(2)
n <- 1500
df <- data.frame(
  x = rnorm(n),
  y = rnorm(n),
  label = paste0("G", seq_len(n))
)

df$outlier <- with(df, abs(x) > 2.5 | abs(y) > 2.5)

plot_ly(df, x = ~x, y = ~y) %>%
  add_markers(marker = list(size = 6, opacity = 0.5)) %>%
  add_text_repel(
    x = ~x, y = ~y, text = ~label,
    visibility = ~outlier,
    priority = ~(abs(x) + abs(y)),
    max_overlaps = 10,
    force = 1.4,
    force_pull = 1.0,
    box_padding = 0.35,
    point_padding = 0.2,
    max_time_ms = 25,
    seed = 1
  )

What I Learned Building This

Two things I would do the same way again:

1. Test driven design: This approach is really powerful with agents. If you have an “source of truth” that the agents can use to check their outputs, you can easily set a feedback loop.
2. Using AI for coding is a really powerful tool, but you still need to do what you are doing. Otherwise, the agent will run in circles.

It’s Beta. Please Stress Test It.

If you try plotly.repel, the best feedback is:

• a plot that breaks it
• a plot where it is almost good but not quite
• a screenshot plus a minimal reproducible example

Repository: bigomics/plotly.repel

It took longer that I thought, but I’m thrilled to share that the core of this work—my main thesis—is now out in Nature Microbiology (link to full paper here). It’s surreal to move from debugging leaky microfluidics chips at 6 a.m. to seeing the story published. Now, let me tell you what it is about.

The leak in the wall

In the world of Gram-negative bacteria, the outer membrane serves as a protective wall, guarding the cell from external threats. Yet like all great walls, it has gates, these are protein channels called porins, that mediate the influx of essential nutrients for the cell and, incidentally, also antibiotics. Since the early studies of porins by Nikaido in the 1980s, these gates were largely considered to be wide open, allowing free passage for small molecules. This assumption, however, presented a fundamental paradox. Every biologist knows that bacteria rely on the proton motor force, that is, the accumulation of a proton gradient in the periplasmic space, as a source for ATP synthesis, motility, and transport. Thus, if porins as always open, then protons can simply leak out through these open gates? This question of energy conservation versus nutrient access led us to investigate what we call the membrane potential paradox.

Following the sugar

Our first step was to watch things flow. We used a fluorescently-tagged glucose molecule (2-NBDG), as a tracer to see how much was getting into the cells. As you’d expect, when we deleted the genes for the main porin proteins (ompF, ompC, ompG, nmpC, phoE), uptake dropped. No gates, no traffic. But then we got our first surprise. While screening a library of knockout mutants, we found one that had nothing to do with porins but still blocked uptake: a mutant for a potassium channel called Kch. [It was a strange and exciting clue.] This pushed us to think about ions. Maybe the porins weren’t just passive pores but were instead sensitive to the ionic environment. We tested changing the pH and potassium levels outside the bacteria, but nothing happened. However, we reasoned, maybe it’s the inside that matters. We used chemicals called ionophores such as CCCP, which makes the membrane leaky to protons, and valinomycin, which does the same for potassium to mess with the periplasmic ion concentrations directly. And it worked! We saw a huge change in the uptake of our fluorescent tracer. This was a big moment, suggesting that porins were indeed regulated by internal ions. But as a scientist, you’re trained to be skeptical of your own results. Ionophores are messy; they can have all sorts of side effects. We struggled with this idea. Using ionophores could have second-order effects. The solution, funnily enough, came from a completely different field: neuroscience. Neuroscientists care deeply about membrane voltage because neural is key for understanding neuron physiology. Thus, they developed this amazing toolbox of light-activated proteins that can be used to control the flow of ions across the membrane using light. We found one called ArchT, a proton pump originally discovered in algae, that you can switch on with a flash of green light. The thought was simple: what if we put this pump into our bacteria and used light to pump protons directly into the periplasm? We set up our bacteria in a microfluidic chip under the microscope. In the dark, the cells were happily swallowing up the fluorescent glucose. Then, we flipped the switch for the 561 nm laser. And just like that, the uptake stopped (Figure 1).

Figure 1. Internal H+ and K+ reshape porin permeability and light-driven ArchT pumping rapidly blocks 2NBDG uptake.

Where the pore flexes

Now we had proof, but we needed a mechanism. Previous literature had focused on the inside L3 loop of porins because it formed a constriction zone. But our molecular dynamics simulations, run by a collaborator, pointed elsewhere; they suggested that protonating charged residues on the periplasmic side for neutral residues (E23A, K27A, D28A, E64A, D69A, D156A, D162A, D289A, K329A) of the porin would cause it to constrict. This was a new idea. To test it, we spent months creating an OmpC mutant where we replaced all those charged residues with neutral alanines. The phenotype of this mutant was the perfect confirmation: it was leakier than wild-type and, crucially, had lost its ability to be regulated by pH (Figure 2).

Figure 2. Periplasmic protonation of conserved charged residues narrows the OmpC pore, explaining the loss of regulation in charge-neutral mutants.

The metabolic engine

It was now logical to ask: Does bacterial physiology have a mechanism to carry out this kind of regulation spontaneously? To find out, we adapted a wide range of genetically encoded fluorescent sensors (pHluorin, pHuji, GcaMp, and Ginko1) to monitor ion concentrations. Our most significant innovation was adapting these sensors for the periplasmic space. We initially thought that exporting them with pelB tags was sufficient, but quickly discovered that proteins do not tolerate the highly oxidative environment of the periplasm. For pHluorin, we found the dynamic range was not wide enough, so we switched to pHuji, which covers lower pH values. For Ginko1, we had multiple failed attempts because it relied on a GFP backbone anchored with a potassium-binding region. The issue with GFP is that it has disulfide bonds that do not form properly in the periplasmic space. Thus, we had to change the GFP for sfYFP while maintaining the potassium-binding region. With these tools, we plugged the cells into our microfluidic devices and began monitoring ion dynamics. We observed spontaneous spikes in membrane potential—bacterial “action potentials,” if you wish—that were mirrored by changes in periplasmic H+ and K+. Crucially, these action potentials were absent in the Kch knockout cells. The most exciting part came when we simultaneously imaged the membrane voltage alongside 2NBDG uptake. This revealed that moments of membrane depolarization correlated perfectly with bursts of 2NBDG uptake (Figure 3). Our mental model to explain this dynamic resembled a 4-piston engine cycle. The cycle begins in the “power stroke” or hyperpolarized state, with the periplasmic space full of protons, representing stored potential energy. These protons are then rapidly consumed to produce ATP or power other biological processes (Exhaust). As the proton gradient is consumed, the membrane begins to depolarize. This drop in potential acts as the “intake valve opening” (fuel intake), allowing nutrients (or 2-NBDG) to rush in through the now-open porin gates. Finally, the bacteria immediately triggers another metabolic cycle (Compression) to pump protons back into the periplasm, re-hyperpolarizing the membrane and closing the porin gates, ready to fire the next “power stroke.” Returning to our original claim that bacteria use the PMF as a source of energy, we tested how different carbon sources would impact these bacterial “action potentials.” We provided the bacteria with sources that would lead to different levels of membrane potential. Interestingly, we observed a higher frequency of membrane voltage oscillations when E. coli was supplied with glucose compared with other carbon sources such as fumarate or lipids. More importantly, the observed synchrony, depolarization correlating with uptake and hyperpolarization with low uptake, was consistent. This was a fascinating discovery suggesting that bacteria could be using these membrane voltage oscillations to dynamically regulate the uptake of small molecules (Figure 4).

Figure 3. Ion-sensor imaging links periplasmic H+/K+ fluctuations with membrane depolarization and bursts of 2NBDG uptake.

Figure 4. Metabolic state controls the frequency of membrane voltage oscillations and porin permeability across carbon sources.

Antibiotic consequences

Now the final question, why would you care about this? Well, it turns out that like sugars, many antibiotics rely on porin to get into the bacterium. Thus, the final phase of the project addressed whether the relationship between membrane voltage and uptake also applied to antibiotics, given that a significant portion of antibiotic resistance is due to changes in drug permeability. We focused on quinolones, a class of naturally fluorescent antibiotics, which made tracking their uptake in single cells easy. First, we confirmed that porins were the main route for quinolone entry, showing that porin knockouts had reduced uptake. The Kch knockout also showed decreased permeability, reinforcing the role of membrane potential. We then measured the EC50 of quinolones in both porin and Kch knockouts compared to wild-type cells, finding that both knockouts were more resistant to quinolones but not to colistin, an antibiotic that enters cells independently of porins. Finally, we revisited our ionophore experiments, treating cells with CCCP to see if altering the internal pH would affect ciprofloxacin uptake. Sure enough, we found that disrupting the proton gradient reduced antibiotic uptake. We hope this work helps explain why bacteria in certain metabolic states become naturally more resistant to drugs. It leaves us with a mechanistic bridge from metabolism to permeability. In lipid-rich environments—like certain intracellular niches—periplasmic acidification or slow bacterial “action potentials” likely close porins and reduce antibiotic entry. Therapeutically, activating Kch or otherwise depleting the proton gradient in the periplasm could, with adjutants, be a potential strategy to enhance antibiotic uptake and combat resistance (Figure 5).

Figure 5. Porin and Kch control ciprofloxacin uptake and resistance, with proton gradient disruption dampening drug entry.

Few thank yous:

Before closing, just to note that this was a true team effort. Immense gratitude to Andres Floto—every success traces back to your ideas and leadership with Floto Lab. Grateful for the support from Cambridge University, the School of Clinical Medicine, and the Laboratory of Molecular Biology. Deep thanks to all co-authors: Stephen Trigg, Georgeos Hardo, Anja Hagting, Ieuan E. Evans, Chris Ruis, Ali F. Alsulami, David Summers, Felicity Crawshay-Williams, Tom L. Blundell, Lucas Boeck, and Somenath Bakshi.

Every month brings new AI coding benchmarks showing superhuman performance. Yet any data scientist who has worked with Claude Code, GitHub Copilot, or similar tools discovers a maddening reality: these supposedly advanced agents must be kept on an impossibly short leash. Despite stellar demos and impressive performance scores, current AI assistants fail catastrophically at the iterative, stateful work that defines serious data analysis.

The problem manifests in two equally frustrating ways. First, we have the chatboxs, this involves asking questiosn and copying and pasting code snippets back and forth—transforms what should be seamless collaboration into a tedious exercise in manual context management. Then, we have the AI agents, but they only know bash and invoke Rscript for every operation, obliterating workspace state and forcing complete pipeline reconstruction for minor changes. Moreover, if you leave then running for long, their erors compound across steps as variables get redefined and assumptions change. Thus, neither approach supports the cumulative, hypothesis-driven exploration that characterizes sophisticated analytical work.

The deeper issue stems from architectural assumptions borrowed from software engineering. AI tools are fundamentally designed for code that lives in executable scripts and runs repeatedly. When you ask an AI agent to modify a plot color in your three-hour analysis session, it doesn’t adjust your existing visualization—it rebuilds your entire analytical pipeline from scratch. This approach is antithetical to exploratory data analysis, where insights emerge through cumulative investigation, hypothesis testing, and iterative refinement spanning hours or days.

Announcing MCPR: The First AI Protocol for Live R Sessions

Today, I’m announcing MCPR, the first Model Context Protocol server designed specifically for interactive R sessions. MCPR solves this problem by enabling AI agents to operate within your live R environment. The breakthrough is treating the AI agent as a translator between Natural English intent to R’s syntax, operating in a persistent session where your workspace state is preserved.

Instead of generating isolated scripts, the agent becomes your “computational hands.” You direct the analytical strategy; the agent executes the code within your established context.

1. It Eliminates Overhead: Exploratory changes no longer require expensive pipeline rebuilds.
2. It Amplifies Judgment: Your time shifts from managing code snippets to high-value tasks like hypothesis refinement and interpretation.
3. It Manages Risk: By operating in a continuous environment, MCPR prevents the compounded errors that arise when context changes between isolated scripts.

The rule for successful AI collaboration has always been to delegate the work, not the thinking. MCPR is the first tool that makes this a practical reality for data scientists.

Architecture Philosophy: Simplicity Scales

MCPR’s design philosophy prioritizes minimal tool interfaces that compose into powerful workflows. Rather than overwhelming agents with dozens of specialized functions, MCPR provides four essential tools: session management, code execution, plot generation with intelligent token optimization, and environment introspection. The communication layer leverages JSON-RPC 2.0 over lightweight sockets, ensuring cross-platform compatibility and non-blocking interactions.

Getting Started: From Installation to Insight

Setting up MCPR takes minutes. Install the package from GitHub:

remotes::install_github("phisanti/MCPR")
library(MCPR)
install_mcpr(agent = "claude")  # Configures MCP integration for claude, copilot, or gemini

Next, start a listener in your R console. This makes your session available to the agent.

# Make your current R session available
MCPR::mcpr_session()

That’s it. Now, you can simply ask your AI assistant to take over. Instruct your AI agent to connect, and watch the collaboration transform. Instead of explaining what you want and manually copying code, simply make request in plain English. The agent doesn’t generate a script for you to copy—it executes directly within your live session: You think: “Let’s filter this dataset for outliers and re-run the regression.” The agent works: It executes the dplyr and lm commands in your live session, using the objects you’ve already created.

The take home message

This isn’t about replacing human analytical judgment. It’s about amplifying it through optimal task delegation. When mechanical execution happens automatically within maintained context, when exploratory changes don’t require expensive rebuilds, when AI agents become computational extensions of analytical thinking rather than separate tools requiring constant management—that’s when human-AI collaboration finally serves discovery rather than creating overhead.

The rule for successful AI collaboration has always been: delegate the work, not the thinking. MCPR is the first tool that makes this philosophy a practical reality.

Acknowledgments

We thank Simon P. Couch(mcptools) for the inspiration to use nanonext and Aleksander Dietrichson (mcpr) for the idea of using roxygen2 for parsing tools.

Introduction

This summer I found myself in unfamiliar territory: building an R project that had absolutely nothing to do with statistical analysis. No regression models, no hypothesis tests, no data visualisations. For the first time in years, I had to venture beyond R’s comfortable functional world into its object-oriented wilderness. And what a wild, confusing, oddly fascinating wilderness it turned out to be.

R’s object-oriented programming landscape looks like the aftermath of some great taxonomic war. You’ve got S3 shambling around like a beloved but decrepit ancestor. S4 standing at attention with its formal methods and rigid protocols. R6 strutting around with its OOP swagger. S7 promising to be the chosen one who will finally bring balance to the Force. And scattered around them, the bones of the fallen: proto (used in the great ggplot2 but to be swapped by S7), R.oo, OOP, referenceClasses (R5?), mutateR, and probably a dozen others I’ve mercifully forgotten.

After some years building this kind of stuff in Python, I truly get the feeling. When you look at well-written Python code, it reads like prose. The syntax is clean, the object model is consistent, everything fits together with an almost zen-like harmony. R’s programming language can be so complex that even R-core developers occasionally get surprised by its behaviour. R treats 1 as a vector of length 1, which means there’s no difference between scalars and vectors except in your head. R has assignment operators that go both ways (<- and ->) because why choose? Lists will return NULL, not an error, if you use a name that hasn’t been defined. However, if you think about assigning NULL to an already existing list, it won’t replace the position with NULL. It will remove that entry. There’s no way to have a list containing NULL after creation. And don’t even get me started on environments: all your code is brought in by “sourcing” it and basically running the code in one single namespace. Compared to Python’s clean import x from y, in R it’s hard to organise routines in logical modules. There’s no hierarchy in organising individual R files when developing a package or project.

Python, meanwhile, was born from completely different philosophical soil. Python is the product of software engineering thinking applied with extraordinary discipline and care. When Guido van Rossum designed Python, he was thinking about how programmers think, how code should read, how systems should be structured. Python is gorgeous. Python makes sense. Python feels like programming should feel—clean, logical, consistent, and elegant. It’s the closest you get to programming in natural language English. But then you actually try to do data wrangling in Python, and something interesting happens:

# Python's methodical approach to data summarisation
quarterly_summary = (sales_data
    .query('transaction_date >= "2023-01-01"')
    .groupby(['sales_rep', 'product_line'])
    .agg({
        'gross_revenue': ['sum', 'mean', 'count'],
        'net_profit': ['sum', 'mean'],
        'customer_satisfaction': 'mean'
    }))

# Navigate the multi-index column hell
quarterly_summary.columns = [f"{col[0]}_{col[1]}" if col[1] != '' else col[0] 
                            for col in quarterly_summary.columns]

# Reshape into something actually usable
final_report = quarterly_summary.pivot_table(
    index='sales_rep',
    columns='product_line',
    values=['gross_revenue_sum', 'net_profit_sum']
).fillna(0).round(2)

The beautiful systematic thinking that makes Python elegant for general programming becomes laborious and verbose when applied to the inherently mathematical world of data manipulation. Moreover, the fact that data science is one of the things amongst many that Python does means that if you jump into another framework, say Polars, then you have to learn another syntax from zero.

What we’re witnessing is the collision between two different ways of thinking about computational problems: engineering thinking and mathematical thinking.

The Functional Sweet Spot: Why R’s Domain-Specific Design is Actually Its Superpower

The worst of R is that it comes from statisticians who needed to crunch numbers (without paying a SAS licence) rather than build the perfect programming language. The early designers weren’t trying to build a “proper” programming language—they were trying to build a mathematical calculator that could handle real data. Thus, these aren’t design flaws—they’re design decisions made by people who prioritised mathematical intuition over computational orthodoxy. Thus, R has data as a first-class citizen. They naturally brought in the functional approach they were familiar with, because this is how mathematical education actually works.

I don’t know about you, but in my high-school math class didn’t learn about objects and methods and inheritance. I learnt about functions: \(f(x) = 2x + 3\), that functions are mappings (given an input, produce an output) and function composition: if you have \(f(x) = 2x + 3\) and \(g(x) = x^2\), then \((g \circ f)(x) = g(f(x)) = g(2x + 3) = (2x + 3)^2\). This is the cognitive foundation that every quantitatively-trained person carries. Mathematics is fundamentally about transformations, mappings, and compositions of functions. Not objects, not methods, not inheritance hierarchies—functions.

The same operation we struggled with in Python becomes natural when expressed in R’s mathematical idiom:

# R expressing the natural flow of mathematical thinking
quarterly_summary <- sales_data %>%
  filter(transaction_date >= as.Date("2023-01-01")) %>%
  group_by(sales_rep, product_line) %>%
  summarise(
    total_gross_revenue = sum(gross_revenue),
    avg_gross_revenue = mean(gross_revenue),
    total_net_profit = sum(net_profit),
    avg_customer_satisfaction = mean(customer_satisfaction),
    transaction_count = n(),
    .groups = "drop"
  ) %>%
  pivot_wider(
    names_from = product_line,
    values_from = c(total_gross_revenue, total_net_profit),
    names_sep = "_"
  )

The functional programming paradigm maps perfectly onto data analysis because data analysis is functional mathematical thinking applied to empirical problems. You transform data through sequences of well-defined mathematical operations. You compose simple functions to build sophisticated analytical procedures. You create computational pipelines because that’s how mathematical reasoning naturally flows from raw data to statistical conclusions.

R’s “weird” design choices aren’t weird—they’re the natural computational expression of mathematical thinking patterns that have been refined through centuries of pedagogical evolution.

The Functional Breaking Point: When Mathematical Thinking Hits Its Limits

But here’s what my summer project taught me: sometimes you need to build systems that exist beyond the elegant world of mathematical transformations. Sometimes you need persistent state that survives across multiple operations. Sometimes you need complex interactions between subsystems that each maintain their own internal logic and coordinate through well-defined interfaces.

Base R excels at transforming data, but it struggles with managing persistent connections, maintaining session state across time, coordinating real-time interactions, or building systems that need to remember complex relationships between their components. These aren’t data analysis problems—they’re systems architecture problems. Moreover, with modern data requirements with interactive dashboards, connections to third-party data sources. Think about what today’s R developers actually build: not just one-off statistical analyses, but living systems that researchers, and non-programmer savvy users (clinicians, wet-lab scientist,…) bio interact with daily. Hence, R needs to grow beyond its statistical and data analysis roots.

Now here comes the crucial insight: R developers currently do not agree on what “properties” should an “object” have—and this disagreement reflects two fundamentally different cognitive approaches to the same technical challenges.

R6: The Engineering Solution

R6 embodies classical object-oriented thinking. An R6 object is a self-contained entity with encapsulated state and behaviour. When you create an R6 object, you’re thinking like a software engineer: “What data does this object need to maintain? What operations should it support? How should it manage its internal consistency?” You tell the object what to do ($connect(), $process_data()), and it manages its own state internally. This is encapsulation in the classical sense—perfect for building complex internal components that need sophisticated state management.

The beauty of R6 lies in its unapologetic embrace of mutable state and familiar object-oriented patterns that will feel natural to programmers coming from other languages. When you need an object that remembers things between method calls, that maintains internal counters, that manages connection pools or caches expensive computations, R6 doesn’t fight you—it gives you the tools to build exactly what you need. This classical OOP approach is particularly valuable for attracting non-R programmers who bring solid architectural knowledge from computer science backgrounds, helping to elevate the engineering practices in R projects. Most R programmers transition from data science to programming rather than the other way around, often lacking the architectural know-how that comes from formal CS training. R6 bridges this gap by providing familiar patterns that encourage good software engineering practices whilst still being accessible within R’s ecosystem.

S7: The R Compromise

S7 represents a different philosophical approach: maintaining functional separation whilst adding object-oriented capabilities. In S7, objects are still primarily data containers, but now they can have formal type definitions and method dispatch. The data (the object) remains separate from the operations (the generic functions), preserving the functional mindset that R users expect. This creates wonderfully consistent user-facing APIs that feel mathematical rather than computational.

The base idea in S7 is how it preserves R’s functional soul while adding the structure that complex systems need, making it an ideal stepping stone for traditional R programmers who want to move into more sophisticated project architectures without abandoning their functional thinking patterns. S7 objects are immutable by default—when you “modify” an S7 object, you get a new object back, just like traditional R operations. This immutability means S7 plays nicely with R’s copy-on-modify semantics and feels natural to R users who are accustomed to functional thinking. For statisticians and data scientists who’ve grown comfortable with generic functions like summary() and plot(), S7 extends this familiar paradigm into more complex domains, allowing them to write classes and build sophisticated systems without the cognitive leap required by classical OOP.

Conclusion

In my opinion, the choice between R6 and S7 isn’t about technical superiority, but rather about cognitive alignment with different problem domains. When building this summer project, I tried the S7 approach, but wasn’t very comfortable with the organisation. Switching to R6 helped me to organise the components and processes. I’m thus not sure if S7 will truly be “the chosen one”, but I think it makes sense to use R6 for the internals when building internal systems that require sophisticated state management and encapsulation. Something that the user will never touch. Then, use S7 when building user-facing APIs that need to feel like mathematical operations. I think this way helps maintain mathematical elegance for data analysis work whilst providing engineering power for systems development. Let’s see what the future of R has for us. I really look forward to the end of the OOP wars.

This weekend I managed to finish a very fun project! I think there are two kinds of bioinformaticians: computer scientists that for one reason or another get into biology, or biologists that for necessity get into computer science. I belong to the second category. Thus, I find myself frequently asking for complex multiflag commands such as find . -name "*.fastq.gz" -exec zcat {} \; | wc -l, or rsync -avz --progress --exclude='*.tmp' /source/data/ user@server:/backup/, or docker run -v $(pwd):/data -v /tmp:/tmp --rm biocontainers/samtools samtools view -bS input.sam | samtools sort -o output.bam. This leads to seconds to minutes to search. The standard solution – web-based or IDE AI assistants – introduces its own inefficiency: context switching. Each lookup requires:

1. Mental context switch from problem-solving to information retrieval
2. Physical context switch from terminal to browser
3. Potential exposure to distractions (notifications, other tabs)
4. Copy-paste operations that introduce error potential
5. Context switch back to terminal and problem state

Thus, in the age of LLMs, I thought, why not build a fast responsive app that is directly there where I need it. My goal was to make something fast, simple and responsive. I thus came up with the idea of CodeDjinn…

CodeDjinn

What is CodeDjinn, or rather, what is the main idea behind CodeDjinn? CodeDjinn is a lightweight-fast CLI assistant that generates shell commands. That’s it. The goal was to be there where you are and come up with quick and simple answers. Building something fast and responsive sounds simple, but it can be complex to implement. Indeed, it taught me a few profound ideas about the nature of tools and human-computer interaction.

First of all, I learned to prioritize – a meditation on essence versus excess. Any of the top-tier models can answer most of the questions that I was intending to address such as code-djinn -a "count total reads in all fastq files". However, it raises a philosophical question: does it make any sense to summon the most powerful models for the simplest of queries? First of all, you’ll spend more resources (and money) than necessary . Second, you’ll spend also a few extra seconds waiting for an answer that smaller models can provide, which would defeat the initial purpose. Thus, I decided to work with the smallest, fastest models I could find, embracing the principle of “sufficient intelligence.”

• Mistral’s Codestral: Mistral has built a reputation for fast, efficient models, making it a natural choice for this use case.
• Google’s Gemini Flash: This model demonstrated superior response times in public benchmarks, with Gemini Flash-lite consistently ranking as a top performer.
• QwQ-32B: Last but not least, I wanted to explore the edges of the ecosystem. Given that Qwen has been making some noise, I thought, why not embrace the experimental spirit?

The second thing that I learned was about optimizing Python code itself – a journey into the temporal dimension of software. I discovered that each time I called the tool from the CLI, I was “invoking a new CodeDjinn”, a fresh consciousness that had to reconstruct its entire reality from scratch. Thus, I implemented a cache system for fast responses – a form of digital memory. I also learned about lazy loading, which meant only the needed libraries were loaded each time.

This was interesting, because in this LLM age, many people have argued that they make us “lazy” and “stupid”. However, I see it differently. These tools are amplifiers of curiosity. I would probably never have learned about concepts like lazy loading if not for the fact that nowadays LLMs can make them easily accessible. They democratize knowledge, turning arcane wisdom into accessible understanding.

Real-World Magic Moments

So, what can CodeDjinn do for you? I focused on a feature set designed for maximum utility. The core functionality includes command generation (ask mode) and safe execution (execute mode). Additionally, I integrated an explanation feature for continuous learning and command understanding. The configuration system supports API key management, shell detection, and prompt customization, allowing users to specify preferred tools and command styles.

After getting done with the building, I started using it in my normal workflow and I was quite amazed about how much it could get done and how it truly helped me to stay in focus on the task I was doing. Here are practical examples demonstrating its capabilities:

# Counting reads across all samples
code-djinn -x "count total reads in all fastq files"
# Instantly: find . -name "*.fastq" -exec wc -l {} + | awk '{s+=$1} END {print s}'

I used to spend minutes crafting that command, each flag a small decision, each pipe a connection between ideas. Now it emerges fully formed, like Athena from Zeus’s head.

# That moment when you need to check memory usage
code-djinn -x "show top 10 memory hungry processes with percentages"
# Conjures: ps aux | sort -rk 4 | head -n 11 

code-djinn -x "show commit history with file changes for last sprint" 
# Magic: git log --oneline --graph --decorate --color --stat --since="2 weeks ago"

And below you can see a quick plot of the execution time of some request. I indeed achieved the “sub-second” response time that I was aiming but note that some of the commands require a lot of time to execute, while others are very fast.

Conclusion: The Philosophy of Simple Tools

Here’s something I discovered while building this: Is this an agent? No, but I think it is really useful. I don’t think we need complex advanced multi-step agents for everything; sometimes getting quick answers can be very useful. CodeDjinn doesn’t try to write your entire analysis pipeline, manage your projects, or have philosophical discussions about the nature of genomics (though that would be fascinating). It translates thoughts into shell commands. That’s it. And that simplicity is its superpower. Indeed, I believe the future is full of small models and soon this will be a default feature of most OS. But this is just an opinion.

I played around with adding more features – “what if it could also edit files?” “what if it could chain multiple commands intelligently?” But each addition made it slower and more complex. The Unix philosophy remains relevant: do one thing well.

If you wanna try, the tool is available at pip install code_djinn – a small djinn in a digital bottle, ready to grant your command-line wishes. If you feel like complaining or asking for more, please open an issue on the GitHub repo.

Bonus

The reason I called it CodeDjinn is because in Arabic mithology a djinn (“جن”) is a supernatural creature that can be summoned to do your bidding. In the same way, CodeDjinn is a tool that you can summon to help you with your coding tasks.

Initially, I leaned on a well-known Python package, PyStackReg, which operates with TurboReg’s fast C++ core to perform subpixel registration. However, as I observed its performance under practical conditions, I recognized that the tool was not delivering the stability and speed that I was familiar when using Fiji. Looking at the images, two little quirks really bugged me: a jump in the very first frame of every movie, and a registration that seemed to circle around its equilibrium point. These small details pushed me to develop a new package.

So, I reached out to Tseng Qing, the creator of the TemplateMatching plugin, and floated the idea of porting his package to Python. I tinkered around his code and realised that OpenCV was the one moving the threads behind scenes. With Python’s excellent API for OpenCV, the solution practically wrote itself. After some trial and error, I succeeded in implementing a new registration algorithm.

My surprise came when I took some metrics and saw that it was crazy fast and very precise (see Figure 1). The key part was to focus on solving the Translation problem fast instead of trying to solve all possible cases. Indeed, after working for a while with MONAI, I think the complex registraction issues can be leave to Deep Learning modesl that do regression on the translation matrix, while the simpler ones can be solved with a more traditional approach. While PyStackReg has to build the pyramid features, try some warp -> gradient -> solve until converge -> repeat, the OpenCV uses a template: it slides the template over the target image (like a 2D convolution), producing a grayscale match-map, and then finds the best match, and you are done.

The end result was good (see the image below). First, I was very happy that the jump in frame 1 disappeared. Second, it was much faster, which really helped us in our analysis pipeline. Finally, the aligned images significantly improved our tracking. Given these good results, I decided to package it. You can download it from PyPI here or if you want to see more details, you can check the GitHub repository. Life is good when you follow your instincts; you never know where the rabbit hole will lead next.

Figure: Comparison showing drift correction results before and after applying the new registration algorithm.

I am very happy this week because, after some time working with it, my contribution to the MONAI project has been accepted! For those unfamiliar, MONAI is a PyTorch-based open-source framework for deep learning in healthcare imaging, and I am proud to be part of such a great community because I use their framework a lot.

My contribution is a flexible implementation of the Restormer model for image restoration. I first stumbled upon this model because, in my current project, we acquire a lot of images (like… A LOT), and very fast. The project consists of acquiring microscopy images of bacteria under multiple conditions across time using well plates. To capture enough bacteria, we use a 40x objective that moves swiftly between positions, making out-of-focus objects really common. The image below (Figure 1) shows an example of the problem. I used a simple Sobel filter to detect the edges of bacteria, just to illustrate how the bacteria in the center are sharper than on the sides of the image:

Figure 1: Source image (left), Sobel filter edge detection (middle), and focus map (right) showing uneven focus across the field.

Since manually focusing on each object is impossible at this scale, I needed an automated solution that could restore focus computationally.

Tackling the issue

To tackle this, I came up with a simple idea: out-of-focus mainly happens because objects are in different planes. So, I take images at different Z-planes and use the model to infer the perfect focal plane from the out-of-focus images (see Figure 2). Given that our images are quite large and almost always contain some objects in focus and some out of focus, I split the image into quadrants and use as target only the quadrants with the most objects in focus.

Figure 2: Z-stack acquisition (left), off-focus slices as input (middle), and in-focus slices as target (right).

The Restormer model

The Restormer, as described by Zamir et al. (2022), is a Vision Transformer specifically designed for high-resolution image restoration. It uses a multi-scale encoder-decoder architecture with skip connections, similar to U-Net, allowing it to capture both global and local image features. The key innovation is that, instead of using spatial attention as in the original Vision Transformer, it uses a combination of spatial and channel attention (what the authors call Multi-Dconv Head Transposed Attention, or MDTA), which allows it to focus on the most relevant features in the image. See Figure 3 for a schematic overview.

Figure 3: Schematic overview of the Restormer architecture, including the UNet-like encoder/decoder architecture, MDTA, and GDFN blocks.

Notice also that one of the key parts is the fact that the original image is also passed to the end of the model. This way, the model can focus on the noise without having to learn the whole image again.

Implementation Details

This implementation extends the original 2D Restormer to support both 2D and 3D operations, making it particularly valuable for volumetric medical imaging. I thought this would be easy because the transformer block does not care about dimensions and the MONAI library already had an UpSample layer using pixel shuffle. However, there was no pixel unshuffle. Thus, I had to implement it myself. For this, I tackle the problem as permutation problem. If you think about it, it is just about moving the dimensions around:

def pixelunshuffle(x: torch.Tensor, spatial_dims: int, scale_factor: int) -> torch.Tensor:
    # ...
    output_size = [batch_size, new_channels] + [d // factor for d in input_size[2:]]
    reshaped_size = [batch_size, channels] + sum([[d // factor, factor] for d in input_size[2:]], [])

    # The eureka moment came when I realized the permutation pattern is just collecting all scale factors first,
    # followed by all spatial dimensions - it's like separating the "what" (features) from the "where" (locations)!
    permute_indices = [0, 1] + [(2 * i + 3) for i in range(spatial_dims)] + [(2 * i + 2) for i in range(spatial_dims)]

    # And then, pass everything to the channel dimension while keeping the spatial dimensions intact.
    x = x.reshape(reshaped_size).permute(permute_indices)
    x = x.reshape(output_size)
    return x

After this, everything else was quite simple because the transformer blocks were already dimension-agnostic by design. The next challenge was to give flexibility to the model and to add support for 3D images. The first point is important because the original Restormer was a generic model trained for all kinds of common RGB images. However, in the scientific and medical domain, it is more common to deal with N-channel images. Thus, the researcher should have space to contract or expand the encoder/decoder steps as required for their project. For example, in our case, we only had 2 steps because our image sampling space is quite homogeneous. The second point was relevant because, in the medical field, it is quite common to deal with 3D images (e.g., CT, MRI, etc.).

To give more flexibility to the Restormer, I only had to closely follow the calculation on how the spatial dimensions and channels are calculated at each step:

class Restormer(nn.Module):

    def __init__(
        self,
        spatial_dims = 2,
        in_channels = 3,
        out_channels = 3,
        dim = 48,
        num_blocks = (1, 1, 1, 1),
        heads = (1, 1, 1, 1),
        num_refinement_blocks = 4,
        ffn_expansion_factor = 2.66,
        bias = False,
        layer_norm_use_bias = True,
        dual_pixel_task = False,
        flash_attention= False,
    ):
        spatial_multiplier = 2 ** (spatial_dims - 1)

        # Define encoder levels
        for n in range(num_steps):
            current_dim = dim * (2) ** (n)
            next_dim = current_dim // spatial_multiplier
            # ...

        # Define decoder levels
        for n in reversed(range(num_steps)):
            current_dim = dim * (2) ** (n)
            next_dim = dim * (2) ** (n + 1)
            # In the decoder, it was also necessary to add an extra convolution step to reduce dimensions to make space for the skip connections.

For the encoder, it was very straightforward: the encoder systematically halves spatial dimensions while multiplying by $2^{(spatial_dims - 1)}$ channel dimensions at each step. This is because each spatial dimension contributes multiplicatively to the total channel increase. For the decoder, it was basically the same, but I also added an extra convolution step to reduce dimensions to make space for the skip connections.

Results

The results were nothing short of amazing. Here’s a before/after comparison (Figure 4):

Figure 4: Left—Input test image (top) and restored image (bottom). Right—Paired SSIM score comparison showing consistent improvement after restoration.

Not only do the restored images look visibly sharper and more defined, but the quantitative results speak for themselves. The SSIM (Structural Similarity Index) scores improved dramatically across the board, as shown in the paired plot. This improvement wasn’t just cosmetic—after restoration, the number of detected objects in our automated pipeline nearly doubled, making downstream analysis much more robust and reliable.

It’s genuinely rewarding to see how a well-designed model can breathe new life into challenging microscopy data. Watching those blurry bacteria snap into focus (both visually and statistically!) was one of those moments that makes all the late-night coding sessions worthwhile.

Things that I learned

Implementing Restormer for MONAI taught me several valuable lessons:

1. Channel attention is powerful: The transposed attention mechanism that operates across feature channels rather than spatial dimensions is remarkably effective while being computationally efficient.
2. Pixel unshuffle is elegant: Using pixel unshuffle/shuffle as downsampling/upsampling mechanisms preserves information by rearranging it between spatial and channel dimensions rather than discarding it.
3. Loss function choice is crucial: Since my goal was to restore images for subsequent segmentation, I used Structural Similarity Index (SSIM) as my loss function. This perceptual metric emphasizes preserving edges and contours rather than just pixel values, which was perfect for my use case. Importantly, I trained a separated model for restoring fluorescent singal, where PSNR was the best loss function because in this case we were only interested in the signal.
4. Transformers are data-hungry but efficient: While they require more training data than CNNs, they converge surprisingly quickly and generalize well to unseen data.
5. Code Quality: Given the fact that this was a constribution to a large-scale project, I had to pay extra attention to code quality. This meant writing unit tests, documentation, and following the MONAI coding standards. This was a great opportunity to learn about best practices in software development and how to write clean, maintainable code.

Conclusion and Future Work

Contributing to MONAI with this Restormer implementation has been one of the most satisfying projects I’ve tackled recently. The model now lives in the MONAI codebase where others can use it for various medical image restoration tasks beyond my application.

I’ve learned that R shines brightest in three areas: statistical analysis, data visualization, and report generation. Data visualization with ggplot2 is a form of art, crafting charts so beautiful that they could bring a tear to a data scientist’s eye. Couple this with RMarkdown, and you have the undisputed champion for scientific reporting. It’s the Swiss Army knife of reproducible research, seamlessly weaving together narrative, code, and results into a single, elegant document. It’s the good, the beautiful, the stuff that makes you fall in love with data analysis.

But then there’s the other side, the bad, the ugly. Trying to make R speak with the rest of the OS, like behaving like a proper CLI tool? That is theoretically possible, but why would you do that to yourself? R wasn’t designed for this. It’s a language built by statisticians for interactive exploration. However, I recently wrote some scripts that made some nice reports from a given dataset, and I found it a bit cumbersome to first write a new Rmd where I was just changing the name of the data path, and second, running:

Rscript -e "rmarkdown::render('./path/to/report', params=list(file='./data.csv'), output_file='report.pdf')"

was too ugly to bear. Thus, the challenge was born. I wanted to create an R package that, when my colleagues installed it, also installed a CLI tool to nicely assemble the automated reports.

The CLI Challenge

The goal was simple: Create a tool where users could install the r package via remotes::install_github('mypackage') and then run:

awesome_report --input data.csv --output report.pdf

However, the R’s standard approach feels more like a workaround:

Rscript myscript.R

Then the classic shebang plus chmod approach:

# Add shebang to the script
#!/usr/bin/Rscript --vanilla


# Then give permissions
chmod +x awesome_report.R

Sure, it works on my machine™, but distributing this to colleagues? That’s where the fun begins.

The Plot Thickens

Enter the R packaging system. Surely someone must have solved this? The cli package looks promising. Littler catches your eye. Even the mighty Dirk Eddelbuettel (yes, that Dirk) says it’s impossible. Here I quote from his (Stack Overflow answer)[https://stackoverflow.com/questions/44566100/installing-executable-scripts-with-r-package]:

Short (and very sad) answer: You cannot. But read on. Reasoning: R will only ever write package content to its own .libPaths() directory (or the first in case several are given), or a directory given by the user. So, say, /usr/local/bin/ is simply out of reach. That is a defensible strategy.

Just when all hope seemed lost, like finding a bug in production code at 4:59 PM on a Friday, I stumbled upon (Rapp)[https://github.com/r-lib/Rapp]. Finally! They say in their intro page:

Rapp (short for “R application”) makes it fun to write and share command line applications in R. It is an alternative front end to R, a drop-in replacement for Rscript that does automatic handling of command line arguments. It converts a simple R script into a command line application with a rich and robust support for command line arguments. It aims to provides a seamless transition from interactive repl-driven development at the R console to non-interactive execution at the command line.

Just what I wanted… But wait - it still needs manual PATH setup?

The Eureka Moment

The solution came together like connecting dots in a complex puzzle. First, while exploring Rapp’s documentation, I found this gem:

Place your app in the exec folder in your package, e.g: exec/myapp. Apps are automatically installed as executable.

Wait - automatic executable permissions? That solves the chmod headache!

Then, diving into R package development docs, I discovered the configure script - a hidden powerhouse that runs during package installation. Perfect timing to set up our PATH.

Finally, Rapp’s own installation instructions provided the missing piece:

export PATH=$(Rscript -e 'cat(system.file("exec", package = "Rapp"))'):$PATH
export PATH=$(Rscript -e 'cat(system.file("exec", package = "my.package.name"))'):$PATH

Combining these three insights, here’s the elegant solution:

#!/bin/bash

# Determine the user's shell configuration file
SHELL_RC="$(basename $SHELL)rc"
USER_RC="$HOME/.$SHELL_RC"

# Define color codes
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'


# Add the exec directory of the package to the PATH
export PATH=$(Rscript -e "cat(system.file(\"exec\", package = \"mypackage\"))"):$PATH
EXPORT_CMD='export PATH=$(Rscript -e "cat(system.file(\"exec\", package = \"mypackage\"))"):$PATH'
if ! grep -Fxq "$EXPORT_CMD" "$USER_RC"; then
    echo "$EXPORT_CMD" >> "$USER_RC"
    # These echo messages are just to inform the users
    echo -e "${GREEN}Added mypackage exec directory to PATH in $USER_RC${NC}"
else
    echo -e "${YELLOW}mypackage exec directory already in PATH in $USER_RC${NC}"
fi
echo -e "${YELLOW}Please run 'source $USER_RC' before using the command.${NC}"
# Source the user's shell configuration file to apply changes
source "$USER_RC"

Place your R scripts in the exec/ folder, and the package installation handles everything else. Now users just need to open his R terminal and run:

install.packages("mypackage")
# OR in my case
remotes::install_github('phisanti/BoeckLabRtools')

And voilà! PDF reports can be buil with.

mypackage build_report data.csv

Is it elegant? Maybe not. Does it work? Absolutely. Remember folks, in R, persistence pays off. And if someone tells you it can’t be done, they probably just haven’t found the right hack yet! If you’re wondering why R makes CLI tools so complicated, remember it was created by statisticians, not programmers!

But let’s take a moment to appreciate the magic behind the scenes - R is lifting all the weight. However, instead of using the ELISA analyzer, this time, I went with the Golem framework. The beauty of this framework is that the app is an R package, making it easier to split the different parts into auxiliary functions. Plus, the app is organized into separate modules. Shiny modules can be tricky at first, but you’ll get used to them. One thing that still amuses me about Shiny is that to pass reactive objects between modules; you have to pass them as functions! Why am I surprised when everything in R is a function? In conclusion, you can find the app at free-prism, or you can check out the code directly at my github. And because the app is an R package, you can even install it on your local device and try it out for yourself. Happy coding, everyone!

If you’re struggling with Shiny, know that you’re not alone. It’s a tricky beast to tame, but with a bit of perseverance, you’ll get there. And when you finally do, the feeling of accomplishment is like no other. So, to all the Shiny users, keep pushing through the frustration. The end result is worth it. And to our friend who just published their first Shiny app, congratulations! We’re all cheering you on and can’t wait to see what other amazing things you’ll create in the future. For those who want to check out our friend’s shiny app and repo, be sure to follow the link they provided. Who knows, maybe it will inspire you to create your own Shiny masterpiece. Link to the app here Link to the repo here

PS, Also, I am working in a bigger Shiny project, so stary tune for the news.