Language Resources

DZone's Featured Languages Resources

Pragmatic Premature Optimization

By Alexander Radzin

“...premature optimization is the root of all evil…” Donald Ervin Knuth Introduction "Premature optimization is the root of all evil." Most software engineers know this, attributed to Donald Knuth, author of The Art of Computer Programming and one of the most influential figures in computer science. Many have also picked up the practical conclusion that followed: "let's make it work first, fix performance later." After all, it's easier to add another EC2 instance than to find the root cause. But here is what Knuth actually wrote: "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%." A little different, isn't it? The second sentence is almost never quoted — and that is convenient, because it turns a careful statement into a simple excuse. Sometimes for laziness. Sometimes because people assume that optimization means sacrificing readability: cryptic bit manipulation, obscure tricks, code that only the author understands at 2 am. I believe Knuth was indeed warning against that kind of optimization. But that assumption is wrong more often than people think. Good, clean code is frequently efficient code too — not by accident, but because choosing the right tool for the job tends to be both clearer and faster. The examples in this article are proof of that. Scope This article focuses on simple, cheap, and foolproof tips that can be applied universally — regardless of your architecture, framework, or domain. In my experience, they carry virtually no risk of making things worse. Architecture, design, networking, database connectivity, threading — these are deliberately out of scope. Not because they are unimportant, but because they are context-dependent. The right answer depends on your specific system, and each of these topics deserves its own article. Examples String Operations We are all familiar with built-in JDK string utilities like: equals(), startsWith(), endsWith(), contains(): Java s1.equals(s2); s1.startsWith(s2); s1.endsWith(s2); s1.contains(s2); Unfortunately, JDK provides only one function for case-insensitive comparison: Java s1.equalsIgnoreCase(s2) There are no functions for case-insensitive startsWith(), endsWith(), contains(). So, often we combine toLowerCase() or toUppserCase() with startsWith(), endsWith(), contains(): Java s1.toLowerCase().startsWith(s2.toLowerCase()); s1.toLowerCase().endsWith(s2.toLowerCase()); s1.toLowerCase().contains(s2.toLowerCase()); A little verbose and null-prone, but just fine if not on the critical path. However, this technique might cause some performance problems. Do not forget that String is an immutable class, so instead of just a char-to-char comparison between two strings, we create two additional strings that then must be garbage-collected. Considering that String is a wrapper over a char array, the memory allocation may become expensive. The solution is to use case-insensitive utilities provided by different libraries, e.g., Apache Lang3: Java startsWithIgnoreCase(s1, s2); endsWithIgnoreCase(s1, s2); containsIgnoreCase(s1, s2); Or, starting from version 3.18.0: Java Strings.CI.startsWith(s1, s2); Strings.CS.startsWith(s1, s2); Where CI exposes case-insensitive and CS — case-sensitive utilities. Many people like regular expressions and use java.util.Pattern class sometimes, not where it is really necessary. For example: Java Pattern.compile("^prefix.+suffix$").matcher(s).find() Instead of: Java s.startsWith("prefix") && s.endsWith("suffix") Or even: Java Pattern.compile("^prefix").matcher(s).find() instead of s.startsWith("prefix") Pattern.compile("suffix$").matcher(s).find() instead of s.endsWith("suffix") Pattern matching is significantly slower than trivial substring matching. The following table shows evaluation time for 1 million operations: Operation * 1 million times Time, ms s.equals("hello") 7 s.startsWith("hello") 6 s.endsWith("hello") 11 s.contains("hello") 24 s.toUpperCase().startsWith("HELLO") 65 s.equalsIgnoreCase("hello") 5 Pattern.compile("hello").matcher(s).find() 238 pattern.matcher(s).find() 31 What can we see from this table? Performance of equals() and startsWith() is similarendsWith() is 2 times more expensivecontains() is 4 times more expensive than equalsChanging case followed by startsWith() is 10 times (!) more expensiveCase-insensitive comparison functions do not have any performance penaltiesSearching for a substring using a precompiled pattern is about 20% more expensive than using a plain contains() method. Compiling the pattern and using it is almost 10 times more expensive than the plain contains() method. So next time you reach for Pattern.compile(), it is worth pausing for a second: is regex actually needed here, or is a plain string method both simpler and faster? If you really need a pattern, at least compile it in advance — better yet, declare it as a private static final class member. Collections Let’s assume that we want to know whether a given list contains the specific element: Java list.contains("red"); In fact, this call invokes code like this: Java int n = list.size(); for (int i = 0; i < n; i++) { if ("red".equals(list.get(i))) { return true; } } Starting from Java 8, we have a streaming API that just hides from us the same gory details: Java list.stream().anyMatch("red"::equals); This is perfectly fine when the list is short, changes frequently, or is searched only occasionally. But if the list is large, stable, and searched repeatedly, a HashSet is the right tool — offering average O(1) lookup instead of O(n). If you cannot change the original data structure, converting it once at initialization time and searching the Set from that point forward is almost always worth it. If both the guaranteed element order and the fast lookup are needed, we can either hold duplicated data structures — a list for ordering and a set for search or just use LinkedHashSet, which solves both problems. Another common case is case-insensitive search. We already saw above that the combination of toLowerCase() or toUpperCase() with comparison significantly reduces the performance. This can be solved by using TreeSet with custom comparator, e.g. String.CASE_INSENSITIVE_ORDER: Java Set<String> set = new TreeSet<>(String.CASE_INSENSITIVE_ORDER); This gives you a sorted, case-insensitive set with no extra allocations - and the same approach works for TreeMap when your data is key-value pairs. Enum Lookups Everyone knows that an enum entry can be found by its name using a built-in method valueOf(s). However, what to do if the given string is lowercase while enum entries following the naming convention are called using capital letters? Some people use a combination of toUpperCase() and valueOf() that work just fine but have the penalty we discussed above. However, very often people prefer to create a special field representing a “custom” name, so the simple enum like: Java enum Color { RED, GREEN, BLUE } Turns into: Java enum Color { RED("red"), GREEN("green"), BLUE("blue"), … } Let’s mention that this design has at least two disadvantages: Duplicate data: The custom name is the same as a built-in but in a different case, which can be solved much more easily. This allows using really custom names that, according to my experience, in most cases are not needed and just create so-called “edge cases” that, in turn, in most cases are just a signal of bad design and might cause a lot of “stupid” bugs. However, let’s continue. How do people often use this custom name? Java public static Color ofColor(String color) { return Arrays.stream(values()) .filter(c -> c.color.equals(color)) .findFirst() .orElseThrow(() -> new IllegalArgumentException("No enum constant %s.%s".formatted(Color.class.getName(), color))); } The implementation looks pretty nice, but this approach means that each call of ofColor() iterates over the list. Yes, in most cases enums are not huge, so the list is short, but anyway, why do this if we can just create a map from the custom name to the enum entry once during initialization and then use it with O(1) complexity? The following example solves both problems at once: it uses a case-insensitive map where the key is the standard name() of the enum entry during initialization: Java private static final Map<String, Color> colors = Arrays.stream(values()).collect(toMap(Enum::name, e -> e, (existing, replacement) -> replacement, () -> new TreeMap<>(CASE_INSENSITIVE_ORDER))); So, now the method ofColor() becomes trivial: Java public static Color ofColor(String color) { return Optional.ofNullable(colors.get(color)) .orElseThrow(() -> new IllegalArgumentException("No enum constant for " + color)); } One can argue that a map-based implementation is not always possible because sometimes the lookup criteria are too complex to be reduced to a simple key. Although I agree in general, I can say in turn that in many (if not in most) cases this is still possible. So far, the lookup key was a simple string. But what if the search criteria is a range rather than an exact value? Consider a more physically accurate model of colors as ranges of electromagnetic waves. Java public enum Color { BLUE(450, 495), GREEN(495, 570), RED(620, 750); …} How to implement the method ofWaveLength(int waveLength)? The straight-forward way is to iterate over the values of the enum and compare the given wave length with the range for each entry, i.e. implement O(n) search. But we can do better using NavigableMap, which is designed exactly for this kind of range query: Java private static final NavigableMap<Integer, Color> wavelengthMap = Arrays.stream(values()) .collect(Collectors.toMap( color -> color.minNm, color -> color, (existing, replacement) -> existing, TreeMap::new )); Unfortunately, the search method is not as trivial as in the previous example, but still very simple and fast: Java public static Color ofWaveLength(int nm) { return Optional.ofNullable(wavelengthMap.floorEntry(nm)) .map(Entry::getValue) .filter(value -> nm <= value.maxNm) .orElseThrow(() -> new IllegalArgumentException("No enum constant for wavelength: " + nm + " nm")); } Now, let’s compare the performance. Operation * 1 million times Time, ms valueOf(s) 34 valueOf(toUpperCase(s)) 78 Iteration with equals() 40 Color.ofColor() iteration 166 Color.ofColor() map 20 Color.ofWaveLength() map 32 The table shows that: As expected, toUpperCase() reduces performance twiceIteration with call of equals is a little bit more expensive than valueOf() although the enum has only three members and will grow linearly as the enum grows. The more members enum has, the more time iteration takes. Map-based implementation is even faster than one based on the built-in valueOf(). Stream-based iteration (ofColor() iteration) is surprisingly slow. Stream setup overhead (boxing, lambda dispatch, spliterator initialization) is non-trivial for tiny collections Pre-Intitialization The principle here is: do not do something several times if you can do it once. The most trivial example is string or numeric constants: Java private static final String FILE_NAME = "config.json"; private static final int MAX_VALUE = 10_000; However, the same principle applies to heavier objects — and that is where it really matters. Let’s take a look at logging. Most people are used to writing the following “magic” line at the beginning of each class (unless we use Lombok’s @Slf4j annotation): Java private static final Logger logger = LoggerFactory.getLogger(MyClass.class); Are all these modifiers (private static final) really needed? Some people try to save typing time: Java private final Logger logger = LoggerFactory.getLogger(MyClass.class); Moreover, if the logger is not static, we can do even more: Java private final Logger logger = LoggerFactory.getLogger(getClass()); This line looks better because it is error-proof: the class here is not hard-coded, so this line can be copied as-is from one class to another or inherited from the base class. So, what’s the problem? The problem is that retrieving the correct logger is potentially expensive due to synchronized registry lookups. Doing this on every instantiation adds up. A friend of mine told me that once in the company where he worked, this change in some critical path improved performance so much that they managed to reduce the AWS cluster by about one hundred large EC2 machines. The same rule applies to pattern compilation. As the benchmark table showed, compiling a pattern on every method call is nearly ten times slower than reusing a precompiled one. The result of Pattern.compile() should always be stored in a static final field. The only exception is the case when the regular expression is generated dynamically, but we should do our best to avoid such a design. Very often we have to format or parse dates. Traditionally I used SimpleDateFormat. What can be more obvious than this: Java private static final String FORMAT = "yyyy-MM-dd HH:mm:ss"; private static final DateFormat format = new SimpleDateFormat(FORMAT); Frankly speaking, I did this many times following the principle I stated above: there is no reason to create the instance every time we need it if we can create it only once. The problem is that SimpleDateFormat is not thread-safe, so sharing the same instance among different threads can cause the problem. Even worse: we can live with this bug for years without knowing about it, since it only happens under high load and in some cases can just produce slightly wrong results that can be lost in an ocean of valid data. So, should we create instances of SimpleDateFormat every time we need it and cause CPU and GC to work hard? Fortunately, starting from Java 8, we can use DateTimeFormatter instead: Java private static final DateTimeFormatter formatter = DateTimeFormatter.ofPattern(DATE_FORMAT); This class is thread-safe, so we can share its instance among different threads and get consistent results. Conclusion We started with a quote that is almost always cited incomplete. Knuth never said ignore performance — he said don't sacrifice clarity for speculative gains, while reminding us not to pass up opportunities in that critical 3%. The examples in this article live in that 3%. None of the performance issues described here should ever appear in production code. They are not hard to avoid — they require no profiler, no benchmarking framework, no architectural discussion. Just the habit of reaching for the right tool. And that habit pays off. Choosing equalsIgnoreCase() over toLowerCase().equals() is cleaner and faster. A static final logger is simpler and cheaper. A pre-built enum map is more readable and O(1). Good code and efficient code are not in conflict here — they are the same code. The only thing required is the habit of pausing for a second and asking: am I doing this n times when once would do? All code examples from this article are available on Gist. More

Running Sentiment Analysis Inside Neo4j With a Java Plugin

By Akmal Chaudhri

CORE

In a chapter of The SingleStore Cookbook, there is a complete sentiment analysis pipeline using Rust compiled to WebAssembly and loaded directly into SingleStore via its Code Engine. The result was clean: one CLI command to deploy, sentiment scoring running inside the database engine alongside the data and a full stock-price-plus-headlines analytical pipeline built on top of it. Can we do the same thing in Neo4j? Neo4j has a fully documented, officially supported extensibility model that lets us write custom functions and procedures in Java and register them directly with the database engine. Java also has a port of Valence Aware Dictionary and sEntiment Reasoner (VADER), the same lexicon-based sentiment analyzer used in the SingleStore Rust implementation. The pieces are all there. The question is how well they would fit together and what the resulting pipeline would look like compared to the SingleStore Wasm approach. This article documents an experiment from start to finish: the UDF implementation, the graph schema, a complete data loading and scoring pipeline, and a full set of analytical queries. Along the way, we also discovered that Neo4j has a second path to sentiment analysis via NLP procedures, and the choice between the two turns out to be an interesting engineering decision in its own right. The goal here isn't to claim a new sentiment-analysis technique. It's to explore what Neo4j's extension model makes possible and how the result compares with the equivalent SingleStore implementation. The full source code is available on GitHub. What We Are Building Figure 1 shows how data moves through the pipeline. CSV files are loaded into Neo4j via LOAD CSV or the Python loader. As each Headline node is created, sentiment.score() is called inline in the same Cypher statement — scoring happens inside the database at ingestion time, not in a separate application step. The resulting graph is then available for the analytical queries covered later in the article. Figure 1. Pipeline data flow The pipeline mirrors the one in the SingleStore book chapter: A VADER-based sentiment function registered with the system and callable from queriesA graph containing synthetic stock price ticks and news headlinesA set of analytical queries: per-headline scoring, daily aggregation, sentiment-vs-price joins, most positive and most negative ranking, and a live consistency check For the example in this article, we'll need a local install of Neo4j, a Docker container, or a server where we can place files and restart the process. How Neo4j Extensibility Works Neo4j lets us extend Cypher with custom Java code packaged as a .jar file. This is a fully documented and supported extensibility path. Neo4j publishes official guidance on setting up a plugin project and maintains a Neo4j Procedure Template on GitHub. Neo4j provides this extensibility model for building custom extensions. There are several extension types: User-defined functions (UDFs) – take inputs, return a single value, called inline in a query like a built-in functionUser-defined aggregation functions (UDAs) – group-level aggregation, analogous to SUM or COLLECTProcedures – more flexible, can return multiple rows and perform side effects, called with CALL For our sentiment use case, a UDF is the right fit. We pass in a string and get back a map of polarity scores. In SingleStore, the equivalent was a Table-Valued Function (TVF) that returned a row set. A Neo4j UDF returning a Map<String, Double> is the closest structural equivalent. One practical note on naming is that Neo4j maintains a list of reserved and deprecated procedure namespaces, such as db.*, dbms.*, graph.* and others. These are off-limits. The sentiment.* namespace is not reserved or deprecated, so it's a safe choice. Check User-defined procedures before choosing a namespace for any new plugin to confirm it doesn't conflict with a built-in namespace. What to Know Before We Build Because a Neo4j UDF runs inside the same JVM as the database engine, it's worth understanding a few practical considerations before diving in. These are the same considerations that apply to any extension of a running JVM process — Neo4j's own plugin authors deal with them too — and being aware of them upfront makes for a smoother build experience. Memory. If a plugin allocates more memory than the JVM has available — for example, loading a very large model file or accumulating state across calls — it can trigger an OutOfMemoryError. The VADER UDF we build here loads a compact lexicon and holds no state, so this is not a concern in practice. For more complex plugins that allocate significant heap memory, Neo4j provides a preview ProcedureMemory API where we can register allocations against the configured transaction memory limits, which prevents uncapped growth from causing database restarts. Uncaught exceptions. An unhandled RuntimeException in a UDF propagates up through the Neo4j query execution engine. Good error handling in the UDF code keeps this from becoming a problem. Infinite loops and thread starvation. A UDF that hangs — waiting on a network call, deadlocked or stuck in a loop — ties up a JVM thread from Neo4j's shared pool. The VADER UDF makes no network calls, holds no state and performs a relatively small amount of computation per call, so this is not a concern here, but it matters for more complex plugins. Dependency conflicts. Because the plugin jar shares the classpath with the database engine, any library bundled into the fat jar must not conflict with libraries Neo4j already ships. This problem was encountered during development and more on that in the build section below, including a straightforward fix. Startup failures. A jar that fails to load prevents the system from starting. The solution is always to test in a development environment first, such as Neo4j Desktop or a local Docker container, before deploying anywhere more critical. Security. A Java plugin has full access to the JVM, filesystem and network. This is the same trust model as Neo4j's own plugins and is appropriate for code we've written and reviewed. For third-party plugins from untrusted sources, the same caution applies as for any third-party code running inside a critical process. AuraDB. AuraDB supports plugins provided and certified by Neo4j, such as APOC, GDS and GenAI, but not arbitrary third-party or custom jars. The Java UDF approach in this article requires self-managed Neo4j, such as Desktop, Docker or a server install. If AuraDB is the target, the Java UDF approach described here is not available; the GenAI plugin or an external service are the alternatives. None of this should discourage us from building a Java UDF. The VADER UDF we build here is small, does one thing, makes no network calls, holds no state and uses a well-tested library. The sensible approach, which applies to any plugin development, is to build and test on a local development instance first, then deploy with confidence. In Neo4j, the steps to deploy our UDF are: Build a fat jarStop the serverCopy the jar file to the server's plugins directoryAdd an allowlist entry to neo4j.confRestart the server The deployment model differs from the Wasm approach — more on that in the build and deploy section below. Setting Up the Project Prerequisites We'll need the following before starting: Java 21 – check with java -version. Java 21 is the version used by the official Neo4j plugin template and by this articleMaven 3.8+ – check with mvn -versionNeo4j 2026.06.0 – the version used for this article, running in one of the ways described below Choosing a Neo4j Install For this experiment, we'll use either Neo4j Desktop or Docker. Neo4j also supports server installs on Linux and Windows — the plugin mechanism is the same — but we did not test that path and don't provide instructions for it here. Neo4j Desktop is the easiest starting point. Download it from Neo4j for Desktop, create a new project and start a local database server. Find the exact path to the plugins directory by clicking Open folder > plugins. Docker is convenient for a clean, throwaway environment. The command below starts Neo4j 2026.06.0 with a plugins volume mounted to a local directory, which is where we'll drop the jar: Shell mkdir -p ~/neo4j/plugins ~/neo4j/data docker run \ --name neo4j-sentiment \ -p 7474:7474 -p 7687:7687 \ -v ~/neo4j/plugins:/plugins \ -v ~/neo4j/data:/data \ -e NEO4J_AUTH=neo4j/password \ -e NEO4J_dbms_security_procedures_allowlist="sentiment.*" \ neo4j:2026.06.0 With Docker we pass the allowlist as an environment variable rather than editing neo4j.conf directly. The jar goes into ~/neo4j/plugins/ on the host. Creating the Project Structure Create a new Maven project directory: Shell mkdir neo4j-sentiment-udf cd neo4j-sentiment-udf The full directory tree should look like this when finished: Plain Text neo4j-sentiment-udf/ ├── pom.xml └── src/ ├── main/ │ └── java/ │ └── sentiment/ │ └── Sentimentable.java └── test/ └── java/ └── sentiment/ └── SentimentableTest.java The sections below cover each part in turn. Next, we'll create both source directories: Shell mkdir -p src/main/java/sentiment mkdir -p src/test/java/sentiment Maven Dependencies We'll create a pom.xml file in the project root. The structure follows the official Neo4j procedure template at Neo4j Procedure Template, with three adjustments specific to this project that are explained below. XML <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>org.neo4j.example</groupId> <artifactId>sentimentable</artifactId> <version>1.0.0-SNAPSHOT</version> <packaging>jar</packaging> <name>Neo4j Sentiment UDF</name> <description>VADER sentiment analysis as a Neo4j user-defined function</description> <properties> <java.version>21</java.version> <maven.compiler.release>${java.version}</maven.compiler.release> <neo4j.version>2026.06.0</neo4j.version> </properties>  <repositories> <repository> <id>jitpack.io</id> <url>https://jitpack.io</url> </repository> </repositories> <dependencies> <dependency> <groupId>org.neo4j</groupId> <artifactId>neo4j</artifactId> <version>${neo4j.version}</version> <scope>provided</scope> </dependency>  <dependency> <groupId>com.github.apanimesh061</groupId> <artifactId>VaderSentimentJava</artifactId> <version>v1.1.1</version> </dependency>  <dependency> <groupId>org.neo4j.test</groupId> <artifactId>neo4j-harness</artifactId> <version>${neo4j.version}</version> <scope>test</scope> </dependency> <dependency> <groupId>org.neo4j.driver</groupId> <artifactId>neo4j-java-driver</artifactId> <version>6.0.2</version> <scope>test</scope> </dependency> </dependencies> <build> <plugins> <plugin> <artifactId>maven-compiler-plugin</artifactId> <configuration> <source>21</source> <target>21</target> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>3.5.4</version> </plugin> <plugin> <artifactId>maven-shade-plugin</artifactId> <version>3.5.1</version> <executions> <execution> <phase>package</phase> <goals><goal>shade</goal></goals> <configuration>  <relocations> <relocation> <pattern>org.apache.commons.lang3</pattern> <shadedPattern>sentiment.shaded.org.apache.commons.lang3</shadedPattern> </relocation> </relocations> <artifactSet> <excludes> <exclude>org.neo4j:*</exclude> </excludes> </artifactSet> <shadedArtifactAttached>false</shadedArtifactAttached> </configuration> </execution> </executions> </plugin> </plugins> </build> </project> The three adjustments from the official template are called out inline as comments. Everything else — groupId convention, provided scope for the Neo4j dependency, the shade plugin structure and the test dependency pattern — follows the official guidance. Writing the UDF We'll create the file src/main/java/sentiment/Sentimentable.java and paste in the following: Java package sentiment; import com.vader.sentiment.analyzer.SentimentAnalyzer; import com.vader.sentiment.analyzer.SentimentPolarities; import org.neo4j.procedure.Description; import org.neo4j.procedure.Name; import org.neo4j.procedure.UserFunction; import java.util.Map; public class Sentimentable { @UserFunction("sentiment.score") @Description("Score a string with VADER. Returns compound, positive, negative, neutral.") public Map<String, Double> score(@Name("text") String text) { if (text == null || text.isBlank()) { return Map.of("compound", 0.0, "positive", 0.0, "negative", 0.0, "neutral", 1.0); } final SentimentPolarities polarities = SentimentAnalyzer.getScoresFor(text); return Map.of( "compound", (double) polarities.getCompoundPolarity(), "positive", (double) polarities.getPositivePolarity(), "negative", (double) polarities.getNegativePolarity(), "neutral", (double) polarities.getNeutralPolarity() ); } } The following implementation details are worth highlighting. The v1.1.1 API uses a static method — SentimentAnalyzer.getScoresFor(text) — rather than a mutable instance. This means there is no shared state between calls, which is what we want in a Neo4j UDF where multiple Cypher queries may invoke the function concurrently. The VADER lexicon is loaded internally by the library on first call and cached for subsequent calls. The @UserFunction("sentiment.score") annotation registers the method as callable from Cypher under that name. The @Name annotation on the parameter provides the argument name for Neo4j's function metadata and documentation — UDFs are always called with positional arguments in Cypher, as shown throughout this article: sentiment.score(row.headline). The return type is Map<String, Double>. In Cypher, this surfaces as a map literal, so callers can destructure it with dot notation: sc.compound, sc.positive and so on. In the SingleStore version, the TVF returned a row set and was used in a FROM clause. Here the UDF is called inline in a WITH or RETURN clause instead. Writing the Tests Following the official Neo4j procedure template pattern, we'll use neo4j-harness to spin up a lightweight embedded Neo4j instance in JUnit, register our UDF with it and run Cypher queries against it — all without deploying to a running database. This is the recommended testing approach in Neo4j's own documentation. We'll create the file src/test/java/sentiment/SentimentableTest.java and paste in the following: Java package sentiment; import org.junit.jupiter.api.AfterAll; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.TestInstance; import org.neo4j.driver.Driver; import org.neo4j.driver.GraphDatabase; import org.neo4j.driver.Session; import org.neo4j.harness.Neo4j; import org.neo4j.harness.Neo4jBuilders; import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertTrue; @TestInstance(TestInstance.Lifecycle.PER_CLASS) public class SentimentableTest { private Neo4j embeddedDatabaseServer; private Driver driver; @BeforeAll void initializeNeo4j() { this.embeddedDatabaseServer = Neo4jBuilders.newInProcessBuilder() .withDisabledServer() .withFunction(Sentimentable.class) .build(); this.driver = GraphDatabase.driver(embeddedDatabaseServer.boltURI()); } @AfterAll void closeNeo4j() { this.driver.close(); this.embeddedDatabaseServer.close(); } @Test void scorePositiveSentence() { try (Session session = driver.session()) { var scores = session.run( "RETURN sentiment.score('The movie was great') AS scores" ).single().get("scores").asMap(); assertTrue((Double) scores.get("compound") > 0.5); assertTrue((Double) scores.get("positive") > 0.0); assertEquals(0.0, (Double) scores.get("negative")); } } @Test void capitalizationIncreasesScore() { try (Session session = driver.session()) { var normal = session.run( "RETURN sentiment.score('The movie was great') AS scores" ).single().get("scores").asMap(); var caps = session.run( "RETURN sentiment.score('The movie was GREAT!') AS scores" ).single().get("scores").asMap(); assertTrue((Double) caps.get("compound") > (Double) normal.get("compound")); } } @Test void emptyStringReturnsNeutral() { try (Session session = driver.session()) { var scores = session.run( "RETURN sentiment.score('') AS scores" ).single().get("scores").asMap(); assertEquals(0.0, (Double) scores.get("compound")); assertEquals(1.0, (Double) scores.get("neutral")); } } @Test void nullStringReturnsNeutral() { try (Session session = driver.session()) { var scores = session.run( "RETURN sentiment.score(null) AS scores" ).single().get("scores").asMap(); assertEquals(0.0, (Double) scores.get("compound")); assertEquals(1.0, (Double) scores.get("neutral")); } } } The four tests mirror the tests we'll run manually in Neo4j Browser, but now they run automatically as part of the build. Neo4jBuilders.newInProcessBuilder() starts a lightweight embedded instance with the Sentimentable function registered; .withDisabledServer() skips the HTTP server since we only need the Bolt connection. The structure follows the official JoinTest.java pattern. Building and Deploying Step 1: Install the Maven Wrapper and build The official Neo4j procedure template uses the Maven Wrapper (mvnw), which means we only need Java installed, not a separate Maven installation. To add the wrapper to the project: Shell mvn wrapper:wrapper Then build and run the tests: Shell ./mvnw clean package Or to skip the tests during development: Shell ./mvnw clean package -DskipTests To use a globally installed Maven directly, mvn clean package -DskipTests works equally well — the wrapper is a convenience, not a requirement. Maven compiles the Java source, runs the Shade plugin and writes two jar files to target/. The one we want is sentimentable-1.0.0-SNAPSHOT.jar — the fat jar with VADER bundled inside. The original-sentimentable-1.0.0-SNAPSHOT.jar is the plain jar without dependencies, so we'll ignore it. If the build fails with a package org.neo4j.procedure does not exist error, check that the pom.xml has <scope>provided</scope> on the Neo4j dependency and that the version matches the running Neo4j instance. Step 2: Copy the Jar to the Plugins Directory Neo4j Desktop: Stop the serverOpen folder > plugins and copy sentimentable-1.0.0-SNAPSHOT.jar into that folderOpen folder > conf > neo4j.conf, find dbms.security.procedures.allowlist= and uncomment the line if it is commented outAdd sentiment.* to the end of the line Docker: Copy to the host directory mounted as /plugins: Shell cp target/sentimentable-1.0.0-SNAPSHOT.jar ~/neo4j/plugins/ Step 3: Whitelist the Function Namespace Neo4j's default dbms.security.procedures.allowlist is *, which loads all plugins. If an allowlist is configured with specific entries, any custom namespace must be included or the function will silently be unavailable — no error on startup, it simply won't exist. It's good practice to configure an explicit allowlist following the principle of least privilege. Our UDF uses only the public Neo4j procedure API, which means it doesn't require the separate dbms.security.procedures.unrestricted setting — that's only needed for extensions that access internal APIs. Step 4: Restart Neo4j Neo4j Desktop: Restart the server using the button in the Desktop UI. If Desktop shows "stopped" immediately after starting, open http://localhost:7474 directly — the server may be running before the UI reflects it. Docker: If this is the initial launch, no restart is needed — the docker run command in the Choosing a Neo4j Install section already starts Neo4j with the jar in place from the mounted plugins directory. If updating the jar after the container is already running, stop the container, replace the jar in ~/neo4j/plugins/ and then restart: Shell docker stop neo4j-sentiment cp target/sentimentable-1.0.0-SNAPSHOT.jar ~/neo4j/plugins/ docker start neo4j-sentiment The clearest confirmation that the plugin loaded correctly is to run the verification queries in step 5 below — if sentiment.score() is visible and returns results, the jar was picked up successfully. Verifying the Function We can interact with Neo4j by entering http://localhost:7474 in the browser. Step 5: Confirm the Function Loaded First, we'll check that Neo4j can see the function at all: Cypher SHOW FUNCTIONS YIELD name WHERE name STARTS WITH 'sentiment' RETURN name; Expected output: Plain Text +-----------------+ | name | +-----------------+ | sentiment.score | +-----------------+ If this returns zero rows, the jar is either not in the plugins directory, the allowlist entry is missing or misspelled or Neo4j was not fully restarted. Step 6: Run the Tests Run the following tests: Cypher RETURN sentiment.score('The movie was great') AS scores; Expected output: JSON { neutral: 0.4230000078678131, negative: 0.0, positive: 0.5770000219345093, compound: 0.6248999834060669 } Now we'll test that VADER's capitalization awareness is working: Cypher RETURN sentiment.score('The movie was GREAT!') AS scores; Expected output: JSON { neutral: 0.36899998784065247, negative: 0.0, positive: 0.6309999823570251, compound: 0.7289999723434448 } The compound score rises with the capitalized GREAT!, exactly as in the Wasm version. For the examples we tested, the Java port produces scores consistent with the Rust crate used in the book chapter. Now, we'll test the null guard. Passing an empty string should return a neutral result rather than an exception: Cypher RETURN sentiment.score('') AS scores; Expected output: JSON { neutral: 1.0, negative: 0.0, positive: 0.0, compound: 0.0 } If all three return the expected values, the UDF is working and we're ready to build the graph schema and load data. Designing the Graph Schema The graph model for this pipeline has three node labels, as shown in Figure 2. A central Stock node connects to Tick nodes via HAS_TICK relationships and to Headline nodes via HAS_HEADLINE relationships. VADER polarity scores are stored directly on each Headline node at ingestion time, making them available to any Cypher query without recomputing. Figure 2. Graph data model Plain Text (:Stock {symbol}) -[:HAS_TICK]-> (:Tick {symbol, ts, open, high, low, close, volume}) -[:HAS_HEADLINE]->(:Headline {id, symbol, ts, headline, url, publisher, compound, positive, negative, neutral}) The Stock node acts as the join key. In SingleStore the queries join tick and stock_sentiment on (symbol, DATE(ts)); in Neo4j that same co-reference is expressed by traversing from a shared Stock node to both Tick and Headline nodes with a date predicate. The relationship replaces the foreign key. Let's now run these commands to create constraints and indexes: Cypher CREATE CONSTRAINT tick_pk IF NOT EXISTS FOR (t:Tick) REQUIRE (t.symbol, t.ts) IS NODE KEY; CREATE CONSTRAINT headline_id IF NOT EXISTS FOR (h:Headline) REQUIRE h.id IS UNIQUE; CREATE CONSTRAINT stock_id IF NOT EXISTS FOR (s:Stock) REQUIRE s.symbol IS UNIQUE; CREATE INDEX tick_symbol_ts IF NOT EXISTS FOR (t:Tick) ON (t.symbol, t.ts); CREATE INDEX headline_symbol_ts IF NOT EXISTS FOR (h:Headline) ON (h.symbol, h.ts); Loading Data and Scoring Headlines Getting the Datasets The datasets, notebook and SQL files for the original SingleStore book chapter are all publicly available in the book's GitHub repository. The two CSV files we need are in the datasets subdirectory: fictitious_stocks.csv – synthetic daily OHLCV stock prices (random-walk model, fictitious symbols)raw_fictitious_headlines.csv – programmatically generated news headlines (templates + ticker symbols + financial events) We'll download both files into our local working directory. Dataset Format fictitious_stocks.csv has seven columns. The date and Name columns are renamed to ts and symbol, respectively, to match the graph schema: Plain Text date,open,high,low,close,volume,Name 2013-01-02,743.98,756.93,736.15,745.68,9142645,BBRQ-FX 2013-01-03,764.41,779.16,757.72,765.16,1208771,BBRQ-FX ... raw_fictitious_headlines.csv has five columns that map directly to the Headline node properties: Plain Text headline,url,publisher,ts,symbol BBRQ-FX stock record revenues after analyst update,http://www.hill.net/,The Stock Chronicle,2014-10-22,BBRQ-FX ... No preprocessing is needed beyond what the loader already does, such as dropping nulls, filtering the one extreme volume outlier and sorting by date. The Python Loader The data_loader.py below reads the two CSV files and writes them into Neo4j via the Python driver. Install the dependencies first if not already done so: Shell pip install -r requirements.txt Then run the loader, substituting the actual paths to the downloaded CSV files. Also replace your_password_here with your actual password. Python # data_loader.py import pandas as pd from neo4j import GraphDatabase from tqdm import tqdm URI = "bolt://localhost:7687" AUTH = ("neo4j", "your_password_here") TICK_CSV = "fictitious_stocks.csv" RAW_CSV = "raw_fictitious_headlines.csv" driver = GraphDatabase.driver(URI, auth=AUTH) def chunks(df, size): for i in range(0, len(df), size): yield df.iloc[i:i+size].to_dict("records") # load tick data tick_df = (pd.read_csv(TICK_CSV) .dropna() .query("volume <= 2_147_483_647") .rename(columns={"date": "ts", "Name": "symbol"}) .sort_values(["ts", "symbol"])) tick_batches = list(chunks(tick_df, 1000)) print(f"Loading {len(tick_df):,} tick rows in {len(tick_batches)} batches...") with driver.session() as session: for batch in tqdm(tick_batches, desc="Ticks", unit="batch"): session.run(""" UNWIND $rows AS row MERGE (s:Stock {symbol: row.symbol}) CREATE (t:Tick {symbol: row.symbol, ts: date(row.ts), open: row.open, high: row.high, low: row.low, close: row.close, volume: toInteger(row.volume)}) CREATE (s)-[:HAS_TICK]->(t) """, rows=batch) # load headlines and score at ingestion time raw_df = pd.read_csv(RAW_CSV) raw_batches = list(chunks(raw_df, 1000)) print(f"Loading {len(raw_df):,} headline rows in {len(raw_batches)} batches...") with driver.session() as session: for batch in tqdm(raw_batches, desc="Headlines", unit="batch"): session.run(""" UNWIND $rows AS row MATCH (s:Stock {symbol: row.symbol}) WITH s, row, sentiment.score(row.headline) AS sc CREATE (h:Headline { id: randomUUID(), symbol: row.symbol, ts: datetime(row.ts), headline: row.headline, url: row.url, publisher: row.publisher, compound: sc.compound, positive: sc.positive, negative: sc.negative, neutral: sc.neutral }) CREATE (s)-[:HAS_HEADLINE]->(h) """, rows=batch) print("Done.") driver.close() Run the Python program: Shell python data_loader.py The key line is sentiment.score(row.headline) AS sc inside the Cypher. This is doing what the sentimentable(i.headline) TVF call does in the SingleStore INSERT ... SELECT — computing scores at the database level in the same operation that writes the record, with no round-trip to the application layer. One important note if we need to re-run the loader is that the script uses CREATE for Tick and Headline nodes, so running it a second time without clearing the database will create duplicates rather than overwriting. Clear the database first with the following Cypher, using the Query tab: Cypher MATCH (n) CALL { WITH n DETACH DELETE n } IN TRANSACTIONS OF 100 ROWS; The batch size of 100 is deliberate — larger values can exceed the default transaction memory limit and fail. After clearing, re-run the schema constraints and indexes before running the loader again. Alternative Loading Directly From GitHub With LOAD CSV To stay entirely within Cypher and avoid Python, Neo4j's LOAD CSV command can fetch the files directly from GitHub over HTTPS. No file copying, no import directory, no Python dependencies. Run both queries using the Query tab in order — ticks first, then headlines, since the headlines query does a MATCH on Stock nodes created by the tick query. Cypher LOAD CSV WITH HEADERS FROM 'https://...' AS row CALL { WITH row MERGE (s:Stock {symbol: row.Name}) CREATE (t:Tick { symbol: row.Name, ts: date(row.date), open: toFloat(row.open), high: toFloat(row.high), low: toFloat(row.low), close: toFloat(row.close), volume: toInteger(row.volume) }) CREATE (s)-[:HAS_TICK]->(t) } IN TRANSACTIONS OF 1000 ROWS; LOAD CSV WITH HEADERS FROM 'https://...' AS row CALL { WITH row MATCH (s:Stock {symbol: row.symbol}) WITH s, row, sentiment.score(row.headline) AS sc CREATE (h:Headline { id: randomUUID(), symbol: row.symbol, ts: datetime(row.ts), headline: row.headline, url: row.url, publisher: row.publisher, compound: sc.compound, positive: sc.positive, negative: sc.negative, neutral: sc.neutral }) CREATE (s)-[:HAS_HEADLINE]->(h) } IN TRANSACTIONS OF 1000 ROWS; LOAD CSV WITH HEADERS reads the first row as column names, so the original names (row.Name, row.date) are mapped directly to the graph property names inline — the same column renaming the Python loader does with rename(). The IN TRANSACTIONS OF 1000 ROWS batching is required for the tick file at ~600,000 rows to avoid the transaction memory limit. The same delete-before-reload rule applies here: re-running either query without clearing the database first will create duplicates. The only requirement is that Neo4j has outbound HTTPS access to reach GitHub, which is the case for Desktop and local Docker. In a network-restricted server environment the Python loader with local files is the safer fallback. Next, some example queries to test using the Query tab. Headline-Level Sentiment Cypher MATCH (h:Headline) RETURN h.symbol AS symbol, date(h.ts) AS ts, left(h.headline, 30) AS headline, round(h.positive, 3) AS positive, round(h.negative, 3) AS negative, round(h.neutral, 3) AS neutral ORDER BY h.symbol, h.ts LIMIT 10; Aggregate Sentiment by Stock and Day Cypher MATCH (h:Headline) WITH h.symbol AS symbol, date(h.ts) AS ts, avg(h.positive) AS avg_positive, avg(h.negative) AS avg_negative, avg(h.neutral) AS avg_neutral, count(h) AS num_headlines RETURN symbol, ts, round(avg_positive, 3) AS avg_positive, round(avg_negative, 3) AS avg_negative, round(avg_neutral, 3) AS avg_neutral, num_headlines ORDER BY symbol, ts LIMIT 10; Join Sentiment With Closing Price In Cypher, the shared Stock node makes the symbol join implicit and we only need a date predicate. Cypher MATCH (t:Tick)<-[:HAS_TICK]-(s:Stock)-[:HAS_HEADLINE]->(h:Headline) WHERE date(t.ts) = date(h.ts) RETURN t.symbol AS symbol, date(t.ts) AS ts, round(t.close, 2) AS close, round(h.positive, 3) AS positive, round(h.negative, 3) AS negative, round(h.neutral, 3) AS neutral ORDER BY t.symbol, t.ts LIMIT 10; Most Positive Headlines Cypher MATCH (h:Headline) RETURN h.symbol, date(h.ts) AS ts, left(h.headline, 30) AS headline, round(h.positive, 3) AS positive ORDER BY h.positive DESC LIMIT 10; Most Negative Headlines Cypher MATCH (h:Headline) RETURN h.symbol, date(h.ts) AS ts, left(h.headline, 30) AS headline, round(h.negative, 3) AS negative ORDER BY h.negative DESC LIMIT 10; In the SingleStore book, CEO scandal headlines dominated the negative ranking across multiple stocks. We see the same pattern here because the underlying VADER lexicon is identical. Validate Stored Scores Against Live UDF Calls This mirrors the consistency check from the SingleStore book, where stored stock_sentiment values were compared against a fresh JOIN LATERAL sentimentable(...) call to confirm the ingestion pipeline was deterministic. Cypher MATCH (h:Headline {symbol: 'BBRQ-FX'}) WITH h, sentiment.score(h.headline) AS live RETURN h.symbol AS symbol, date(h.ts) AS ts, left(h.headline, 30) AS headline, CASE WHEN round(h.positive, 3) = round(live.positive, 3) AND round(h.negative, 3) = round(live.negative, 3) AND round(h.neutral, 3) = round(live.neutral, 3) THEN 'match' ELSE 'not match' END AS comparison LIMIT 10; Daily Average Sentiment vs. Closing Price The CTE-style aggregation from the book translates naturally to Cypher's WITH chaining. Cypher MATCH (h:Headline) WITH h.symbol AS symbol, date(h.ts) AS ts, avg(h.positive) AS avg_positive, avg(h.negative) AS avg_negative, avg(h.neutral) AS avg_neutral MATCH (t:Tick {symbol: symbol}) WHERE date(t.ts) = ts RETURN symbol, ts, round(t.close, 2) AS daily_close, round(avg_positive, 3) AS avg_positive, round(avg_negative, 3) AS avg_negative, round(avg_neutral, 3) AS avg_neutral ORDER BY symbol, ts LIMIT 10; What We Learned The experiment was a clear success. VADER runs inside Neo4j, scores headlines at ingestion time via a simple Cypher call and all the analytical queries from the SingleStore book have direct equivalents in Cypher. For the examples we tested, the Java port produces scores consistent with the Rust crate used in the SingleStore book — although independent language ports may differ in edge cases due to differences in tokenization or floating-point handling. The graph model handles the stock-tick-plus-headlines domain naturally and in several respects the Cypher queries are more expressive than their SQL counterparts — the relationship traversal from a shared Stock node replaces a keyed SQL join in a way that reflects the actual structure of the domain rather than just being an implementation detail. The graph model is a genuine advantage for the join queries. Replacing JOIN tick ON (symbol, DATE(ts)) with a graph traversal through a shared Stock node is not just syntactic preference — it reflects the actual structure of the domain. A stock symbol connects ticks and headlines naturally as a graph entity and Cypher expresses that more directly than a keyed SQL join. In-database scoring works. Calling sentiment.score(row.headline) inside the Cypher CREATE statement means scoring and ingestion happen in the same operation, with no round-trip to an application layer. This is the same goal the SingleStore Wasm pipeline achieves and the Java UDF delivers it cleanly. The dependency conflict is a one-time fix. We hit the commons-lang3 version conflict during development and it stopped the server from starting. The fix — relocating the bundled classes to a private namespace using the Maven Shade plugin — is straightforward once we know what to look for and the solution is baked into the pom.xml in this article. There are also honest differences from the SingleStore Wasm approach. Deployment requires a restart. SingleStore uses a tool that loads a function into a live database with no downtime. Neo4j requires a jar build, a file copy, a config edit and a restart. For an initial Docker launch, the jar is picked up automatically — but any subsequent update to the jar requires a container restart. The Maven Wrapper and the clear deployment steps in this article make the process repeatable. No execution sandbox. SingleStore runs each Wasm function instance in its own isolated process with a hard memory boundary. The Neo4j UDF runs in the same JVM as the server. For a small, well-behaved plugin like the VADER UDF this makes no practical difference, but it's a meaningful architectural distinction for more complex or heavyweight plugins. Language is JVM-based. The Wasm approach accepts any language that compiles to the Wasm core spec. Neo4j's extensibility model is JVM-only. For teams that want to bring existing Python or Rust models into the database, that is worth knowing about upfront. Alternative Approaches The Java UDF is the focus of this article, but it's not the only way to bring sentiment scoring close to Neo4j data. We considered several alternatives during the experiment. Some are compelling for specific use cases and others less so. Knowing the options helps us choose the right tool for our situation. Pre-scoring outside the database. Score all headlines before loading. Add the polarity scores as columns in the CSV and load everything with LOAD CSV. Nothing custom runs inside Neo4j at all. For a batch pipeline like this one, where data are loaded once and queried many times, this is entirely practical and requires no Java knowledge. The only thing we give up is the ability to call sentiment.score() inline in Cypher at query time. For many teams this will be the right answer and it's the simplest path to a working pipeline. External microservice. Deploy a small Python or Rust service that runs VADER and exposes an HTTP endpoint. An external microservice can expose VADER through an HTTP API, with the application layer calling the service before or during ingestion. This gives us complete process isolation — a crash in the sentiment service cannot touch the database — and works with AuraDB. The tradeoff is network latency on every call and the operational overhead of running a separate service. For lower-volume or interactive use cases it's a clean, flexible pattern. Neo4j GenAI plugin. Neo4j's GenAI plugin supports calling embedding and LLM APIs — OpenAI, Azure OpenAI and compatible endpoints — directly from Cypher. It's fully managed by Neo4j, works on AuraDB and requires no Java. To use a cloud LLM for sentiment classification rather than VADER’s lexicon is a well-supported, low-friction path. The tradeoff is API cost and the opacity of a large language model compared to VADER's fully transparent, inspectable lexicon — which matters in regulated domains where we need to explain a score. GraalVM native compilation. GraalVM can ahead-of-time compile Java UDFs to native binaries, reducing JVM startup overhead and memory footprint. This is a performance optimization rather than an architectural change — the code still runs inside the Neo4j process — and adds significant build complexity for modest gain in this use case. It is worth knowing about for larger, more heavyweight plugins, but not the right choice here. Wasm runtime embedded inside a Java UDF. Theoretically, we could embed a Wasm runtime such as wasmtime inside a Java UDF and execute the VADER Wasm module from within Neo4j, getting Wasm's sandbox guarantees inside Neo4j's plugin model. It's technically feasible but no published working example appears to exist and the complexity cost is high relative to the alternatives. An interesting idea to watch, but not practical today. The table below shows how these approaches compare on the dimensions that matter most. ApproachCompute locationAuraDBLanguage choiceOperational complexityPre-score outside DBCompleteYesAnyLowExternal microserviceCompleteYes (via APOC)AnyMediumAPOC NLP (cloud API)Remote serviceNo (APOC Extended required)N/ALowGenAI pluginRemote serviceYesN/ALowJava UDF (this article)Shared JVMNoJVM-basedMediumWasm-in-Java (theoretical)Wasm sandboxNoAny (via Wasm)Very high The Java UDF sits in the middle of this table — it's uniquely capable of calling sentiment.score() inline from any Cypher query without application-layer involvement and it runs entirely within the system without external API calls or network latency. Whether that inline, self-contained capability is what our use case needs is the key question. For development, experimentation and pipelines where the data and team are well understood, it's a compelling and practical approach. For other situations, the alternatives above offer different but equally valid tradeoffs. A Second Path Is APOC NLP Procedures The two approaches differ in where the computation happens, as shown in Figure 3. With the Java UDF, the VADER lexicon is bundled in the jar and scoring runs inside the Neo4j JVM — no network call, no external dependency, no per-call cost. With APOC NLP, Neo4j orchestrates calls to an external cloud API and receives scores back over the network. That single architectural difference drives most of the tradeoffs covered in this section. Figure 3. Java UDF vs. APOC NLP Neo4j already has sentiment analysis capability — it just works quite differently and it lives not in GDS but in APOC Extended, a separate component from APOC Core. APOC's NLP procedures act as wrappers around cloud-based Natural Language APIs. The supported providers are AWS Comprehend, Azure Cognitive Services and Google Cloud Natural Language. The calling pattern is straightforward. With AWS, for example: Cypher MATCH (h:Headline {symbol: 'BBRQ-FX'}) CALL apoc.nlp.aws.sentiment.stream(h, { key: $apiKey, secret: $apiSecret, nodeProperty: 'headline' }) YIELD value RETURN h.headline, value.sentiment, value.sentimentScore; And with Azure: Cypher MATCH (h:Headline {symbol: 'BBRQ-FX'}) CALL apoc.nlp.azure.sentiment.stream(h, { key: $apiKey, url: $apiUrl, nodeProperty: 'headline' }) YIELD value RETURN h.headline, value.sentiment, value.sentimentScore; The graph variant goes one step further and writes the sentiment result back as a node property automatically, with write: true in the config map. Choosing Between the Two Java VADER UDFAPOC NLP (AWS / Azure / GCP)Where scoring runsInside Neo4j JVMExternal cloud APINetwork call per batchNoYesCost per callNo API chargeAPI pricing appliesModel qualityLexicon-based (VADER)Cloud NLP / ML modelsAuraDB compatibleNoNo (APOC Extended not available in AuraDB)Java knowledge neededYesNoOffline / air-gappedYesNoDeterministic resultsYesProvider-dependentDomain tuningLimited (lexicon)Better (ML models handle context) The Java UDF is the stronger choice when scoring volume is high, API costs matter, the text is short social-media-style content that VADER was designed for, or an offline/air-gapped environment is required. The VADER lexicon is fully transparent — we can inspect why a string received a given score, which matters in regulated domains. APOC NLP is the stronger choice when Java knowledge is limited, the text requires linguistic nuance beyond VADER’s lexicon (negation, sarcasm, domain-specific vocabulary), or cloud NLP APIs are already in use for other workloads. One important constraint applies to both: APOC NLP is part of APOC Extended, not APOC Core. AuraDB includes APOC Core by default, but APOC Extended is not available in AuraDB — so neither the Java UDF nor APOC NLP works there. The GenAI plugin or an external microservice are the practical AuraDB paths. GDS, Neo4j's Graph Data Science library, does not include text-level sentiment analysis — it's graph-algorithm-oriented. Text scoring in Neo4j is either in-database via a Java UDF or delegated to a cloud NLP service via APOC. Summary The experiment confirms that Neo4j's Java extensibility model is a capable platform for in-database compute. The VADER UDF works, the graph model is a natural fit for the stock-tick-plus-headlines domain and the analytical queries translate cleanly from SQL to Cypher — in some cases more expressively, because the relationship between prices and headlines is explicit in the graph schema rather than inferred at query time through a join predicate. The more interesting engineering question is when to use a Java UDF versus the alternatives. The answer depends primarily on four factors: Deployment model (self-managed Neo4j only for UDFs)Latency and network requirements (the UDF has none; APOC NLP and external microservices introduce both)Model sophistication (VADER's lexicon is transparent and fast but limited; cloud NLP APIs offer better linguistic coverage)Operational constraints (Java knowledge, plugin management and the restart-on-update requirement all have a cost) There is no universally correct choice — the table in the APOC NLP section lays out the tradeoffs and reasonable teams will land in different places depending on their priorities. What the article does establish is that the approach works and is officially supported. Building a plugin is documented and templated. For development, experimentation and well-understood production pipelines, it's a practical and interesting path. To go further, the official Neo4j Procedure Template is an excellent starting point, neo4j-harness makes unit testing UDFs straightforward without needing a running database instance and the full Neo4j Java Reference covers procedures, aggregation functions and the complete extensibility API in depth. The full source code is available on GitHub. More

Containerizing Spark and Lakehouse Development with Docker

By Aniket Abhishek Soni

Working With Spreadsheets in Java: A Practical Overview

By Hawk Chen

CORE

Designing Rayfall: One Expression Language for a Columnar Database

By Anton Kundenko

Demystifying Thread Hopping With Swift 6.2

Ever since Swift Concurrency was introduced, its main mission has been clear: keep memory safe without making us write callback hell. But if we’re being honest, context switching-specifically thread hopping-has always been a bit of a head-scratcher. How many times have you marked an async function as nonisolated on a @MainActor class, only to watch it instantly jump off to the cooperative global pool for no obvious reason? Swift 6.2 addresses this head-on with Approachable Concurrency and its underlying flag, NonisolatedNonsendingByDefault. Let’s break down what actually changes under the hood, how @concurrent fits into the picture, and what this all looks like when stepping through real code. What Changes With NonisolatedNonsendingByDefault Before Swift 6.2 (or with Approachable Concurrency turned off), any nonisolated async function would immediately yield its execution to Swift’s global cooperative executor whenever you called await. That meant constant, often unnecessary thread switching. With Approachable Concurrency enabled (APPROACHABLE_CONCURRENCY = YES), that default behavior flips. Ordinary async methods now behave much like their synchronous counterparts. They stay on the caller’s executor by default instead of hopping away. A few quick rules to keep in mind: nonsending: The function isn’t bound to a specific actor’s isolation domain, but it keeps the execution context of whoever called it.@concurrent: The explicit opt-in attribute telling the compiler, “No, seriously, run this on the global concurrent pool.”Good to know: @concurrent automatically implies nonisolated, so writing both is redundant. Comparing the Flags: A Basic Test Let’s look at a straightforward example to see the difference in practice: Swift @MainActor class ViewModel { var title = "Hello" func updateData() { print("1:", Thread.isMain) } nonisolated func helperMethod() async { print("2:", Thread.isMain) } @concurrent func thirdMethod() async { print("3:", Thread.isMain) } } // Calling it from a MainActor context: Task { let viewModel = ViewModel() viewModel.updateData() await viewModel.helperMethod() await viewModel.thirdMethod() } Quick Compiler Tip: When testing thread execution across different isolation contexts, you might run into compiler warnings or errors when accessing Thread.isMainThread. To cleanly check the main thread without triggering actor isolation warnings, use a nonisolated helper extension: Swift extension Thread { static nonisolated var isMain: Bool { Thread.isMainThread } } Here’s what gets printed depending on your project settings: OutputAPPROACHABLE_CONCURRENCY = NOAPPROACHABLE_CONCURRENCY = YES1: updateData()truetrue2: helperMethod()false (Background)true (Main Thread)3: thirdMethod()false (Background)false (Background) What’s happening here? When set to NO: Calling helperMethod() drops off the main actor and executes on a background thread (false).When set to YES: helperMethod() isn’t isolated, but thanks to nonsending, it inherits the caller’s context. Since the calling Task runs on @MainActor, helperMethod() stays right there on the main thread.thirdMethod() is marked @concurrent, so it always hops to a background worker thread regardless of the build setting. Deep Dive: Following the Execution Chain To really see how thread hopping behaves during nested calls and returns, let’s trace a slightly more complex scenario involving a custom global actor: Swift @globalActor actor BackgroundActor { static let shared = BackgroundActor() } @MainActor class ViewModel { var name = "Swift 6" // 1. Synchronous isolated method func runTest() { print("1:", Thread.isMain) Task { await complexHelper() } } // 2. Async nonisolated helper nonisolated func complexHelper() async { print("2:", Thread.isMain) // Jumping over to our custom actor await BackgroundActor.shared.doWork { print("3:", Thread.isMain) } print("4:", Thread.isMain) // Calling a sync nonisolated helper syncHelper() } // 3. Synchronous nonisolated helper nonisolated func syncHelper() { print("5:", Thread.isMain) } } extension BackgroundActor { func doWork(_ operation: @Sendable () -> Void) async { operation() Task { print("6:", Thread.isMain) } } } Side-by-Side Execution Trace: StepAPPROACHABLE_CONCURRENCY = NOAPPROACHABLE_CONCURRENCY = YES1truetrue2falsetrue ← Stays on caller’s thread3falsefalse ← Hopped to BackgroundActor4falsetrue ← Returned to caller context5falsetrue ← Synchronous call from step 46falsefalse ← Task spawned inside BackgroundActor Why steps 2, 4, and 5 change in Swift 6.2: Step 2 (complexHelper): Because the caller is on @MainActor, complexHelperstarts executing on the main thread (true).Step 3 (doWork): We explicitly await a method on BackgroundActor, so execution correctly hops over to a background thread (false).Step 4 (After await doWork): Here’s the key difference. When doWorkfinishes, control resumes in complexHelper. Under Swift 6.2, the method remembers where it was called from, so it hops back to the Main Thread (true).Step 5 (syncHelper): This is a plain synchronous call made right after step 4, so it stays on the main thread (true). Wrapping Up Swift 6.2’s Approachable Concurrency makes writing async Swift feel a lot more natural: Fewer random context switches: Your app spends less time hopping back and forth across threads when it doesn’t need to.Predictable execution: Async code holds onto its caller’s context until you explicitly use @concurrent or call into a different actor.Easier mental model: Async methods now align much closer with how we expect synchronous code to flow, removing a big chunk of the concurrency learning curve.

By Nikita Vasilev

Text Analysis Without a Backend: Replacing an LLM Call With Intl.Segmenter and 60 Lines of JavaScript

Last spring, I had six small text features to build: flag filler phrases in a draft, score sentence-length variation, format a citation, check a document against a rubric. My first design put all six behind an API route that called a model. It worked in an afternoon. Then I priced it. Anthropic lists Claude Fable 5 at $10 per million input tokens and $50 per million output. A 700-word draft plus instructions runs about 1,500 input tokens, and users hit the button five or six times per session while they edit. The bill is survivable. The rest of the tradeoff is not. Every keystroke a user typed would leave their machine and land in someone else's logs. Every click added 900ms of round trip to something that should feel like a spellchecker. And two runs over identical input returned different advice, which turns "did my edit help?" into an unanswerable question. I rewrote all six as deterministic browser code. No API route, no server, no network. This is what that took, and where the approach breaks. What a Heuristic Actually Catches The honest framing is that heuristics and models solve different problems, and half the features people route to an LLM belong in the first category. A model is worth paying for when the task needs world knowledge or judgment: Is this argument coherent, does this paragraph follow from the last one, is this claim supported? A regular expression cannot do any of that. But "does this text contain the phrase in order to" is a lookup. "How much do sentence lengths vary" is arithmetic. "Should of be capitalized in this title" is a rule from a style manual, written down, unchanged since 2019. Sending those to a probabilistic system buys you latency and nondeterminism in exchange for nothing. The six tools I run in production all fall in the second category. They ship as static pages with inline scripts, no build-time secrets, and no runtime dependencies. Sentence Segmentation Without a Regex You Will Regret Every metric below needs sentence boundaries, so this is the piece to get right first. Splitting on /[.!?]+\s+/ collapses under real prose. Run it over four ordinary lines and watch: Code language: Text Plain Text IN : The file cost $3.50. It shipped on Jan. 5 anyway. naive: ["The file cost $3.50", "It shipped on Jan", "5 anyway."] IN : He said "stop." Then he left. naive: ["He said \"stop.\" Then he left."] One false split, one missed split, and the abbreviation list you are about to write will never end. The browser ships an ICU-backed segmenter instead: Code language: JavaScript JavaScript const SEG = new Intl.Segmenter('en', { granularity: 'sentence' }); const raw = (text) => [...SEG.segment(text)].map((s) => s.segment.trim()).filter(Boolean); ICU gets both of those cases right, along with 9 a.m., decimals and section numbers like 2.1. It has one failure I hit in production, and it is worth knowing before you ship: it breaks after title abbreviations. Code language: Text Plain Text IN : She met Dr. Chen last week. The draft grew by 3.5 pages. ICU : ["She met Dr.", "Chen last week.", "The draft grew by 3.5 pages."] The repair is a merge pass over the output rather than a rewrite of the splitter. If a segment ends in a known title, glue the next one onto it: Code language: JavaScript JavaScript const TITLE_END = /(^|\s)(Dr|Mr|Mrs|Ms|Prof|Sr|Jr|St|vs|Fig|No)\.$/i; function sentences(text) { return raw(text).reduce((out, part) => { const prev = out[out.length - 1]; if (prev && TITLE_END.test(prev)) out[out.length - 1] = `${prev} ${part}`; else out.push(part); return out; }, []); } Verified against the cases above: Code language: Text Plain Text ["Dr. Chen wrote 3.5 pages.", "She revised twice."] ["She met Dr. Chen last week.", "The draft grew by 3.5 pages."] ["The file cost $3.50.", "It shipped on Jan. 5 anyway."] ["We deployed at 9 a.m.", "Nobody noticed."] ["He said \"stop.\"", "Then he left."] ["Prof. Ada Lovelace vs. Mr. Babbage.", "Round one."] That is a twelve-entry list against the open-ended one the naive regex demands, because ICU already covers the numeric and punctuation cases that make abbreviation lists grow. Intl.Segmenter landed in Chrome 87, Safari 14.1 and Firefox 125, so a 2026 audience has it. It also does granularity: 'word', which matters the moment a user writes in Thai or Japanese, where whitespace tokenization returns one enormous token. Guard it if you support older embedded webviews: Code language: JavaScript JavaScript const hasSegmenter = typeof Intl !== 'undefined' && 'Segmenter' in Intl; Phrase Matching That Does Not Fire on Substrings The naive filler checker uses indexOf, then reports "just" inside "adjustment" and loses the user's trust in the first thirty seconds. Build one alternation with word boundaries, compile it once, and keep the phrase list in data rather than code: Code language: JavaScript JavaScript const FILLERS = [ 'in order to', 'it is important to note', 'at the end of the day', 'due to the fact that', 'a wide variety of', 'needless to say', ]; const escapeRe = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); const FILLER_RE = new RegExp( '\\b(' + FILLERS.map(escapeRe).join('|') + ')\\b', 'gi' ); function findFillers(text) { return [...text.matchAll(FILLER_RE)].map((m) => ({ phrase: m[0], index: m.index, })); } Two details that cost me a rewrite. Compile the RegExp outside the function, because a global-flagged regex carries lastIndex state and rebuilding it per call hides that bug instead of fixing it. And use matchAll rather than a while (re.exec()) loop, which is where that state bites. The phrase list is the whole product here. Mine came from marking up 200 real drafts by hand, not from asking a model what filler looks like. Measuring Variation, and the Trap Next to It Uniform sentence length reads as flat prose. The metric is standard deviation over word counts: Code language: JavaScript JavaScript function rhythm(text) { const lens = sentences(text).map((s) => s.split(/\s+/).length); if (lens.length < 2) return null; const mean = lens.reduce((a, b) => a + b, 0) / lens.length; const variance = lens.reduce((a, n) => a + (n - mean) ** 2, 0) / lens.length; return { mean, sd: Math.sqrt(variance), count: lens.length }; } Low standard deviation is a useful writing signal. It is also, and this is where teams get into trouble, one of the two features commercial AI-text detectors lean on, alongside token-level perplexity. Do not ship it as one. A peer-reviewed study in Patterns tested seven commercial detectors and found they misclassified more than half of TOEFL essays written by non-native English speakers as machine-generated, while scoring near-perfect on native-speaker samples (full text). Steady sentence patterns are what a second-language writer produces under pressure. If your product tells that user their own writing looks synthetic, you have built a discrimination engine with a progress bar on it. Report the number as rhythm. Let the writer decide. Make "No Network" a Test, Not a Promise Claiming a tool runs locally is easy. Proving it survives the next dependency bump is the engineering. Two layers. Content Security Policy on the tool pages: Code language: HTML HTML <meta data-fr-http-equiv="Content-Security-Policy" content="default-src 'self'; connect-src 'none'; img-src 'self' data:;"> connect-src 'none' kills fetch, XMLHttpRequest, WebSocket and sendBeacon. If you run first-party analytics on the same origin, drop to connect-src 'self' and lean harder on the second layer. That second layer is a Playwright spec that fails the build if anything leaves the origin: Code language: JavaScript JavaScript test('clarity checker makes no offsite requests', async ({ page }) => { const offsite = []; page.on('request', (req) => { if (new URL(req.url()).origin !== BASE) offsite.push(req.url()); }); await page.goto(`${BASE}/tools/clarity-checker/`); await page.fill('#draft', 'In order to be clear, it is important to note this.'); await page.click('#analyze'); expect(offsite).toEqual([]); }); This caught a real regression for me: a font subset I added later pulled from a CDN, which meant the browser advertised the visitor's IP and user agent to a third party on a page whose whole selling point was that nothing left the device. The CSP would have blocked the request in a browser that enforced it. The test told me before a user did. The Comparison, With Numbers LLM API routeBrowser heuristicFirst response600–1,200 msunder 5 msMarginal cost~$0.001 per runzeroSame input, same outputnoyesUser text leaves deviceyesnoWorks offlinenoyesHandles novel phrasingyesnoJudges argument qualityyesnoShips without a backendnoyes The last row decided it for me. Six static pages on a CDN have no runtime to patch, no key to rotate, and no bill that scales with traffic. When to Call the Model Anyway I still reach for one, on three conditions. The task needs judgment rather than lookup. Restructuring an argument, catching a claim the writer never supported, spotting that paragraph four repeats paragraph two. No word list gets there. The user asked for it explicitly, with the data boundary stated in plain language on the button. Silent exfiltration dressed as a feature is how teams end up in a compliance review. And the output gets checked. For anything structured, constrain the response with a schema and validate it before it touches your UI, because a model that returns prose where your parser expects an object will do it on a Friday. Everything else stayed in the browser. Six features, roughly 400 lines of JavaScript total, zero infrastructure, and a p99 that is a rounding error. The default in 2026 is to reach for an API key first. Check whether the problem is a lookup before you do.

By Kevin Brown

A Practical Guide to Using Java Virtual Threads With JMS Listeners

Scaling JMS Listeners With Java Virtual Threads Event-driven architecture is widely used in enterprise systems to decouple services, absorb traffic spikes, and move work out of request paths. Java Message Service (JMS), now standardized as Jakarta Messaging, remains common in systems built around ActiveMQ, IBM MQ, Solace, TIBCO EMS, and similar brokers. Java 21 virtual threads give these systems another scaling option. A JMS listener often spends more time waiting on a database, HTTP service, cache, or file system than it spends using the CPU. Moving that blocking work to virtual threads can reduce platform-thread pressure without forcing the application into a reactive programming model. However, virtual threads do not make the broker, database, or downstream services unlimited. They also do not change acknowledgment, transaction, redelivery, or ordering semantics. A safe design combines virtual threads with bounded JMS consumer concurrency, explicit resource limits, idempotency, and production metrics. This article explains what virtual threads change for Spring JMS listeners, how to configure them explicitly, and how to avoid moving the bottleneck from the JVM into the rest of the system. The Traditional JMS Listener Model A typical queue-based flow moves messages from the broker through a Spring listener container and into a handler that calls downstream systems. Figure 1 compares how that handler work occupies platform threads with how it runs when the container's consumer-invoker tasks use virtual threads. Figure 1. Platform threads compared with virtual-thread consumer invokers in a Spring JMS listener. The container manages JMS connections, sessions, consumers, acknowledgments, and listener invocation. The handler contains the business logic: Java @JmsListener( destination = "orders.created", containerFactory = "jmsListenerContainerFactory" ) public void handle(OrderCreatedEvent event) { Customer customer = customerClient.getCustomer(event.customerId()); inventoryService.reserve(event.orderId(), customer); orderRepository.markAsProcessing(event.orderId()); } This code is easy to read, but each downstream operation may block. With platform threads, an operating-system-backed thread remains occupied while a query or network call is waiting. When enough listener threads are blocked, new messages wait even if the CPU is not saturated. The application has become thread-bound rather than CPU-bound. Before virtual threads, teams usually increased the listener thread pool, scaled out more service instances, or rewrote the flow around asynchronous or reactive APIs. Those options remain valid, but each has a cost. Larger platform-thread pools use more memory and add scheduling overhead. More instances increase infrastructure and operational work. Reactive code can scale efficiently, but it changes libraries, control flow, debugging, and error handling. What Virtual Threads Change A virtual thread is still a java.lang.Thread, but it is scheduled by the JVM rather than being permanently tied to one operating-system thread. The platform thread that temporarily runs a virtual thread is called its carrier. When a virtual thread blocks on supported I/O, the JVM can unmount it from the carrier. The carrier is then free to run another virtual thread. This lets an application maintain straightforward, sequential code while supporting many concurrent blocking operations. As Figure 1 shows, virtual threads that are waiting on supported I/O can unmount from their carriers, leaving those carriers available to execute other ready work. Virtual threads can improve throughput when platform-thread scarcity is the limiting factor. They do not make an individual database call or HTTP request faster, and they do not add CPU capacity. Good candidates include handlers dominated by: JDBC callsBlocking REST or gRPC clientsCache lookupsFile or object-storage operationsLegacy synchronous SDKsSynchronous orchestration across downstream systems Weak candidates include handlers dominated by: CPU-heavy transformationsEncryption or compressionImage or video processingMachine learning inferenceLarge in-memory aggregation The JDK guidance is to create a virtual thread per task rather than pool virtual threads. Limited resources should be protected with explicit mechanisms such as semaphores, rate limiters, connection pools, and framework concurrency settings. The JMS Detail That Changes the Design For Spring's DefaultMessageListenerContainer, a listener thread normally belongs to a consumer invoker. That invoker owns or reuses a JMS Session and MessageConsumer and may process many messages during its lifetime. Therefore, enabling virtual threads does not necessarily create one new virtual thread for every message. It places the container's consumer tasks on virtual threads. The distinction matters because raising concurrency also raises the number of active JMS consumers and sessions. Those broker-side resources are not as cheap as virtual threads. The right side of Figure 1 models this relationship explicitly: a configured consumer-invoker task runs on a virtual thread and may process multiple messages during its lifetime. This architecture is still useful. A consumer can unmount from its carrier while its handler waits on downstream I/O. But the listener container's concurrency remains the primary control over how many messages can be processed at once. Configure the JMS Executor Explicitly Spring Boot can enable virtual threads for several Boot-managed execution paths with spring.threads.virtual.enabled=true. Do not assume that this property alone proves that a JMS listener container uses virtual threads. Configure the JMS container's executor explicitly and verify it at runtime. Figure 2 separates the application wiring from the runtime flow. The explicit connection between the virtual-thread-enabled TaskExecutor and the JMS listener factory is the important step; the container's concurrency setting continues to bound active consumers and sessions. Figure 2. Explicit Spring JMS virtual-thread wiring and runtime message flow. The following example uses Java 21 or later and Spring Framework 6.1 or later. It supplies a virtual-thread-enabled SimpleAsyncTaskExecutor to the listener container factory: Java import java.util.concurrent.Executor; import jakarta.jms.ConnectionFactory; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.core.task.SimpleAsyncTaskExecutor; import org.springframework.jms.config.DefaultJmsListenerContainerFactory; @Configuration(proxyBeanMethods = false) class JmsConfiguration { @Bean("jmsVirtualThreadExecutor") SimpleAsyncTaskExecutor jmsVirtualThreadExecutor() { SimpleAsyncTaskExecutor executor = new SimpleAsyncTaskExecutor("jms-vt-"); executor.setVirtualThreads(true); return executor; } @Bean DefaultJmsListenerContainerFactory jmsListenerContainerFactory( ConnectionFactory connectionFactory, @Qualifier("jmsVirtualThreadExecutor") Executor executor ) { DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory(); factory.setConnectionFactory(connectionFactory); factory.setTaskExecutor(executor); // Example limits only. Derive these from load tests and // the safe capacity of the broker and downstream systems. factory.setConcurrency("10-100"); // Prefer transactional JMS acknowledgment when redelivery // on listener failure is required. factory.setSessionTransacted(true); return factory; } } SimpleAsyncTaskExecutor.setVirtualThreads(true) requires Java 21. Spring Framework 6.2 also added DefaultMessageListenerContainer.setVirtualThreads(true) for applications that construct the listener container directly and use its internal default executor. If a Spring Boot application uses Boot's DefaultJmsListenerContainerFactoryConfigurer, apply it before the explicit executor, concurrency, and transaction overrides so that other Boot JMS properties are retained. Virtual threads are daemon threads. In a non-web worker where no other non-daemon thread keeps the JVM alive, use Spring Boot's spring.main.keep-alive=true or an equivalent application-lifecycle mechanism. Do not rely on incidental threads created by a broker client to keep the process running. A small startup test can confirm the execution mode: Java if (!Thread.currentThread().isVirtual()) { throw new IllegalStateException( "The JMS listener is not running on a virtual thread" ); } Use this as a test or temporary diagnostic rather than performing it for every production message. Also confirm the active container factory when an application defines more than one. Bound Concurrency Around Real Capacity Virtual threads reduce thread scarcity. They do not remove resource scarcity. A listener can still be limited by: JMS sessions and consumersBroker prefetch, consumer windows, or creditDatabase connectionsHTTP client connectionsDownstream rate limitsMemory used by in-flight payloadsTransaction locksCPU A useful first estimate comes from Little's Law: Shell required concurrency ~= target throughput x average processing time If the target is 200 messages per second and the average handler time is 250 milliseconds, the initial estimate is: Shell 200 messages/second x 0.25 seconds = 50 concurrent handlers That value is only a starting point. It must be capped by the safe capacity of every dependency. If each message holds a database connection and the usable pool capacity is 30, setting listener concurrency to 100 may only create 70 additional waiters. If a payment API permits 40 concurrent requests, protect that call separately with a semaphore or rate limiter. The concurrency range 10-100 in the example means that the container can maintain a baseline and scale to a maximum. It does not guarantee that 100 is safe, and a maximum of 100 may be much too high for some brokers or workloads. Broker flow-control settings matter as well. Excessive prefetch can move a large backlog from the broker into consumers, increase the number of unacknowledged messages, and make recovery less predictable. Keep enough prefetched work to feed consumers, but avoid using prefetch as an unbounded application queue. Acknowledgment and Transactions Must Be Deliberate Virtual threads do not change message-delivery guarantees. This is especially important with Spring's DefaultMessageListenerContainer. In its default AUTO_ACKNOWLEDGE mode, the container acknowledges before listener execution, so a listener exception does not cause redelivery. If the application requires rollback and redelivery after a handler failure, use a transacted JMS session or an appropriately configured external transaction manager. A local JMS transaction covers JMS receipt and JMS sends performed through the same session. It does not automatically include a database transaction. A database commit can succeed, and the JMS commit can fail, causing the message to be delivered again. There are three common strategies: Use idempotent handlers and local transactions.Use an inbox/outbox design to make database effects repeatable and outbound publication reliable.Use JTA/XA when atomic coordination across JMS and another transactional resource is required, and its operational cost is justified. Figure 3 shows the inbox/outbox lifecycle, including the duplicate path, the separate JMS acknowledgment boundary, broker-managed redelivery, and dead-letter handling. Figure 3. Idempotent JMS processing, acknowledgment, retry, and dead-letter lifecycle. Do not treat @Transactional on a database service as proof that the JMS acknowledgment participates in the same transaction. Verify which transaction manager is active and which resources it coordinates. Make the Consumer Idempotent Redelivery can occur after broker failover, transaction rollback, application restart, timeout, or a failure between two resource commits. Higher concurrency also makes race conditions in duplicate detection easier to expose. An inbox table is a common solution. As shown in Figure 3, the application atomically inserts the message ID and applies the business changes in the same database transaction. A duplicate key follows a safe no-op path instead of repeating the business effect. The database must enforce a unique constraint on the message ID. A separate exists() check is not enough because two concurrent deliveries can both observe that the row is absent. Java @Transactional public void process(OrderCreatedEvent event) { boolean firstDelivery = processedMessageRepository.tryInsert(event.messageId()); if (!firstDelivery) { return; } orderService.apply(event); } tryInsert should use an atomic insert-if-absent operation protected by a unique key and report a duplicate without committing a separate transaction. Avoid catching a generic constraint exception if the persistence provider marks the whole transaction rollback-only. If the business update fails, the transaction should roll back both the inbox insert and the business changes. External side effects need their own idempotency strategy. For example, send an idempotency key to a payment API or persist an operation state before invoking a service that cannot participate in the local transaction. Keep Transactions and Retries Short Avoid holding a database or JMS transaction open while a slow external service retries for minutes. The risky pattern begins a transaction, calls an external API, waits and retries, and only then updates the database and commits. This can hold locks, database connections, JMS sessions, and unacknowledged messages. A virtual thread makes the waiting thread cheaper, but it does not release those resources. A safer design, illustrated in Figure 3, commits the business update and outbox record as local intent and continues asynchronously through an outbox publisher. The database update and outbox insert occur in one local transaction. A separate publisher sends pending outbox records and marks them complete. If the inbound JMS message is redelivered after the database commit, the inbox key prevents the business update and outbox insert from being repeated. Long retry delays should normally be handled with broker redelivery delay, a retry queue, or a scheduler. Sleeping a virtual thread is cheap from a carrier-thread perspective, but the listener may still hold a JMS consumer, session, transaction, and message during the delay. Classify errors before retrying: Failure typeTypical responseTransient network or dependency failureRetry with exponential backoff and jitterRate limitHonor the server's delay and reduce concurrencyInvalid message schemaSend to a dead-letter queueMissing required business dataDead-letter or route for correctionRepeated unknown failureStop after a bounded attempt count and alert Every production listener should define a maximum redelivery count, dead-letter destination, replay procedure, and owner for investigating poison messages. Do Not Detach Work From the Listener Carelessly A tempting design is to let the JMS listener receive a message, submit the real work to another executor, and return immediately. This can create more parallelism, but it can also acknowledge the message before the work finishes. It may also cross thread boundaries with a JMS Session, which is single-threaded by contract. Transaction context, error propagation, and redelivery behavior can all be lost. Let the listener container own the handler's execution unless the application deliberately implements a handoff protocol. A safe handoff usually means persisting the message or command durably before the listener returns, not merely placing a Runnable in an in-memory executor. Preserve Ordering Where It Matters Higher concurrency changes ordering behavior. Once a queue has multiple active consumers, messages can complete in a different order from the order in which the broker delivered them. Choose the ordering scope explicitly: Keep concurrency at one for strict global ordering.Partition or route messages by a business key.Serialize processing for the same key.Add sequence checks when events can arrive out of order.Design state transitions to reject stale events. Virtual threads are easiest to adopt when messages are independent or when ordering is limited to a partition or business key. For topics, do not increase consumer concurrency as if the destination were a queue. Depending on subscription configuration, additional topic consumers can receive additional copies of each message. Review durable and shared subscription semantics for the broker and container. Test the Bottleneck, Not Just the Thread Count An illustrative order-processing workload may perform one database read, two HTTP calls, one database update, and one outbound event for each message. Compare platform threads and virtual threads with: The same message corpus and payload distributionThe same acknowledgment and transaction settingsThe same database and HTTP pool limitsThe same broker prefetch or creditThe same retry and dead-letter policyA controlled concurrency ramp Measure more than throughput: metricwhat it revealsQueue depth and oldest-message ageBacklog and user-visible delayConsume rateSustainable throughputHandler p50, p95, and p99 latencyNormal and tail behaviorScheduled and active JMS consumersActual container concurrencyPlatform and virtual thread countsWhether thread pressure movedCarrier CPU and pinned-thread eventsScheduler or compatibility problemsDatabase pool utilization and wait timeDatabase saturationHTTP pool utilization and timeoutsOutbound connection pressureDownstream throttlingRate-limit pressureRedelivery and DLQ countsFailure amplificationHeap and garbage collectionCost of in-flight work Virtual threads are successful when the system sustains the required throughput with lower platform-thread pressure and without increasing timeouts, throttling, redelivery, or tail latency. If throughput rises while downstream errors rise faster, the system is not healthier. It is only delivering overload more efficiently. Diagnose Pinning and Provider Compatibility On Java 21, a virtual thread can pin its carrier when it blocks while executing certain synchronized or native code. Occasional short pinning is usually harmless. Frequent long pinning can reduce scalability. Use Java Flight Recorder's jdk.VirtualThreadPinned event or run a load test with: Shell -Djdk.tracePinnedThreads=full Do this with the actual JMS provider, JDBC driver, HTTP client, monitoring agents, and security libraries used in production. Compatibility cannot be inferred from a synthetic Thread.sleep benchmark. JDK 24's JEP 491 removes nearly all pinning caused by synchronized methods and blocks, but native or foreign-function interactions and third-party behavior still deserve testing. Decision Matrix scenariovirtual-thread fitBlocking JDBC callsStrongBlocking REST or gRPC callsStrongLegacy synchronous SDKsStrongHigh-volume, I/O-bound queue listenersStrong with bounded consumersCPU-heavy transformationWeakStrict global orderingLimitedSmall downstream capacityUseful only with strict limitsWeak acknowledgment or retry designFix delivery semantics firstNo observabilityAdd measurements first Production Checklist Before enabling virtual threads for JMS listeners, confirm that: The application runs on Java 21 or later.The JMS executor is explicitly configured and verified as virtual.Listener concurrency is capped by measured downstream capacity.Broker prefetch, consumer window, or credit is tuned.Acknowledgment and transaction behavior is documented and tested.Duplicate processing is prevented with an atomic idempotency mechanism.Retries are bounded, delayed, and classified.A dead-letter queue and replay process exist.Ordering requirements are explicit.Load tests use real drivers and representative dependencies.Queue age, tail latency, pool saturation, redelivery, and pinned-thread events are monitored. Conclusion Virtual threads are a strong fit for JMS listeners that spend much of their time waiting on blocking I/O. They let teams preserve simple, imperative Java code while reducing the platform-thread cost of concurrent message processing. The safe adoption pattern is not “turn on virtual threads and remove the limits.” It is: Put the listener container's consumer tasks on virtual threads.Bound consumer concurrency using broker and downstream capacity.Make acknowledgment, transactions, and idempotency explicit.Test with the real provider and dependencies.Measure where the bottleneck moves. When those controls are in place, virtual threads can modernize an established JMS application without requiring a reactive rewrite. They make waiting cheaper. The architecture still has to decide how much work the system can safely accept. References JEP 444: Virtual ThreadsOracle Java 21 Virtual Threads GuideSpring Framework: DefaultMessageListenerContainerSpring Framework: Processing JMS Messages Within TransactionsSpring Boot 3.2 Release Notes: Virtual Thread SupportJakarta Messaging 3.1 SpecificationJEP 491: Synchronize Virtual Threads Without Pinning

By Krishna Kandi

Why Is the Agent Card Important?

Let's begin with the definition of an AI agent. Agents are software entities that perform tasks autonomously on behalf of a user or another program. Another way to say it is that agents can perceive the environment, think, and act to achieve a specific goal with minimal human intervention. Action is the key here. For example, if I ask my agent to book a flight from Bengaluru to Delhi. The agent will perform the following tasks. Check the flight availabilityCompare priceAsk for confirmation (Human in the loop)Book the ticket (Action) Now, can we use the same agent for every kind of action? The answer is no. It will be akin to building a monolithic application. Rather, we will prefer an architecture similar to microservices or multiple APIs designed for different functionalities. We will create multiple agents specialized for acting on specific tasks. Let's extend our previous example and think about multiple agents to build a complete travel solution. We have agents such as: Travel Agent → books flightsHotel Agent → reserves hotelFinance Agent → checks budget Now, if we have to achieve a common business goal (booking a flight and hotel after comparing the price), there will be a need for agents' collaboration and interaction. This is where the A2A protocol comes in. A2A is an open protocol that complements Anthropic's Model Context Protocol (MCP). This means MCP standardizes how AI applications connect to data sources, databases, and APIs. A2A focuses on how specialized, autonomous agents (e.g., a "Sales Agent" and a "Finance Agent") "talk" and exchange information to achieve a goal, even if they are built by different providers (OpenAI, Anthropic, Google) and on different frameworks. Agent Card is one of the key capabilities that facilitates communication between Client Agent and Remote Agent. In other words, Agent Card makes A2A possible. Agents can advertise their capabilities using an “Agent Card” in JSON format, allowing the client agent to identify the best agent that can perform a task and leverage A2A to communicate with the remote agent. We can understand agent card with an analogy. You might have seen WSDL file when there is a soap web service is exposed or open api specification for RESTFul apis. WSDL or Open API Specification describes the operations, methods, input, output etc. Similar to this Agent Card make the Agent discoverable which means the agent can actively broadcast its presence, capabilities, and endpoints so that other AI agents or orchestrators can find it and use it automatically, without a human developer having to manually hardcode the connection. (This is analogy is completely from two different software architecture. I have used this for simplifying the visualisation of Agent Card). Agent Card defines the following: What does the agent do?When should this agent be used?What input does this agent expect?What output does it return?What security schemes are supported by the agent?What is the endpoint to call this agent? If we take the previous analogy of an API, each API has a contract that defines input, output, endpoints, methods, etc. Similarly, you can understand an Agent Card as a clear contract for an Agent. JSON { "url": "https://api.travelbot-ai.com/v1/a2a", "documentationUrl": "https://docs.travelbot-ai.com/guide", "capabilities": { "streaming": true, "pushNotifications": true, "stateTransitionHistory": false }, "authentication": { "type": "bearer", "description": "JWT token obtained via OAuth2 client credentials flow." }, "defaultInputModes": ["text"], "defaultOutputModes": ["text", "data"], "skills": [ { "id": "skill-find-flights", "name": "Search Flights", "description": "Finds available flights based on origin, destination, and dates.", "tags": ["travel", "flights", "search"], "InputModes": ["text", "data"], "OutputModes": ["data"], "examples": [ "Find me a one-way flight from JFK to LAX on October 12th." ] }, { "id": "skill-book-hotel", "name": "Reserve Hotel Room", "description": "Books a specific hotel room for given check-in/check-out dates.", "tags": ["travel", "hotels", "booking"], "InputModes": ["data"], "OutputModes": ["text", "data"], "examples": [ "Book the Deluxe King Room at The Grand Hotel from Nov 1 to Nov 5." ] } ] } To see exactly how an Agent Card operates, it helps to look at its structure. In an Agent-to-Agent (A2A) workflow, a client agent requests this card from a server agent before sending a task, establishing exactly how they will interact. The key fields of the agent card are: URL: Where to connect to the agentDocumentationUrl: The user manual/guideCapabilities: What special features it supports (like live streaming or notifications)Authentication: How to securely log in (e.g., passwords, tokens)DefaultInputModes / DefaultOutputModes: How it talks and listens by default (text, audio, data)Skills: A list of specific jobs the agent can do, including details on how each job works To demonstrate this, we can build an agent with an agent card. I will use MuleSoft A2A Task Listener to demonstrate this. Do remember, Agent Card makes Agent-to-agent communication seamless; however, it is not limited to a2a. Any client that we want to connect to an agent and use it will be utilizing the Agent Card to understand the capabilities and skills of the agent. Step 1: Create a project in MuleSoft using the A2A Task Listener. Step 2: Configure A2A. Step 3: Configure the HTTP Listener. Step 4: Deploy the server. Step 5: Retrieve the agent-card using the local URL (http://localhost:8081/support-agent/.well-known/agent-card.json). Step 6: Deploy the code to CloudHub and test it again. You will receive the response as provided below: JSON { "name": "Travel Agent", "description": "Handles flight and hotel booking task.", "url": "https://travel-agent-of3h9v.5sc6y6-3.usa-e2.cloudhub.io/support-agent", "provider": { "organization": "MuleSoft", "url": "https://www.mulesoft.com" }, "version": "1.0.0", "capabilities": { "streaming": false, "pushNotifications": false, "stateTransitionHistory": false }, "defaultInputModes": [ "application/json", "text/plain" ], "defaultOutputModes": [ "application/json", "text/plain" ], "skills": [ { "id": "skill-find-flights", "name": "Search Flights", "description": "Finds available flights based on origin, destination, and dates.", "tags": [ "Flight Booking" ] }, { "id": "skill-book-hotel", "name": "Reserve Hotel Room", "description": "Books a specific hotel room for given check-in/check-out dates.", "tags": [ "Hotel Booking" ] } ], "supportsAuthenticatedExtendedCard": false, "preferredTransport": "JSONRPC", "protocolVersion": "0.3.0" } This will be used by the Client Agent to discover the skills of other agents and send the task request. Please watch the video for step-by-step implementation: I hope this helps. Let me know if you liked it.

By Ajay Singh

Java Enterprise Is Already Ready for the AI Era

Artificial intelligence is changing software engineering, impacting automation, user interaction, data analysis, and application development. Developers are evaluating how their technology stacks fit with these changes. For Java developers in enterprise settings, a main question is whether the Java enterprise ecosystem is prepared for AI. The short answer is yes. You do not need to abandon Java or wait for a new platform to build AI-enabled applications. Java already provides a mature ecosystem of AI libraries, model providers, APIs, and integration patterns. Jakarta EE offers the capabilities required to deploy these technologies in production-grade enterprise systems today. The ecosystem is evolving, with new initiatives exploring perfect integration of AI concepts within Jakarta EE APIs and programming models. This article reviews existing capabilities, Jakarta EE’s role within modern AI architectures, and potential future developments. AI and Software Engineering When applying artificial intelligence in software engineering, it is important to distinguish the different ways AI can be used throughout the development lifecycle. AI can assist with documentation, testing, code reviews, architecture exploration, and code generation. Architecturally, these uses fall into two categories: using AI to develop software and integrating AI within the software itself. The first category, AI-assisted software development, is currently the most common. Developers use AI tools to generate, explain, refactor, or test code. While these tools can boost productivity, they also introduce risks if not used with proper engineering discipline. Insufficient context, unreviewed code, or tools lacking architectural constraints can cause defects, security issues, complexity, or inconsistent design. AI does not replace the engineering team; it remains their responsibility to use it effectively. New methodologies are emerging to structure this interaction. Approaches like vibe coding focus on rapid development through conversational AI, while Spec-Driven Development offers explicit requirements, constraints, and context before code generation. Agent-based workflows increasingly use repositories with instructions, specifications, and Markdown files to give coding agents the required context. These approaches do not require abandoning Java; Java projects can already employ these techniques. The second category entails integrating AI within the application itself, making AI part of the application's runtime behavior rather than just assisting developers. Applications may use a large language model (LLM) to classify information, generate content, extract structured data, retrieve knowledge, execute tools, or make decisions within business workflows. This combination delivers a fundamental architectural change. Traditional enterprise applications are predominantly deterministic: developers define process flow using methods, conditions, rules, workflows, and state changes. With the same inputs and state, the execution path is predictable. In contrast, AI-enabled applications can present a dynamic execution model, where some behavior is determined at runtime via the LLM. However, not every AI-enabled application should surrender control to the model. In practice, AI architectures exist on a spectrum of autonomy. At one end, the model functions within a tightly controlled deterministic workflow. As autonomy increases, the model can select tools, plan steps, evaluate results, and coordinate more complex actions. This evolution is reflected in the Core Autonomy Patterns, which start with deterministic directed acyclic graph (DAG) workflows and progress toward more autonomous approaches such as retrieval-augmented generation (RAG), reflection, planning, ReAct, multi-agent systems, and Model Context Protocol (MCP) integrations. As flexibility increases, so does the architectural responsibility for observability, security, testing, governance, failure handling, and control. Recognizing this distinction is essential when evaluating Jakarta EE’s readiness for AI. The first category already integrates naturally with Java development tools. The second stresses the importance of the enterprise platform: AI applications still require dependency injection, configuration, REST APIs, persistence, messaging, transactions, security, observability, asynchronous execution, and integration with external systems. These are the capabilities Jakarta EE was designed to provide. Jakarta EE and AI Now Java and Jakarta EE are ready for the AI era. Integrating AI does not require leaving the enterprise Java ecosystem or waiting for new specifications. Jakarta EE applications can already use large language models (LLMs), embed AI in business workflows, and employ these capabilities within the wider enterprise platform. This is evident inside real-world applications. For example, Skillwell Simulate, a Jakarta EE-based platform, integrates with AWS services and uses Amazon Bedrock for AI features. This shows that Jakarta EE applications can adopt modern AI services while retaining the benefits of established enterprise architecture. At the lowest abstraction level, applications can integrate directly with AI providers such as OpenAI, Anthropic, Google, and Amazon Bedrock using their APIs or Java SDKs. This approach delivers full access to provider-specific features but increases coupling. Each provider uses different API models, configurations, formats, authentication, and features. Supporting multiple providers can add boilerplate and increase complexity. Enterprise developers are familiar with this challenge. Different vendors and technologies offer different capabilities, so abstractions provide a unified programming model. AI integration is now adopting a similar approach. OmniHai is a lightweight Java AI library for Jakarta EE and MicroProfile applications. Instead of requiring each vendor's SDK, OmniHai provides a consistent AIService abstraction and communicates directly with provider REST APIs. It currently supports OpenAI, Anthropic, Google AI, xAI, Mistral, Meta AI, Azure OpenAI, OpenRouter, Hugging Face, Ollama, and custom providers. With CDI, an AI provider can be injected directly into a Jakarta EE component: Java @Inject @AI(provider = AIProvider.ANTHROPIC,apiKey = "your-anthropic-api-key") private AIService claude; The application interacts with AIService instead of provider-specific APIs. This enables chat interactions to use a consistent programming model across providers: Java String response = claude.chat( "Explain microservices", ChatOptions.newBuilder() .systemPrompt("You are a helpful software architect.") .temperature(0.5) .maxTokens(500) .build() ); OmniHai also supports asynchronous and streaming operations through the same abstraction. Conceptually, this approach is similar to abstractions like EntityManager in Jakarta Persistence: the application uses a common API while implementation details remain hidden. Although not a perfect comparison, it illustrates OmniHai’s role in managing multiple AI providers. LangChain4j CDI offers a higher-level programming model. Instead of working directly with an AIService object, developers define an AI service as a Java interface. LangChain4j CDI detects interfaces annotated with @RegisterAIService and supplies their implementations as CDI beans. For example: Java @RegisterAIService public interface AssistantService { @SystemMessage("You are a helpful assistant.") String chat(String userMessage); } Developers do not write implementation classes. The infrastructure generates the implementation and connects the interface to the configured language model. The resulting service can be injected as any other CDI bean: Java @Path("/assistant") public class AssistantResource { @Inject AssistantService assistant; @GET @Path("/chat") public String chat(@QueryParam("message") String message) { return assistant.chat(message); } } This programming model will be familiar to Jakarta EE developers. It is similar to the repository abstraction in Jakarta Data, where developers define the contract through an interface and the infrastructure supplies the implementation. Although the technologies address different needs, this model reduces the amount of infrastructure code developers must write. LangChain4j goes beyond basic model invocation. It offers unified APIs for over 20 LLM providers and includes abstractions for tools, Retrieval-Augmented Generation (RAG), chat memory, structured outputs, agents, embedding stores, and other AI features. Supported integrations include Amazon Bedrock, Anthropic, Azure OpenAI, Google AI Gemini, OpenAI, Mistral, OCI Generative AI, among others. These options represent different levels of abstraction: OmniHai serves as a lightweight template-style abstraction, allowing the application to invoke operations through a common AIService. LangChain4j CDI advances this by supplying a declarative interface-based model, where developers describe the AI service and the infrastructure provides its implementation. Both approaches ensure the application stays a Jakarta EE application. Once an AI capability is available as a CDI bean, it integrates perfectly with the platform. REST endpoints can expose it, Jakarta Persistence or Jakarta NoSQL can supply data, Jakarta Security can protect its operations, Jakarta Messaging can trigger asynchronous workflows, and other Jakarta EE APIs continue their roles. The question is no longer whether Jakarta EE can integrate with AI; it already does. The key architectural decision is now the required level of abstraction: direct provider integration for maximum control, a lightweight common API like OmniHai, or a richer AI programming model such as LangChain4j CDI. Jakarta EE and Future Jakarta EE already supports AI integration, and the platform continues to evolve. Jakarta EE 12 focuses on improving the data layer, with updates to Jakarta Data, Jakarta Persistence, Jakarta NoSQL, and the new Jakarta Query specification. These improvements are especially important for AI applications that rely on enterprise data, persistence, retrieval, and contextual content. The primary AI-focused initiative is Jakarta Agentic AI, which has released its first milestone. Its purpose is not to replace LangChain4j or provider SDKs, but to offer a standard programming model for building AI agents with Jakarta EE. The specification defines a small set of concepts to structure agent workflows based on annotations, thus making the developer's life way easier: APIPurpose @Agent Declares an agent class @Trigger Defines the workflow entry point @Decision Determines whether and how the workflow proceeds @Action Defines a step in the workflow @Outcome Marks the end of the workflow @HandleException Handles exceptions inside the workflow @WorkflowScoped Provides one CDI context per workflow execution LargeLanguageModel Injectable facade for interacting with an LLM Result Represents the result of a decision This example presents a simplified fraud-detection agent and illustrates how Jakarta Agentic AI integrates with the Jakarta EE programming model. The agent uses the LargeLanguageModel facade for AI interaction and leverages Jakarta Persistence and Jakarta NoSQL to access enterprise data. As a result, AI capabilities are incorporated as part of the application, not as a separate programming environment. Java @Agent public class FraudDetectionAgent { @Inject LargeLanguageModel model; @Inject EntityManager entityManager; @Inject Template template; @Trigger private void handleTransaction( @Valid BankTransaction transaction) { } @Decision private Result checkFraud(BankTransaction transaction) { CustomerHistory history = template .find(CustomerHistory.class, transaction.customerId()) .orElse(null); String output = model.query( """ Analyze this transaction for potential fraud using the transaction and customer history. """, transaction, history); return new Result(isFraud(output), null); } @Action private void handleFraud( Fraud fraud, BankTransaction transaction) { if (fraud.isSerious()) { alertBankSecurity(fraud); } } @Outcome private void markTransaction( BankTransaction transaction) { BankTransaction managed = entityManager.merge(transaction); managed.markAsSuspect(); } } Conclusion Enterprise Java is prepared for AI today, with Jakarta EE already supporting this integration. Developers can add AI using provider SDKs, OmniHai, or LangChain4j CDI, while continuing to leverage Jakarta EE features for persistence, security, messaging, transactions, REST APIs, and enterprise data. AI enhances the existing platform as an integrated capability, rather than requiring replacement. The ecosystem continues to advance. Jakarta EE 12 enhances the data foundation, and Jakarta Agentic AI is introducing a structured programming model for building agents that integrate seamlessly with the platform. Jakarta EE is ready for AI now, and its capabilities will keep improving as the platform evolves.

By Otavio Santana

CORE

Building an Identity-Aware MCP Server in Python

The Model Context Protocol connects AI agents to your databases, APIs, and file systems. Out of the box, it connects them with no identity, no scoping, and no audit trail. The MCP specification acknowledges this gap explicitly. Its OAuth 2.1 authorization spec marks authentication as optional. The result, according to research published on Security Boulevard in April 2026, is that 53 percent of open-source MCP implementations ship with static API keys. Eighty-eight percent require backend authentication, but only 8.5 percent implement proper credential management. Every one of those static keys is a credential waiting to be stolen, a scope waiting to be abused, and an audit entry that will read "unknown agent executed query" when the incident report is written. This article builds the alternative. We will build an MCP server in Python that accepts tool calls only from authenticated agents, validates OAuth 2.1 Bearer tokens using stateless JWKS-based validation, enforces tool-level scopes and roles, maintains an infrastructure-level tool allow-list, and logs every access decision with the full delegation chain back to the human who authorized it. The complete companion project, roughly 350 lines of Python with a 13-test suite, is available on GitHub. Prerequisites You will need Python 3.12 or later and an OIDC-compatible identity provider. The examples use Auth0 (free tier works), but Okta, Keycloak, Entra ID, or any provider that exposes a /.well-known/jwks.json endpoint will work. Basic familiarity with OAuth 2.1 concepts and MCP server architecture is assumed. All code shown is extracted from the companion project. File paths reference code/src/. Architecture Every tool call flows through five gates before reaching your business logic: Architecture: Five-gate MCP tool call authorization pipeline. Gates two and three are infrastructure-level controls. System prompts are not security controls. An MCP server the agent has not been explicitly authorized to call should be unreachable. Period. Regardless of what the LLM decides to invoke. Part 1: JWKS-Based Token Validation The foundation of an identity-aware MCP server is stateless JWT validation. Every request carries a Bearer token issued by your OAuth 2.1 authorization server. The MCP server validates it against the provider's JSON Web Key Set, a public key document that lets you verify signatures without a network call to the IdP on every request. The JWKS Cache Create src/auth/middleware.py. We start with a cache that fetches the JWKS once and holds it in memory, refreshing every five minutes or on-demand when an unknown key ID appears (key rotation): Python class JWKSCache: """Cached JWKS with automatic refresh on unknown key id.""" def __init__(self, jwks_url: str, cache_ttl: int = 300): self._url = jwks_url self._ttl = cache_ttl self._keys: dict[str, dict] = {} self._last_fetch: float = 0 async def get_key(self, kid: str) -> dict: if not self._keys or (time.monotonic() - self._last_fetch) > self._ttl: await self._refresh() key = self._keys.get(kid) if key is None: logger.info("Unknown kid '%s', forcing JWKS refresh", kid) await self._refresh() key = self._keys.get(kid) if key is None: raise AuthError(f"Key '{kid}' not found in JWKS", 401) return key async def _refresh(self) -> None: if self._url.startswith("http"): async with httpx.AsyncClient() as client: resp = await client.get(self._url, timeout=10) resp.raise_for_status() jwks = resp.json() else: with open(self._url) as fh: jwks = json.load(fh) self._keys = {k["kid"]: k for k in jwks.get("keys", [])} self._last_fetch = time.monotonic() The get_key method is where the key rotation logic lives. When a token arrives with a kid the cache has never seen, we force a refresh before rejecting it. An unknown kid could mean a legitimate rotation, not an attack. We try once more before failing. In practice, this means you never need to restart your MCP server when your identity provider rotates signing keys. The Token Validator The validator uses the cache to verify every Bearer token. It checks five things, and the order matters: header validity, signature, issuer, audience, and expiry: Python class TokenValidator: def __init__(self, jwks_url: str, issuer: str, audience: str, clock_tolerance: int = 30): self._jwks = JWKSCache(jwks_url) self._issuer = issuer self._audience = audience self._clock_tolerance = clock_tolerance async def validate(self, token: str) -> ValidatedToken: # 1. Decode header to get the key id. unverified = jwt.get_unverified_header(token) kid = unverified.get("kid") if not kid: raise AuthError("Token header missing 'kid' claim", 401) # 2. Fetch the matching public key. jwk = await self._jwks.get_key(kid) # 3. Verify signature + standard claims. claims = jwt.decode( token, jwk, algorithms=["RS256"], issuer=self._issuer, audience=self._audience, options={"verify_exp": True, "require": ["exp", "iss", "sub", "aud"]}, ) # 4. Clock-tolerance check (belt-and-suspenders with the library). now = int(time.time()) if claims["exp"] + self._clock_tolerance < now: raise AuthError("Token has expired", 401) # 5. Extract scopes, roles, and delegation chain. scope_str = claims.get("scope", "") token_scopes = set(scope_str.split()) roles = claims.get("roles", []) delegation_chain = self._extract_delegation(claims) return ValidatedToken( subject=claims["sub"], email=claims.get("email"), roles=roles, scopes=token_scopes, delegation_chain=delegation_chain, ) The iss (issuer) check prevents tokens from a different authorization server from being accepted. The aud (audience) check prevents tokens intended for a different service from being replayed against yours. The exp check with clock tolerance handles the reality that clocks drift. Thirty seconds of tolerance is the pragmatic default recommended by the Upstash MCP OAuth deep-dive. The delegation chain extraction is worth examining separately. When an agent acts on behalf of a human who authorized it, RFC 8693's act claim carries that nesting. We recursively unpack it: Python def _extract_delegation(self, claims: dict) -> list[str]: chain = [] act = claims.get("act", {}) while act: sub = act.get("sub", "") if sub: chain.append(sub) act = act.get("act", {}) return chain A token issued directly to a human will have an empty delegation chain. A token issued to an agent acting on behalf of "[email protected]" will carry ["[email protected]"]. A multi-hop chain, human to orchestrator agent to sub-agent, carries both identifiers in order. This is what lets your audit logs trace every action back to a person. Part 2: The Two Mandatory Discovery Endpoints An MCP client connecting to your server needs to discover two things: that authentication is required, and where to get tokens. The MCP specification mandates two well-known endpoints for this, defined in RFC 9728 and RFC 8414, respectively. Create src/auth/discovery.py: Python def build_discovery_routes( resource_url: str, authorization_server_url: str, scopes_supported: list[str] | None = None, ) -> dict: async def protected_resource(request: Request) -> JSONResponse: return JSONResponse({ "resource": resource_url, "authorization_servers": [authorization_server_url], "bearer_methods_supported": ["authorization_code"], }) async def authorization_server(request: Request) -> JSONResponse: return JSONResponse({ "issuer": authorization_server_url, "authorization_endpoint": f"{authorization_server_url}/authorize", "token_endpoint": f"{authorization_server_url}/oauth/token", "jwks_uri": f"{authorization_server_url}/.well-known/jwks.json", "scopes_supported": scopes_supported or [ "database.read", "database.write", "email.send", "admin.users.read", ], "response_types_supported": ["code"], "grant_types_supported": ["authorization_code", "client_credentials"], "code_challenge_methods_supported": ["S256"], "token_endpoint_auth_methods_supported": ["none"], }) return { "/.well-known/oauth-protected-resource": protected_resource, "/.well-known/oauth-authorization-server": authorization_server, } Without these endpoints, MCP clients cannot auto-discover your authentication configuration. The client first hits your server without a token, receives a 401 with a WWW-Authenticate header pointing to the protected resource metadata, fetches it to confirm auth is required, then reads the authorization server metadata to learn the token endpoint and supported grant types. code_challenge_methods_supported: ["S256"] is not optional. MCP clients are public clients. They cannot keep a client secret, so PKCE is the only defense against authorization code interception. The NAPTHA AI reference implementation explicitly documents this. Part 3: Tool Definitions With Scope and Role Requirements Now we define the tools themselves. Each tool declares what scopes and roles are required to invoke it. These declarations live alongside the tool code, not in a separate config file. Proximity reduces the chance of drift between a tool and its authorization requirements. Create src/tools/database.py: Python # Each tool is a handler with declared requirements. TOOL_REGISTRY: dict[str, tuple[list[str], list[str], callable]] = { "read_customer_record": ( ["database.read"], # required scopes [], # required roles read_customer_record, # handler ), "update_customer_plan": ( ["database.write"], [], update_customer_plan, ), "list_all_customers": ( ["admin.users.read"], ["admin"], # admin role required list_all_customers, ), } A developer with database.read scope can read customer records but cannot update plans. A contractor with no scopes gets blocked from everything. An admin with admin.users.read scope and the admin role can list all customers. The registry is the single source of truth for access control. The server enforces it at request time without consulting a database. Here is one tool handler showing resource-level constraint enforcement: Python async def read_customer_record(customer_id: int, *, _token=None) -> dict: # Optional: enforce per-resource constraints from the token. if _token and hasattr(_token, "raw_claims"): constraint = _token.raw_claims.get("resource_constraints", {}) allowed_id = constraint.get("customer_id") if allowed_id is not None and customer_id != allowed_id: raise PermissionError( f"Token scoped to customer {allowed_id}, " f"requested customer {customer_id}" ) record = _CUSTOMER_DB.get(customer_id) if record is None: raise ValueError(f"Customer {customer_id} not found") return record The resource_constraints claim in the token is what turns "this agent can read customer data" into "this agent can read customer 48291 for the next sixty seconds." It is the difference between scoping to a database table and scoping to a row. Part 4: The Tool Allow-List Gate System prompts are not security controls. A prompt injection can rewrite an agent's intent mid-session and convince it to call a tool it was never meant to access. The only reliable defense is an infrastructure-level allow-list that rejects unauthorized tool calls regardless of what the LLM decides. The allow-list is derived directly from the tool registry. Any tool not in the registry is unreachable: Python ALLOWED_TOOLS: set[str] = set(TOOL_REGISTRY.keys()) This set is checked before scope and role evaluation. A tool that is not in the registry cannot be called, period. A tool that is in the registry but requires scopes the token does not carry gets a 403. A tool that is in the registry and the token carries the right scopes goes through. The distinction between "tool not in allow-list" and "tool forbidden for this agent" matters for debugging and audit. The first indicates a misconfiguration or an attack. The second indicates a legitimate agent attempting an unauthorized operation, which itself is worth logging. Part 5: The Audit Logger Every tool call, successful or blocked, produces an audit log entry with the full delegation chain. The format is JSON Lines: one JSON object per line, ingestible by any SIEM, Splunk, or grep. Create src/audit/logger.py: Python class AuditLogger: def __init__(self, filepath: str | Path = "audit.log") -> None: self._path = Path(filepath) self._path.touch(exist_ok=True) def record(self, event: str, token: ValidatedToken, tool_name: str = "", tool_args: dict | None = None, result_summary: str = "", error: str = "") -> None: entry = { "timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()), "event": event, "correlation_id": str(uuid.uuid4()), "subject": token.subject, "email": token.email, "roles": token.roles, "scopes": sorted(token.scopes), "delegation_chain": token.delegation_chain, "tool": tool_name, "tool_args": tool_args or {}, "result": result_summary, "error": error, } with open(self._path, "a") as fh: fh.write(json.dumps(entry, default=str) + "\n") When an auditor asks "who authorized this data access," the answer is in the log, not in a code review three weeks later. A correctly logged tool call looks like this: Python { "timestamp": "2026-06-14T14:04:00Z", "event": "tool_call", "subject": "alice-developer", "email": "[email protected]", "roles": ["developer"], "scopes": ["database.read", "email.send"], "delegation_chain": ["bob-admin"], "tool": "read_customer_record", "tool_args": {"customer_id": 1001}, "result": "ok" } Delegation chain flow: Human → Orchestrator Agent → Sub-Agent → MCP Server. The delegation chain reads: Bob (admin) delegated to Alice's developer agent, which called read_customer_record for customer 1001 at 14:04 UTC. If your logs cannot produce that sentence, your AI identity program is not operational. Part 6: Assembling the Server The main server wires together the token validator, the tool allow-list, the scope and role checks, the tool handlers, and the audit logger. Every request flows through them in order. Create src/server.py. Here is the core request path: Python token_validator = TokenValidator( jwks_url=OIDC_JWKS_URL, issuer=OIDC_ISSUER, audience=OIDC_AUDIENCE, clock_tolerance=30, ) audit = AuditLogger(AUDIT_LOG_FILE) async def mcp_tool_endpoint(request: Request) -> JSONResponse: # 1 — Extract and validate the Bearer token. auth = request.headers.get("Authorization", "") if not auth.startswith("Bearer "): raise AuthError("Missing Bearer token", 401) token_str = auth[7:] try: token = await token_validator.validate(token_str) except AuthError: audit.record("auth_failure", ...) raise # 2 — Parse the tool invocation. body = await request.json() tool_name = body.get("tool", body.get("name", "")) tool_args = body.get("arguments", body.get("args", {})) # 3 — Tool allow-list enforcement. if tool_name not in ALLOWED_TOOLS: audit.record("tool_allow_list_block", token, tool_name=tool_name) return JSONResponse( {"error": f"Tool '{tool_name}' is not authorized"}, status_code=403, ) # 4 — Scope + role authorization. required_scopes, required_roles = get_tool_requirements(tool_name) if required_scopes and not token.has_any_scope(required_scopes): return JSONResponse( {"error": "Insufficient scopes", "required": required_scopes, "granted": sorted(token.scopes)}, status_code=403, ) if required_roles: if not (set(token.roles) & set(required_roles)): return JSONResponse( {"error": "Insufficient role", "required_one_of": required_roles, "have": sorted(token.roles)}, status_code=403, ) # 5 — Execute and audit. handler = TOOL_REGISTRY[tool_name][2] result = await handler(**tool_args, _token=token) audit.record("tool_call", token, tool_name=tool_name, tool_args=tool_args, result_summary=str(result)[:200]) return JSONResponse({"result": result}) The 401 response format is specified by the MCP specification. The WWW-Authenticate header with resource_metadata is how clients discover that authentication is required: Python async def auth_error_handler(request, exc): return Response( content='{"error":"' + exc.args[0] + '"}', status_code=401, media_type="application/json", headers={ "WWW-Authenticate": ( f'Bearer resource_metadata=' f'"{AUDIENCE}/.well-known/oauth-protected-resource",' f'error="invalid_token"' ), }, ) Part 7: The Demo Agent To verify the server end-to-end without configuring a real OAuth provider, the companion project includes a demo agent that generates self-signed tokens for three simulated identities. Run it with python demo/agent.py --demo. The demo creates three agents with progressively restricted access: Plain Text Agent 1: Alice — developer, scopes: database.read + email.send ✓ Can read customer records ✗ Cannot update plans (missing database.write) ✗ Cannot list all customers (missing admin role) Agent 2: Bob — admin, scopes: database.read + database.write + admin.users.read ✓ Can read customer records ✓ Can update plans ✓ Can list all customers Agent 3: Carol — contractor, scopes: (none) ✗ Blocked from everything This is not a theoretical exercise. In the Stryker attack of March 2026, a compromised admin credential, one identity, over-privileged, with no scoping, allowed attackers to remotely wipe 200,000 devices across 79 countries. The attack did not use malware. It used the platform's own legitimate wipe functionality. The credential had no scope limiting it to a subset of devices, no short lifetime, and no audit trail that would have surfaced the anomaly before tens of thousands of endpoints were erased. Part 8: Testing The companion project includes a 13-test suite that verifies every security gate. Run it with: Python python -m pytest tests/ -v The test matrix covers the decision table exhaustively: TestConditionExpectedNo tokenMissing Authorization header401Invalid tokenMalformed JWT401Expired tokenexp in the past401Valid token + correct scopedatabase.read calling read_customer_record200Valid token + wrong scopeemail.send calling read_customer_record403Valid token + missing scopedatabase.read calling update_customer_plan403Valid token + correct scopesdatabase.read database.write calling update_customer_plan200Valid token + wrong roledeveloper role calling list_all_customers403Valid token + correct roleadmin role calling list_all_customers200Unknown tooldelete_everything not in allow-list403Discovery: protected resourceUnauthenticated GET200Discovery: authorization serverUnauthenticated GET200Audit log entriesTool call with delegation chainWritten with full chain Each test generates a real RSA key pair, signs a JWT with it, loads a matching JWKS, and sends a request through the full server stack using Starlette's TestClient. No mocking of the auth layer. The tests exercise the actual token validation code path. Part 9: Common Pitfalls localhost vs 127.0.0.1 redirect URI mismatch. MCP clients running locally often register 127.0.0.1 as their redirect URI, but the authorization server redirects to localhost (or vice versa). The Upstash OAuth deep-dive documents this as the most common integration failure. Normalize both addresses at registration and at token exchange. Cursor re-registers OAuth clients on every connection. The Dynamic Client Registration endpoint must handle the same client identity registering repeatedly. Store by client identity, not by registration request. Idempotency is critical. Clock skew causing spurious rejections. A 30-second clockTolerance is the pragmatic default. Distributed systems have clock drift. Rejecting a valid token because the IdP's clock is 12 seconds ahead of yours is a self-inflicted outage. Forgetting to serve discovery endpoints over HTTPS. MCP clients will refuse to fetch well-known URIs over plain HTTP in production. If your server is behind a load balancer, ensure the resource_url reflects the externally visible HTTPS URL, not the internal service name. Logging Bearer tokens. Sanitize the Authorization header from request logs. A leaked Bearer token in your logging pipeline is an identity compromise waiting to happen. The audit logger in this project intentionally records the validated identity, never the raw token. Production Hardening Before deploying to production, lock down the following: PKCE (S256) is mandatory. MCP clients are public clients without a client secret. PKCE is the only defense against authorization code interception.Short-lived tokens. Fifteen to sixty minutes, with refresh token rotation. Each use of a refresh token invalidates the previous one.HTTPS only. HTTP must be rejected at the network level. The MCP security best practices specification explicitly prohibits plaintext.Session-based authentication is prohibited. The MCP spec mandates token-based authentication. No cookies, no sessions.Audit log rotation and retention. JSON Lines accumulate quickly at production throughput. Configure log rotation and feed the audit stream to your SIEM. What We Built We built an MCP server that accepts tool calls only from authenticated agents. It validates OAuth 2.1 Bearer tokens using stateless JWKS-based validation with automatic key rotation. It enforces tool-level scopes and roles. A developer with database.read cannot write. A contractor with no scopes gets blocked from everything. An admin with the right role and scope can list all records. It maintains an infrastructure-level tool allow-list that rejects unauthorized tool calls regardless of what the LLM decides. It logs every access decision with the full delegation chain, so an auditor can trace any action back to the human who authorized it. The standards to do this at scale are maturing rapidly. SPIFFE handles workload identity. RFC 8693 covers token exchange with delegation chains. The IETF AIMS framework addresses agent identity. The engineering to do it in a single Python file is deployable today. The companion project is available on GitHub with setup instructions, a working demo, and a 13-test suite. Clone it, configure your OAuth provider, and you have an identity-aware MCP server in under 200 lines of application code. GitHub repository: github.com/pravin-khandke/identity-aware-mcp-server Clone it and run the demo in under two minutes: Shell git clone https://github.com/pravin-khandke/identity-aware-mcp-server.git cd identity-aware-mcp-server python3 -m venv .venv && source .venv/bin/activate pip install -r requirements.txt python demo/agent.py --demo All code shown in this article is extracted from the repository. See src/auth/middleware.py for the JWKS validator, src/server.py for the full request pipeline, and tests/test_server.py for the 13-test suite.

By Pravin Khandke

Beyond JSON: Benchmarking TOON and TOON-LD for LLMs

JSON has been the default structured-data format for APIs, configuration, event streams, and application integration for decades. It is portable, readable, widely supported, and easy to validate. However, JSON was not designed for LLMs. When structured data is placed inside an LLM prompt, every quotation mark, repeated field name, brace, comma, and nested structure contributes to the prompt’s token count. For a small request, this overhead may be insignificant. For applications that send thousands of records, tool results, or knowledge-graph entities to an LLM, it can consume a meaningful portion of the context window. Token-Oriented Object Notation, or TOON, proposes a different representation. It encodes the same objects, arrays, and primitive values as JSON but uses a compact, line-oriented syntax designed for LLM prompts. TOON combines indentation for nested structures with tabular representations for homogeneous arrays. Its strongest use case is a collection of objects that share the same fields. TOON-LD applies a related idea to Linked Data. It is intended to represent JSON-LD knowledge graphs more compactly while retaining Linked Data constructs such as @context, @id, @type and @graph. This tutorial explains the differences among JSON, TOON, JSON-LD & TOON-LD and shows how to benchmark their token consumption, serialized size, conversion overhead and round-trip correctness. Why JSON Consumes Additional LLM Tokens Consider the following incident records: JSON { "incidents": [ { "id": "INC-000001", "service": "checkout", "severity": "critical", "region": "ap-south-1", "owner": "platform" }, { "id": "INC-000002", "service": "payments", "severity": "high", "region": "eu-west-1", "owner": "payments" } ] } The field names id, service, severity, region, and owner appear in every record. An application parser needs those repeated keys to reconstruct each JSON object, but an LLM prompt pays for their repeated tokenization. A corresponding TOON representation can declare the fields once and place the values in rows: Plain Text incidents[2]{id,service,severity,region,owner}: INC-000001,checkout,critical,ap-south-1,platform INC-000002,payments,high,eu-west-1,payments The exact encoded output depends on the TOON specification and encoder version, so production applications should generate TOON through a library rather than manually constructing it. The important difference is structural: JSON repeats the complete object syntax for every row, whereas TOON can amortize that structure across a uniform collection. The TOON project describes the format as a lossless representation of the JSON data model and identifies uniform arrays of objects as its primary efficiency advantage. It also notes that deeply nested or non-uniform data may not receive the same benefit and can sometimes remain more efficient in JSON. JSON and TOON Serve Different Architectural Purposes TOON should not automatically replace JSON across an application. JSON remains appropriate for: Public and internal APIsApplication configurationPersistent storageEvent exchangeSchema-based validationBrowser and programming-language interoperabilityObservability logs and audit records TOON is better evaluated as a representation used at the LLM boundary. A practical architecture is: The application continues to use JSON internally. Only the structured context inserted into the prompt is converted to TOON. This approach reduces migration risk and confines the new format to the part of the architecture where token efficiency matters. What Is JSON-LD? JSON-LD is a W3C-standardized JSON-based format for Linked Data. It adds semantic meaning to ordinary JSON through globally identifiable concepts and relationships. The JSON-LD 1.1 specification is a W3C Recommendation and is designed to integrate Linked Data into JSON-based programming environments and web services. Consider the following example: JSON-LD { "@context": { "ex": "https://example.org/", "affects": { "@id": "ex:affects", "@type": "@id" }, "ownedBy": { "@id": "ex:ownedBy", "@type": "@id" } }, "@graph": [ { "@id": "ex:incident-101", "@type": "ex:Incident", "ex:severity": "critical", "affects": "ex:checkout" }, { "@id": "ex:checkout", "@type": "ex:Service", "ownedBy": "ex:platform-team" } ] } This document contains more than two nested JSON objects. It describes a graph: An LLM can use this structure for questions such as: Which team owns the service affected by incident 101? JSON-LD is therefore useful for knowledge graphs, semantic search, Graph-RAG, interoperable metadata, and agent systems that must traverse relationships among entities. What Is TOON-LD? TOON-LD is an emerging format that extends TOON with Linked Data semantics. Its implementation describes TOON-LD as a compression representation for JSON-LD knowledge graphs used in LLM context windows. It supports JSON-LD constructs and provides conversions between JSON-LD and TOON-LD. A simplified TOON-LD representation of a uniform graph may resemble: Plain Text @context: ex: https://example.org/ @graph[2]{@id,@type,ex:severity,ex:affects}: ex:incident-101,ex:Incident,critical,ex:checkout ex:incident-102,ex:Incident,high,ex:payments The main optimization again comes from declaring a common shape once instead of repeating every JSON-LD field for every entity. TOON-LD should nevertheless be assessed differently from JSON-LD. JSON-LD is a mature W3C standard with established processors and semantic-web tooling. TOON-LD is considerably newer and should be evaluated for library stability, interoperability, and semantic preservation before production use. JSON, TOON, JSON-LD and TOON-LD Compared Format Data model Main objective Typical use JSON Object and array tree Universal structured-data exchange APIs, events, configuration and storage TOON JSON-compatible object and array tree Reduce tokens in LLM context Prompt records, RAG context and tool results JSON-LD RDF-compatible linked graph Semantically interoperable Linked Data Knowledge graphs and semantic metadata TOON-LD Token-oriented linked graph Reduce JSON-LD context tokens Graph-RAG and knowledge-driven agents TOON should be compared with JSON. TOON-LD should primarily be compared with JSON-LD. Comparing TOON-LD only with ordinary JSON would mix two different data models and could produce a misleading conclusion. Designing a Fair Benchmark Token-efficiency claims should not be evaluated with one carefully selected payload. The accompanying benchmark uses four datasets: Flat homogeneous incident recordsNested homogeneous incident recordsIrregular and sparse incident recordsJSON-LD incident knowledge graphs Each dataset is generated at multiple scales: 10 records,100 records, 1000 records, 10000 records This exposes an important characteristic of token-oriented formats: their benefits can depend significantly on the shape and scale of the input. Flat Homogeneous Data The flat dataset contains records with identical fields: JSON { "id": "INC-000001", "service": "service-01", "severity": "critical", "region": "ap-south-1", "owner": "platform", "latency_ms": 450, "retryable": true } This is likely to be the strongest scenario for TOON because the schema can be declared once and reused for all rows. Nested Data The nested dataset includes workload, metric, and status objects: JSON { "id": "INC-000001", "workload": { "namespace": "team-1", "deployment": "service-01", "pod": "service-01-000001" }, "metrics": { "cpu_percent": 72, "memory_mib": 850, "latency_ms": 450 }, "status": { "severity": "critical", "acknowledged": false } } This tests whether TOON’s reduced punctuation compensates for indentation and nested structural markers. Irregular Data The irregular dataset intentionally varies fields across records: JSON [ { "id": "INC-000001", "service": "checkout", "severity": "critical" }, { "id": "INC-000002", "dependencies": ["postgresql", "kafka"], "retry_after_seconds": 30 }, { "id": "INC-000003", "error": { "code": 503, "message": "upstream unavailable" } } ] This is important because tabular formats perform best when records share a schema. Sparse or heterogeneous structures can reduce or eliminate that advantage. Linked-Data Graph The final dataset contains incidents, services, teams, and relationships expressed through JSON-LD. This evaluates TOON-LD against the representation it is intended to optimize. Metrics Used in the Experiment The benchmark records the following metrics. Serialized Characters This is the number of Unicode characters in the encoded document. Character count is easy to understand, but it is not a substitute for token count. Different tokenizers divide the same text differently. UTF-8 Bytes The benchmark measures the encoded byte length using: len(serialized_value.encode("utf-8")). This helps estimate storage and network-transfer overhead. Token Count Token count is measured using the selected tokenizer. The repository defaults to the o200k_base tokenizer but allows another tokenizer to be configured. For linked data, JSON-LD replaces JSON in the calculation. Token savings are tokenizer-specific. A result measured with one tokenizer should not be presented as universally applicable to every model family. Encoding Latency Encoding latency measures the time required to convert an in-memory object to JSON, TOON, JSON-LD, or TOON-LD. The benchmark reports: median encoding latency;95th-percentile encoding latency. Decoding Latency Decoding latency measures the time required to reconstruct the application data from its serialized representation. This matters because reducing prompt tokens may introduce additional CPU overhead in the application. Peak Memory Python’s tracemalloc module records the peak memory observed during serialization. Round-Trip Correctness For every measured iteration, the benchmark verifies: source data == decode(encode(source data)) A format that produces a smaller prompt but cannot reliably reconstruct the source data is unsuitable for lossless interchange. Running the Benchmark Clone the repository: Shell git clone https://github.com/jojustin/json-toon-toonld-benchmark.git cd json-toon-toonld-benchmark Create a virtual environment: Shell python -m venv .venv source .venv/bin/activate Install the dependencies: Shell pip install -r requirements.txt Run a small validation experiment first: Shell python -m src.run_benchmark --sizes 10 100 --iterations 5 Run the complete benchmark: Shell python -m src.run_benchmark --sizes 10 100 1000 10000 --iterations 30 To calculate percentage reductions and encoding overhead: Shell python -m src.summarize Run the automated tests: Shell pytest -q Why the Benchmark Uses Minified JSON A TOON comparison can be exaggerated by comparing it only with pretty-printed JSON. Pretty-printed JSON contains indentation and line breaks intended for human readability: JSON { "id": 1, "name": "Alice" } Minified JSON removes optional whitespace: JSON {"id":1,"name":"Alice"} Since production systems can easily minify JSON before placing it in a prompt, minified JSON is the appropriate primary baseline. Pretty-printed JSON can still be reported as a separate readability baseline, but it should not be the only comparison. Interpreting the Expected Results The benchmark results show that token-oriented serialization is not uniformly more efficient than JSON. Its effectiveness depends strongly on the structure of the input data. TOON performs best when the input consists of flat, homogeneous records that share the same fields, while compact JSON remains more efficient for irregular and deeply nested structures. TOON and TOON-LD also introduce measurable conversion overhead because their encoders must analyze the input structure and generate a more specialized representation. Token Efficiency For the flat dataset, TOON reduced the token count from approximately 39,500 tokens to 23,000 tokens, corresponding to a reduction of about 42%. This result represents TOON’s intended use case: a large collection of records sharing a common schema. Rather than repeating every field name for each record, TOON declares the fields once and represents the values in a tabular form. The result was different for irregular data. Compact JSON required approximately 27,300 tokens, while TOON required about 33,000 tokens — an increase of approximately 21%. Because the records contained different fields and structures, TOON could not efficiently amortize a shared schema across the collection. The additional structural notation therefore outweighed the savings obtained by removing JSON punctuation. A similar pattern appeared in the nested dataset. TOON used approximately 74,000 tokens compared with 64,000 tokens for compact JSON, representing an increase of around 16%. The result indicates that deeply nested objects are not necessarily well suited to tabular token-oriented encoding. Indentation, nested object markers, and repeated hierarchical structures can make TOON less compact than minified JSON. For the linked-data dataset, TOON-LD reduced the representation from approximately 40,000 JSON-LD tokens to 28,500 tokens, a saving of about 29%. This demonstrates the potential of schema-aware linked-data compression. However, the token reduction must be interpreted together with the round-trip validation results. In the tested implementation, the reconstructed TOON-LD output did not preserve valid JSON-LD semantics. The observed token saving therefore represents compression potential, but not a verified lossless transformation for this workload. Encoding Performance JSON consistently encoded faster than TOON. For the flat dataset, compact JSON required approximately 6 milliseconds, whereas TOON required around 27 milliseconds. TOON was therefore about four times slower, despite producing a substantially smaller token representation. The irregular dataset showed a similar pattern. JSON encoding took approximately 5 milliseconds, while TOON required nearly 30 milliseconds. In this case, TOON introduced significant processing overhead while also producing more tokens, making compact JSON preferable on both efficiency and runtime grounds. For the nested dataset, JSON required approximately 12 milliseconds and TOON approximately 55 milliseconds. This was the highest TOON encoding time observed among the datasets. The additional processing required to traverse and represent deeply nested structures contributed to both higher runtime and higher token count. JSON-LD encoding required approximately 6 milliseconds for the linked-data dataset, compared with about 17 milliseconds for TOON-LD. TOON-LD was therefore around three times slower to encode, although its absolute processing time remained below 20 milliseconds for 1,000 records. These results show that reduced token count is not computationally free. TOON and TOON-LD shift some work from the LLM prompt to the application’s serialization layer. End-to-End Conversion Overhead For flat data, TOON introduced approximately 29.14 milliseconds of additional conversion time compared with JSON. For irregular data, the overhead increased to 31.94 milliseconds. The linked-data comparison produced the lowest overhead: TOON-LD added approximately 11.59 milliseconds relative to JSON-LD. The nested dataset generated the largest conversion overhead at 58.93 milliseconds. This finding is consistent with the encoding-time and token-count results: nested structures were both slower to process and less token-efficient in TOON. Although these overheads are small compared with the end-to-end latency of many remote LLM requests, they may still matter in high-throughput systems, local inference pipelines, or workflows that repeatedly serialize and deserialize large payloads. Conversion cost should therefore be evaluated relative to the expected inference savings and request volume. Overall Interpretation The combined results reveal three distinct workload categories. Workload Token outcome Conversion outcome Recommendation Flat, homogeneous records About 42% fewer tokens About 29 ms additional conversion time Strong candidate for TOON Irregular records About 21% more tokens About 32 ms additional conversion time Prefer compact JSON Deeply nested records About 16% more tokens About 59 ms additional conversion time Prefer compact JSON Linked data About 29% fewer tokens About 12 ms additional conversion time Promising, but semantic validation must pass The strongest result is that data shape is the primary determinant of TOON efficiency. TOON is effective for uniform, tabular collections because it avoids repeating field names. It is less suitable for sparse, irregular, or deeply nested data, where compact JSON can require fewer tokens and substantially less conversion time. The linked-data result should be treated cautiously. Although TOON-LD reduced token usage and introduced relatively modest conversion overhead, the tested implementation failed semantic round-trip validation. It should therefore not be presented as a lossless JSON-LD replacement for this experiment. A practical selection policy derived from the results is: Flat and homogeneous records → TOON Irregular or nested records → Compact JSON Linked-data graphs → JSON-LD unless TOON-LD semantic validation passes Overall, the benchmark supports using TOON as a selective prompt-boundary optimization, rather than as a universal replacement for JSON. The appropriate decision should consider token reduction, conversion overhead, structural correctness, and semantic preservation together. Extending the Benchmark With LLM accuracy The repository focuses on deterministic, provider-neutral measurements. A second experiment can assess how well an LLM understands each representation. Use semantically identical questions for JSON and TOON: List the IDs of all critical incidents owned by the platform team. Return only a JSON array of incident IDs. For JSON-LD and TOON-LD, include multi-hop questions: Which teams own services affected by critical incidents? Measure: Input tokensOutput tokensTime to first tokenTotal response latencyExact-match accuracyPrecision, recall, and F1Invalid-output rateHallucination rateCost per request Keep these variables constant: Model and model versionSystem promptQuestionTemperatureMaximum output tokensDatasetNumber of repeated trials Randomize the order of JSON and TOON trials so that temporary service conditions do not consistently favor one format. When Should TOON Be Considered? TOON is worth evaluating when: Large homogeneous datasets are repeatedly placed in promptsPrompt-token cost is significantContext-window capacity is constrainedThe application controls both encoding and decodingStructured context is primarily read by the modelBenchmarked accuracy remains acceptable TOON may be less attractive when: Payloads are smallObjects are deeply nested or highly irregularStandard interoperability is more important than token savingsThe model must reliably generate complex TOON outputDownstream tools require JSON directlyConversion complexity exceeds measurable savings When Should TOON-LD Be Considered? TOON-LD may be useful when: A Graph-RAG pipeline inserts many JSON-LD entities into promptsRepeated graph entities share common shapesA semantic agent receives linked relationships as contextPreserving @context, identifiers, and graph relationships is essentialJSON-LD token consumption limits useful graph size It should be approached cautiously when: External systems expect standards-compliant JSON-LD directlyRDF canonicalization and semantic round trips have not been testedPackage maturity and long-term compatibility are criticalThe linked-data graph contains complex or highly heterogeneous structures Security Considerations Structured-data compression does not eliminate prompt-security concerns. Before inserting TOON or TOON-LD content into a prompt: Treat serialized values as untrusted dataSeparate instructions from retrieved contentValidate decoded responsesEnforce output schemas where possibleLimit graph traversal and retrieved entity countsPrevent untrusted content from altering system instructionsLog the canonical JSON or JSON-LD source for auditability For TOON-LD, external contexts and linked identifiers should also be controlled. Applications should avoid dereferencing arbitrary remote contexts or URLs without appropriate allowlists, timeouts and content validation. Conclusion JSON remains the correct default for general-purpose application integration. It has unmatched interoperability, mature tooling, schema support and broad developer familiarity. TOON addresses a narrower problem: reducing the token overhead of structured data passed to language models. Its strongest potential advantage is in large, homogeneous collections where repeated JSON keys consume substantial context. TOON-LD applies the same general principle to JSON-LD knowledge graphs. It may allow Graph-RAG and semantic-agent systems to place more linked data in an LLM context, but it is newer and requires careful testing for semantic equivalence and implementation maturity. The key decision should not be based on token reduction alone. A production evaluation should measure: Token countSerialized bytesEncoding and decoding overheadMemory usageRound-trip correctnessLLM comprehensionStructured-output reliabilityEnd-to-end latencyCost at realistic request volumes A practical adoption pattern is to retain JSON or JSON-LD as the canonical application representation and introduce TOON or TOON-LD only as an explicitly measured prompt-boundary optimization. The accompanying benchmark provides a reproducible starting point for making that decision with evidence rather than assumptions.

By Josephine Eskaline Joyce

CORE

A Framework-Agnostic Approach to SSR for Microfrontends

On one of our projects, we were building microfrontends, and at some point we wanted to add SSR. The reasons were the usual ones: better first paint, fewer layout shifts, real content for crawlers, less JS to load before something appears on screen. Setting it up turned out to be harder than I expected. There was no obvious out-of-box path that fit our setup, and most of the approaches I found either assumed a shared build or asked us to add new infrastructure on top of what we already had. That is what made me start sketching a small package. Something any team could drop in and get SSR for their microfrontend without rewriting either side. The result is @mf-toolkit/mf-ssr. The rest of this is about the approach behind it, since I think that is the interesting part. What I Wanted I started from a short list, taken straight from how I'd want to use such a thing: MF content on first paint. The remote's HTML should arrive inside the host's server response, not be fetched from the client after JS loads. No empty slot, no layout shift, real content in crawlers.No shared build, no central orchestrator. Each team builds and deploys their remote on their own schedule. The host should not need a special Node process that imports every remote into one bundle, and remote teams should not need to rewrite their bundler config to fit a central setup.Two paths for two setups, one host component. I wanted both scenarios covered. url mode for when the remote team runs their own server and wants to own SSR on their side (and possibly use a non-React framework). loader mode for when the remote only ships a static React bundle and the host server can do the SSR for it. The host code should look almost the same in either case, with just a single prop telling the component which path to use.Any framework, any runtime. The remote might be React, but it could be Vue, Svelte, or anything else. The host shouldn't care. And on the server, the same code should run on Node, Bun, Cloudflare Workers, or Vercel Edge with no rewrites.Host state still drives the remote after hydration. When the host re-renders with new props, the remote should re-render too. No re-fetch, no re-mount, no shared store between bundles.Honest failure modes. A timeout when the remote is slow, retry when a request fails, an explicit fallback for total failure, and a cache that respects auth boundaries. The things that decide whether SSR is a win or a regression when one team has a bad deploy. The last bullet is what most articles skip. SSR is easy in the happy path. The interesting code is what happens when one of the remotes is slow, down, or returning garbage. How It Works The idea is small: Instead of importing remote components into the host server, the host pulls the rendered output in over HTTP at SSR time and streams it into its own response. The browser gets a full page on first paint. How that "pull" happens depends on how the remote is deployed. The package supports two modes for that: url mode – the remote has its own HTTP endpoint that returns rendered HTML. The host fetches that HTML during SSR.loader mode – the remote is a static React bundle on a CDN or S3, no server behind it. The host imports the component directly during SSR and renders it inline. Same host component (<MFBridgeSSR>) in both cases, just one prop changes. Both modes can live on the same page. The interesting part is what happens after hydration. The host has to push prop changes into the remote without re-fetching anything. I will get to that in a moment. I'll start with url mode since it is the more general case (any framework on the remote side, any runtime on the server), and then cover loader mode separately. url mode: Remote With Its Own HTTP Endpoint In url mode, the remote server does the SSR. The remote team runs their own runtime (Node, Bun, a Cloudflare Worker, a Next.js Route Handler, whatever they prefer) and exposes an HTTP endpoint that returns rendered HTML for the given props. The host's SSR pass just calls that endpoint and inlines the response into the page. Each microfrontend owns its own rendering pipeline. Remote Handler TypeScript-JSX import { createMFReactFragment } from '@mf-toolkit/mf-ssr/fragment' import { CheckoutWidget } from './CheckoutWidget' export const handler = createMFReactFragment(CheckoutWidget) handler is a plain Web fetch handler: (req: Request) => Promise<Response>. It reads props from the query string, renders the component to a stream with renderToReadableStream, and writes the props into a small <script> tag so the client can hydrate without going back to the network. One nuance worth flagging: those props go inside a <script> tag, so a raw </script> inside a string prop would close the tag prematurely and let user-controlled values escape into the HTML context. The handler escapes <, >, &, and U+2028/U+2029 to their \uXXXX equivalents before embedding. JSON.parse on the client treats them the same as the originals, but the browser's HTML parser never sees a closing tag. It is a few lines of code that close a real XSS hole. You wire the handler into whatever HTTP framework the remote team already uses. Hono, a Next.js Route Handler, Bun, plain Node, a Cloudflare Worker. The handler doesn't know about any of them. And because the whole thing is Web Streams, it runs on Cloudflare Workers, Vercel Edge, Bun, and Node 18+ without changes. Non-React Remotes createMFReactFragment is a React-only helper. If the remote is Vue, Svelte, Solid, or vanilla JS, the team writes their own fetch handler instead, but it has to produce the same HTML shape the host expects: TypeScript-JSX <div data-mf-ssr="checkout"> <script type="application/json" data-mf-props>{"orderId":"42"}</script> <div data-mf-app></div> </div> The team uses their framework's SSR renderer (renderToString for Vue, Svelte's SSR API, and so on) to produce the inner HTML, and serializes props into the <script data-mf-props> tag, applying the same < / > / & escaping. On the client, the remote mounts itself into [data-mf-app] and reads initial props from [data-mf-props]. If it needs prop updates from the host after hydration, it listens on the same DOMEventBus (exported from @mf-toolkit/mf-bridge). The bus is a thin wrapper over native CustomEvent, with no React dependency, so it works fine for any framework. This path is more work than createMFReactFragment, but the contract is small and explicit. The host doesn't care which framework produced the inner HTML — as long as the wrapper structure matches, hydration finds the right slots. Host Component TypeScript-JSX <MFBridgeSSR url="https://checkout.acme.com/fragment" namespace="checkout" props={{ orderId, step } fallback={<CheckoutSkeleton />} /> During SSR, the host fetches the remote's HTML and streams it into the response. Each <MFBridgeSSR> lives in its own Suspense boundary, so a slow checkout doesn't block the header. They stream as they resolve. On the client, the host hydrates, then waits for prop changes coming from React. Prop Updates After Hydration This was the part I cared about most. The remote is in its own React root, often in its own bundle, sometimes in a completely different framework. You can't re-render it like a normal child. So I used the one thing both sides already share at runtime: the DOM node the remote is mounted into. When the host re-renders with new props, the host fires a CustomEvent on that node. The remote listens for it and re-renders its root with the new props. No re-fetch, no global state, no coupling between bundles beyond a shared namespace string. TypeScript-JSX // remote client entry import { hydrateWithBridge } from '@mf-toolkit/mf-bridge/hydrate' import { CheckoutWidget } from './CheckoutWidget' hydrateWithBridge(CheckoutWidget, { namespace: 'checkout' }) I picked this because it is isolated by construction. If a page has several MF slots, each one has its own mount node, so events never leak between them. And it is just DOM, so there is no bundler magic to debug when something goes wrong. Events and Commands Prop streaming is one direction. For the other direction, the same bus works in reverse. The host passes onEvent to receive events the remote emits, and a commandRef it can use to send imperative commands back: TypeScript-JSX const resetRef = useRef<((type: string, payload?: unknown) => void) | null>(null) <MFBridgeSSR url="https://checkout.acme.com/fragment" namespace="checkout" props={{ orderId } onEvent={(type, payload) => { if (type === 'orderPlaced') navigate('/thanks') } commandRef={resetRef} /> // somewhere in host code, e.g. when the user switches accounts: resetRef.current?.('reset') On the remote, hydrateWithBridge accepts an onCommand handler, and DOMEventBus (exported from @mf-toolkit/mf-bridge) lets the remote send events back: TypeScript-JSX import { hydrateWithBridge } from '@mf-toolkit/mf-bridge/hydrate' import { DOMEventBus } from '@mf-toolkit/mf-bridge' hydrateWithBridge(CheckoutWidget, { namespace: 'checkout', onCommand: (type) => { if (type === 'reset') store.reset() }, }) // inside the widget, after a successful payment: const container = document.querySelector<HTMLElement>('[data-mf-namespace="checkout"]')! new DOMEventBus(container, 'checkout').send('event', { type: 'orderPlaced', payload: { orderId }, }) The channel is the same DOMEventBus, just with extra event names on top of propsChanged. So everything I said earlier about isolation still holds: events on one slot don't reach another, even when the remote is the same. loader mode: Remote as a Static Bundle In loader mode, the host server does the SSR for the remote. The remote team ships only a static React bundle (CDN, S3, or a Module Federation host) and runs no server of their own. When the host renders its page server-side, it imports the remote component and renders it inline, the same way it renders any other component in the host tree. The remote has no SSR runtime and no rendering responsibility; the host does all the work.ё Host Component JSX const loadCheckout = () => import('checkout/Widget').then(m => m.CheckoutWidget) <MFBridgeSSR loader={loadCheckout} props={{ orderId, step } fallback={<CheckoutSkeleton />} /> That is everything. No namespace, no errorFallback tricks needed for hydration, no client entry to write on the remote side. The package wraps the loader in React.lazy and renders the component inside the host's React tree, both server-side and after hydration. Props, Events, Commands Since the remote lives inside the host's React tree, every kind of communication is just React: Props – re-render normally. When the host's parent component re-renders with new props, the remote re-renders too. No DOMEventBus, no hydrateWithBridge, no propsChanged events.Events from remote to host – pass a callback through props. The remote calls it like any other handler.Commands from host to remote – pass them through props as well, or expose a ref through forwardRef. If you find yourself wanting onEvent / commandRef here, you are probably reaching for url mode. Requirements A few constraints come with this mode: Host must be able to resolve the loader on the server. The package calls your loader() function as-is. It doesn't fetch bundles from URLs itself. In practice, this means Module Federation runtime on the host (or some other server-side dynamic import mechanism that knows how to find checkout/Widget). Without that, the import fails in Node before any rendering happens.React only. The host literally calls the component during SSR, so the remote has to be a React component. For Vue/Svelte/vanilla remotes, use url mode.SSR-safe import. The remote's exposed module has to be importable on the server, which means no window, document, or other browser globals at the module top level. Move that code inside useEffect or behind a typeof window check.Stable loader reference. Define loadCheckout at module scope or wrap it in useCallback. The package caches the resulting React.lazy by loader reference so Suspense retries reuse the same promise. A new function on every render would break that and trigger an infinite retry loop. When to Pick Which CategoryURL modeLoader modeRemote infrastructureOwn HTTP endpoint: Node.js, Bun, Worker, etc.Static bundle on CDN, S3, or Module Federation hostRemote frameworkAny: React, Vue, Svelte, vanilla JavaScriptReact onlyIsolationSeparate React root inside the remote bundleRendered inline in the host React treeProp updatesDOM events through DOMEventBusNative React re-renderEvents and commandsonEvent and commandRefReact props and refsBest forIndependent teams, mixed frameworks, and polyreposSimple React remotes with no extra infrastructure Both modes use the same <MFBridgeSSR> and can be mixed freely on the same page. The Corner Cases I Spent Time On A few production scenarios I wanted to make sure the package handled honestly. Graceful Degradation When the Remote Is Down A remote can be slow, return a 5xx, or simply not respond. The host page shouldn't break because of one bad slot. mf-ssr accepts an errorFallback, and the trick is that the fallback can be the same remote mounted on the client through mf-bridge: TypeScript-JSX import { MFBridgeSSR } from '@mf-toolkit/mf-ssr' import { MFBridgeLazy } from '@mf-toolkit/mf-bridge' <MFBridgeSSR url="https://checkout.acme.com/fragment" namespace="checkout" props={{ orderId } timeout={2000} errorFallback={ <MFBridgeLazy register={() => import('checkout/entry').then(m => m.register)} props={{ orderId } fallback={<CheckoutSkeleton />} /> } /> If the SSR fetch times out, the user still gets the widget. Just on the client, the same way it would have worked without mf-ssr at all. The page doesn't break. The slot loses its first-paint optimization, for that one request. When the remote recovers, the next render uses SSR again with no code change on either side. I like this case because it inverts the usual SSR-or-nothing tradeoff. SSR becomes the fast path, with a working client-side path sitting right behind it. Auth-Isolated Caching The host caches fragments by url + props + timeout. Fine for public content. Not fine when each user gets different HTML — they would share a cache slot and see each other's pages. So there is a cacheKey prop you set when the request carries auth: TypeScript-JSX <MFBridgeSSR url="https://account.acme.com/fragment" namespace="account" props={{ view: 'orders' } fetchOptions={{ headers: { authorization: `Bearer ${token}` } } cacheKey={userId} /> The other side of the same coin is public fragments. The remote's fragment endpoint accepts a cacheControl option, so you can serve a product card as public, s-maxage=60, stale-while-revalidate=30 and let a CDN cache it for everyone: TypeScript-JSX export const handler = createMFReactFragment(ProductCard, { cacheControl: 'public, s-maxage=60, stale-while-revalidate=30', vary: 'Accept-Language', }) One pattern handles per-user fragments, the other handles cacheable public ones. Same component on both sides. Multiple Instances of the Same Remote Header, sidebar, and a content slot can all be the same remote on one page. The reason I sent prop updates through the mount DOM node, instead of a global event bus, is exactly this case: each <MFBridgeSSR> has its own DOM node, so events stay scoped to it. No filtering by instance id, no manual subscription bookkeeping. Warming the Cache From RSC If you know a fragment is going to be needed, you can start the fetch before <MFBridgeSSR> even renders. Suspense then skips the fallback entirely: TypeScript-JSX import { preloadFragment } from '@mf-toolkit/mf-ssr' // In a Server Component or route loader preloadFragment('https://checkout.acme.com/fragment', { orderId }) By the time the component renders down the tree, the HTML is already there. Where It Fits If your microfrontends share one build (a single bundler config that imports every remote), you don't need any of this. Use whatever your framework gives you. mf-ssr is for the case where each team builds and deploys independently. Different repos or not, the point is that there is no shared build step pulling everything into one Node process — and you still want a full page on first paint. The bet is that HTTP is a good enough boundary between teams, and that DOM events are a good enough way to keep host state in sync with remote rendering after hydration. The CSS isolation question, by the way, lives in mf-bridge, not here: it has shadowDom and adoptHostStyles props that wrap the remote in a Shadow DOM and forward host stylesheets (including Tailwind / CSS-in-JS chunks injected after mount) into the shadow root. SSR fragments don't use it by default since the HTML is inlined into the host response, but the option exists if you want it. Try It The package is published as @mf-toolkit/mf-ssr. The repo has runnable examples, and I've also made a demo repo where you can play with all my tools. If you've solved the same problem in a different way, I'd be curious to compare notes.

By Vitaly Zheltko

GraphQL Isn’t Dead Yet, AI Agents Revived It

We all saw the rise and fall of GraphQL. The technology was hip at the time, and then we discovered it was slow, very complex, and it was easy to shoot yourself in the foot on security. REST won that fight. One major factor that went in favor of REST was that every language speaks it, every developer understands it, and you don’t need to run a special server just to serve a GraphQL API. But does this still stand true in the age of AI? Let us try to unpack this question and see if this time it could be different for GraphQL? There’s a New API Consumer, and It Doesn’t Think Like Humans For years, APIs had two audiences: first, the services (predictable, hard-coded integrations) like APIs talking to APIs, and humans using apps (who don’t mind a bit of extra data; nobody notices 40 fields traveling across the wire while the screen only renders 10 fields). AI agents are a third audience, and they behave nothing like the first two. Think of it like this: a human browsing a shopping site doesn’t care if the product page quietly loads size charts, reviews, and shipping data if the human is not interested in those. An AI agent, though, has to read every field it’s handed, and every one of those fields sits in its memory, costing money and crowding out the things it actually needs to think about. It’s less like browsing and more like being handed the whole filing cabinet when you asked for one folder. Over-Fetching Isn’t Just Wasteful for Agents; It’s Expensive in a Different and Costly Currency Let us assume an agent asks “who manages this account?” A typical REST endpoint hands back the entire user record, the email, address, and ten other fields. This is because building a trimmed-down endpoint for every possible question is a lot of upfront engineering work. A human skims past the noise. An agent has to carry it around for the rest of the conversation, like packing your whole closet for a weekend trip because folding a smaller bag felt like too much effort. GraphQL flips that: the agent asks for exactly “manager name and email,” and that’s all that comes back. The N+1 Problem, Agent Edition Anyone who’s worked with databases knows the pain: you fetch a list of 10 orders, then make 10 more calls to get customer details for each one. REST APIs often have the same shape. For an agent, every one of those round trips is another context-window hit and another few seconds of latency, like sending ten separate texts instead of one paragraph. GraphQL lets the agent ask for orders and their customers in a single request. A Schema the Agent Can Actually Read REST documentation is a promise: “this is what the API looks like, we hope, as of whenever someone last updated the docs.” When it drifts out of date, an agent’s fallback is basically the same as a stressed junior developer’s: search the web, then go read the source code. GraphQL bakes the documentation into the API itself. The agent can ask the server, at runtime, “What exists, what does it need, what’s deprecated?” It’s the difference between asking a new coworker to guess your team’s tools from an outdated wiki page, versus just asking the tool itself how it works. Security That Matches How Agents Actually Work Most REST permission systems are coarse, calendar.read, repos.write and so on. Fine for a human logging into one app with one role. But an agent might handle customer support in one breath and billing cleanup in the next, and you don't want it holding a master key for both. GraphQL checks access field-by-field, not just endpoint-by-endpoint. That means you can grant an agent “read the customer’s name” without also granting “read their payment history”, even if both live on the same object. It’s the difference between giving someone a key to the building versus a key to one specific drawer. Errors an Agent Can Actually Act On REST failure: “400 Bad Request.” Sometimes JSON, sometimes an HTML page; format varies by provider and sometimes even within the same provider. GraphQL failure, “the field user.team.name failed, no read access on team 7." That's something an agent can act on directly; it can even retry a different query, ask for permission, or explain the problem to a person instead of burning another model call just to figure out what went wrong. Where REST Still Wins, and Probably Always Will This isn’t “GraphQL beats REST.” Caching is nearly free with REST; every CDN on earth understands it natively. GraphQL caching is a genuine engineering project. Uploading a file or streaming video over REST is simple; doing it over GraphQL is awkward. And running a GraphQL server is real operational overhead REST doesn’t have. So what’s the actual comeback? Not GraphQL replacing REST for humans and services. More like this shape, Human or agent → MCP server / CLI tool → GraphQL → your actual backend Today, most people bolt an MCP server onto REST, then hand-build the exact “shape” of every response, field by field, tool by tool, basically reinventing what GraphQL already does natively. Put GraphQL underneath instead, and the MCP layer can just pass the agent’s query straight through, precise fields, typed schema, field-level permissions, structured errors, all included. I’m not saying rip out your REST APIs. I’m saying the layer sitting between AI agents and your systems might quietly end up looking a lot like GraphQL, and if you’re building tools for agents right now, this is worth an experiment.

By Akash Lomas

Database Bottlenecks Nobody Talks About: Optimizing SQL Queries Beyond Indexing

Every performance guide starts the same way. "Add an index." And yes, indexes matter. But I've spent years fixing production databases, and here's the truth: indexing is the easy 20%. The hard 80% is everything nobody writes blog posts about. I once spent three days chasing a query that had a perfect index. The index wasn't the problem. The problem was that the database's own statistics were lying to it. This article is about that other 80%. Why This Problem Keeps Coming Back Most teams treat database performance as a one-time task. Add indexes during launch week. Move on. But databases are not static. Data grows. Traffic patterns shift. Your "small lookup table" from six months ago now has four million rows. The query that ran in 2ms during testing can quietly become a 4-second query in production. Nobody notices until users complain. Here's the uncomfortable part: indexing advice assumes your query planner always makes good decisions. It doesn't. Query planners are guessing machines. They guess based on statistics, and statistics go stale. Why Developers Struggle With This Most backend engineers learn SQL as a language, not as an execution engine. You write SELECT * FROM orders WHERE customer_id = 123, it returns rows, and that feels like magic. But behind that query is a planner making dozens of decisions: Should it use an index or scan the whole table?Should it join tables in this order or that order?Should it use a hash join or a nested loop? Developers rarely see this decision-making. So when performance drops, the first (and often only) fix is "add an index." Sometimes that helps. Often it doesn't touch the real issue. The Real Problem: Stale Statistics Most relational databases (Postgres, MySQL, SQL Server) use cost-based optimizers. These optimizers don't know your data. They estimate it using statistics — sampled snapshots of your table's shape. If those statistics are outdated, the optimizer makes bad guesses. It might think a column has 10 distinct values when it actually has 10 million. Here's a real example from a Postgres system I worked on: SQL -- Table: events (48 million rows) EXPLAIN ANALYZE SELECT * FROM events WHERE event_type = 'checkout_completed' AND created_at > NOW() - INTERVAL '7 days'; The plan showed a sequential scan, even though we had an index on event_type. Why? The table statistics thought checkout_completed made up 40% of rows. In reality, it was 0.3%. The fix wasn't a new index. It was this: SQL ANALYZE events; One command. Query time dropped from 6.2 seconds to 90 milliseconds. Lesson: An index is only useful if the planner trusts it's worth using. Common Mistakes Developers Make Let's go through the mistakes I see over and over, across different companies and different stacks. 1. Trusting SELECT * Pulling every column, even ones you don't need, forces the database to read more data pages than necessary. On wide tables, this alone can double query time. 2. Ignoring the N+1 Query Pattern This one is everywhere in ORM-heavy codebases. Python # Bad: 1 query for orders + N queries for customers orders = Order.objects.all() for order in orders: print(order.customer.name) # triggers a new query each time Python # Good: 1 query total orders = Order.objects.select_related("customer").all() for order in orders: print(order.customer.name) If you have 500 orders, the bad version runs 501 queries. The good version runs 1. 3. Deep Pagination With OFFSET SQL -- Gets slower as the offset grows SELECT * FROM products ORDER BY id LIMIT 20 OFFSET 100000; The database still has to scan and discard 100,000 rows before returning your 20. On a page 5,000 request, this crawls. Better approach — keyset pagination: SQL SELECT * FROM products WHERE id > 100000 ORDER BY id LIMIT 20; This uses the index directly. No wasted scanning. Pagination MethodPerformance at Page 10Performance at Page 5000ComplexityOFFSET/LIMITFastVery slowLowKeyset (cursor-based)FastFastMediumPrecomputed pagesFastFastHigh (needs caching) 4. Doing Math on Indexed Columns SQL -- Index on created_at is useless here SELECT * FROM orders WHERE DATE(created_at) = '2026-07-20'; Wrapping a column in a function usually breaks the database's ability to use its index. SQL -- This keeps the index usable SELECT * FROM orders WHERE created_at >= '2026-07-20' AND created_at < '2026-07-21'; Small rewrite. Big difference. How Modern Systems Actually Solve This Real production systems don't rely on a single trick. They layer several defenses. Plain Text Client Request │ ▼ API Layer │ ▼ Query Cache (Redis) ├── cache hit? return here ▼ Connection Pool (PgBouncer) │ ▼ Read Replica (for reads) ──── Primary DB (for writes) │ ▼ Query Planner + Statistics │ ▼ Storage Engine Each layer exists to reduce pressure on the layer below it. Miss the cache, and you hit the pool. Miss the primary's write load, and reads go to a replica. Connection Pooling Matters More Than People Think Opening a raw database connection is expensive. It involves a TCP handshake, authentication, and memory allocation on the database side. Without pooling, a burst of traffic can create hundreds of connections in seconds. Postgres, for example, starts choking well before 500 connections. Plain Text # pgbouncer.ini [databases] mydb = host=127.0.0.1 port=5432 dbname=mydb [pgbouncer] pool_mode = transaction max_client_conn = 1000 default_pool_size = 25 With transaction pooling mode, PgBouncer hands out a real database connection only for the duration of a transaction, then returns it to the pool. This lets 1,000 app connections share just 25 real ones. Lock Contention: The Silent Killer This is the bottleneck that almost nobody talks about, because it doesn't show up in slow query logs the same obvious way. Here's what happened to us. A "quick" query started timing out during peak hours: SQL UPDATE inventory SET stock = stock - 1 WHERE product_id = 42; Individually, this query was fast. But during a flash sale, hundreds of these updates hit the same row at the same time. Each transaction had to wait for the previous one to release its row lock. The queries weren't slow. They were queued. Plain Text Time Transaction A Transaction B Transaction C 0ms LOCK row 42 waiting... waiting... 5ms UPDATE + COMMIT LOCK row 42 waiting... 6ms UPDATE + COMMIT LOCK row 42 7ms UPDATE + COMMIT How we fixed it: Moved to an eventual-consistency model for stock counts (queue-based decrement)Used SELECT ... FOR UPDATE SKIP LOCKED for job-queue-style tablesBatched decrements instead of doing them one row at a time SQL -- Instead of 100 individual UPDATE statements UPDATE inventory SET stock = stock - sub.qty FROM ( VALUES (42, 3), (43, 1), (44, 7) ) AS sub(product_id, qty) WHERE inventory.product_id = sub.product_id; One batched statement instead of a hundred lock acquisitions. Isolation Levels: A Trade-off, Not a Setting You Ignore Most engineers leave the isolation level at whatever the database defaults to. That's usually fine — until it isn't. Isolation LevelPreventsPerformance CostCommon Use CaseRead UncommittedNothing muchLowestRarely used, riskyRead CommittedDirty readsLowDefault in Postgres, most web appsRepeatable ReadNon-repeatable readsMediumFinancial reports, reconciliationSerializablePhantom readsHighestBanking transactions, inventory locks Higher isolation means more correctness guarantees. It also means more locking, more retries, and lower throughput. Don't default to Serializable "to be safe." You'll pay for it in throughput, and most apps don't need it. Query Plan Reading: A Skill Most Engineers Skip If you only remember one thing from this article, remember this: learn to read EXPLAIN ANALYZE output. It tells you the truth. Everything else is a guess. SQL EXPLAIN ANALYZE SELECT o.id, c.name FROM orders o JOIN customers c ON o.customer_id = c.id WHERE o.status = 'pending'; Sample output to watch for: SQL Hash Join (cost=120.50..3400.22 rows=850 width=64) (actual time=12.100..340.556 rows=42000 loops=1) Hash Cond: (o.customer_id = c.id) -> Seq Scan on orders o (cost=0.00..2900.00 rows=850) (actual time=0.020..300.100 rows=42000 loops=1) Notice the gap: the planner estimated 850 rows. The actual count was 42,000. That's a 49x miss. When estimated and actual rows differ by a wide margin, that's your signal. Stale statistics, bad indexes, or a query shape the planner can't reason about well. Denormalization: Sometimes the Right Move Normalization is taught as the "correct" way to design schemas. In practice, strict normalization can hurt performance on read-heavy systems. We had a dashboard query joining six tables to compute one number: total revenue per region. SQL SELECT r.name, SUM(o.total) FROM orders o JOIN customers c ON o.customer_id = c.id JOIN regions r ON c.region_id = r.id JOIN order_items oi ON oi.order_id = o.id JOIN products p ON oi.product_id = p.id JOIN categories cat ON p.category_id = cat.id GROUP BY r.name; This ran in 4 seconds. Dashboard needed sub-second response. We added a summary table, updated by a nightly job: SQL CREATE TABLE revenue_by_region ( region_name TEXT PRIMARY KEY, total_revenue NUMERIC, updated_at TIMESTAMP ); Dashboard query became: SQL SELECT region_name, total_revenue FROM revenue_by_region; From 4 seconds to 8 milliseconds. The trade-off: data is now up to 24 hours stale. This only works if your business can tolerate staleness. For real-time fraud detection, this approach would be wrong. Know your consistency requirements before you denormalize. Performance Considerations Checklist Before shipping a query to production, run through this: ✔ Did you check EXPLAIN ANALYZE, not just EXPLAIN? ✔ Are your table statistics current (ANALYZE run recently)? ✔ Does the query avoid functions wrapped around indexed columns? ✔ Are you selecting only the columns you need? ✔ Is pagination using keyset instead of large OFFSET values? ✔ Are batch writes used instead of row-by-row loops? ✔ Is the isolation level appropriate for the use case, not just the default? ✔ Have you tested this query against production-sized data, not a dev sample? Security Considerations Performance work sometimes creates security gaps. Watch for these: Dynamic query building for "flexible filters" often leads to string concatenation, which opens SQL injection risk. Use parameterized queries even for performance-tuned raw SQL.Read replicas used for reporting sometimes get looser access controls because "it's just a read replica." That's still your data.Caching layers (Redis, Memcached) can leak sensitive data if you cache full row objects without checking what's in them. Scaling Challenges As systems grow, new problems appear that indexing can't fix: Plain Text Single DB Instance │ ▼ Growing write load │ ▼ Read Replicas (helps reads, not writes) │ ▼ Still hitting write limits │ ▼ Sharding (splits writes across nodes) │ ▼ Cross-shard joins become painful Sharding solves write throughput but creates a new problem: joins across shards don't work the way they used to. You end up doing joins in application code, which is slower and more error-prone than letting the database do it. This is why teams delay sharding as long as possible. It's a last resort, not a first optimization. What We Learned A few honest lessons from years of doing this: Statistics decay silently. Schedule ANALYZE (or your database's equivalent) as a routine job, not an afterthought.The slowest part of a query is often not the query itself. It's lock waiting, connection exhaustion, or network round trips.ORMs hide problems well. They also hide the N+1 pattern extremely well. Turn on query logging in staging and actually read it.Caching isn't free. Cache invalidation bugs have cost us more debugging time than the queries we were trying to avoid.Nobody reads execution plans until something breaks. Read them earlier. It's a habit, not a rescue tool. When Not to Use These Techniques Not every optimization belongs in every system. Don't denormalize a table that changes every second the sync job will never catch up.Don't add read replicas if your write load, not read load, is the actual bottleneck.Don't reach for sharding if a bigger instance and better indexing would solve it for the next two years.Don't tune isolation levels down for "performance" on a system handling money movement. Optimization without a clear bottleneck measurement is just guessing with extra steps. Final Thoughts Indexing is the first lesson in database performance, not the last one. The real bottlenecks stale statistics, lock contention, bad pagination, and isolation level mismatches don't show up in a "10 SQL Tips" listicle. They show up at 2 AM, during a traffic spike, when your on-call phone rings. The next challenge for most teams isn't learning these techniques. It's building the habit of checking for them before a query becomes a production incident. That habit reading EXPLAIN ANALYZE, tracking replication lag, watching lock wait times matters more than any single trick in this article.

By Muhammad Awais Arshad

Languages

DZone's Featured Languages Resources

Monthly Top Languages Experts

The Latest Languages Topics