First, the question arises, what makes these JEPAs “energy based”? They say “An EBM defines a scalar energy function E(x,y) measuring compatibility between inputs x and outputs y, where low energy indicates high compatibility. Learning consists of shaping the energy landscape so that correct input-output pairs have lower energy than incorrect ones.” and thus you can achieve training by minimizing E, subject to some regularization to prevent universally minimizing E. This seems like a more general concept than energy based models, in which your scalar energy function actually represents the un-normalized distribution (perhaps why they call them energy based JEPAs and not energy based models). It isn’t clear whether this energy objective carries any practical impact that would normally be associated with energy based models (since EBMs have certain properties, e.g. sampling, composability, etc.). Perhaps on a smaller problem it could be actually measurable, but does the energy learned by these JEPA regularized approaches actually look like the energy landscape? Does it match the true energy better or worse than other, more explicit, EBM methods (e.g. contrastive divergence)? Can we sample from these models? These are just some of the interesting questions that the energy based framing of SSL brings up, although I don’t seem them frequently addressed.

But perhaps the more interesting question is, what makes these EB-JEPAs JEPAs? If you look at Figure 2 of the I-JEPA paper, you will see that a JEPA is distinguished from a Joint-Embedding Architecture (JEA) through its predictive component (in the embedding space), and in the background, Section 2, the authors remark that JEAs can be cast in this energy minimization framework. Many more common forms of JEA exist, e.g. SimCLR, but come with tradeoffs as the paper outlines. In the EB-JEPA, they present an image JEPA that looks quite different (their image example looks like Fig 1 of SimCLR, along with the usage of image augmentations, unlike I-JEPA). It seems at first glance to be more of a JEA than a JEPA. However, these are not solid lines, and equations (1) and (5) reveal more insight into these differences. Although the image case has no $g_\phi$ (predictor), it would seem like a JEA, but really a JEA is a subclass of JEPAs (in which predictors are identity functions).

In jepax we are able to recreate similar performance to the published results, see the figure below.

Interestingly, ViT based models seem to substantially underperform ResNet/CNN architectures. We investigated a variety of architecture choices (e.g. Batch Norm, Group Norm, depth, size, loss weightings, etc.) and found this performance to be generally representative. However, it is often said that ViT (and transformer based models more broadly) require more data to achieve high quality performance, so this may simply be an example of that phenomenon. Additionally, we saw similar results with I-JEPA, with the ViT performing much better on ImageNet1k than on CIFAR-100.

With the above in mind, we are hopeful that the addition of EB-JEPAs in jepax will allow for faster and more fruitful explorations of JEPAs and further support the growth of JAX based JEPA researchers.

Changelog

1. February 25, 2026: Published initial version.

State management is not always pretty, and leads to unergonomic interfaces such as flax:

logits, updates = state.apply_fn(
      {'params': params, 'batch_stats': state.batch_stats},
      x=batch['image'], train=True, mutable=['batch_stats'])

or somewhat complicated implementations that require careful operations for edge cases.

But what if there was a way to simplify this? Thanks to the experimental hijax features, I think there will be. A further explanation of what hijax is will be in a future blog, but for now check out the JAX devlabs keynote for more. Basically, hijax is a new interface to working with intermediate levels of operations, between pytrees and primitives. It’s experimental and changing, so the interface is subject to change (this code is with v0.9.0 JAX), but the idea has a lot of potential to simplify these workloads.

Similar to how array refs allow for stateful operations (see my previous blog), hijax has features that enable functionality in that direction. Specifically, the idea of Quasi Dynamic Data (QDD) which allows for stateful operations from the user perspective. Under the hood, when a trace requires lowering, hijax primitives call their to_lojax methods. For Box, box_get_p.to_lojax simply returns the stored value’s leaves, while box_set_p.to_lojax updates the internal _val attribute. This means the mutable semantics the user programs get compiled down to direct value access. This allows a nice simplification of batch norm implementations of state. For example,

import jax
import jax.numpy as jnp
from jax.tree_util import register_pytree_node_class
from jax._src.hijax import Box

@register_pytree_node_class
class BatchNorm:

    def __init__(self, num_features, momentum=0.1, eps=1e-5, axis_name=None):
        self.running_mean = Box(jnp.zeros(num_features))
        self.running_var = Box(jnp.ones(num_features))
        self.gamma = jnp.ones(num_features)
        self.beta = jnp.zeros(num_features)
        self.momentum = momentum
        self.eps = eps
        self.num_features = num_features
        self.axis_name = axis_name

    def __call__(self, x, training=True):
        if training:
            if self.axis_name is not None:
                mean = jax.lax.pmean(x, axis_name=self.axis_name)
                mean_of_squares = jax.lax.pmean(x ** 2, axis_name=self.axis_name)
                var = mean_of_squares - mean ** 2
            else:
                mean = x.mean(axis=0)
                var = x.var(axis=0)

            rm = self.running_mean.get()
            rv = self.running_var.get()
            new_rm = jax.lax.stop_gradient(rm * (1 - self.momentum) + mean * self.momentum)
            new_rv = jax.lax.stop_gradient(rv * (1 - self.momentum) + var * self.momentum)
            self.running_mean.set(new_rm)
            self.running_var.set(new_rv)
        else:
            mean = self.running_mean.get()
            var = self.running_var.get()

        x_norm = (x - mean) / jnp.sqrt(var + self.eps)
        return self.gamma * x_norm + self.beta

With a little sprinkling of vmap rules¹, like we did with refs, we can even get this to replace the somewhat complicated substate/vmap setup of equinox. Basically, we just have to tell JAX that we have a vmap rule for this Box, in which we just want to broadcast/propagate the vmaping to the elements of the Box. Certainly there would be other ways to approach this, but for the simple Counter example this is good enough.

from jax._src.interpreters import batching
from jax._src.interpreters.batching import not_mapped
from jax._src.hijax import (
    new_box_p,
    box_get_p,
    box_set_p,
)

def _new_box_batching(axis_data, batched_args, batch_dims, *, treedef):
    box = new_box_p.bind(treedef=treedef)
    return box, not_mapped


def _box_get_batching(axis_data, batched_args, batch_dims, *, avals):
    (box,), (box_bdim,) = batched_args, batch_dims

    if box_bdim is not not_mapped:
        raise ValueError("Box cannot be batched")

    results = box_get_p.bind(box, avals=avals)
    out_bdims = (not_mapped,) * len(results)
    return results, out_bdims


def _box_set_batching(axis_data, batched_args, batch_dims, *, treedef):
    box, *vals = batched_args
    box_bdim, *val_bdims = batch_dims

    if box_bdim is not not_mapped:
        raise ValueError("Box cannot be batched")

    box_set_p.bind(box, *vals, treedef=treedef)
    return [], []

batching.fancy_primitive_batchers[new_box_p] = _new_box_batching
batching.fancy_primitive_batchers[box_get_p] = _box_get_batching
batching.fancy_primitive_batchers[box_set_p] = _box_set_batching

@register_pytree_node_class
@dataclass(init=False)
class Counter:
    index: Box
    _hash: int
    _counter: ClassVar[int] = 0

    def __init__(self, index):
        self.index = Box(index)
        self._hash = Counter._counter
        Counter._counter += 1

    def __hash__(self):
        return self._hash

    def __call__(self, x):
        value = self.index.get()
        new_x = x + value
        self.index.set(value + 1)
        return new_x

    def tree_flatten(self):
        children = (self.index,)
        aux_data = self._hash
        return children, aux_data

    @classmethod
    def tree_unflatten(cls, aux_data, children):
        (index_box,) = children
        obj = object.__new__(cls)
        obj.index = index_box
        obj._hash = aux_data
        return obj


counter = Counter(jnp.array(0))
x = jnp.array(2.3)
f = jax.jit(counter)

print(counter.index.get())

_ = f(x)
print(counter.index.get())

_ = f(x)
print(counter.index.get())


class Model(eqx.Module):
    linear: eqx.nn.Linear
    counter: Counter
    v_counter: Counter

    def __init__(self, key):
        # Not-stateful layer
        self.linear = eqx.nn.Linear(2, 2, key=key)
        # Stateful layer.
        self.counter = Counter(jnp.array(0))
        # Vmap'd stateful layer. (Whose initial state will include a batch dimension.)
        self.v_counter = jax.vmap(Counter, out_axes=None)(jnp.array([0, 0]))

    def __call__(self, x):
        assert x.shape == (2,)
        x = self.linear(x)
        x = self.counter(x)
        x = jax.vmap(self.v_counter)(x)
        return x


key = jr.key(0)
model = Model(key)
x = jnp.array([5.0, -1.0])
_ = model(x)
print(model.counter.index.get())
print(model.v_counter.index.get())

As you can see, this replicates the effect of the vmap/substate of equinox in a much simpler and elegant manner (of course, the custom vmap rules are annoying, but this is mostly because it is experimental and requires only a single implementation for this Box type). This is just the beginnings of the cool stuff you can do with hijax, and I look forward to seeing more mainstream JAX package adoption, for example see the Flax hijax talk.

Changelog

1. February 2, 2026: Published initial version.
2. February 17, 2026: Updated to reflect upstream cur_qdd fix.

Footnotes

This post is mostly an exploration of JAX, and the solutions presented below probably come with a lot of footguns (especially with any sort of sharding/distributed training) and this is meant to be pedagogical rather than practical (in fact, I even prefer the explicit key management of JAX, usually).

So, how does numpy/torch handle randomness and how does JAX differ? The traditional approach of numpy is to have the pseudo-random generator (PRNG) hidden from the user, so each time you call np.random.normal it implicitly handles the generation to be unique. It also allows for you to set a global seed with np.random.seed(int). The JAX approach is to explicitly manage the key which is the driver of the pseudo-random generator (the motivations for JAX’s PRNG design are outlined here). Historically, these impure functions have not meshed well with JAX’s design. While JAX’s design has a lot of benefits, sometimes you just want a random number and don’t want to think about the key (case in point: https://docs.jax.dev/en/latest/pytrees.html#example-of-jax-tree-map-with-ml-model-parameters).

However, now that array refs are part of the exposed API, we can make things a little more numpy-like. Now, we all know what arrays are, but what is an array ref? That is beyond the scope of this blog (a clever way of saying: I’m not really sure myself), so let’s just use the physicist explanation of spin and say an array ref is like a C pointer except it’s not a pointer, and it’s not a first class citizen¹.

With array refs, it’s quite straightforward to make a more numpy style array interface (in fact, there’s a JEP for this exact thing). We can just make a stateful ref counter, and increment that for stateful randomness. To be specific, consider the following:

import jax
from jax import numpy as jnp
from dataclasses import dataclass, field

@jax.tree_util.register_dataclass
@dataclass
class RNG:
    key: jax.Array
    counter: jax.Ref

    def __init__(self, seed):
        self.key = jax.random.key(seed)
        self.counter = jax.new_ref(jnp.array(0, dtype=jnp.int32))

    def normal(self, shape = (), dtype = None):
        self.counter[...] = self.counter[...] + 1
        key = jax.random.fold_in(self.key, self.counter[...])
        return jax.random.normal(key, shape=shape, dtype=dtype)

    def set_seed(self, seed):
        self.counter[...] = jnp.array(seed)


rng = RNG(0)

Then, when we use it in a package, we might see something like,

from jox import rng as jr
jr.set_seed(0)

weights = jr.normal((10, 10))

However, right off the bat, this comes with a problem. We can’t vmap over it. If you try, you see a Exception: performing a set/swap operation with vmapped value on an unbatched array reference of type Ref{int32[]}. Move the array reference to be an argument to the vmapped function? error. The reason for that is the ref is being closed over, and so when the vmap-ing happens, JAX doesn’t know what the ref should become (since many operations in parallel are happening to the ref). Vmap is like 1/3 of the JAX transforms, so this is a bit of a problem. There’s (at least) two possible solutions. First, we can make the actual PRNG object be passed around and allow for splitting the object so it can also be vmap-ed over (if the ref is an argument of the function, you can vmap over refs). This is what the current JEP does. But the second route is to double down and embrace the numpy interface (the whole point of this was to avoid managing an explicit state as a user, key or otherwise). We can accomplish this second route because we know something specific about the outcome of our ref: it doesn’t depend on the inputs. The usual problem with closed over refs is if you do some operation on the inputs to the vmap and the ref depends on them, it’s unclear what the output should be. However, we know our ref doesn’t depend on the values of the array (we can just increment the ref by the batch size, or even just by 1).

One way to accomplish this second approach is with a custom vmap rule. However, since we aren’t explicitly vmap-ing over the jr.normal, the custom vmap rule won’t necessarily trigger. So we have to make our implementation recognize when it is inside of vmap. This can be done using axis names, via the following:

@jax.custom_batching.custom_vmap
def _normal(counter, key, batch_idx, shape, dtype):
    counter[...] = counter[...] + 1
    key = jax.random.fold_in(key, counter[...])
    return jax.random.normal(key, shape=shape, dtype=dtype)

@_normal.def_vmap
def _normal_vmap_rule(axis_size, in_batched, counter, key, batch_idx, shape, dtype):
    # Add by batch size (could just be 1?)
    counter[...] += axis_size
    k = jax.random.fold_in(key, counter[...])
    result = jax.random.normal(k, shape=(axis_size, *shape), dtype=dtype)
    return result, True

@jax.tree_util.register_dataclass
@dataclass
class RNG:
    key: jax.Array
    counter: jax.Ref
    axis: str = field(metadata=dict(static=True))

    def __init__(self, seed, axis="i"):
        self.key = jax.random.key(seed)
        self.counter = jax.new_ref(jnp.array(0, dtype=jnp.int32))
        self.axis = axis

    def normal(self, shape = (), dtype = None):
        try:
            batch_idx = jax.lax.axis_index(self.axis)
        except:
            batch_idx = jnp.array(0)
        return _normal(self.counter, self.key, batch_idx, shape, dtype)

    def set_seed(self, seed):
        self.counter[...] = jnp.array(seed)


rng = RNG(0)

This allows for vmap support, albeit not perfect since the user has to specify the axis name (also this isn’t robust to nested vmaps but could be made to be)². However, this is only at the vmap level and is a minor change (that could also be optimized at the interface). If they don’t specify it, the code will run, but all the values will be identical (silent PRNG failures are always preferable to raising errors right?).

def f(x):
    return jr.normal() + x

jax.vmap(f, axis_name="i")(jnp.ones(10)) # works!

Technically, we have one more hurdle to go through. If you set the shape (even outside of a jit), you will see an error like TypeError: Shapes must be 1D sequences of concrete values of integer type, got (JitTracer<~int32[]>, JitTracer<~int32[]>). This is because the custom vmap code is immediately traced for a jaxpr, which means the static integers get promoted to tracers. It is not clear to me whether this is necessary, and the fix is to add something like static arguments (like custom vjp) or whether there is a simpler solution³. However, we can skirt around this for now by also capturing the shapes:

def normal(self, shape = (), dtype = None):
    try:
        batch_idx = jax.lax.axis_index(self.axis)
    except:
        batch_idx = jnp.array(0)

    @jax.custom_batching.custom_vmap
    def _normal(counter, key, batch_idx):
        counter[...] = counter[...] + 1
        key = jax.random.fold_in(key, counter[...])
        return jax.random.normal(key, shape=shape, dtype=dtype)

    @_normal.def_vmap
    def _normal_vmap_rule(axis_size, in_batched, counter, key, batch_idx):
        # Add by batch size
        counter[...] += axis_size
        k = jax.random.fold_in(key, counter[...])
        result = jax.random.normal(k, shape=(axis_size, *shape), dtype=dtype)
        return result, True

    return _normal(self.counter, self.key, batch_idx)

To prove this approach actually works, here is a pure JAX VAE, with absolutely no jax.random.key and no jax.random.split. We can visualize the latent space in the same way as https://keras.io/examples/generative/vae/,

import jax
import jax.numpy as jnp
from jox import rng as jr
import optax
import matplotlib.pyplot as plt
import numpy as np
from torchvision import datasets

train = datasets.MNIST(root='./data', train=True, download=True)
test = datasets.MNIST(root='./data', train=False, download=True)

x_all = np.concatenate([train.data.numpy(), test.data.numpy()], axis=0)
x_all = x_all.astype(np.float32) / 255.0
x_all = x_all.reshape(-1, 1, 28, 28)

batch_size = 100
num_samples = (len(x_all) // batch_size) * batch_size
x = jnp.array(x_all[:num_samples].reshape(-1, batch_size, 1, 28, 28))

print(f"Data shape: {x.shape}")
plt.matshow(x[0][0].reshape((28, 28)), cmap='gray')
plt.show()

def init_linear(in_features, out_features):
    stddev = jnp.sqrt(2.0 / (in_features + out_features))
    w = jr.normal((in_features, out_features)) * stddev
    b = jnp.zeros(out_features)
    return {'w': w, 'b': b}


def linear(params, x):
    return x @ params['w'] + params['b']

def init_vae(input_dim=784, hidden_dims=[512, 256], latent_dim=2):
    params = {'encoder': [], 'decoder': [], 'z_mean': None, 'z_log_var': None}
    key_idx = 0
    
    enc_dims = [input_dim] + hidden_dims
    for i in range(len(enc_dims) - 1):
        params['encoder'].append(init_linear(enc_dims[i], enc_dims[i+1]))
        key_idx += 1
    
    params['z_mean'] = init_linear(hidden_dims[-1], latent_dim)
    key_idx += 1
    params['z_log_var'] = init_linear( hidden_dims[-1], latent_dim)
    key_idx += 1
    
    dec_dims = [latent_dim] + hidden_dims[::-1] + [input_dim]
    for i in range(len(dec_dims) - 1):
        params['decoder'].append(init_linear(dec_dims[i], dec_dims[i+1]))
        key_idx += 1
    
    return params

def encode(params, x):
    h = x.flatten()
    for layer_params in params['encoder']:
        h = jax.nn.relu(linear(layer_params, h))
    z_mean = linear(params['z_mean'], h)
    z_log_var = linear(params['z_log_var'], h)
    z = z_mean + jnp.exp(0.5 * z_log_var) * jr.normal(shape=z_mean.shape)
    return z_mean, z_log_var, z


def decode(params, z):
    h = z
    for layer_params in params['decoder'][:-1]:
        h = jax.nn.relu(linear(layer_params, h))
    logits = linear(params['decoder'][-1], h)
    return logits.reshape(1, 28, 28)


def forward(params, x):
    z_mean, z_log_var, z = encode(params, x)
    logits = decode(params, z)
    return z_mean, z_log_var, logits

def binary_cross_entropy(labels, logits):
    # https://github.com/google/flax/blob/main/examples/vae/train.py
    logits = jax.nn.log_sigmoid(logits)
    return -jnp.sum(labels * logits + (1.0 - labels) * jnp.log(-jnp.expm1(logits)))


def vae_loss_single(logits, z_mean, z_log_var, data):
    recon_loss = jnp.sum(binary_cross_entropy(data, logits))
    kl_loss = -0.5 * jnp.sum(1 + z_log_var - jnp.square(z_mean) - jnp.exp(z_log_var))
    return recon_loss + kl_loss

def loss_fn(params, data):
    def single_loss(x):
        z_mean, z_log_var, logits = forward(params, x)
        return vae_loss_single(logits, z_mean, z_log_var, x)
    
    losses = jax.vmap(single_loss, axis_name="i")(data)
    return jnp.mean(losses)

@jax.jit
def step(params, opt_state, data):
    loss_value, grads = jax.value_and_grad(loss_fn)(params, data)
    updates, opt_state = optimizer.update(grads, opt_state, params)
    params = optax.apply_updates(params, updates)
    return params, opt_state, loss_value

params = init_vae(input_dim=784, hidden_dims=[512, 256], latent_dim=2)
optimizer = optax.adam(learning_rate=0.001)
opt_state = optimizer.init(params)

losses = []
for i in range(100):
    if i % 10 == 0:
        print(i)
    batch_losses = []
    for batch in x:
        params, opt_state, l = step(params, opt_state, batch)
        batch_losses.append(l)
    losses.append(np.mean(np.array(batch_losses)))

plt.plot(losses)
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.show()

def plot_latent_space(params, n=30, figsize=15):
    digit_size = 28
    scale = 1.0
    figure = np.zeros((digit_size * n, digit_size * n))
    
    grid_x = jnp.linspace(-scale, scale, n)
    grid_y = jnp.linspace(-scale, scale, n)[::-1]

    for i, yi in enumerate(grid_y):
        for j, xi in enumerate(grid_x):
            z_sample = jnp.array([xi, yi])
            x_decoded = jax.nn.sigmoid(decode(params, z_sample))
            digit = x_decoded[0].reshape(digit_size, digit_size)
            figure[
                i * digit_size : (i + 1) * digit_size,
                j * digit_size : (j + 1) * digit_size,
            ] = digit

    plt.figure(figsize=(figsize, figsize))
    start_range = digit_size // 2
    end_range = n * digit_size + start_range
    pixel_range = jnp.arange(start_range, end_range, digit_size)
    sample_range_x = [float(round(i, 1)) for i in grid_x.tolist()]
    sample_range_y = [float(round(i, 1)) for i in grid_y.tolist()]
    plt.xticks(pixel_range, sample_range_x)
    plt.yticks(pixel_range, sample_range_y)
    plt.xlabel("z[0]")
    plt.ylabel("z[1]")
    plt.imshow(figure, cmap="Greys_r")
    plt.show()

plot_latent_space(params)

Changelog

1. December 15, 2025: Published initial version.
2. February 2, 2026: Fixed some typos.

Footnotes

The first, and most trivial case, is when you have more classes than quantized vectors. If you are quantizing your vectors to a set of $N$ vectors, and you have $N + 1$ classes, then naturally two classes have to have overlap in this quantized space. However, such a reduced quantized space isn’t as common in machine learning problems.

For the second example, consider the XOR problem. This is known to be not linearly separable. So in the case where we have $2$ classes and $2$ quantization vectors, then if class 1 maps to $(0, 1)$ and $(1, 0)$ and class 2 maps to $(1, 1)$ and $(0, 0)$ in quantized space, but the non-quantized vectors are separable then we have shown our goal. Given these four quantization vectors, consider the following embedding points, class 1: $(0.75, -2)$, $(-2, 0.8)$, class 2: $(-0.75, 0.25)$, $(0.8, 0.8)$, these embed to $(1, 0)$, $(0, 1)$, $(0, 0)$, $(1, 1)$. Clearly these embeddings are not linearly separable, but with $f(x, y) = -0.225x - 0.37y - 0.1$, the non-quantized vectors are.

Changelog

1. November 29, 2025: Published initial version.
2. February 2, 2026: Updated to use math mode for equations and vectors.

However, as a contrarian thought experiment, here I will posit an argument for why the bubble will not pop. Why will the bubble never pop? A simple argument: AI (specifically LLMs) in their current forms (without need for any future improvements) are the single greatest tool in the history of human civilization for surveillance and control of individuals at a massive society level scale. Thus, the bubble will never pop because governments and big tech will continue to operate AI, even at a seemingly substantial loss, due to the power it provides.

This doesn’t rely on any increases in performance, or any changes in how the technology works, and is purely based on the LLMs as they exist today. What makes them the most powerful tool for control? One of their biggest strengths is control of information, especially as enabled by social media. Already, on social media, specialized algorithms are deployed to surveil, manipulate, and control users. LLMs supercharge these capabilities. LLMs can directly interact, engage, and influence users through bot profiles. While existing bot/troll farms have been successful for years (1, 2, 3), LLMs enable the large scale production and delivery of propaganda without the engineering requirements of bot farms, and are capable delivering more sophisticated messages. Additionally, as LLMs increasingly become a source of truth, they provide big tech with a means to further centralize narrative control. LLMs can be made to parrot anything, they have no intrinsic proclivity or bias for truthful statements. Thus, once companies with LLMs have monopolized trust (and squashed competing alternatives), they can begin to use this trust to manipulate and influence the public (i.e. the exact same playbook that was run for social media and has influenced politics on a global scale). In fact, even if those who control the LLMs are wholly benevolent, LLMs are still being used as tools of information control (powered by the blind collecting of training data). To be clear, this is not blind speculation, already LLMs are being adopted as tools for surveillance, “suppress[ing] dissenting arguments”, and propaganda. The full scale and power of these approaches is certainly underestimated by publicly available information.

Much of the above was also true for existing social media algorithms (albeit not at the same scale), but a key unique property of LLM technology is the individuals’ interactions. Many people use these models to serve a variety of needs, such as travel planner, doctor, email writer, therapist, friend, financial planner, etc, etc, etc. This is both a massive amount of information that would otherwise be inaccessible, and a potent tool for influence. Already, LLMs are influencing people in a variety of powerful and unprecedented ways. There is the obvious case of “AI psychosis”, in which AI taps into users delusions and causes them to spiral, which has been reported on at length (e.g. 4, 5). More relevant though, is the recent trend of LLMs influencing particularly vulnerable people. While these current cases perhaps stem more from negligence than malice, they highlight the essential point about the capability of LLMs for influencing mental states and physical choices. Chatbots can be deployed to guide users to any action or any conclusion or any mental state (certainly they are not omnipotent mind control machines, but they wield substantial influence as people grow to depend on them). Imagine a therapist who is available 24/7, 365 days a year, ready to answer and help you through any crises, through any feeling, through any breakup. That is normally where the VC pitch ends, but the key element is that this therapist is not a real therapist, it is an LLM deployed by a company that doesn’t share your goals but seeks to profit off of you. Through a chat window, users are (willingly) dumping unbelievable amounts of personal data. Never before have peoples’ feelings, desires, plans, travel ideas, emails, health documents, legal documents, and more, been compiled so neatly. But this gold mine doesn’t stop there. Not only do companies and governments have data that would otherwise be difficult to come by (regardless of the extent they are currently taking advantage of this data), they can directly use it not just analyze for large scale patterns, but influence every single individual in a unique and personalized manner. No longer does big tech need crude algorithms to take advantage of vulnerable teenage girls, or rely on huge engineering efforts to deploy spyware to surveil users encrypted chats, or exploit adware to collect, log, and sell peoples’ personal health information. Now people simply ask the AI to help them when they are feeling vulnerable, they have an AI browser or OS that can simply see any encrypted messages, they upload their doctors notes to chatbots to ask questions about them.

Probably the “best” case is that the result of all this control and manipulation is aggressive and invasive advertising (those who have read more science fiction books can probably imagine more creatively unpleasant use cases). But these ads will likely be exceedingly effective. While ads on most websites are easy to distinguish, the continual stream of text of a chatbot lends itself brilliantly to ads. If you ask an LLM, “what is the best printer?” you expect it to merely tell you an answer from an opaque computing process, and as such being served an ad in this format is all but impossible to determine.

Additional Reading

• How AI and Human Behaviors Shape Psychosocial Effects of Extended Chatbot Use: A Longitudinal Randomized Controlled Study
• A.I. as normal technology (derogatory)
• Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task
• The Artificiality of Alignment
• I Am An AI Hater

Changelog

1. October 28, 2025: Published initial version.
2. November 29, 2025: Removed comment about market (ir)rationality.