This is the second part of this series on open problems in decentralized AI infrastructure. The first part was about verifiable computations and the third part (to be published) will be about data, ownership and privacy.

We now discuss issues related to decentralized system design such as communication, resource allocation and incentives.

Although machine learning computations are highly parallelizable and hence a good candidate for being decentralized, there are many problems that come with decentralized training.

First, as with any distributed systems, communication overhead is a big issue which usually leads to higher latency compared to centralized setups.

Second, in realistic scenarios, the compute nodes can have varying compute and network capabilities. Efficiently leveraging and load balancing across such heterogeneous workers is challenging and impacts the overall training time.

Lastly, there is the issue of how to design incentives that drive the correct type of behavior from all the actors in the system.

Communication

Latency is a significant challenge in decentralized computing, and even more so for ML workloads. The decentralized nature of the system introduces communication overhead, leading to slower processing times compared to centralized setups. For example, Gensyn found that (on CPU) their decentralized training protocol adds a 46% time overhead compared to native training (for MNIST image classification). However, this overhead comes in great part from their verification mechanism, which would not be necessary in a trusted setting.

Aside from the verification overhead, techniques exist that can significantly reduce latency compared to centralized setups, however they come with their own set of challenges. Edge computing, for instance, could be used for inferences that require lighter computation but lower latency. This decentralized approach brings computation closer to the data source and end-user, enabling faster and more responsive processing. By leveraging geographically distributed edge nodes, edge computing can reduce data transmission round-trip time (at least up to 16.7%), which is particularly relevant for inference workloads that require low latency for optimal user experience.

However, realizing the full potential of decentralized edge computing comes with technical challenges. Ensuring network reliability, addressing bandwidth limitations, and handling intermittent connectivity are crucial for seamless edge computing operations. Although proximity to the data and users could also ensure independent operation, even during sporadic internet outage, providing a reliable and uninterrupted experience.

Managing and storing large volumes of data generated at the edge is challenging due to the limited storage and computational capacity of edge devices. Edge devices come in various forms (e.g., smartphones, IoT) each with different hardware and software configurations. Managing this heterogeneity and ensuring interoperability is complex, especially due to the lack of common standards. Interoperability between devices is also a significant challenge for other types of decentralized computing. To address these challenges and open problems, ongoing research and development efforts are required.

Projects like PETALS and StableHorde are already exploring novel approaches to decentralized inference and model serving at the edge. PETALS uses model parallelization, meaning that the model is spread across a few nodes, with each node holding different layers of the models. PETALS uses quantization (a technique we discussed in the first part of this series as a solution to non-determisticity) to both reduce the size of the model and to improve on communication overhead by compressing the hidden states exchanged between different nodes (using dynamic blockwise quantization). For example, it reduces the number of servers needed for BLOOM-176B from 44 to 22, cuts latency in half and halves the bandwidth requirements without noticeably affecting generation quality. This helps deal with memory, which is usually the bottleneck, more than computation, for inference workload. Similarly to edge computing, each client prioritizes connecting to nearby servers to minimize network latency in each communication round.

For training and fine-tuning, data parallelism and model parallelism are the general approaches. Pipeline parallelism was introduced to reduce the communication complexity (and especially circumvent an all-to-all form of communication). Instead of allocating each device with a subset of each layer to compute, each device is assigned one or several layers of the model such that each node only needs to communicate with their adjacent layers.
One issue with pipeline parallelism is that the node with the weakest compute or network capabilities slows down the system and, therefore, nodes with higher capabilities. This problem is specific to heterogeneous networks with possible crashes. To solve this issue, the SWARM protocol introduces temporary dynamic pipelines such that each node can potentially interact with multiple predecessors or successors to maximize their utilization (i.e., they are not waiting on slower nodes). Similarly, this allows handling crash faults. The figure below illustrates the SWARM protocol. Nodes within a stage then run an All-Reduce protocol to average the results of their Stochastic Gradient Descent (SGD) and perform the optimization.

We note that decentralized optimization algorithms that operate asynchronously with unreliable workers introduce new theoretical questions in terms of their convergence behavior compared to centralized SGD. Providing tight theoretical convergence guarantees is an open problem.

Other techniques such as ZeroOffload can be used to optimize GPU use and offloading data and compute to CPU.

Allocation

One of the main challenges in decentralized machine learning is the allocation of compute and storage. We want to efficiently match inference or training requests with available nodes while ensuring fair allocation, prevent bottlenecks, and optimize for low latency and compute cost.

In PETALS, the allocation of work happens through a load balancing mechanism and a distributed hash table (DHT). The system tries to distribute the transformer blocks evenly across the available servers. The goal is to maximize the total throughput of the model by identifying any bottlenecks and assigning servers to the Transformer blocks with the worst performance.

Servers periodically announce which Transformer blocks they’re currently handling through a DHT. When a new server joins, it checks this DHT to find which block is most needed and it claims this range for itself. After a server has picked its blocks, it broadcasts its own network and compute throughput info to the DHT.

Since servers can drop out or crash at any time, all the nodes in the system regularly check if reshuffling the block assignments would significantly improve the overall throughput. If it would, they reconfigure blocks around until the throughput is as high as possible. If there’s a sudden exodus of servers handling certain blocks, this reallocation process quickly redistributes the remaining servers to plug the holes. So, in essence, the system is always working to balance the load and keep the throughput as high as possible.

The DHT acts as the central bulletin board that lets servers coordinate and divvy up the work. Clients then need to efficiently find the fastest sequence of servers for model execution. For inference, clients ping nearby servers to measure latency and use beam search to find the optimal path. If a server fails, clients remove it, and rerun the routing protocol for a replacement, during inference they would send previous inputs to the new server to maintain attention keys and values.
This approach works well for a system that serves a limited number of models but does not scale very well when the number of nodes and models increase.

For fine-tuning and training, SWARM uses the Interleaved Weighted Round-Robin scheduler for assigning data as well as a DHT that is re-updated every few minutes to account for leaving and joining nodes. In the Interleaved Weighted Round-Robin, each node is assigned a weight proportional to their compute and memory capacity and, as the name suggests, the scheduler assigned works, in a round robin fashion to each node proportionally to their weight. However, to avoid bursty service, instead of assigning all tasks to a single worker before moving to the next, the scheduler interleaves the assignments. It assigns a small batch of tasks to each worker in the current iteration, then moves to the next worker in the circular order, and so on, until all tasks or data are assigned.

In Horde AI, each request is queued and prioritized relative to the amount of kudos, a token specific to the horde ai network. Once ordered, the requests are sent to the first available node.

Another approach to assigning workloads, suggested by SAKSHI, is to use a set of scheduler nodes (which are called routers in their paper) that perform load balancing.

Schedulers maintains the following attributes about each server: its location, hardware capacity, request load (number of requests the provider is currently serving), model capacity (i.e., the set of AI models that the server can provide inference for), and potentially content moderation rules (e.g., NSFW). However, it is important to note that these attributes are not always provably verifiable. Location can be loosely verified by using some form of proof-of-location, e.g., random pings or a decentralized coordinate system.

For attributes that cannot be directly verified, a reputation system can be implemented to reflect whether a compute node performs in accordance with its claimed attributes. Schedulers would then also maintain local reputation scores for each server. Developing proof systems for these other attributes remains an open problem.

The scheduler is trusted with two main tasks. First, it should maintain a correct list of nodes and their attributes, although, as already noted, some attributes are not reliably measurable. Hence, the scheduler should maintain correct attributes to the best of its knowledge. Second, it should route the requests to the best possible nodes according to criteria such as location (to minimize latency) and cost.

Scheduler nodes could, however, collude with some compute nodes and route more jobs toward them, even if they are not the best option in terms of location and/or price. This impacts the fairness of the protocol and its performance, as it could increase latency and costs for users. For the well-being of a decentralized system, especially one with financial rewards, fairness is an important property that schedulers should maintain. Schedulers shouldn’t unfairly prioritize certain nodes over others, i.e., it shouldn’t always be the same nodes that get the jobs. We could incorporate verifiable decentralized randomness to help ensure enough rotation between servers of similar attributes (similarly to how leader election protocols work in proof-of-stake systems), however this is not trivial to design and is an open problem.

An easier approach is to consider a reputation system where users and compute nodes can rate schedulers, whether centralized or peer-to-peer (e.g., using some kind of web of trust architecture).
Another open problem would be to design a fraud-proof system where malicious schedulers can be slashed directly on the blockchain. The main difficulty is to create proofs that attest to misbehavior and can be distinguished from, for example, bad latency. Furthermore, as noted in SAKSHI, compute nodes that observe malicious behavior from a scheduler node could opt-out from this server, enforcing a fairer dynamic market for routing.

The figure below illustrates the (simplified) flow between users, who connect to a scheduler, who then load balance the users’ request to the compute nodes, whose work would be verified by challengers’ nodes (in the case of optimistic machine learning) that use the blockchain as a settlement layer.

Another interesting approach to load balancing and job allocation, although in a centralized setting, is that of ZeroGPU, introduced by HuggingFace. Instead of dedicating a single GPU to each HuggingFace Space, ZeroGPU efficiently shares GPUs across multiple Spaces. It dynamically allocates and releases GPUs as needed, based on the workload demands of each Space. A similar approach could be employed in a decentralized network to ensure that GPUs are not “stuck” on a single application.

Incentives

In many decentralized systems, participants are motivated to contribute and behave honestly through various incentive designs. While some systems rely on altruistic behavior (e.g., PETALS or Tor) or non-financial incentives (e.g., Horde AI), incorporating financial incentives is a popular approach to attract more participants and encourage honest participation in the network.

Financial incentives seem quite natural in the case of a decentralized marketplace for ML compute. Rewards are designed to incentivize various actors within the system to fulfill their respective roles effectively. Compute nodes are incentivized to perform computations correctly and are compensated for their expenses. In the case of optimistic machine learning (that we discussed in the first part of this series), challenger nodes are incentivized to verify computations and, if necessary, initiate an on-chain challenge. Scheduler nodes are incentivized to perform the routing and allocation of tasks efficiently.

Slashing mechanisms are implemented to punish certain undesirable behaviors within the network such as compute nodes performing incorrect computations; challenger nodes spamming the network with baseless challenges; schedulers denying service to clients or favoring a specific set of compute nodes, which would result in other nodes not receiving any work or rewards. As we have already noted, for slashing to be implementable it is important to be able to create concrete proofs of misbehavior, which is not always trivial.

Nodes in the network can receive rewards and compensation from two primary sources, possibly used conjointly: from the users requesting the compute work and from newly minted tokens, as is used in proof-of-work or proof-of-stake minting mechanisms.
SAKSHI proposed the use of unidirectional payment channels for decentralized inferences, as they consist of direct, repeated interactions between users and the compute nodes that execute their tasks. In the optimistic case, where everyone behaves honestly, only a single settlement transaction should be recorded on the blockchain.

Furthermore, a fee of the final transaction could go to the schedulers, incentivizing an open and fair market for schedulers. Users may choose the scheduler they connect to based on location, reputation, UI or fees.

Pricing of requests can be done in a similar way as the Ethereum gas mechanism. Each basic operation corresponds to a specific amount of gas. Multidimensional approaches could also be applicable here. However, designing such a fee mechanism is still an open problem.

Designing an incentive scheme that appropriately rewards and slashes compute nodes, schedulers and challengers (i.e., verifier nodes in the “optimistic approach”) while keeping fees competitive for users is currently an open problem.

Another issue that arises is that the high volatility associated with crypto prices may be an undesirable property for many users. Stablecoins could come into play here to solve this issue. Minted coins on the other hand can help with bootstrapping a new network, attracting more compute nodes from the start of the platform.

I would like to thank Alfonso de la Rocha and Mahmoud Shehata for useful discussions on this topic and Jorge Soares, Alexander Hicks and Sergey Fedorov for feedback on the first draft of this post.

Over the past year, decentralized AI has been attracting a lot of attention and interest, particularly on the infrastructure side. The main arguments in favor of building decentralized infrastructure for AI are compelling: it fights censorship, can help mitigate bias, provides transparency in how data is used and ensures competitive pricing. In this post, we examine several key open problems in the field of decentralized AI infrastructure, exploring potential solutions and the challenges that lie ahead. By examining these issues, we aim to shed light on the current state of the field and the exciting possibilities that decentralized AI infrastructure holds for the future of AI.

The post consists of three parts: the first one discusses verifiable computations. We explore several approaches to verifiable computations in machine learning: cryptographic, optimistic, and TEE-based.

In the second part we discuss communication, resource allocation, and incentives. The third part (to be published) will be about data, ownership and privacy.

Verifiable Computations

One of the key arguments for decentralizing compute, on top of the other reasons we’ve discussed above, is that compute power is a scarce resource right now. There’s a huge amount of untapped compute capability out there in the world, whether it’s in data centers or people’s personal GPUs (that could even be accessed through browsers with tech like WebGPU). By building a decentralized network of compute resources, we can tap into all that latent compute power that’s currently sitting idle. This could allow us to train massive AI models that would compete with giant AI oligopolies such as OpenAI, Meta & Anthropic, or compute oligopolies such as AWS, Google, etc.

Decentralized compute networks come with many technical challenges. Although the field of machine learning has made some significant progress in the areas of distributed machine learning (e.g., in federated learning and distributed training), all of those existing approaches assume a “crash-fault” model and are not Byzantine-fault tolerant: they can handle nodes going offline unexpectedly, but they’re not designed to deal with actively malicious actors (see, for example, Petals, Hivemind or Stable Horde). In a decentralized permissionless system, we have to assume that some nodes will be acting in bad faith, especially when financial incentives are at stake. Nodes might try to sabotage the training process, poison the model, or just freeride without doing any useful work.

One of the main techniques traditionally used to make a system resistant to Byzantine (i.e., malicious) actors is to use cryptographic tools to enforce good behavior (e.g., proving the correct execution of a computation with a SNARK). If nodes have to cryptographically prove that they did the work correctly, it’s a lot harder for them to cheat. By requiring nodes to provide cryptographic evidence of their adherence to the protocol, the system can ensure that participants act honestly or detect malicious actors and take appropriate actions against them (e.g., their input will not be included in the final result).

Another approach is to extend the threat model to include rationality and use incentives to deter malicious behavior (e.g., rational cryptography and optimistic approaches). Instead of trying to make cheating impossible (through cryptography), the goal is to make it economically irrational.
Adapting these techniques to ML workloads is non-trivial as they involve heavy and non-deterministic computations, with results that are hard to verify.

Cryptographic approaches

Zero-knowledge proof technologies provide very powerful tools to cryptographically prove that a computation was executed correctly (e.g., SNARKs or STARKs). When it comes to using these tools to enforce honest behavior, the scale and complexity of computations in ML pose a huge challenge.

For inference, the computations are lightweight compared to training, so generating a SNARK for inference appears reasonable, although it still adds an overhead. This approach is used by projects like EZKL and Modulus Labs. There’s still plenty of room for optimization and this is an active area of research. Proof computation and verification time as well as proof size are key metrics to keep an eye on. Even small efficiency gains can make a big difference at scale.

For training and fine-tuning, the computations are much more intensive as they require huge numbers of matrix multiplications, gradient calculations, and back-propagation passes over massive datasets. Generating a SNARK or other type of ZK-proofs is much less realistic, especially since compute hardware is so scarce.

So while SNARKs and other ZK-proof systems are a powerful tool in the toolbox for trustless computation, they’re not a silver bullet. Other cryptographic techniques such as multi-party computations (including secret sharing and distributed ZK) have also been explored in the context of decentralized training. Multi-part computations, although less computational intensive, incur a large communication overhead and scale poorly in the number of participants. They would be suitable for smaller networks but not for huge decentralized networks with thousands of nodes.

For training and fine-tuning, the path forward will likely involve some combination of different cryptographic techniques (e.g., ZK techniques and MPC,) as well as hardware acceleration. We can anticipate significant advancements in this field in the coming months, for example from Gensyn. According to their litepaper, they plan to publish their new findings in the near future, so it’s definitely something to keep a close eye on.

Bagel Network presents another approach to verifying inference results in decentralized ML systems by leveraging watermarking and fingerprinting techniques used in AI. These tools are used to provide proof of ownership for an inference and help authenticate models. Verifiers can check the watermark and fingerprint to confirm that a result came from a specific model. This establishes trust and accountability without requiring re-execution of the inference itself and incurs less overhead than using ZK tech. Formal proofs establishing the security guarantees of these techniques in this context are still an open research problem.

In addition to ensuring the integrity of the system, ZK-proofs could play a role in tackling the privacy concerns associated with decentralized AI, a topic that we will explore in the third part of this series, which focuses on data and privacy.

Rational approaches

Optimistic approaches have been gaining a lot of traction in the web3 world lately. They offer a pragmatic way to scale up decentralized systems without relying on expensive cryptographic proofs for every single transaction. The core idea is simple, but powerful. Instead of requiring nodes to constantly prove that they’re behaving honestly, you assume they are by default. The system operates “optimistically” under the assumption that everyone is acting in good faith. It’s only when someone raises an alarm and claims that a node has misbehaved that the system steps in to arbitrate. Nodes must stake a portion of their tokens to participate in the network. If they’re caught misbehaving, their stake is slashed as a punishment. The threat of losing money keeps everyone in line.

Although initially proposed in the context of rollups, this paradigm can be adapted for machine learning computations. In order for a node to get slashed, there needs to be proof of their misbehavior. In the case of rollups, the proof is objective, based on cryptographic evidence that a node broke the rules (e.g., they signed two conflicting transactions). Having a cryptographic proof is paramount in optimistic systems, as the arbitration usually happens through a smart contract, which needs to be able to automatically and deterministically verify the validity of the proof to execute the appropriate consequences (such as slashing).

Bisection protocols help pinpoint the exact computational step where the result is erroneous for a set of sequential computations, such that a smart contract can arbitrate a computation dispute at a minimal cost. In a bisection protocol, when a dispute arises, the parties involved iteratively divide the computation into smaller and smaller segments, comparing their results at each step until they identify the specific step where their results diverge.
To illustrate this, if the initial assertion covers 1000 computation steps, the challenger would split it into two assertions, each covering 500 steps. If the challenger’s result matches the challenged result at step 500, the divergence must occur in the latter half of the computation. The challenger would then repeat the process on the second half: divide it into two segments of 250 computations each and continue the bisection.

Recently, EigenLayer introduced the concept of “intersubjective proof”, which are proofs that cannot be mathematically verified (such as ZK proof) but “where there is a broad-based agreement among all reasonable active observers of the system” e.g., in the case of censorship and oracles. The idea here is that some claims might not be objectively provable with cryptography alone, but they can still be evaluated by a decentralized network of peers. Intersubjective proofs open up interesting possibilities for handling tricky cases that don’t fit neatly into the objective proof paradigm. But they also come with their own set of challenges around things like collusion resistance and subjectivity resolution.

In the context of decentralized ML, both types of proofs could come into play. For some simpler tasks like hosting models or running inference, objective proofs might be the best option. But for more complex tasks like training or dataset curation, intersubjective proofs could be a valuable tool in the toolkit.

Inferences

As the SAKSHI paper points out, AI models can be seen as a sequence of functions, representing the layers in a neural network. This structure makes them especially well-suited for the bisection protocol used in optimistic rollups, as described above. However, adapting this approach from optimistic rollups to enable optimistic inferences in decentralized ML systems introduces a new set of challenges that go beyond those faced in traditional optimistic rollup designs.

First, inference computations are not entirely deterministic: different hardware architectures, software implementations, or runtime conditions may have slightly different implementations of floating-point arithmetic, which can lead to discrepancies in the results of inference computations. Even when using the same randomness seed, two different inference computations on two separate machines may produce different results.

One way to address this issue is to use a technique called quantization. Quantization involves converting the weights and activations of a neural network from floating-point values to a lower precision (e.g. fixed-point values), which reduces the precision and accuracy of the computations but removes the randomness introduced by floating-point arithmetic, thereby guaranteeing that the same input will always produce the same output regardless of the specific hardware or software implementation. Although this technique induces an acceptable loss of accuracy, addressing the randomness of inference results without loss of accuracy is still an open problem.

One potential alternative to quantization could be generating a probabilistic proof instead of a deterministic one. For instance, a proof could demonstrate that an inference result deviates from the expected distribution or is “too far” from an honest inference based on a suitable metric. While this probabilistic approach, which has been suggested for training (discussed in the next section), presents various challenges, it would be intriguing to explore whether relaxing the requirements from deterministic to probabilistic proofs for inference could enable new methods that maintain perfect accuracy.

Lastly, as mentioned in the previous section, intersubjective proofs have been proposed when an objective proof is not feasible. While this approach could be investigated for inference, it might be an overkill, given the heavy “social consensus” they appear to require.

Another issue of using a blockchain-based optimistic approach is that it requires the computations being verified to be compatible with the relevant virtual machine (e.g., EVM, RISC-V, or Wasm). However, these virtual machines usually do not support frameworks such as CUDA, ROCm or MPS, that allow for optimizing the computations to specific hardware.

Two-phase bisection protocols address this limitation. The two-phase bisection protocol works by first representing the computations as a set of higher-level operations to narrow down disputes to a single high-level operation in Phase 1, using a bisection protocol as explained above. Then, Phase 2 zooms in to pinpoint the specific disputed basic operation within that high-level operation (still using a bisection protocol), minimizing overall communication rounds and data structure size. The bisection protocol in this case is performed twice, once on the “large instructions” (corresponding to multiple basic operations) and once in the basic operations to pinpoint the exact computations. This technique was first introduced by Agatha and is implemented by Ora.

Other approaches could be used here to even further improve the efficiency of the protocol. As an example, Ora suggested using a multi-phase bisection protocol with more than two phases. This is currently an open problem.

Designing a GPU-optimized VM could also be another approach to solve this issue. I am not aware of any project currently tackling this but I wouldn’t be surprised to see this happening soon.

Let’s note that optimistic inference offers a key advantage compared to optimistic rollups: in the case of an inference, a malicious computation could be obvious to spot (e.g., if the result is garbage or completely unrelated to the initial prompt). In the case of rollups, it is not possible to know if someone double-spent without examining all the other transactions. This means that not every single inference needs to be independently verified, as any incorrect computation would stand out on its own. If a node produces an obviously incorrect output, the user who requested that inference will likely quickly notice the problem. They can then submit a challenge to flag the issue, triggering the dispute resolution process. We do not necessarily require a network of challengers to oversee every inference request.

However, it’s important to recognize that optimistic approaches are best suited for certain use cases, particularly decentralized marketplaces for inference services, but may not be suitable for other applications that require faster finality, such as on-chain AI agents.

Training and Fine-tuning

Applying optimistic approaches to training and fine-tuning workloads is significantly more challenging compared to inference. The key issue is the computational intensity involved. Training a large-scale ML model often requires an immense amount of processing power and time, making it unreasonable to assume that a challenger could redo the entire computation just for verification purposes.

Moreover, ML training involves randomization. Even if two nodes start with the same initial parameters and data, they will likely end up with slightly different results due to factors like random initialization, data shuffling, and floating-point precision. This makes it much harder to determine whether a given set of training results is “correct” or not.
To address these challenges, researchers have proposed a new technique called Proof-of-Learning (PoL). The key idea behind PoL is to make the verification process probabilistic. Instead of requiring the challenger to recompute the entire training process, they only need to redo a small, randomly selected subset of the computation.

Furthermore, since quantization techniques used in inference to ensure consistency aren’t applicable to the stochastic nature of training, PoL relies on distance-based metrics for verification. If the results computed by the challenger on the subset are sufficiently close to the results reported by the training node, the training is considered valid. If the distance exceeds some predefined threshold, the training is considered suspicious.

While PoL is a promising approach, it’s important to note that it’s still an active area of research with ongoing challenges. A couple of recent papers have highlighted potential weaknesses in PoL schemes. The key takeaway is that there needs to be further advancements in the field before we can have strong security guarantees.

One interesting direction is the idea of PoL with incentives, as explored in a recent paper. The insight here is that even if an attacker is able to generate a fake PoL proof, the cost of doing so would exceed the cost of performing the actual training. Essentially, cheating becomes economically irrational. The attacker would be better off just doing the work honestly.

Lastly, as we’ve discussed above, intersubjective proofs could be used in the context of ML, whether for inference or training; however, designing and implementing such a system is currently an open problem.

Trusted Hardware

Finally, it’s worth mentioning a somewhat controversial approach in the web3 space: leveraging trusted execution environments (TEEs) to ensure the integrity of computations. TEEs, such as Intel’s SGX, provide a secure enclave where code can be executed in isolation, shielded from tampering by the host system.

However, relying on TEEs introduces centralization risks, as they are typically produced by a single hardware vendor like Intel. While some open-source alternatives have been proposed, they have yet to gain significant traction.

Moreover, TEEs are not foolproof and have been shown to be vulnerable to various side-channel attacks and exploits (techniques that exploit unintended information leakage through timing, power consumption, electromagnetic emissions, or other channels to extract sensitive data). The complexity of modern ML workloads and the potential for data leakage through side-channels make it particularly challenging to ensure the security of TEE-based ML systems.

As a result, the use of TEEs for machine learning computations remains an open problem. While they offer some compelling security properties, the centralization risks and potential vulnerabilities make them a controversial choice for decentralized AI.

Head to part 2 of this series, where I discuss communication, resource allocation and incentives!

I would like to thank Chad Fowler and Alexander Hicks for their feedback on the draft of this post.