ChatGPU

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

Bring your code. ChatGPU adapts, optimizes, and runs it across a network of GPUs.

We want people to rent out their GPUs when they aren't using them, making spare capacity, including consumer chips, available for training, simulations, and experimentation.

But personal machines can disconnect at any time and vary enormously in hardware and software. Users cannot tune their code for every possible device. Our agents handle that adaptation, checking changes against the original behaviour and using telemetry to troubleshoot failures and improve future runs.

How to use it

1. Create a new job. Drop in any relevant files, tell ChatGPU what you want it to do, and click Submit project.
2. Watch your agent work. Follow along as it plans, tests, and supervises your run.
3. Reap the rewards. Review your results and download the outputs, such as your trained model and evaluation metrics.

How it works

Network

Each contributor runs a worker that connects to ChatGPU and reports its runtime, memory, and supported workloads. Workers initiate authenticated outbound connections, so contributors do not need to expose their computers to incoming connections. Owners can pause their workers when they need their machines back.

The scheduler matches tasks to compatible, available workers. Supported environments include CPU, NVIDIA CUDA, AMD ROCm, and Apple MPS, with eligibility checked for each workload. The dashboard receives live updates, while PostgreSQL stores job state, assignments, accepted results, and execution history.

Agent to run: from request to results

We use GPT-6 Astra to power job planning and run supervision. The planning agent reads the uploaded code and the user's request, identifies dependencies and output requirements, and chooses compatible hardware. If a requirement is unclear, it asks the user. It can also consult specialist agents about dependencies, parallelization, and validation before deciding on a plan.

Adaptation, migration, and optimization form one preparation stage. The agent applies the supported steps the workload needs, using small worker runs to measure and validate its choices. Once preparation passes, workers execute the full workload, validate its outputs, and save the results for download. A run supervisor follows progress and investigates problems throughout the pipeline.

Inside adaptation, migration, and optimization

The loop starts with the original program and fixed acceptance criteria. Candidates pass code checks before running on the relevant hardware. Compilation errors, numerical differences, and performance measurements become feedback for the next proposal. The agent can revise supported parts of the implementation within its budget, while the definition of a correct result stays fixed.

• Adaptation changes how work is organized. Independent computations can be divided into batches, with checks that their combined results agree with the original on the tested cases.
• Migration adapts supported hardware-specific code for a different chip. The target's numerical results are checked against a reference from the original environment.
• Optimization compares alternative implementations using repeated measurements. A candidate must preserve the required behaviour and meet its performance criteria to replace the accepted version.

A rejected optional optimization leaves the last accepted code in place. A required adaptation or migration must pass before full execution can begin. Accepted code is frozen for the full run, and the user's requested workload and quality requirements remain unchanged. The transformations available depend on the workload and runtime; code that already fits can proceed without rewriting.

When a run fails: the Sentry feedback loop

We use Sentry as our telemetry platform across the system, collecting errors, logs, traces, and performance measurements. Failures can come from lost workers, missing dependencies, compilation errors, resource limits, or outputs that fail validation. ChatGPU records the affected job, worker, and attempt so the supervisor can connect Sentry evidence to the run it is investigating.

The supervisor reads this evidence alongside current job state and its saved findings. Its response depends on the cause:

What happened	How the system responds
A temporary failure or worker loss	Retry eligible work within existing limits, or arrange compatible replacement capacity where the execution policy permits it.
A candidate fails compilation or validation	Feed the evidence back to the preparation agent for a bounded revision. Keep the accepted version when an optional optimization is rejected.
A required change cannot pass, or the workload cannot fit	Stop with an explanation, or ask the user for a missing decision. Preserve the original output and quality requirements.
Sentry is unavailable	Continue investigating from stored execution logs and job state.

The planner owns code adaptation and validation; the supervisor manages the run's supported recovery actions. Every action is checked against the job's scope and limits, and the next observation establishes whether recovery worked. Findings, actions, and follow-ups persist across supervisor runs. The scheduler handles heartbeats, leases, and routine retries independently of model calls.

Using telemetry to improve model optimization and migration

The same execution data can help us improve future runs. Preparation records include hardware and runtime details, original and candidate code identities, numerical checks, timings, rejection reasons, and final outcomes. Successful, failed, skipped, and fallback outcomes are saved, with structured Sentry Logs providing a way to search and investigate them.

We plan to use this evidence to improve our model optimization and migration algorithms: identify which changes help particular workloads and chips, learn from recurring compatibility failures, and turn those failures into regression cases. Candidate changes can then be evaluated against both correctness and performance evidence before adoption. A faster result must still meet the original quality requirements.

Our existing development agent already investigates recorded incidents and proposes fixes with tests for human review. We want to build on that foundation with better migration strategies and optimization choices informed by previous runs. Broader learning across runs and training agents on these records are future uses of the data.

Example flow: NVIDIA training code on an AMD GPU

A user brings a training project with supported GPU code written for NVIDIA and asks to run it on AMD. The model, data, training settings, and acceptance criteria must stay consistent. With compatible source and target workers available, the flow is:

1. Establish a reference. Run a small, unchanged workload on NVIDIA and save its numerical results and measurements.
2. Optimize where useful. Test supported implementation changes against the original on the source GPU. Keep a candidate only when repeated measurements and correctness checks justify it; otherwise, retain the original.
3. Migrate the accepted code. Translate supported CUDA code to HIP for AMD. Compilation and execution feedback guide any permitted revisions.
4. Validate on the target. Run the bounded workload on AMD and compare its results with the fixed NVIDIA reference. A failed required migration returns to the revision loop within the job's limits, or stops with an explanation.
5. Run the full job. Freeze the validated code and train on AMD for the requested workload. Reload and evaluate the final checkpoint, then return the model and results if they pass the original checks.

The small preparation runs establish evidence for the execution strategy. The full training run still performs the requested work, and its final result must satisfy the original validation requirements. This flow prepares code for the target before the full run begins.

Keeping runs recoverable and under user control

Worker assignments have renewable leases. A machine must keep reporting to retain ownership. If it disappears, eligible work can be retried or replanned within the job's limits. Each attempt has a distinct identity, so a late result from an old worker cannot overwrite the accepted result. Work tied to a specific validated machine must continue to respect that placement requirement.

Plans, execution history, validation evidence, and accepted files remain available after worker cleanup. Users can inspect what happened, download their outputs, or cancel outstanding work. Optional spending caps are expressed in Canadian dollars and stop work when estimated usage reaches the cap. Enforcement is periodic, so these are soft limits. Current estimates use CPU/RAM rates; owner payouts and payment processing remain future marketplace work.

The same upload flow can host supported Python HTTP services when the user asks for an endpoint. Readiness checks control routing, and compatible replacement workers can restart the service behind the same authenticated URL.

Challenges we ran into

This was the most technically challenging project we have ever built. There were so many moving parts: the dashboard, scheduler, agents, database, remote workers, GPU runtimes, and telemetry. Getting each piece working was a challenge. Getting them to agree on what was happening throughout a job was an even bigger one.

• Connections were not on our side. Unreliable connections made integration and debugging harder. A worker could become unreachable while a job was still in progress, leaving us to distinguish slow execution from a lost machine. Heartbeats, renewable assignments, and recovery rules became essential.
• Every machine brought different constraints. Hardware, available memory, dependencies, and GPU runtimes all affected whether a workload could run. Choosing a worker meant checking what the code actually needed, then testing the execution plan on that hardware before committing to the full job.
• Code changes needed evidence. An agent could propose a migration or optimization, but we still needed to establish whether it preserved the original behaviour on the tested cases. That meant fixed acceptance criteria, reference results, numerical checks, and repeated performance measurements. A failed optional optimization also needed a reliable path back to accepted code.
• Recovery created its own coordination problems. Retries, delayed updates, and returning workers all had to fit into one consistent job history. We had to track individual attempts, reject stale results, and keep agent actions within the user's workload and spending limits.
• Debugging meant following a job across the whole system. A visible failure could originate in planning, environment setup, compilation, execution, or validation. Connecting Sentry telemetry to the relevant job, worker, and attempt gave both us and the supervising agent the context to investigate.

Through all of this, we wanted the user experience to stay simple: upload files, describe the task, and collect the results. Making the system explain its progress, surface useful failures, and return downloadable outputs took work across every layer. The hardest part was making all those pieces work together from submission through to a validated result.

What we learned

How to stay awake.

What's next

Sleep.

For Sponsors

Sentry: traces and logs across distributed jobs

We used Sentry primarily for traces and logs. With jobs crossing the backend and remote workers, these helped us understand where time was spent and what happened when execution failed.

• Tracing: Python workers wrap execution in task.execute transactions, tagged with job, task, worker, and attempt identifiers. This connects execution timing to a particular run.
• Logs: Execution messages provide context around setup and runtime failures, helping us follow a job across components and investigate what went wrong.

Implementation: telemetry setup and worker instrumentation.

OpenAI: GPT-6 Astra inside an execution and feedback loop

GPT-6 Astra turns a user's request into decisions the system can execute and check. It powers planning and run supervision through the OpenAI Responses API. Uploaded code, available hardware, execution evidence, and the user's constraints provide the context for choosing the next step.

Our model adapter sends JSON Schema function definitions and returns tool calls to a separate execution loop, following the Responses API function-calling pattern. The backend validates tool arguments and authority before dispatch. Supervisor tools include get_job, list_eligible_workers, search_logs, take_action, and remember, so an investigation can progress from observation to action to a saved finding.

The engineering is in how the loop handles real execution. The application saves model output and pending tool activity before side effects, then saves results before the next model call. Requests use store=false with caller-owned history and encrypted reasoning items retained for continuation. Calls are sequential, and the default shared loop limits a turn to eight model calls, eight tool calls, and 90 seconds. Incomplete model responses cannot dispatch partially generated calls.

If execution is interrupted, the loop records uncertain outcomes and inspects durable task and action identities before further work. Duplicate tool-call IDs are rejected. The supervisor also takes a per-job database lock to prevent concurrent investigations from issuing competing actions. These mechanisms make long-running supervision possible across multiple short model turns.

Correctness comes from execution evidence. The preparation pipeline checks supported code changes against the original program's acceptance criteria. Compiler errors, numerical comparisons, and measured performance guide revisions. The supervisor works within the submitted limits, while the scheduler manages leases and routine retries. Tests use a replaceable model client to exercise malformed calls, interrupted turns, timeouts, and persistence behaviour.

Codex with GPT-6 Astra was central to building ChatGPU. We ran 16 Astra worktrees in parallel across our team for implementation. This was the most technically complex project we had attempted, and what impressed us most was Astra's ability to navigate all the moving parts and their dependencies.

We used Codex from the planning stage to scope the project and reason through the architecture, then throughout implementation. With the dashboard, scheduler, workers, agents, and telemetry all interacting, understanding how a change fit into the wider system was a major part of the work. Astra helped us work through that complexity while our team built different parts in parallel.

Testing with screenshots was a highlight. We could show Codex the interface, review what the user would actually see, and iterate with visual feedback. That was especially valuable for a flow spanning job submission, agent supervision, and results. One of us described it as the best testing experience they had ever had.

Implementation: model transport, agent loop, and supervisor tools.

Built With

• ai-agents
• amazon-web-services
• amd
• baseten
• codex
• cuda
• distributed-systems
• fastify
• gpu
• inference
• llm
• migration
• mlops
• networking
• node.js
• nvidia
• onnx
• openai
• openai-agents-sdk
• postgresql
• rocm
• rox
• sentry
• typescript
• vllm