Quick answer: AIXI is a theoretical ideal for sequential decision-making, not a computable Python algorithm. A useful Python experiment should name its approximations: environment model, history, planning horizon, policy search, reward, stochasticity, and resource budget.
AIXI Python is best understood as a learning exercise around universal agents, not as a normal Python package you import and run in production. AIXI is Marcus Hutter’s theoretical model for a reinforcement-learning agent that combines Solomonoff-style universal induction with sequential decision making. In plain terms, the agent keeps a full interaction history, considers computable environment programs that could explain that history, weights simpler explanations more strongly, and chooses the action with the best expected future reward.
The important technical limit is that exact AIXI is not computable. Hutter’s original AIXI paper and the Universal Artificial Intelligence book page both make that point while describing AIXI as a mathematical reference model. Legg and Hutter’s universal intelligence definition gives related background on evaluating agents across broad classes of environments. For practical approximations, see Veness, Ng, Hutter, Uther, and Silver’s Monte-Carlo AIXI approximation.
The pyaixi repository is relevant because it describes a pure Python implementation of the MC-AIXI-CTW approximation. Treat it as research code and study material before using it as a dependency. This guide keeps every snippet offline-safe and executable with the standard library only. The code below demonstrates tiny agent ideas: histories, rewards, simple model weights, and short planning. It is explicitly not a full AIXI implementation.
Represent Interaction History
AIXI is defined over complete histories of actions, observations, and rewards. A toy Python version can start with a list of records. This is only bookkeeping, but it is the core shape that later planning code will read.
def add_step(history, action, observation, reward):
item = {
"action": action,
"observation": observation,
"reward": reward,
}
return history + [item]
history = []
history = add_step(history, "left", "coin", 1)
history = add_step(history, "right", "empty", 0)
total_reward = sum(step["reward"] for step in history)
print(total_reward, history[-1]["observation"])
The list is deliberately small and transparent. Real AIXI uses a formal sequence of actions and percepts, where a percept usually bundles observation and reward. For Python practice, a readable dictionary record is enough to make the data flow visible.
Make A Tiny Environment
An agent needs something to interact with. This two-button world rewards one action and returns a short observation. There is no hidden framework, network call, or third-party install.
class TwoButtonWorld:
def __init__(self, goal="left"):
self.goal = goal
self.turn = 0
def step(self, action):
self.turn += 1
reward = 1 if action == self.goal else 0
observation = "match" if reward else "miss"
return observation, reward
world = TwoButtonWorld(goal="left")
for action in ["right", "left", "left"]:
observation, reward = world.step(action)
print(world.turn, action, observation, reward)
This environment is too small to be interesting as AI, but it is useful for checking terms. The agent sends an action. The environment replies with an observation and a reward. The agent then appends that result to its history before choosing again.
Weight Simple Explanations
AIXI uses algorithmic probability, where shorter programs receive higher prior weight. Python cannot enumerate all programs, so this toy version scores only two hand-written explanations. That narrow list is the main simplification.
programs = [
{
"name": "left_world",
"length": 4,
"predict": lambda action: ("match", 1) if action == "left" else ("miss", 0),
},
{
"name": "right_world",
"length": 5,
"predict": lambda action: ("match", 1) if action == "right" else ("miss", 0),
},
]
history = [("right", "miss", 0), ("left", "match", 1)]
def compatible(program, past):
return all(
program["predict"](action) == (observation, reward)
for action, observation, reward in past
)
weights = {
program["name"]: 2 ** (-program["length"])
for program in programs
if compatible(program, history)
}
normalizer = sum(weights.values())
posterior = {name: weight / normalizer for name, weight in weights.items()}
print(posterior)
This mirrors the intuition without claiming the full theorem. A real AIXI agent reasons over all computable environments relative to a universal machine. The snippet only shows how past evidence can remove explanations that no longer match the observed history.
Choose A One-Step Action
Once an agent has weighted explanations, it can score actions by expected reward. This one-step choice ignores longer futures, but it makes the action selection rule concrete.
world_models = {
"left_world": (
0.8,
lambda action: ("match", 1) if action == "left" else ("miss", 0),
),
"right_world": (
0.2,
lambda action: ("match", 1) if action == "right" else ("miss", 0),
),
}
def expected_reward(action):
score = 0.0
for probability, predict in world_models.values():
_observation, reward = predict(action)
score += probability * reward
return score
actions = ["left", "right"]
scores = {action: expected_reward(action) for action in actions}
best_action = max(scores, key=scores.get)
print(best_action, scores)
Full AIXI looks ahead over a horizon and averages over possible future percepts. The one-step version is a teaching bridge: it shows why beliefs and reward estimates both matter before the agent acts.
Search A Short Horizon
The formal equation interleaves maximization over actions with summation over future percepts. A small brute-force horizon search can demonstrate that pattern for two actions and two possible worlds.
from itertools import product
def simulate(goal, action_plan):
return sum(1 if action == goal else 0 for action in action_plan)
beliefs = {"left": 0.6, "right": 0.4}
actions = ["left", "right"]
horizon = 2
plan_scores = {}
for action_plan in product(actions, repeat=horizon):
plan_scores[action_plan] = sum(
probability * simulate(goal, action_plan)
for goal, probability in beliefs.items()
)
best_plan = max(plan_scores, key=plan_scores.get)
print(best_plan[0], best_plan, plan_scores[best_plan])
Brute force works here because the action space and horizon are tiny. The real AIXI expression is enormous even before considering all computable environment programs, which is why the exact model is a specification rather than a practical algorithm.
Run A Guarded Toy Loop
For experimentation, keep the loop bounded and reproducible. This final example uses a small number of turns and a local rule for choosing the action with the better observed average reward.
class DemoWorld:
def __init__(self):
self.payoff = {"left": [1, 0, 1], "right": [0, 1, 0]}
self.turn = 0
def step(self, action):
pattern = self.payoff[action]
reward = pattern[self.turn % len(pattern)]
self.turn += 1
return f"turn-{self.turn}", reward
def choose_action(stats):
averages = {}
for action, rewards in stats.items():
averages[action] = sum(rewards) / len(rewards) if rewards else 0.5
return max(averages, key=averages.get)
def run_demo(rounds=6):
world = DemoWorld()
stats = {"left": [], "right": []}
trace = []
for _ in range(rounds):
action = choose_action(stats)
observation, reward = world.step(action)
stats[action].append(reward)
trace.append((action, observation, reward))
return trace
if __name__ == "__main__":
print(run_demo())
This loop is not Bayesian, not Solomonoff induction, and not MC-AIXI-CTW. It is a safe local harness for learning the anatomy of agent code: bounded interaction, action selection, reward collection, and a trace you can inspect.
Practical Takeaways
Use AIXI as a formal reference point for thinking about general agents. It tells you what an idealized reward-maximizing learner would consider if computation were unlimited and the environment were computable. That is valuable theory, but it does not remove engineering choices about representation, exploration, resource budgets, or safety checks.
Use pyaixi or MC-AIXI-CTW papers when you want to study approximation strategies. Use the toy code above when you want executable Python examples that explain the ingredients without pretending to implement the full model. For most Python projects, the lesson is architectural: keep histories explicit, keep environment boundaries small, score actions from evidence, and keep every experiment bounded enough to debug.
Separate Theory From Implementation
The ideal model uses unbounded or incomputable components, so an implementation is an AIXI-inspired approximation. Use precise language about what is simulated and what is omitted.
Define The Agent Interface
Represent observations, actions, rewards, terminal states, and history explicitly. A small environment interface lets you test an agent against deterministic fixtures before adding stochastic behavior.
Bound Planning And Models
Choose a finite horizon, model class, candidate policy set, rollout budget, or other approximation. The bound is part of the algorithm and should be reported with results.
Measure More Than Reward
Track cumulative reward, regret or baseline comparison when meaningful, action distribution, computation time, memory, failures, and sensitivity to the environment. A single reward curve can hide unstable behavior.
Keep Environments Reproducible
Seed randomness, version environment transitions, record histories, and separate training or planning data from evaluation episodes. Do not let a mutable global random state decide the result silently.
Test The Loop
Test reset, step validation, terminal behavior, reward accounting, action bounds, horizon cutoff, deterministic policies, stochastic seeds, and agent-environment separation with small hand-checkable cases.
Use the AIXI overview for the theoretical context and the Gymnasium environment API for practical agent-environment interfaces. Related Python Pool references include tests and history sequences.
For related agent experiments, compare environment tests, history sequences, and state mappings before evaluating an AIXI-inspired policy.
For the authoritative API and current behavior, consult the AIXI research paper.
Frequently Asked Questions
What is AIXI?
AIXI is a theoretical model of an agent that combines sequential decision-making with an idealized search over environments; it is not a practical algorithm that can be computed exactly.
Can I implement AIXI exactly in Python?
No practical implementation can perform the unbounded universal search and planning described by the ideal model, so experiments must use explicit approximations and resource bounds.
What should an AIXI-inspired experiment define?
Define observations, actions, rewards, history representation, environment dynamics, planning horizon, model class, computation budget, and evaluation metrics.
How do I make agent experiments reproducible?
Seed stochastic components, version the environment and policy, record action and reward histories, fix budgets, and test the environment independently from the agent.
