CueTip 🎱

CueTip is a library for natural language interaction and planning with a pool simulation. It enables language models to interact with PoolTool, a fully 3D highly-accurate physics simulation of pool/billiards.

See our paper here for more details and come see us at SIGGRAPH 2025!

• Natural Language Processing: Language models interpret the results of shots through the emission of natural language events
• Planning & Optimization: Models describe target shots through a list of similar NL events, then black box optimization tunes shot parameters so that the outcome matches the target
• Physics Simulation: Integration with PoolTool for accurate physics-based shot outcomes
• Neural Surrogate: Fast neural surrogate model of SOTA pool agents for efficient optimization
• Expert Grounding: Optimisation utilises expert knowledge to enable grounded explanations of shots

• Python 3.10+
• Language model API key (e.g. OpenAI) or local LM
• Optional: W&B account with authorize key

1. Install uv package manager:

curl -fsSL https://astral.sh/uv/install.sh | sh

2. Create a virtual environment:

uv venv

3. Install required packages:

uv sync --all-extras --frozen

Optionally using the argument --cache-dir=YOUR_CACHE_DIR to specify a specific cache directory.

4. Note: PoolTool may require a font that is not installed by default. You can install it by running:

wget -O HackNerdFontMono-Regular.ttf https://github.com/ryanoasis/nerd-fonts/raw/master/patched-fonts/Hack/Regular/HackNerdFontMono-Regular.ttf
mv HackNerdFontMono-Regular.ttf .venv/lib/python3.10/site-packages/pooltool/ani/fonts/

Optionally, before continuing, you may want to login with W&B as follows:

wandb login YOUR_WANDB_AUTHORIZE_KEY

Before running the examples, train the neural surrogate model:

uv run train_neural_surrogate.py [--use_wandb]

This model helps optimize shot parameters efficiently during simulation. If you have used the optional argument --use_wandb and have previously logged in, then you can check on the training of the neural surrogate model in your W&B account.

To verify your installation is working correctly:

uv run test_shot_optimisation.py

this will run a simple shot optimisation example to ensure everything is working correctly.

Run the function agent example which utilizes the trained neural surrogate to optimize shots:

uv run example_function_agent.py

Run the language model agent example where an LLM plans potential shots and the neural surrogate selects the best outcome:

uv run example_llm_agent.py

Note: The LLM agent requires an OpenAI API key or a compatible local LLM setup in your environment variables. The DSPy library is used for LM inference and supports many backends.

For containerized usage with GPU support:

# Build the Docker image
docker build -t cuetip:main .

# Run with NVIDIA GPU support
docker run --gpus all -it cuetip:main bash

This project is licensed under the MIT License - see the LICENSE file for details.

Built with ♥ for pool players and AI enthusiasts