ROS Agentic Operating System: Control robots with LLMs through MCP with Reachy Mini as the interface.

Using Reachy Mini Lite for easy media stream.

The client supports a local OpenAI-compatible LLM (e.g. vLLM), the OpenAI API, the Groq API, or the Anthropic API. Choose one via CLI or environment variables.

For local inference, run an OpenAI-compatible server (e.g. vLLM) and point the client at it:

# On the machine with the GPU (or with port forwarding):
vllm serve openai/gpt-oss-120b --tool-call-parser openai --enable-auto-tool-choice --port 6000

To verify the endpoint: curl http://localhost:6000/v1/models (or use https if your server uses TLS).

Groq provides inference with a free tier (with limits). You must set an API key to use Groq:

• macOS/Linux: export GROQ_API_KEY=your_key
• Windows (PowerShell): $env:GROQ_API_KEY="your_key"

Get a key at console.groq.com/keys.

Supported Groq tool-use models include: llama-3.1-8b-instant, llama-3.3-70b-versatile, openai/gpt-oss-120b, openai/gpt-oss-20b, moonshotai/kimi-k2-instruct-0905, qwen/qwen3-32b, and meta-llama/llama-4-scout-17b-16e-instruct. Default is openai/gpt-oss-120b.

Required for image analysis and better TTS experience.

OpenAI is the default hosted agent model provider:

• macOS/Linux: export OPENAI_API_KEY=your_key
• Windows (PowerShell): $env:OPENAI_API_KEY="your_key"

To route OpenAI-compatible requests through a custom endpoint while keeping the OpenAI provider path, also set OPENAI_BASE_URL:

export OPENAI_BASE_URL=https://your-endpoint.example/v1
scripts/reachy_mini_env/bin/python -m client --openai

OpenAI API usage is billed through the API platform, separately from ChatGPT Free/Plus/Pro subscriptions. A ChatGPT subscription does not provide API quota for rosaOS.

Developed with Python 3.12.

Cloning this repo requires the use of the recursive flag to download all submodules (ros-mcp-server). Further instructions to setup ros-mcp-server are in the rosaOS setup file found in the submodule directory.

git clone https://github.com/lilyjge/reachy-mcp.git --recursive
cd reachy-mcp
uv venv --python 3.12 scripts/reachy_mini_env

# Install dependencies
uv pip install -p scripts/reachy_mini_env/bin/python -r requirements.txt

For fresh macOS + Reachy Mini Lite setup details, including camera permissions and voice/STT keys, see docs/macos-reachy-mini-setup.md.

Start all services at once:

./scripts/start_all.sh

This will start:

• Reachy Mini daemon (port 8000)
• MCP server (port 5001)
• Process manager MCP server (port 7001)
• RAG agent (port 8765)

Logs are saved to the scripts/logs/ directory. To stop all services:

./scripts/stop_all.sh

Alternatively, start each service manually:

Start Reachy Mini's robot daemon server on the default port 8000:

scripts/reachy_mini_env/bin/reachy-mini-daemon

Start the Reachy Mini's MCP server on port 5001. For TTS, we support ElevenLabs API, Groq API, or the local pyttsx3 package.

scripts/reachy_mini_env/bin/python -m server
scripts/reachy_mini_env/bin/python -m server --tts-elevenlabs --tts-voice M4zkunnpRihDKTNF0D7f # Use ElevenLabs

Start the operating system's client (default port 8765). To use your own OpenAI compatible endpoint for the agents, start the client with --local and optionally --endpoint (port, default 6000). To use OpenAI, Groq, or Anthropic, start the client with --provider or a shortcut flag and optionally specify a model with --model.

scripts/reachy_mini_env/bin/python -m client                    # OpenAI (requires OPENAI_API_KEY)
scripts/reachy_mini_env/bin/python -m client --local             # Local LLM at port 6000
scripts/reachy_mini_env/bin/python -m client --provider groq --model moonshotai/kimi-k2-instruct-0905 # Groq
scripts/reachy_mini_env/bin/python -m client --anthropic --model claude-sonnet-4-6 # Anthropic API
scripts/reachy_mini_env/bin/python -m client --openai --model gpt-5.2 # OpenAI API

Now you can talk to the Reachy Mini directly.

To chat via CLI instead of the robot:

scripts/reachy_mini_env/bin/python -m client.chat.client_cli
# Optional: --base-url http://localhost:8765  (or set RAG_AGENT_PORT)

Or, when the agent is running, visit http://localhost:8765/ in your browser (or the port you set with --port / RAG_AGENT_PORT).

All ports and the LLM source can be overridden by environment variables so scripts and deployed setups don't rely on CLI flags.

Agent system prompts and robot config live under the config directory (or ROSAOS_CONFIG_DIR):

• config/kernel.txt — System prompt for the kernel agent (one placeholder: {robot_list}).
• config/process.txt — System prompt template for process agents (placeholders: {robot_instructions}, {kernel_instructions}).
• config/drivers.json — MCP server names, URLs, and descriptions. If you change REACHY_MCP_PORT, update the reachy-mini URL in this file to match (e.g. http://localhost:5001/mcp).
• config/prompts/<server_name>.txt — Per-robot instructions for the LLM (e.g. reachy-mini.txt).

Edit these files to customize behavior without changing code.

Debug MCP servers using the MCP Inspector Tool (requires Node installation):

npx @modelcontextprotocol/inspector

rosaOS is structured like a minimal operating system: a kernel schedules and supervises processes (LLM workers) that perform tasks, while a device layer (MCP server) exposes hardware (Reachy Mini) as callable tools. The LLM is the “CPU” that executes kernel and process logic.

1. User input → Speech via Reachy mic is transcribed by the server’s STT loop and POSTed to client /stt; or text is sent via CLI or the UI.
2. Kernel receives an event ([User said] ... or [Worker callback] ...). It runs the kernel agent (LLM) with tools from the process server, typically calling launch_process(system_prompt) to start a worker.
3. Process manager starts a worker subprocess with WORKER_ID, WORKER_SYSTEM_PROMPT, and CALLBACK_URL (client /event).
4. Worker runs the process agent (LLM) with tools from the Reachy MCP server: move, see, speak, etc. When done, it POSTs { worker_id, message, done } to /event.
5. Kernel gets a [Worker callback] event and can respond to the user (e.g. via another launched process that uses speak) or launch further work. Primary communication to the user is through Reachy speaking; outgoing messages are also pushed to /updates for the UI/CLI to poll.

So: kernel = one agent that only launches processes; processes = short-lived agents that use the robot and report back via callbacks.

See docs/architecture.md for a diagram (Mermaid) of the same layout.