Overview

Relevant source files

• .github/banner_dark.png
• .github/banner_light.png
• README.md
• examples/README.md
• examples/voice_agents/README.md
• livekit-agents/livekit/agents/__init__.py
• livekit-agents/livekit/agents/llm/chat_context.py
• livekit-agents/livekit/agents/voice/__init__.py
• livekit-agents/livekit/agents/voice/agent.py
• livekit-agents/livekit/agents/voice/agent_activity.py
• livekit-agents/livekit/agents/voice/agent_session.py
• livekit-agents/livekit/agents/voice/audio_recognition.py
• livekit-agents/livekit/agents/voice/events.py
• livekit-agents/livekit/agents/voice/generation.py
• tests/fake_io.py
• tests/fake_session.py
• tests/fake_stt.py
• tests/test_agent_session.py

This document introduces the LiveKit Agents framework, a Python library for building realtime, programmable voice AI agents that run on servers README.md23-31 It explains the framework's purpose, high-level architecture, and core components.

For information on installing and running your first agent, see Quick Start. For details on the monorepo structure, plugin architecture, dependency management, and versioning across packages, see Project Structure and Versioning. For in-depth coverage of the runtime architecture, see Core Architecture.

---

Purpose and Scope

LiveKit Agents is designed for building conversational, multi-modal voice agents that can see, hear, and understand README.md27-29 The framework provides:

• Flexible AI Integration: Standardized abstraction layers for Speech-to-Text (STT), Large Language Models (LLM), Text-to-Speech (TTS), Voice Activity Detection (VAD), and Realtime APIs README.md35
• Integrated Scheduling & Dispatch: Built-in task scheduling and worker distribution via AgentServer and job management APIs README.md36
• Voice Pipeline Management: Semantic turn detection, preemptive generation, interruption handling, and audio/text synchronization README.md40-42
• Ecosystem & Telephony: Seamless compatibility with WebRTC client SDKs, Model Context Protocol (MCP) servers, and LiveKit's telephony stack README.md37-41

The framework handles low-level streaming logic and tool orchestration so developers can focus on defining agent instructions and implementing custom function tools README.md71-77

Sources: README.md23-44 livekit-agents/livekit/agents/__init__.py148-153

---

System Architecture

LiveKit Agents uses a layered architecture organized by functional concerns: deployment infrastructure, agent orchestration, voice interaction processing, I/O management, and provider abstractions.

Overall System Architecture and Component Relationships

Sources: README.md71-77 livekit-agents/livekit/agents/__init__.py33-40 livekit-agents/livekit/agents/voice/agent_session.py48-51

Agent Deployment and Execution Flow

Sources: README.md107-137 livekit-agents/livekit/agents/worker.py149-152

Layer Responsibilities

Layer	Key Components	Responsibilities
Deployment	AgentServer, WorkerOptions	Process management, job distribution, worker registration livekit-agents/livekit/agents/worker.py148-152
Orchestration	AgentSession, Agent	Conversation lifecycle, state transitions, prompt and tool management livekit-agents/livekit/agents/voice/agent.py38-61 livekit-agents/livekit/agents/voice/agent_session.py48-51
Pipeline	AudioRecognition, TurnHandlingOptions	VAD, STT, semantic endpointing, interruption handling livekit-agents/livekit/agents/voice/audio_recognition.py158-186 livekit-agents/livekit/agents/voice/turn.py141-147
I/O	RoomIO	LiveKit room connections, participant stream management livekit-agents/livekit/agents/voice/room_io.py1-46
Abstraction	LLM, STT, TTS	Standardized interfaces for multi-provider model integration livekit-agents/livekit/agents/__init__.py23

Sources: README.md23-44 livekit-agents/livekit/agents/__init__.py23-274

---

Child Pages

For more details on specific topics covered in this section of the framework, refer to the following child pages:

• Quick Start: Quick Start — A minimal working example demonstrating how to configure AgentServer, AgentSession, and Agent to deploy a basic voice agent README.md85-137
• Project Structure and Versioning: Project Structure and Versioning — An explanation of the monorepo layout, plugin packages, dependency management via uv, and synchronized versioning.

Sources: README.md85-137