This page contains reference documentation for Middleware. See the docs for conceptual guides, tutorials, and examples on using Middleware.
LangChain provides prebuilt middleware for common agent use cases:
| CLASS | DESCRIPTION |
|---|---|
SummarizationMiddleware |
Automatically summarize conversation history when approaching token limits |
HumanInTheLoopMiddleware |
Pause execution for human approval of tool calls |
ModelCallLimitMiddleware |
Limit the number of model calls to prevent excessive costs |
ToolCallLimitMiddleware |
Control tool execution by limiting call counts |
ModelFallbackMiddleware |
Automatically fallback to alternative models when primary fails |
PIIMiddleware |
Detect and handle Personally Identifiable Information |
TodoListMiddleware |
Equip agents with task planning and tracking capabilities |
LLMToolSelectorMiddleware |
Use an LLM to select relevant tools before calling main model |
ToolRetryMiddleware |
Automatically retry failed tool calls with exponential backoff |
LLMToolEmulator |
Emulate tool execution using LLM for testing purposes |
ContextEditingMiddleware |
Manage conversation context by trimming or clearing tool uses |
ShellToolMiddleware |
Expose a persistent shell session to agents for command execution |
FilesystemFileSearchMiddleware |
Provide Glob and Grep search tools over filesystem files |
AgentMiddleware |
Base middleware class for creating custom middleware |
Create custom middleware using these decorators:
| DECORATOR | DESCRIPTION |
|---|---|
@before_agent |
Execute logic before agent execution starts |
@before_model |
Execute logic before each model call |
@after_model |
Execute logic after each model receives a response |
@after_agent |
Execute logic after agent execution completes |
@wrap_model_call |
Wrap and intercept model calls |
@wrap_tool_call |
Wrap and intercept tool calls |
@dynamic_prompt |
Generate dynamic system prompts based on request context |
@hook_config |
Configure hook behavior (e.g., conditional routing) |
Core types for building middleware:
| TYPE | DESCRIPTION |
|---|---|
AgentState |
State container for agent execution |
ModelRequest |
Request details passed to model calls |
ModelResponse |
Response details from model calls |
ClearToolUsesEdit |
Utility for clearing tool usage history from context |
InterruptOnConfig |
Configuration for human-in-the-loop interruptions |
SummarizationMiddleware types:
| TYPE | DESCRIPTION |
|---|---|
ContextSize |
Union type |
ContextFraction |
Summarize at fraction of total context |
ContextTokens |
Summarize at token threshold |
ContextMessages |
Summarize at message threshold |
Summarizes conversation history when token limits are approached.
This middleware monitors message token counts and automatically summarizes older messages when a threshold is reached, preserving rec
Human in the loop middleware.
Tracks model call counts and enforces limits.
This middleware monitors the number of model calls made during agent execution and can terminate the agent when specified limits are reached. It supports
Track tool call counts and enforces limits during agent execution.
This middleware monitors the number of tool calls made and can terminate or restrict execution when limits are exceeded. It supports
Automatic fallback to alternative models on errors.
Retries failed model calls with alternative models in sequence until
success or all models exhausted. Primary model specified in create_agent.
Detect and handle Personally Identifiable Information (PII) in conversations.
This middleware detects common PII types and applies configurable strategies to handle them. It can detect emails, credit
Middleware that provides todo list management capabilities to agents.
This middleware adds a write_todos tool that allows agents to create and manage
structured task lists for complex multi-step op
Uses an LLM to select relevant tools before calling the main model.
When an agent has many tools available, this middleware filters them down to only the most relevant ones for the user's query. This
Middleware that automatically retries failed tool calls with configurable backoff.
Supports retrying on specific exceptions and exponential backoff.
Emulates specified tools using an LLM instead of executing them.
This middleware allows selective emulation of tools for testing purposes.
By default (when tools=None), all tools are emulated. You
Automatically prune tool results to manage context size.
The middleware applies a sequence of edits when the total input token count exceeds configured thresholds.
Currently the ClearToolUsesEdit
Middleware that registers a persistent shell tool for agents.
The middleware exposes a single long-lived shell session. Use the execution policy to match your deployment's security posture:
Provides Glob and Grep search over filesystem files.
This middleware adds two tools that search through local filesystem:
Base middleware class for an agent.
Subclass this and implement any of the defined methods to customize agent behavior between steps in the main agent loop.
State schema for the agent.
Model request information for the agent.
Response from model execution including messages and optional structured output.
The result will usually contain a single AIMessage, but may include an additional
ToolMessage if the model used a
Configuration for clearing tool outputs when token limits are exceeded.
Configuration for an action requiring human in the loop.
This is the configuration format used in the HumanInTheLoopMiddleware.__init__
method.
Decorator used to dynamically create a middleware with the before_agent hook.
Decorator used to dynamically create a middleware with the before_model hook.
Decorator used to dynamically create a middleware with the after_model hook.
Decorator used to dynamically create a middleware with the after_agent hook.
Async version is aafter_agent.
Create middleware with wrap_model_call hook from a function.
Converts a function with handler callback into middleware that can intercept model calls, implement retry logic, handle errors, and rewr
Create middleware with wrap_tool_call hook from a function.
Async version is awrap_tool_call.
Converts a function with handler callback into middleware that can intercept tool calls, implement r
Decorator used to dynamically generate system prompts for the model.
This is a convenience decorator that creates middleware using wrap_model_call
specifically for dynamic prompt generation. The de
Decorator to configure hook behavior in middleware methods.
Use this decorator on before_model or after_model methods in middleware classes
to configure their behavior. Currently supports specify
