Cloud platform features

This page describes the platform features that apply only to Cloud deployments. For self-hosted equivalents, see Deploy to self-hosted.

​

Data region

Deployments can be created in two data regions: US and EU. The data region for a deployment is implied by the data region of the LangSmith organization where the deployment is created. Deployments and the underlying database for the deployments cannot be migrated between data regions.

​

Static IP addresses

All traffic from deployments created after January 6, 2025 comes through a NAT gateway. This NAT gateway has several static IP addresses depending on the data region.

​

Allowlist IP addresses

Use the following static IP addresses to allowlist traffic for each region: For automated updates, use the egress IP inventory (JSON). Filter by service: langsmith-deployment, your cloud, and your data region. Allowlist every matching cidr. See Egress IP inventory for the field definitions and update guidance.

​

Payload size

The maximum payload size for all requests sent to Cloud deployments is 25 MB. A request with a payload larger than 25 MB returns a 413 Payload Too Large error.

​

Deployment types

The control plane offers two deployment types: Serverless and Dedicated. Each is available in three sizes: Small, Medium, and Large. Agent Server is fault-tolerant on both deployment types; it automatically recovers from transient Redis or Postgres interruptions and retries failed background runs.

​

Serverless

A Serverless deployment runs one replica that scales to zero when idle and wakes on the next request. It suits background agents, development, and testing, plus production workloads that can tolerate cold-start latency. Set Scale down after to 1 hour, 1 day, or 1 week (the default) at creation or in the Scaling dialog. Shorter windows reduce idle compute charges; longer windows reduce cold starts. The platform checks for scheduled runs and enabled cron jobs due within 10 minutes to start a scaled-down deployment ahead of time. This reduces cold-start delays but does not guarantee readiness. Requests during startup wait up to 60 seconds by default, then time out. The replica does not scale out. Its queue worker handles up to 10 concurrent runs by default, configurable with N_JOBS_PER_WORKER; additional runs queue.

​

Dedicated

A Dedicated deployment stays on and autoscales separate API server and queue worker replicas. Each has its own database with automatic backups, point-in-time recovery, and high availability. It suits latency-sensitive workloads that need always-on capacity. Dedicated deployments are billed separately for runtime compute and database capacity, including while idle. Autoscaling changes runtime charges, not database charges. For rates and examples, see LangSmith Deployment billing. To request more resources, contact support. Increases depend on your use case and available capacity.

​

Sizes

These tables show requested vCPU and memory for billable compute on LangSmith Cloud in GCP. Redis is excluded because it is not billed on any deployment type or size; see Compute billing. A Serverless deployment runs one replica containing an API server and a queue worker, plus a separate Redis container: A Dedicated deployment runs separate pools of API server and queue worker replicas, a single Redis container, and its own database instance: Minimum billable runtime totals include 2 API server replicas and 1 queue worker. Database capacity is billed separately and is not included in these totals. Autoscaling adds runtime replicas at the listed per-replica size. On a Dedicated deployment, the compute size and the database size can be set independently from the Scaling dialog. A size change rolls out as a new revision and keeps the deployment’s threads, checkpoints, and API URL. For the price of each size, see the pricing page, which includes a deployment cost calculator. For how Serverless and Dedicated deployments are billed, see Manage billing.

​

Run multiple agents on one deployment

Deployments are billed on the resources they consume, not on the number of agents they serve. To run several agents on the resources of one deployment, define multiple graphs in one langgraph.json. This works well for Dedicated: one always-on instance with its own database can host several production agents. All graphs in a deployment share its compute, database, autoscaling, and deployment type. They also ship together as one revision. Give an agent its own deployment when it needs a different size, latency profile, or isolation from the other agents.

​

Database provisioning

The control plane and data plane listener application coordinate to automatically create a Postgres database for each Cloud deployment. The database serves as the persistence layer for the deployment. When implementing a LangGraph application, a checkpointer does not need to be configured. A checkpointer is automatically configured for the graph. Any checkpointer configured for a graph is replaced by the one that is automatically configured. There is no direct access to the database. All access to the database occurs through the Agent Server. The database is never deleted until the deployment itself is deleted. For self-hosted deployments, see custom PostgreSQL configuration.

​

Scaling

Cloud deployments autoscale automatically. You do not configure queue workers, replicas, or pool sizes directly. Serverless scales between zero and one replica. A Dedicated deployment adds and removes replicas up to the maximum for its size. API servers and queue workers are separate pools that scale independently, so read traffic does not slow run submission and vice versa:

• API servers scale on CPU. The target is 50% of the CPU limit, which is twice the requested vCPU in the sizes table. The minimum is 2 replicas.
• Queue workers scale on CPU or on run backlog, whichever needs more replicas. The CPU target is the same. The backlog target counts pending and running runs at 75% of N_JOBS_PER_WORKER, rounded down. With the default of 10, the target is 7 runs per replica; 30 runs require 5 workers. The minimum is 1 replica.

Memory does not trigger scaling. Redis and database compute do not autoscale. To avoid rapid scaling changes, scale-down waits 15 minutes, then removes at most 10% of replicas (at least one) every 2 minutes. It pauses during revision rollouts. Replicas remain billable until they scale down. Autoscaling changes the number of replicas, but the CPU and memory available to each replica are fixed by the deployment’s size. If a deployment is under sustained CPU pressure, or its containers run out of memory, upgrade it to a larger size. A size change rolls out as a new revision with no downtime; the deployment type cannot be changed. Application-level scaling levers (durability modes, async patterns, avoiding synchronous blocking, using /join instead of polling) apply to Cloud the same as to self-hosted. See Scaling on self-hosted for the underlying concepts; the Helm and resource configurations there do not apply to Cloud.

​

Next steps

• Create a deployment: Deploy from a GitHub repository or with the langgraph deploy CLI.
• Manage billing: See how deployment compute and databases are billed.

---