Cloud architecture and scalability
This section is only relevant for the cloud-managed LangSmith services available at https://smith.langchain.com and https://eu.smith.langchain.com.
For information on the self-hosted LangSmith solution, please refer to the self-hosted documentation.
LangSmith is deployed on Google Cloud Platform (GCP) and is designed to be highly scalable. Many customers run production workloads on LangSmith for both LLM application observability and evaluation.
Architecture
The US-based LangSmith service is deployed in the us-central1 (Iowa) region of GCP.
NOTE: The EU-based LangSmith service is now available (as of mid-July 2024) and is deployed in the europe-west4 (Netherlands) region of GCP.
If you are interested in an enterprise plan in this region, please contact us at sales@langchain.dev.
Regional storage
The resources and services in this table are stored in the location corresponding to the URL where sign-up occurred (either the US or EU). Cloud-managed LangSmith uses Supabase for authentication/authorization and ClickHouse Cloud for data warehouse.
| US | EU | |
|---|---|---|
| URL | https://smith.langchain.com | https://eu.smith.langchain.com |
| API URL | https://api.smith.langchain.com | https://eu.api.smith.langchain.com |
| GCP | us-central1 (Iowa) | europe-west4 (Netherlands) |
| Supabase | AWS us-east-1 (N. Virginia) | AWS eu-central-1 (Germany) |
| ClickHouse Cloud | us-central1 (Iowa) | europe-west4 (Netherlands) |
| LangGraph Cloud | us-central1 (Iowa) | europe-west4 (Netherlands) |
See the Regions FAQ for more information.
Region-independent storage
Data listed here is stored exclusively in the US:
- Payment and billing information with Stripe and Metronome
GCP services
LangSmith is composed of the following services, all deployed on Google Kubernetes Engine (GKE):
- LangSmith Frontend: serves the LangSmith UI.
- LangSmith Backend: serves the LangSmith API.
- LangSmith Platform Backend: handles authentication and other high-volume tasks. (Internal service)
- LangSmith Playground: handles forwarding requests to various LLM providers for the Playground feature.
- LangSmith Queue: handles processing of asynchronous tasks. (Internal service)
LangSmith uses the following GCP storage services:
- Google Cloud Storage (GCS) for runs inputs and outputs.
- Google Cloud SQL PostgreSQL for transactional workloads.
- Google Cloud Memorystore for Redis for queuing and caching.
- Clickhouse Cloud on GCP for trace ingestion and analytics. Our services connect to Clickhouse Cloud, which is hosted in the same GCP region, via a private endpoint.
Some additional GCP services we use include:
- Google Cloud Load Balancer for routing traffic to the LangSmith services.
- Google Cloud CDN for caching static assets.
- Google Cloud Armor for security and rate limits. For more information on rate limits we enforce, please refer to this guide.
Scalability
LangSmith is designed to be scalable and performant.
As of load testing done in February 2024, LangSmith can comfortably process 500K+ runs (spans) per minute. We anticipate that LangSmith can process 750K+ runs per minute with the optimizations we've made since then.