🦜🛠️ LangSmith MCP Server
A production-ready Model Context Protocol (MCP) server that provides seamless integration with the LangSmith observability platform. This server enables language models to fetch conversation history, prompts, runs and traces, datasets, experiments, and billing usage from LangSmith.
📋 Example Use Cases
The server enables powerful capabilities including:
- 💬 Conversation History: "Fetch the history of my conversation from thread 'thread-123' in project 'my-chatbot'" (paginated by character budget)
- 📚 Prompt Management: "Get all public prompts in my workspace" / "Pull the template for the 'legal-case-summarizer' prompt"
- 🔍 Traces & Runs: "Fetch the latest 10 root runs from project 'alpha'" / "Get all runs for trace <uuid> (page 2 of 5)"
- 📊 Datasets: "List datasets of type chat" / "Read examples from dataset 'customer-support-qa'"
- 🧪 Experiments: "List experiments for dataset 'my-eval-set' with latency and cost metrics"
- 📈 Billing: "Get billing usage for September 2025"
🛠️ Available Tools
The LangSmith MCP Server provides the following tools for integration with LangSmith.
💬 Conversation & Threads
| Tool Name | Description |
|---|
get_thread_history | Retrieve message history for a conversation thread. Uses char-based pagination: pass page_number (1-based), and use returned total_pages to request more pages. Optional max_chars_per_page and preview_chars control page size and long-string truncation. |
📚 Prompt Management
| Tool Name | Description |
|---|
list_prompts | Fetch prompts from LangSmith with optional filtering by visibility (public/private) and limit. |
get_prompt_by_name | Get a specific prompt by its exact name, returning the prompt details and template. |
push_prompt | Documentation-only: how to create and push prompts to LangSmith. |
🔍 Traces & Runs
| Tool Name | Description |
|---|
fetch_runs | Fetch LangSmith runs (traces, tools, chains, etc.) from one or more projects. Supports filters (run_type, error, is_root), FQL (filter, trace_filter, tree_filter), and ordering. When trace_id is set, returns char-based paginated pages; otherwise returns one batch up to limit. Always pass limit and page_number. |
list_projects | List LangSmith projects with optional filtering by name, dataset, and detail level (simplified vs full). |
📊 Datasets & Examples
| Tool Name | Description |
|---|
list_datasets | Fetch datasets with filtering by ID, type, name, name substring, or metadata. |
list_examples | Fetch examples from a dataset by dataset ID/name or example IDs, with filter, metadata, splits, and optional as_of version. |
read_dataset | Read a single dataset by ID or name. |
read_example | Read a single example by ID, with optional as_of version. |
create_dataset | Documentation-only: how to create datasets in LangSmith. |
update_examples | Documentation-only: how to update dataset examples in LangSmith. |
🧪 Experiments & Evaluations
| Tool Name | Description |
|---|
list_experiments | List experiment projects (reference projects) for a dataset. Requires reference_dataset_id or reference_dataset_name. Returns key metrics (latency, cost, feedback stats). |
run_experiment | Documentation-only: how to run experiments and evaluations in LangSmith. |
📈 Usage & Billing
| Tool Name | Description |
|---|
get_billing_usage | Fetch organization billing usage (e.g. trace counts) for a date range. Optional workspace filter; returns metrics with workspace names inline. |
📄 Pagination (char-based)
Several tools use stateless, character-budget pagination so responses stay within a size limit and work well with LLM clients:
- Where it’s used:
get_thread_history and fetch_runs (when trace_id is set).
- Parameters: You send
page_number (1-based) on every request. Optional: max_chars_per_page (default 25000, cap 30000) and preview_chars (truncate long strings with "… (+N chars)").
- Response: Each response includes
page_number, total_pages, and the page payload (result for messages, runs for runs). To get more, call again with page_number = 2, then 3, up to total_pages.
- Why it’s useful: Pages are built by JSON character count, not item count, so each page fits within a fixed size. No cursor or server-side state—just integer page numbers.
🐳 Docker Deployment (HTTP-Streamable)
The LangSmith MCP Server can be deployed as an HTTP server using Docker, enabling remote access via the HTTP-streamable protocol.
Building the Docker Image
docker build -t langsmith-mcp-server .
Running with Docker
docker run -p 8000:8000 langsmith-mcp-server
The API key is provided via the LANGSMITH-API-KEY header when connecting, so no environment variables are required for HTTP-streamable protocol.
Connecting with HTTP-Streamable Protocol
Once the Docker container is running, you can connect to it using the HTTP-streamable transport. The server accepts authentication via headers:
Required header:
LANGSMITH-API-KEY: Your LangSmith API key
Optional headers:
LANGSMITH-WORKSPACE-ID: Workspace ID for API keys scoped to multiple workspacesLANGSMITH-ENDPOINT: Custom endpoint URL (for self-hosted or EU region)
Example client configuration:
from mcp import ClientSession
from mcp.client.streamable_http import streamablehttp_client
headers = {
"LANGSMITH-API-KEY": "lsv2_pt_your_api_key_here",
# Optional:
# "LANGSMITH-WORKSPACE-ID": "your_workspace_id",
# "LANGSMITH-ENDPOINT": "https://api.smith.langchain.com",
}
async with streamablehttp_client("http://localhost:8000/mcp", headers=headers) as (read, write, _):
async with ClientSession(read, write) as session:
await session.initialize()
# Use the session to call tools, list prompts, etc.
Cursor Integration
To add the LangSmith MCP Server to Cursor using HTTP-streamable protocol, add the following to your mcp.json configuration file:
{
"mcpServers": {
"HTTP-Streamable LangSmith MCP Server": {
"url": "http://localhost:8000/mcp",
"headers": {
"LANGSMITH-API-KEY": "lsv2_pt_your_api_key_here"
}
}
}
}
Optional headers:
{
"mcpServers": {
"HTTP-Streamable LangSmith MCP Server": {
"url": "http://localhost:8000/mcp",
"headers": {
"LANGSMITH-API-KEY": "lsv2_pt_your_api_key_here",
"LANGSMITH-WORKSPACE-ID": "your_workspace_id",
"LANGSMITH-ENDPOINT": "https://api.smith.langchain.com"
}
}
}
}
Make sure the server is running before connecting Cursor to it.
Health Check
The server provides a health check endpoint:
curl http://localhost:8000/health
This endpoint does not require authentication and returns "LangSmith MCP server is running" when the server is healthy.
🧪 Development and Contributing
Prerequisites
- Python 3.10+ (3.11+ recommended)
- uv – install with
curl -LsSf https://astral.sh/uv/install.sh | sh
- LangSmith API key – from smith.langchain.com
- Node.js (optional) – only if you want to use MCP Inspector to test the server (stdio or streamable-http)
Development workflow
- Edit code in
langsmith_mcp_server/ or tests/.
- Format and lint (required before committing):
make format
make lint
- Run tests:
make test
# Or a single file:
make test TEST_FILE=tests/tools/test_dataset_tools.py
- Type-check (optional):
uv run mypy langsmith_mcp_server/
Testing with MCP Inspector
You can test the server with MCP Inspector using either stdio or streamable-http.
-
Start MCP Inspector:
npx @modelcontextprotocol/inspector@latest
Open http://localhost:6274 in your browser.
-
Connect in the Inspector:
- Stdio: Choose stdio transport and configure the server command (e.g.
uv run langsmith-mcp-server) and set LANGSMITH_API_KEY in the environment.
- Streamable HTTP: Start the server first (
uv run uvicorn langsmith_mcp_server.server:app --host 0.0.0.0 --port 8000 or Docker), then choose streamable-http, URL http://localhost:8000/mcp, and add header LANGSMITH-API-KEY = your API key.
Load testing
A session-based load test opens many MCP sessions and calls the list_prompts tool in each, using langchain-mcp-adapters. Run from the CLI (no UI). The server must be running first.
uv sync --group load
# Terminal 1: start the server
uv run uvicorn langsmith_mcp_server.server:app --host 0.0.0.0 --port 8000
# Terminal 2: run the load test
uv run python tests/load_test_sessions.py --sessions 20 --calls-per-session 3
Options
| Option | Default | Description |
|---|
--url | http://localhost:8000/mcp | MCP endpoint URL |
--api-key | from .env | LANGSMITH_API_KEY (or set in project root .env) |
--sessions | 10 | Number of concurrent sessions |
--calls-per-session | 3 | list_prompts calls per session |
--debug | off | Print step-by-step logs and first error traceback |
--report PATH | — | Write a report after the run (see below) |
Report
Use --report PATH to write a JSON report after the test (e.g. --report load_test_report creates load_test_report.json with config, summary, per-session results, and first error).
uv run python tests/load_test_sessions.py --sessions 5 --report load_test_report
# Creates: load_test_report.json (in current directory)
Contributing checklist
Before opening a PR:
For more detail (adding tools, code standards, troubleshooting), see CLAUDE.md.
📄 License
This project is distributed under the MIT License. For detailed terms and conditions, please refer to the LICENSE file.
Made with ❤️ by the LangChain Team