Evals MCP Server
by cyanheadsio.github.cyanheads/evals-mcp-serverv0.1.5
Author verifiable eval records through a draft→review→revise→submit loop with enforced graders.
context tax
queued
security
queued
cold start
queued
freshness
Active2d ago
Install Evals MCP server
Install in Claude Code
claude mcp add evals -- npx -y @cyanheads/evals-mcp-serverInstall in Cursor
{
"mcpServers": {
"evals": {
"command": "npx",
"args": [
"-y",
"@cyanheads/evals-mcp-server"
]
}
}
}Add to ~/.cursor/mcp.json (global) or .cursor/mcp.json (project).
Install in Claude Desktop
{
"mcpServers": {
"evals": {
"command": "npx",
"args": [
"-y",
"@cyanheads/evals-mcp-server"
]
}
}
}Settings → Developer → Edit Config (claude_desktop_config.json), then restart.
Install in VS Code
{
"servers": {
"evals": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"@cyanheads/evals-mcp-server"
]
}
}
}Add to .vscode/mcp.json in your workspace.
Install in Windsurf
{
"mcpServers": {
"evals": {
"command": "npx",
"args": [
"-y",
"@cyanheads/evals-mcp-server"
]
}
}
}Add to ~/.codeium/windsurf/mcp_config.json.
Configuration
| Variable | Required | Secret | Description |
|---|---|---|---|
| EVALS_DATA_DIR | — | — | Root folder for record JSON. The store manages drafts/, submitted/, and exports/ subdirs under it. |
| EVALS_REQUIRE_CONFIRMATION | — | — | When 'true', evals_submit_draft requests human confirmation before finalizing. |
| EVALS_DEFAULT_LICENSE | — | — | Default metadata.license applied when a draft omits one. |
| EVALS_CAPTURE_DIR | — | — | Directory of framework-written tool-call captures; when set, captures EvalsIDs resolve to full dumps. |
| MCP_LOG_LEVEL | — | — | Sets the minimum log level for output (e.g., 'debug', 'info', 'warn'). |
| MCP_TRANSPORT_TYPE | — | — | Selects the HTTP transport. |
| MCP_HTTP_HOST | — | — | The hostname for the HTTP server. |
| MCP_HTTP_PORT | — | — | The port to run the HTTP server on. |
| MCP_HTTP_ENDPOINT_PATH | — | — | The endpoint path for the MCP server. |
| MCP_AUTH_MODE | — | — | Authentication mode to use: 'none', 'jwt', or 'oauth'. |
Freshness
Active — last maintenance signal 2d ago. The newest of the signals below sets the band.
Last commit (default branch)
2026-10-07 · 2d ago · GitHub
Latest release
2026-10-07 · 2d ago · GitHub · v0.1.5
Package published
no data · npm/PyPI
Registry entry updated
2026-10-07 · 2d ago · official registry · v0.1.5
FAQ
›How do I install the Evals MCP server in Claude Code?
Run: claude mcp add evals -- npx -y @cyanheads/evals-mcp-server. For Cursor, VS Code, Claude Desktop and Windsurf, use the install tabs above.
›Does Evals require an API key?
No required environment variables are declared in its published metadata.
›Can I use Evals as a remote (hosted) MCP server?
Yes — it offers both a hosted endpoint and a local stdio package.
›Is Evals in the official MCP registry?
Yes, as io.github.cyanheads/evals-mcp-server.