Eval Runner MCP Server

Install in Claude Code

claude mcp add eval-runner -e YOUR_API_KEY='<your-api-key>' -- npx -y mcp-eval-runner

Install in Cursor

{
  "mcpServers": {
    "eval-runner": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-eval-runner"
      ],
      "env": {
        "YOUR_API_KEY": "<your-api-key>"
      }
    }
  }
}

Add to ~/.cursor/mcp.json (global) or .cursor/mcp.json (project).

Install in Claude Desktop

{
  "mcpServers": {
    "eval-runner": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-eval-runner"
      ],
      "env": {
        "YOUR_API_KEY": "<your-api-key>"
      }
    }
  }
}

Settings → Developer → Edit Config (claude_desktop_config.json), then restart.

Install in VS Code

{
  "servers": {
    "eval-runner": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "mcp-eval-runner"
      ],
      "env": {
        "YOUR_API_KEY": "<your-api-key>"
      }
    }
  }
}

Add to .vscode/mcp.json in your workspace.

Install in Windsurf

{
  "mcpServers": {
    "eval-runner": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-eval-runner"
      ],
      "env": {
        "YOUR_API_KEY": "<your-api-key>"
      }
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json.