TheCrawler MCP Server
by manchittlabio.github.manchittlab/thecrawlerv0.5.8
Universal web scraper with LLM-ready markdown, RAG chunking, PDF/DOCX support.
context tax
queued
security
queued
cold start
queued
freshness
Slowing3mo ago
Install TheCrawler MCP server
Install in Claude Code
claude mcp add thecrawler -- npx -y thecrawlerInstall in Cursor
{
"mcpServers": {
"thecrawler": {
"command": "npx",
"args": [
"-y",
"thecrawler"
]
}
}
}Add to ~/.cursor/mcp.json (global) or .cursor/mcp.json (project).
Install in Claude Desktop
{
"mcpServers": {
"thecrawler": {
"command": "npx",
"args": [
"-y",
"thecrawler"
]
}
}
}Settings → Developer → Edit Config (claude_desktop_config.json), then restart.
Install in VS Code
{
"servers": {
"thecrawler": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"thecrawler"
]
}
}
}Add to .vscode/mcp.json in your workspace.
Install in Windsurf
{
"mcpServers": {
"thecrawler": {
"command": "npx",
"args": [
"-y",
"thecrawler"
]
}
}
}Add to ~/.codeium/windsurf/mcp_config.json.
Freshness
Slowing — last maintenance signal 3mo ago. The newest of the signals below sets the band.
Last commit (default branch)
2026-07-02 · 3mo ago · GitHub
Latest release
no data · GitHub
Package published
no data · npm/PyPI
Registry entry updated
2026-07-02 · 3mo ago · official registry · v0.5.8
FAQ
›How do I install the TheCrawler MCP server in Claude Code?
Run: claude mcp add thecrawler -- npx -y thecrawler. For Cursor, VS Code, Claude Desktop and Windsurf, use the install tabs above.
›Does TheCrawler require an API key?
No required environment variables are declared in its published metadata.
›Can I use TheCrawler as a remote (hosted) MCP server?
No hosted endpoint is published; it runs locally over stdio.
›Is TheCrawler in the official MCP registry?
Yes, as io.github.manchittlab/thecrawler.
Alternatives to TheCrawler
Other memory MCP servers.
FetchActiveWeb content fetching and conversion for efficient LLM usageMemoryActiveKnowledge graph-based persistent memory for maintaining context.
Claude FlowActiveAI orchestration with hive-mind swarms, neural networks, and 87 MCP tools for enterprise dev.
HindsightActiveHindsight: long-term memory for AI agents modeled on human memory.
Codebase MemoryActiveIndexes a repo into a persistent local knowledge graph of functions, classes, call chains, HTTP routes and cross-service links, using tree-sitter and LSP.
BrowserActiveA fast, lightweight MCP server that empowers LLMs with browser automation via Puppeteer’s structured accessibility data, featuring optional vision mode for complex visual understanding and flexible, cross-platform configuration.