searxNcrawl MCP server
MCP server and CLI tools for web search and crawling, built on SearXNG and Crawl4AI
What searxNcrawl does
Summarised from the project README; check the repository for the latest details.
Features
- Single page, multi-page, and site crawling (DFS with depth/page limits)
- Production-tested extraction config optimized for documentation sites
- Configurable timeouts with graceful error handling
- Markdown deduplication — `exact` (default) removes repeated blocks, `off` disables it
- Link removal — strip all links for cleaner LLM context (`--remove-links`)
- Dedup guardrails — non-destructive metadata signals when removal is unusually aggressive
Repository: DasDigitaleMomentum/searxNcrawl
Website: https://www.das-digitale-momentum.de/en/open-source/#searxncrawl
Stars: ★167
Language: Python
Last updated: 2026-10-14
crawl4aillm-toolsmcpmcp-serverpythonsearxngweb-crawlerweb-search
Website: https://www.das-digitale-momentum.de/en/open-source/#searxncrawl
Stars: ★167
Language: Python
Last updated: 2026-10-14
crawl4aillm-toolsmcpmcp-serverpythonsearxngweb-crawlerweb-search
How to install searxNcrawl
No install config was found in this README. See the repository for setup steps, then follow our guides for Claude Desktop, Cursor or VS Code.
Featured in: Best MCP servers for web scraping
Listed on Socketlist – add this badge to your README:
Maintain this project? Get it featured for free during launch →
Related Web Search & Scraping MCP servers
gpt-researcher by assafelovic · ★30100
An autonomous agent that conducts deep research on any data using any LLM providers
Web Search & Scraping
An autonomous agent that conducts deep research on any data using any LLM providers
Web Search & Scraping
Douyin_TikTok_Download_API by Evil0ctal · ★20643
🚀 抖音、TikTok 数据采集与无水印视频下载 API，自托管，支持 MCP 调用与 Docker 一键部署。| Self-hosted TikTok & Douyin scraper and no-watermark video downloader — async RES
Web Search & Scraping
🚀 抖音、TikTok 数据采集与无水印视频下载 API，自托管，支持 MCP 调用与 Docker 一键部署。| Self-hosted TikTok & Douyin scraper and no-watermark video downloader — async RES
Web Search & Scraping
apify-mcp-server by apify · ★11266
The Apify MCP server enables your AI agents to extract data from social media, search engines, maps, e-commerce sites, or any other website
Web Search & Scraping
The Apify MCP server enables your AI agents to extract data from social media, search engines, maps, e-commerce sites, or any other website
Web Search & Scraping
firecrawl-mcp-server by firecrawl · ★7606
🔥 Official Firecrawl MCP Server - Adds powerful web scraping and search to Cursor, Claude and any other LLM clients.
Web Search & Scraping
🔥 Official Firecrawl MCP Server - Adds powerful web scraping and search to Cursor, Claude and any other LLM clients.
Web Search & Scraping
wigolo by KnockOutEZ · ★5476
The go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no cloud, $0/query. Public beta.
Web Search & Scraping
The go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no cloud, $0/query. Public beta.
Web Search & Scraping
Weekly MCP digest – new and trending MCP servers in your inbox.