Knowledge infrastructure to make your agents smarter. Search, scrape and 550+ data providers through one API.
-
Updated
Nov 18, 2026 - TypeScript
Knowledge infrastructure to make your agents smarter. Search, scrape and 550+ data providers through one API.
Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Python scraper based on AI
Best headless browser for AI agents. Lite, Fast, High-Compatibility. Built in Rust
Turn any website into a structured API. Extract, automate, search and monitor the web.
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
A collection of awesome web crawler,spider in different languages
Self-hosted webscraper.
Professional-grade, local-first browser extension that transforms any webpage into high-fidelity, LLM-friendly Markdown entirely within the browser's runtime. Zero-cloud, privacy-centric, and designed specifically for AI workflows.
A list of practical knowledge-building projects.
Scrape Google AI Mode responses without blocks on a large scale.
A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama
PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs
The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.
Foundational low latency web data collecting in Rust
Generate and download e-books from online sources.
Web Scraper in Go, similar to BeautifulSoup
A guide for extracting titles, authors, and citations from Google Scholar using Python and Oxylabs SERP Scraper API.
A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.
To associate your repository with the web-scraper topic, visit your repo's landing page and select "manage topics."