Search Results - RepositoryStats

firecrawl mendableai

2.3k

27.6k

agpl-3.0

152

🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.

ai llm rag data crawler scraper markdown scraping ai-scraping web-crawler webscraping html-to-markdown

Created 2024-04-15

3,001 commits to main branch, last one 19 hours ago

Scrapling D4Vinci

178

2.7k

bsd-3-clause

25

🕷️ An undetectable, powerful, flexible, high-performance Python library that makes Web Scraping easy again!

Created 2024-10-13

355 commits to main branch, last one 5 days ago

CyberScraper-2077 itsOwen

145

1.6k

mit

12

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

llm openai scraper gemini-api ai-scraping llm-scraper web-scraper webscraping

Created 2024-08-17

71 commits to main branch, last one 28 days ago

parsera raznem

64

1.0k

gpl-2.0

16

Lightweight library for scraping web-sites with LLMs

ai llm python scraping opensource playwright ai-scraping webscraping data-extraction

Created 2024-08-12

123 commits to main branch, last one 21 days ago

firecrawl-app-examples mendableai

27

120

unknown

2

🔥 This repository contains complete application examples, including websites and other projects, developed using Firecrawl.

ai llm rag data examples markdown scrapers templates ai-scraping web-crawler html-to-markdown

Created 2024-06-22

18 commits to main branch, last one 6 days ago

abx-dl ArchiveBox

4

65

mit

5

⬇️ A simple all-in-one CLI tool to download EVERYTHING from a URL (like youtube-dl/yt-dlp, forum-dl, gallery-dl, simpler ArchiveBox). 🎭 Uses headless Chrome to get HTML, JS, CSS, images/video/audio/s...

cli curl wget chrome yt-dlp cli-tool crawling headless scraping puppeteer archivebox downloader gallery-dl playwright youtube-dl ai-scraping http-client internet-archiving

Created 2024-10-21

92 commits to main branch, last one 2 months ago