Statistics for topic crawler

RepositoryStats tracks 605,968 Github repositories, of these 576 are tagged with the crawler topic. The most common primary language for repositories using this topic is Python (274). Other languages include: Go (70), JavaScript (52), TypeScript (29), Java (28), PHP (28), C# (16), HTML (16), Rust (11)

Stargazers over time for topic crawler

Most starred repositories for topic crawler (view more)

10.6k

53.8k

bsd-3-clause

1.8k

Scrapy, a fast high-level web crawling & scraping framework for Python.

python crawler crawling scraping framework web-scraping hacktoberfest web-scraping-python

Created 2010-02-22

10,680 commits to master branch, last one 12 hours ago

EasySpider NaiboWang

4.5k

37.1k

other

233

A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。

Created 2020-07-18

616 commits to master branch, last one 12 days ago

3.0k

28.2k

mit

381

👾 Fast and simple video download library and CLI tool written in Go

go qq iqiyi video youku golang tumblr crawler scraper youtube bilibili download downloader

Created 2018-02-24

857 commits to master branch, last one 3 months ago

1.8k

23.6k

apache-2.0

332

Elegant Scraper and Crawler Framework for Golang

go golang spider crawler scraper crawling scraping framework

Created 2017-09-29

666 commits to master branch, last one 7 months ago

proxy_pool jhao104

5.2k

21.9k

mit

445

Python ProxyPool for web spider

http proxy redis spider crawler

Created 2016-11-25

652 commits to master branch, last one 4 months ago

firecrawl mendableai

1.7k

21.8k

agpl-3.0

117

🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.

ai llm rag data crawler scraper markdown scraping ai-scraping web-crawler webscraping html-to-markdown

Created 2024-04-15

2,715 commits to main branch, last one 4 hours ago

Trending repositories for topic crawler (view more)