Statistics for topic crawling

RepositoryStats tracks 579,129 Github repositories, of these 94 are tagged with the crawling topic. The most common primary language for repositories using this topic is Python (41). Other languages include: Go (15)

Stargazers over time for topic crawling

Most starred repositories for topic crawling (view more)

10.5k
53.0k
bsd-3-clause
1.8k
Scrapy, a fast high-level web crawling & scraping framework for Python.
Created 2010-02-22
10,592 commits to master branch, last one 2 days ago
1.8k
23.3k
apache-2.0
332
Elegant Scraper and Crawler Framework for Golang
Created 2017-09-29
666 commits to master branch, last one 5 months ago
664
15.5k
apache-2.0
103
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and o...
Created 2016-08-26
4,679 commits to master branch, last one 13 hours ago
2.1k
14.1k
mit
385
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
Created 2013-11-25
651 commits to master branch, last one 4 years ago
List of libraries, tools and APIs for web scraping and data processing.
Created 2015-08-12
534 commits to master branch, last one 10 days ago
302
5.7k
apache-2.0
102
Declarative web scraping
Created 2018-08-23
845 commits to master branch, last one 19 days ago

Trending repositories for topic crawling (view more)