Statistics for topic crawling

RepositoryStats tracks 584,796 Github repositories, of these 94 are tagged with the crawling topic. The most common primary language for repositories using this topic is Python (41). Other languages include: Go (15)

Stargazers over time for topic crawling

Most starred repositories for topic crawling (view more)

10.6k
53.2k
bsd-3-clause
1.8k
Scrapy, a fast high-level web crawling & scraping framework for Python.
Created 2010-02-22
10,621 commits to master branch, last one a day ago
1.8k
23.3k
apache-2.0
333
Elegant Scraper and Crawler Framework for Golang
Created 2017-09-29
666 commits to master branch, last one 5 months ago
670
15.7k
apache-2.0
103
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and o...
Created 2016-08-26
4,692 commits to master branch, last one a day ago
2.1k
14.2k
mit
385
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
Created 2013-11-25
651 commits to master branch, last one 4 years ago
List of libraries, tools and APIs for web scraping and data processing.
Created 2015-08-12
534 commits to master branch, last one 24 days ago
302
5.7k
apache-2.0
101
Declarative web scraping
Created 2018-08-23
846 commits to master branch, last one 12 days ago

Trending repositories for topic crawling (view more)