Insutanto / scrapy-distributed

A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.

Date Created 2020-08-02 (4 years ago)

Commits 28 (last one about a year ago)

Stargazers 55 (0 this week)

Watchers 6 (0 this week)

Forks 11

License unknown

Ranking

RepositoryStats indexes 579,555 repositories, of these Insutanto/scrapy-distributed is ranked #431,347 (26th percentile) for total stargazers, and #296,754 for total watchers. Github reports the primary language for this repository as Python, for repositories using this language it is ranked #81,866/115,123.

Insutanto/scrapy-distributed is also tagged with popular topics, for these it's ranked: python (#17,538/21870), redis (#1,419/1694), kafka (#670/837), crawler (#473/560), rabbitmq (#376/452), scraping (#258/340), spider (#288/332), scrapy (#138/162)

Other Information

Insutanto/scrapy-distributed has Github issues enabled, there are 10 open issues and 3 closed issues.

There have been 3 releases, the latest one was published on 2023-04-23 (about a year ago) with the name 2023.04.23 Support Scrapy 2.6+.

All Topics

kafka redis python scrapy spider crawler crawling rabbitmq scraping redisbloom rabbitmq-pipeline distributed-spider

Star History

Github stargazers over time

Watcher History

Github watchers over time, collection started in '23

Recent Commit History

2 commits on the default branch (master) since jan '22

Yearly Commits

Commits to the default branch (master) per year

Issue History

Languages

The only known language in this repository is Python

updated: 2024-09-23 @ 12:59am, id: 284488864 / R_kgDOEPT0oA