tonywangcn / scaleable-crawler-with-docker-cluster

a scaleable and efficient crawelr with docker cluster , crawl million pages in 2 hours with a single machine

tonywangcn
tonywangcn
Date Created 2017-02-28 (8 years ago)
Commits 10 (last one 3 years ago)
Stargazers 97 (0 this week)
Watchers 6 (0 this week)
Forks 26
License unknown
Ranking

RepositoryStats indexes 627,578 repositories, of these tonywangcn/scaleable-crawler-with-docker-cluster is ranked #311,175 (50th percentile) for total stargazers, and #288,366 for total watchers. Github reports the primary language for this repository as Python, for repositories using this language it is ranked #58,779/127,525.

tonywangcn/scaleable-crawler-with-docker-cluster is also tagged with popular topics, for these it's ranked: python (#13,463/23247),  docker (#3,816/6563),  crawler (#386/591),  rabbitmq (#293/473),  distributed (#298/391)

Other Information

tonywangcn/scaleable-crawler-with-docker-cluster has 1 open pull request on Github, 1 pull request has been merged over the lifetime of the repository.

Github issues are enabled, there is 1 open issue and 2 closed issues.

Star History

Github stargazers over time

100100909080807070606050504040303020201010002018201820192019202020202021202120222022202320232024202420252025

Watcher History

Github watchers over time, collection started in '23

7777776.56.566666620232023Jul '23Jul '2320242024Jul '24Jul '2420252025

Recent Commit History

2 commits on the default branch (master) since jan '22

22221111110000Jul '22Jul '2220232023Jul '23Jul '2320242024Jul '24Jul '2420252025

Yearly Commits

Commits to the default branch (master) per year

443.53.5332.52.5221.51.5110.50.50020172017201820182019201920202020202120212022202220242024

Issue History

Total Issues
Open Issues
Closed Issues
332.52.5221.51.5110.50.5002018201820192019202020202021202120222022202320232024202420252025

Languages

The only known language in this repository is Python

PythonPython

updated: 2025-03-07 @ 12:14am, id: 83430073 / R_kgDOBPkKuQ