Search Results - RepositoryStats

Gerapy Gerapy

642

3.4k

mit

125

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js

vue vuejs django docker gerapy scrapy spider scrapyd dashboard webspider distributed

Created 2017-06-30

631 commits to master branch, last one 3 months ago

scrapydweb my8100

568

3.2k

gpl-3.0

73

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO :point_right:

scrapy spider scrapyd dashboard scrapyd-ui log-parsing scrapyd-api log-analysis scrapyd-admin scrapyd-keeper scrapyd-manage scrapyd-control scrapyd-monitor scrapy-log-analysis scrapy-visualization scrapyd-log-analysis scrapyd-visualization scrapyd-cluster-management

Created 2018-09-30

70 commits to master branch, last one 2 months ago

SpiderKeeper DormyMo

508

2.7k

unknown

107

admin ui for scrapy/open source scrapinghub

scrapy spider scrapyd dashboard scrapy-ui scrapyd-ui scrapyd-dashboard

Created 2016-01-18

104 commits to master branch, last one 6 years ago

house-renting kezhenxu94

142

797

apache-2.0

36

Possibly the best practice of Scrapy 🕷 and renting a house 🏡

docker python scrapy scrapyd scrapy-spider scrapy-crawler

This repository has been archived (exclude archived)

Created 2018-05-12

73 commits to master branch, last one 3 years ago

ScrapyDouban baabaaox

203

740

unknown

22

豆瓣电影/豆瓣读书 Scarpy 爬虫

docker douban python scrapy pymysql python3 scrapyd douban-book douban-movie douban-scrapy

Created 2016-09-11

12 commits to master branch, last one 2 years ago

spider-admin-pro mouday

80

567

mit

11

spider-admin-pro 一个集爬虫Scrapy+Scrapyd爬虫项目查看和爬虫任务定时调度的可视化管理工具，SpiderAdmin的升级版

scrapy spider python3 scrapyd

Created 2021-01-25

182 commits to v3.0 branch, last one about a month ago

NewsCrawl casual-silva

142

556

unknown

10

狠心开源企业级舆情新闻爬虫项目：支持任意数量爬虫一键运行、爬虫定时任务、爬虫批量删除；爬虫一键部署；爬虫监控可视化; 配置集群爬虫分配策略；👉 现成的docker一键部署文档已为大家踩坑

web docker python scrapy script spider scrapyd scrapydweb apscheduler scrapy-crawler

Created 2022-07-08

116 commits to main branch, last one 11 months ago

crawlab-lite crawlab-team

75

224

bsd-3-clause

12

Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台

scrapy spider crawlab crawler scrapyd platform scrapy-ui scrapyd-ui web-crawler crawling-tasks crawler-management

Created 2020-03-20

113 commits to master branch, last one 2 years ago

estela bitmakerla

13

174

mit

15

estela, an elastic web scraping cluster 🕸

react django docker python scrapy scraper scrapyd crawling requests scraping kubernetes web-scraping hacktoberfest python-requests webscraping-python scrapy-visualization

Created 2022-03-03

432 commits to main branch, last one about a month ago

logparser my8100

22

89

gpl-3.0

5

A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.

scrapy scrapyd log-parser log-analyse log-parsing visualization scrapy-log-analysis scrapyd-log-analysis

Created 2019-01-20

22 commits to master branch, last one 4 years ago

ARGUS datawizard1337

25

88

gpl-3.0

6

ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks...

python scrapy scrapyd crawling scraping webcrawling webscraping

Created 2018-05-18

186 commits to master branch, last one 2 years ago