ZSAIm / VideoCrawlerEngine

起源于旧项目爱奇艺解析器(iqiyi-parser)在开发、维护和扩展的过程中遇到的一些问题，而实现的一个基于任务流式的可视化爬虫引擎。引擎的执行单元是节点。脚本节点(script)作为根节点来完成对节点和流程的描述，经由任务节点(task)解析流程描述并生成节点的执行队列，最后交由工作者执行池处理。整个过程可视可控，所有节点处理器都以插件的形式导入，以最大程度实现易扩展性。

ZSAIm
ZSAIm
Date Created 2020-06-09 (4 years ago)
Commits 24 (last one 4 years ago)
Stargazers 475 (0 this week)
Watchers 15 (0 this week)
Forks 89
License unknown
Ranking

RepositoryStats indexes 632,894 repositories, of these ZSAIm/VideoCrawlerEngine is ranked #98,677 (84th percentile) for total stargazers, and #142,377 for total watchers. Github reports the primary language for this repository as Python, for repositories using this language it is ranked #16,769/129,085.

ZSAIm/VideoCrawlerEngine is also tagged with popular topics, for these it's ranked: python (#5,089/23414)

Other Information

ZSAIm/VideoCrawlerEngine has 21 open pull requests on Github, 6 pull requests have been merged over the lifetime of the repository.

Github issues are enabled, there are 4 open issues and 6 closed issues.

All Topics
python taskflow

Star History

Github stargazers over time

500500450450400400350350300300250250200200150150100100505000Jul '20Jul '2020212021Jul '21Jul '2120222022Jul '22Jul '2220232023Jul '23Jul '2320242024Jul '24Jul '2420252025

Watcher History

Github watchers over time, collection started in '23

171717171616161616161515151520232023Jul '23Jul '2320242024Jul '24Jul '2420252025

Recent Commit History

0 commits on the default branch (master) since jan '22

Inactive

No recent commits to this repository

Yearly Commits

Commits to the default branch (master) per year

18181616141412121010886644220020202020202120212022202220242024

Issue History

Total Issues
Open Issues
Closed Issues
10109988776655443322110020212021Jul '21Jul '2120222022Jul '22Jul '2220232023Jul '23Jul '2320242024Jul '24Jul '2420252025

Languages

The primary language is Python but there's also others...

PythonPythonVueVueJavaScriptJavaScriptHTMLHTML

updated: 2025-03-26 @ 09:12am, id: 271067003 / R_kgDOECgnew