ZSAIm / VideoCrawlerEngine

起源于旧项目爱奇艺解析器(iqiyi-parser)在开发、维护和扩展的过程中遇到的一些问题，而实现的一个基于任务流式的可视化爬虫引擎。引擎的执行单元是节点。脚本节点(script)作为根节点来完成对节点和流程的描述，经由任务节点(task)解析流程描述并生成节点的执行队列，最后交由工作者执行池处理。整个过程可视可控，所有节点处理器都以插件的形式导入，以最大程度实现易扩展性。

ZSAIm
ZSAIm
Date Created 2020-06-09 (4 years ago)
Commits 24 (last one 3 years ago)
Stargazers 462 (0 this week)
Watchers 16 (0 this week)
Forks 90
License unknown
Ranking

RepositoryStats indexes 597,394 repositories, of these ZSAIm/VideoCrawlerEngine is ranked #97,343 (84th percentile) for total stargazers, and #138,081 for total watchers. Github reports the primary language for this repository as Python, for repositories using this language it is ranked #16,236/119,856.

ZSAIm/VideoCrawlerEngine is also tagged with popular topics, for these it's ranked: python (#5,019/22362)

Other Information

ZSAIm/VideoCrawlerEngine has 21 open pull requests on Github, 6 pull requests have been merged over the lifetime of the repository.

Github issues are enabled, there are 4 open issues and 6 closed issues.

All Topics
python taskflow

Star History

Github stargazers over time

Watcher History

Github watchers over time, collection started in '23

Recent Commit History

0 commits on the default branch (master) since jan '22

Inactive

No recent commits to this repository

Yearly Commits

Commits to the default branch (master) per year

Issue History

Languages

The primary language is Python but there's also others...

updated: 2024-12-23 @ 06:46pm, id: 271067003 / R_kgDOECgnew