3 results found Sort:
A next-generation crawling and spidering framework.
Created
2021-01-02
1,077 commits to main branch, last one 24 days ago
gocrawler, go分布式爬虫框架
Created
2024-02-09
34 commits to main branch, last one 5 months ago
分布式爬虫项目，本项目支持个性化定制页面解析器二次开发，项目整体采用微服务架构，通过消息队列实现消息的异步发送，使用到的框架包括：redigo, gorm, goquery, easyjson, viper, amqp, zap, go-micro，并通过Docker实现容器化部署，中间爬虫节点支持水平拓展。
Created
2020-08-17
129 commits to master branch, last one about a year ago