postmodern / spidr

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

Date Created 2009-03-08 (15 years ago)

Commits 886 (last one 10 months ago)

Stargazers 807 (0 this week)

Watchers 27 (0 this week)

Forks 106

License mit

Ranking

RepositoryStats indexes 585,332 repositories, of these postmodern/spidr is ranked #62,337 (89th percentile) for total stargazers, and #81,390 for total watchers. Github reports the primary language for this repository as Ruby, for repositories using this language it is ranked #1,193/7,666.

postmodern/spidr is also tagged with popular topics, for these it's ranked: ruby (#509/2374), web (#436/2086), scraper (#81/569), crawler (#134/566), spider (#89/333), web-scraping (#33/219)

Other Information

postmodern/spidr has 3 open pull requests on Github, 13 pull requests have been merged over the lifetime of the repository.

Github issues are enabled, there are 13 open issues and 50 closed issues.

All Topics

web ruby spider crawler scraper web-spider web-crawler web-scraper spider-links web-scraping

Star History

Github stargazers over time

Watcher History

Github watchers over time, collection started in '23

Recent Commit History

72 commits on the default branch (master) since jan '22

Yearly Commits

Commits to the default branch (master) per year

Issue History

Languages

The only known language in this repository is Ruby

updated: 2024-11-22 @ 09:56pm, id: 145726 / R_kgDOAAI5Pg