adamdehaven / fetchurls

A bash script to spider a site, follow links, and fetch urls (with built-in filtering) into a generated text file.

Date Created 2018-01-13 (7 years ago)
Commits 50 (last one 3 years ago)
Stargazers 129 (0 this week)
Watchers 7 (0 this week)
Forks 44
License mit
Ranking

RepositoryStats indexes 622,366 repositories, of these adamdehaven/fetchurls is ranked #254,530 (59th percentile) for total stargazers, and #274,768 for total watchers. Github reports the primary language for this repository as Shell, for repositories using this language it is ranked #6,566/16,265.

adamdehaven/fetchurls is also tagged with popular topics, for these it's ranked: website (#399/1006),  spider (#215/345),  shell-script (#113/287)

Other Information

adamdehaven/fetchurls has Github issues enabled, there are 2 open issues and 11 closed issues.

There have been 15 releases, the latest one was published on 2021-12-26 (3 years ago) with the name wget --reject-regex.

Homepage URL: https://www.adamdehaven.com/blog/easily-crawl-a-website-and-fetch-all-urls-with-a-shell-script/

Star History

Github stargazers over time

14014012012010010080806060404020200020192019202020202021202120222022202320232024202420252025

Watcher History

Github watchers over time, collection started in '23

776.56.5665.55.5554.54.54420232023Feb '23Feb '23Apr '23Apr '23Jun '23Jun '23Aug '23Aug '23Oct '23Oct '23Dec '23Dec '23Feb '24Feb '24Apr '24Apr '24Jun '24Jun '24Aug '24Aug '24Oct '24Oct '24Dec '24Dec '24Feb '25Feb '25

Recent Commit History

0 commits on the default branch (master) since jan '22

Inactive

No recent commits to this repository

Yearly Commits

Commits to the default branch (master) per year

25252020151510105500201820182019201920202020202120212022202220242024

Issue History

Total Issues
Open Issues
Closed Issues
1414121210108866442200202020202021202120222022202320232024202420252025

Languages

The only known language in this repository is Shell

ShellShell

updated: 2025-02-13 @ 08:35pm, id: 117303814 / R_kgDOBv3qBg