adamdehaven / fetchurls

A bash script to spider a site, follow links, and fetch urls (with built-in filtering) into a generated text file.

adamdehaven
adamdehaven
Date Created 2018-01-13 (7 years ago)
Commits 50 (last one 3 years ago)
Stargazers 129 (0 this week)
Watchers 6 (0 this week)
Forks 44
License mit
Ranking

RepositoryStats indexes 636,587 repositories, of these adamdehaven/fetchurls is ranked #258,645 (59th percentile) for total stargazers, and #290,625 for total watchers. Github reports the primary language for this repository as Shell, for repositories using this language it is ranked #6,667/16,584.

adamdehaven/fetchurls is also tagged with popular topics, for these it's ranked: website (#408/1022),  spider (#217/347),  shell-script (#116/290)

Other Information

adamdehaven/fetchurls has Github issues enabled, there are 2 open issues and 11 closed issues.

There have been 15 releases, the latest one was published on 2021-12-26 (3 years ago) with the name wget --reject-regex.

Homepage URL: https://www.adamdehaven.com/blog/easily-crawl-a-website-and-fetch-all-urls-with-a-shell-script/

Star History

Github stargazers over time

14014012012010010080806060404020200020192019202020202021202120222022202320232024202420252025

Watcher History

Github watchers over time, collection started in '23

776.56.5665.55.5554.54.54420232023Jul '23Jul '2320242024Jul '24Jul '2420252025

Recent Commit History

0 commits on the default branch (master) since jan '22

Inactive

No recent commits to this repository

Yearly Commits

Commits to the default branch (master) per year

25252020151510105500201820182019201920202020202120212022202220242024

Issue History

Total Issues
Open Issues
Closed Issues
1414121210108866442200202020202021202120222022202320232024202420252025

Languages

The only known language in this repository is Shell

ShellShell

updated: 2025-03-07 @ 06:24am, id: 117303814 / R_kgDOBv3qBg