6 results found Sort:

iipc
awesome-web-archiving iipc
153
1.9k
cc0-1.0
91
An Awesome List for getting started with web archiving
awesome awesome-list webarchiving
Created 2017-06-16
139 commits to main branch, last one about a month ago
akamhy
waybackpy akamhy
32
450
mit
10
Wayback Machine API interface & a command-line tool
osint cdx-api savepagenow webarchiving web-archiving archive-webpage wayback-machine archive-webpages internet-archive internet-archiving wayback-machine-api wayback-machine-python
Created 2020-05-02
497 commits to master branch, last one about a year ago
harvard-lil
warc-gpt harvard-lil
18
211
mit
12
WARC + AI - Experimental Retrieval Augmented Generation Pipeline for Web Archive Collections.
ai rag warc webarchiving
Created 2023-10-23
97 commits to main branch, last one 23 days ago
N0taN3rd
node-warc N0taN3rd
20
92
mit
9
Parse And Create Web ARChive (WARC) files with node.js
warc pupeteer warc-files webarchive web-archives webarchiving web-archiving chrome-remote-interface
Created 2017-05-21
114 commits to master branch, last one 5 years ago
ArchiveTeam
wget-lua ArchiveTeam
14
88
gpl-3.0
20
Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.
ftp lua warc wget zstd crawl spider crawler scraper crawlers crawling scraping wget-lua archiving downloader archiveteam webarchiving
Created 2019-09-07
4,428 commits to v1.21.3-at branch, last one 6 months ago
machawk1
awesome-memento machawk1
8
81
cc0-1.0
8
A list of things related to software, literature, and other content for 🕣 Memento
awesome memento memento-rfc awesome-list webarchiving
Created 2016-09-16
64 commits to main branch, last one about a month ago