centic9 / CommonCrawlDocumentDownload

A small tool which uses the CommonCrawl URL Index to download documents with certain file types or mime-types. This is used for mass-testing of frameworks like Apache POI and Apache Tika

Date Created 2015-04-22 (9 years ago)

Commits 296 (last one 10 days ago)

Stargazers 63 (0 this week)

Watchers 13 (0 this week)

Forks 18

License bsd-2-clause

Ranking

RepositoryStats indexes 598,732 repositories, of these centic9/CommonCrawlDocumentDownload is ranked #403,796 (33rd percentile) for total stargazers, and #167,080 for total watchers. Github reports the primary language for this repository as Java, for repositories using this language it is ranked #21,403/28,658.

centic9/CommonCrawlDocumentDownload is also tagged with popular topics, for these it's ranked: java (#6,081/7782)

Other Information

There have been 6 releases, the latest one was published on 2023-01-15 (about a year ago) with the name 1.0.0.10.

All Topics

java warc cdx-files mime-types commoncrawl

Star History

Github stargazers over time

Watcher History

Github watchers over time, collection started in '23

Recent Commit History

89 commits on the default branch (master) since jan '22

Yearly Commits

Commits to the default branch (master) per year

Issue History

Languages

The primary language is Java but there's also others...

updated: 2024-12-21 @ 12:07pm, id: 34407138 / R_kgDOAg0C4g