16 results found Sort:

restic
restic restic
1.5k
24.1k
bsd-2-clause
239
Fast, secure, efficient backup program
go backup dedupe restic deduplication secure-by-default
Created 2014-04-27
8,032 commits to master branch, last one a day ago
borgbackup
borg borgbackup
729
10.7k
other
150
Deduplicating archiver with compression and authenticated encryption.
c ssh backup cython dedupe python python-3 borgbackup encryption compression deduplication
Created 2015-05-12
8,433 commits to master branch, last one 16 hours ago
dedupeio
dedupe dedupeio
543
4.0k
mit
120
:id: A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.
dedupe python datamade clustering de-duplicating dedupe-library python-library record-linkage entity-resolution
Created 2012-04-20
3,303 commits to main branch, last one 5 months ago
scinos
yarn-deduplicate scinos
55
1.4k
apache-2.0
23
Deduplication tool for yarn.lock files
yarn dedupe lock-file yarn-lock duplicates duplicated-packages
Created 2018-06-17
341 commits to master branch, last one about a year ago
J535D165
recordlinkage J535D165
150
916
bsd-3-clause
32
A powerful and modular toolkit for record linkage and duplicate detection in Python
dedupe python privacy similarity data-matching deduplication python-library record-linkage string-distance machine-learning entity-resolution utrecht-university
Created 2015-10-18
912 commits to master branch, last one 10 months ago
zinggAI
zingg zinggAI
109
903
agpl-3.0
18
Scalable identity resolution, entity resolution, data mastering and deduplication using ML
ml etl spark dedupe datalake identity analytics fuzzymatch masterdata dataquality data-science deduplication fuzzy-matching dataengineering entity-resolution modern-data-stack data-transformation identity-resolution data-transformations analytics-engineering
Created 2021-08-25
1,818 commits to main branch, last one 17 days ago
nil0x42
duplicut nil0x42
90
799
gpl-3.0
22
Remove duplicates from MASSIVE wordlist, without sorting it (for dictionary-based password cracking)
c uniq dedupe hashes unique hashcat cracking password wordlist wordlists dictionary password-cracking remove-duplicates wordlist-generator duplicate-detection
Created 2014-08-24
290 commits to master branch, last one about a year ago
blakeembrey
free-style blakeembrey
29
701
mit
14
Make CSS easier and more maintainable by using JavaScript
js css hash dedupe css-in-js css-string javascript typescript minification
Created 2014-12-29
284 commits to master branch, last one 4 months ago
Zygo
bees Zygo
55
603
gpl-3.0
20
Best-Effort Extent-Same, a btrfs dedupe agent
btrfs dedup dedupe
Created 2016-11-17
664 commits to master branch, last one about a month ago
dedupeio
dedupe-examples dedupeio
216
395
mit
26
:id: Examples for using the dedupe library
dedupe python record-linkage entity-resolution
Created 2014-04-02
1,348 commits to master branch, last one 2 years ago
knjcode
imgdupes knjcode
24
336
unknown
5
Identifying and removing near-duplicate images using perceptual hashing.
image dedupe deduplicate perceptual-hashes perceptual-hashing
Created 2018-05-24
161 commits to master branch, last one 4 years ago
kornelski
dupe-krill kornelski
10
183
mit
8
A fast file deduplicator
dupes macos dedupe hardlinks rust-library file-deduplication
Created 2017-03-31
91 commits to main branch, last one 9 months ago
Lakshmipathi
dduper Lakshmipathi
18
163
gpl-2.0
11
Fast block-level out-of-band BTRFS deduplication tool.
btrfs dedupe btrfs-progs btrfs-tools deduplication
Created 2020-06-16
110 commits to master branch, last one 6 months ago
kdeldycke
mail-deduplicate kdeldycke
38
159
gpl-2.0
10
📧 CLI to deduplicate mails from mail boxes.
mh cli mail mbox mmdf babyl email dedupe python cleanup mailbox maildir deduplication
Created 2013-03-25
1,420 commits to main branch, last one a day ago
jason89521
daxus jason89521
0
90
mit
5
Daxus is a server state management library for React that provides full control over data, leading to a better user experience.
data hook cache react dedupe revalidate user-experience server-state-management
Created 2023-04-11
945 commits to main branch, last one 10 days ago
jRimbault
yadf jRimbault
0
50
mit
3
Yet Another Dupes Finder
dedupe fdupes dupes-finder deduplication file-deduplication duplicate-detection
Created 2020-08-03
296 commits to main branch, last one 3 months ago