16 results found Sort:

restic
restic restic
1.6k
27.1k
bsd-2-clause
240
Fast, secure, efficient backup program
go backup dedupe restic deduplication secure-by-default
Created 2014-04-27
8,739 commits to master branch, last one 7 days ago
dedupeio
dedupe dedupeio
552
4.2k
mit
119
:id: A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.
dedupe python datamade clustering de-duplicating dedupe-library python-library record-linkage entity-resolution
Created 2012-04-20
3,332 commits to main branch, last one about a month ago
scinos
yarn-deduplicate scinos
55
1.4k
apache-2.0
22
Deduplication tool for yarn.lock files
yarn dedupe lock-file yarn-lock duplicates duplicated-packages
Created 2018-06-17
341 commits to master branch, last one about a year ago
J535D165
recordlinkage J535D165
153
975
bsd-3-clause
32
A powerful and modular toolkit for record linkage and duplicate detection in Python
dedupe python privacy similarity data-matching deduplication python-library record-linkage string-distance machine-learning entity-resolution utrecht-university
Created 2015-10-18
912 commits to master branch, last one about a year ago
zinggAI
zingg zinggAI
122
968
agpl-3.0
16
Scalable identity resolution, entity resolution, data mastering and deduplication using ML
ml etl spark dedupe datalake identity analytics fuzzymatch masterdata dataquality data-science deduplication fuzzy-matching dataengineering entity-resolution modern-data-stack data-transformation identity-resolution data-transformations analytics-engineering
Created 2021-08-25
2,354 commits to main branch, last one a day ago
nil0x42
duplicut nil0x42
90
887
gpl-3.0
22
Remove duplicates from MASSIVE wordlist, without sorting it (for dictionary-based password cracking)
c uniq dedupe hashes unique hashcat cracking password wordlist wordlists dictionary password-cracking remove-duplicates wordlist-generator duplicate-detection
Created 2014-08-24
290 commits to master branch, last one 2 years ago
blakeembrey
free-style blakeembrey
29
707
mit
14
Make CSS easier and more maintainable by using JavaScript
js css hash dedupe css-in-js css-string javascript typescript minification
Created 2014-12-29
284 commits to master branch, last one 11 months ago
Zygo
bees Zygo
56
704
gpl-3.0
26
Best-Effort Extent-Same, a btrfs dedupe agent
btrfs dedup dedupe
Created 2016-11-17
726 commits to master branch, last one 17 days ago
dedupeio
dedupe-examples dedupeio
214
408
mit
25
:id: Examples for using the dedupe library
dedupe python record-linkage entity-resolution
Created 2014-04-02
1,356 commits to main branch, last one 4 months ago
knjcode
imgdupes knjcode
24
353
unknown
5
Identifying and removing near-duplicate images using perceptual hashing.
image dedupe deduplicate perceptual-hashes perceptual-hashing
Created 2018-05-24
161 commits to master branch, last one 4 years ago
kornelski
dupe-krill kornelski
10
187
mit
8
A fast file deduplicator
dupes macos dedupe hardlinks rust-library file-deduplication
Created 2017-03-31
91 commits to main branch, last one about a year ago
kdeldycke
mail-deduplicate kdeldycke
39
170
gpl-2.0
11
📧 CLI to deduplicate mails from mail boxes
mh cli mail mbox mmdf babyl email dedupe python cleanup mailbox maildir deduplication
Created 2013-03-25
1,517 commits to main branch, last one 27 days ago
Lakshmipathi
dduper Lakshmipathi
18
170
gpl-2.0
12
Fast block-level out-of-band BTRFS deduplication tool.
btrfs dedupe btrfs-progs btrfs-tools deduplication
Created 2020-06-16
111 commits to master branch, last one about a month ago
jason89521
daxus jason89521
1
95
mit
5
Daxus is a server state management library for React that provides full control over data, leading to a better user experience.
data hook cache react dedupe revalidate user-experience server-state-management
Created 2023-04-11
959 commits to main branch, last one 5 months ago
jRimbault
yadf jRimbault
0
55
mit
3
Yet Another Dupes Finder
dedupe fdupes dupes-finder deduplication file-deduplication duplicate-detection
Created 2020-08-03
304 commits to main branch, last one about a month ago
Senzing
awesome Senzing
2
53
apache-2.0
8
Curated list of awesome software and resources for Senzing, The First Real-Time AI for Entity Resolution.
dedupe entity awesome senzing entities identity resource fuzzymatch resolution data-matching deduplication de-duplicating disambiguation entity-linking fuzzy-matching record-linkage entity-matching entity-resolution senzing-community identity-resolution
Created 2018-11-05
146 commits to main branch, last one 13 days ago