16 results found Sort:

restic
restic restic
1.6k
26.7k
bsd-2-clause
239
Fast, secure, efficient backup program
go backup dedupe restic deduplication secure-by-default
Created 2014-04-27
8,693 commits to master branch, last one 9 days ago
dedupeio
dedupe dedupeio
551
4.1k
mit
120
:id: A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.
dedupe python datamade clustering de-duplicating dedupe-library python-library record-linkage entity-resolution
Created 2012-04-20
3,332 commits to main branch, last one 19 days ago
scinos
yarn-deduplicate scinos
55
1.4k
apache-2.0
22
Deduplication tool for yarn.lock files
yarn dedupe lock-file yarn-lock duplicates duplicated-packages
Created 2018-06-17
341 commits to master branch, last one about a year ago
J535D165
recordlinkage J535D165
152
967
bsd-3-clause
32
A powerful and modular toolkit for record linkage and duplicate detection in Python
dedupe python privacy similarity data-matching deduplication python-library record-linkage string-distance machine-learning entity-resolution utrecht-university
Created 2015-10-18
912 commits to master branch, last one about a year ago
zinggAI
zingg zinggAI
120
959
agpl-3.0
16
Scalable identity resolution, entity resolution, data mastering and deduplication using ML
ml etl spark dedupe datalake identity analytics fuzzymatch masterdata dataquality data-science deduplication fuzzy-matching dataengineering entity-resolution modern-data-stack data-transformation identity-resolution data-transformations analytics-engineering
Created 2021-08-25
2,218 commits to main branch, last one 20 hours ago
nil0x42
duplicut nil0x42
90
881
gpl-3.0
22
Remove duplicates from MASSIVE wordlist, without sorting it (for dictionary-based password cracking)
c uniq dedupe hashes unique hashcat cracking password wordlist wordlists dictionary password-cracking remove-duplicates wordlist-generator duplicate-detection
Created 2014-08-24
290 commits to master branch, last one 2 years ago
blakeembrey
free-style blakeembrey
29
707
mit
14
Make CSS easier and more maintainable by using JavaScript
js css hash dedupe css-in-js css-string javascript typescript minification
Created 2014-12-29
284 commits to master branch, last one 10 months ago
Zygo
bees Zygo
56
685
gpl-3.0
25
Best-Effort Extent-Same, a btrfs dedupe agent
btrfs dedup dedupe
Created 2016-11-17
664 commits to master branch, last one 7 months ago
dedupeio
dedupe-examples dedupeio
214
406
mit
25
:id: Examples for using the dedupe library
dedupe python record-linkage entity-resolution
Created 2014-04-02
1,356 commits to main branch, last one 3 months ago
knjcode
imgdupes knjcode
24
350
unknown
5
Identifying and removing near-duplicate images using perceptual hashing.
image dedupe deduplicate perceptual-hashes perceptual-hashing
Created 2018-05-24
161 commits to master branch, last one 4 years ago
kornelski
dupe-krill kornelski
10
186
mit
8
A fast file deduplicator
dupes macos dedupe hardlinks rust-library file-deduplication
Created 2017-03-31
91 commits to main branch, last one about a year ago
Lakshmipathi
dduper Lakshmipathi
18
168
gpl-2.0
12
Fast block-level out-of-band BTRFS deduplication tool.
btrfs dedupe btrfs-progs btrfs-tools deduplication
Created 2020-06-16
111 commits to master branch, last one 17 days ago
kdeldycke
mail-deduplicate kdeldycke
37
165
gpl-2.0
11
📧 CLI to deduplicate mails from mail boxes
mh cli mail mbox mmdf babyl email dedupe python cleanup mailbox maildir deduplication
Created 2013-03-25
1,484 commits to main branch, last one about a month ago
jason89521
daxus jason89521
1
95
mit
5
Daxus is a server state management library for React that provides full control over data, leading to a better user experience.
data hook cache react dedupe revalidate user-experience server-state-management
Created 2023-04-11
959 commits to main branch, last one 4 months ago
jRimbault
yadf jRimbault
0
55
mit
3
Yet Another Dupes Finder
dedupe fdupes dupes-finder deduplication file-deduplication duplicate-detection
Created 2020-08-03
304 commits to main branch, last one 16 days ago
Senzing
awesome Senzing
2
52
apache-2.0
9
Curated list of awesome software and resources for Senzing, The First Real-Time AI for Entity Resolution.
dedupe entity awesome senzing entities identity resource fuzzymatch resolution data-matching deduplication de-duplicating disambiguation entity-linking fuzzy-matching record-linkage entity-matching entity-resolution senzing-community identity-resolution
Created 2018-11-05
144 commits to main branch, last one 22 days ago