Search Results - RepositoryStats

520

6.9k

apache-2.0

51

Open source real-time translation app for Android that runs locally

nllb onnx android offline whisper mobile-app translator android-app onnxruntime translation bluetooth-le transformers sentencepiece realtime-translator

Created 2020-03-30

262 commits to v2.00 branch, last one 10 days ago

70

286

mit

19

Fast and customizable text tokenization library with BPE and SentencePiece support

bpe cpp icu python unicode tokenizer tokenization sentencepiece machine-translation natural-language-processing

Created 2017-02-14

601 commits to master branch, last one about a year ago

25

232

mit

7

🌿 An easy-to-use Japanese Text Processing tool, which makes it possible to switch tokenizers with small changes of code.

nlp kytea mecab janome sudachi japanese sentencepiece text-processing natural-language-processing

Created 2018-08-22

431 commits to main branch, last one 6 months ago

16

111

unknown

2

使用sentencepiece中BPE训练中文词表，并在transformers中进行使用。

tokenization chinese-vocab sentencepiece

Created 2023-06-24

2 commits to main branch, last one about a year ago