Search Results - RepositoryStats

368

5.0k

apache-2.0

36

Open source real-time translation app for Android that runs locally

nllb onnx nllb200 whisper mobile-app translator whisper-ai android-app onnxruntime translation bluetooth-le transformers sentencepiece realtime-translator

Created 2020-03-30

185 commits to v2.00 branch, last one 20 hours ago

67

269

mit

19

Fast and customizable text tokenization library with BPE and SentencePiece support

bpe cpp icu python unicode tokenizer tokenization sentencepiece machine-translation natural-language-processing

Created 2017-02-14

601 commits to master branch, last one 11 months ago

22

222

mit

7

🌿 An easy-to-use Japanese Text Processing tool, which makes it possible to switch tokenizers with small changes of code.

nlp kytea mecab janome sudachi japanese sentencepiece text-processing natural-language-processing

Created 2018-08-22

431 commits to main branch, last one about a month ago

14

97

unknown

2

使用sentencepiece中BPE训练中文词表，并在transformers中进行使用。

tokenization chinese-vocab sentencepiece

Created 2023-06-24

2 commits to main branch, last one about a year ago