2 results found Sort:

LanguageMachines
ucto LanguageMachines
13
65
gpl-3.0
13
Unicode tokeniser. Ucto tokenizes text files: it separates words from punctuation, and splits sentences. It offers several other basic preprocessing steps such as changing case that you can all use to...
nlp folia language tokeniser punctuation computational-linguistics natural-language-processing
Created 2013-03-26
1,579 commits to master branch, last one about a month ago
andreihar
taibun andreihar
1
25
mit
2
Taiwanese Hokkien Transliterator and Tokeniser
tl nlp poj taigi python zhuyin hokkien taiwanese tokeniser tokenizer nlp-library romanisation romanization tokenisation tokenization transliterator transliteration natural-language-processing
Created 2023-06-14
269 commits to main branch, last one 2 months ago