2 results found Sort:

lionsoul2014
friso lionsoul2014
93
483
apache-2.0
33
High performance Chinese tokenizer with both GBK and UTF-8 charset support based on MMSEG algorithm developed by ANSI C. Completely based on modular implementation and can be easily embedded in other...
c tokenizer cjk-tokenizer php-tokenizer full-text-search korean-tokenizer chinese-tokenizer japanese-tokenizer chinese-word-segmentation
Created 2014-03-31
148 commits to master branch, last one about a year ago
howl-anderson
MicroTokenizer howl-anderson
22
147
mit
9
一个轻量且功能全面的中文分词器，帮助学生了解分词器的工作原理。MicroTokenizer: A lightweight Chinese tokenizer designed for educational and research purposes. Provides a practical, hands-on approach to understanding NLP concepts, fe...
tokenizer chinese-nlp dag-network chinese-tokenizer educational-project nlp-machine-learning chinese-word-segmentation
Created 2018-06-12
396 commits to master branch, last one about a month ago