3 results found Sort:

lionsoul2014
friso lionsoul2014
91
495
apache-2.0
32
High performance Chinese tokenizer with both GBK and UTF-8 charset support based on MMSEG algorithm developed by ANSI C. Completely based on modular implementation and can be easily embedded in other...
c tokenizer cjk-tokenizer php-tokenizer full-text-search korean-tokenizer chinese-tokenizer japanese-tokenizer chinese-word-segmentation
Created 2014-03-31
148 commits to master branch, last one about a year ago
howl-anderson
MicroTokenizer howl-anderson
22
149
mit
8
一个轻量且功能全面的中文分词器，帮助学生了解分词器的工作原理。MicroTokenizer: A lightweight Chinese tokenizer designed for educational and research purposes. Provides a practical, hands-on approach to understanding NLP concepts, fe...
tokenizer chinese-nlp dag-network chinese-tokenizer educational-project nlp-machine-learning chinese-word-segmentation
Created 2018-06-12
396 commits to master branch, last one 5 months ago
samzshi0529
HanziNLP samzshi0529
3
25
mit
1
A NLP package for Chinese text：Preprocessing, Tokenization, Chinese Fonts, Word Embeddings, Text Similarity and Sentiment Analysis 轻量级中文自然语言处理软件包
nlp hanzi chinese hanzi-pinyin tokenization machine-learning chinese-tokenizer sentiment-analysis nlp-machine-learning
Created 2023-10-07
198 commits to main branch, last one 4 months ago