Search Results - RepositoryStats

sentencepiece google

1.2k

10.3k

apache-2.0

127

Unsupervised text tokenizer for Neural Network-based text generation.

word-segmentation neural-machine-translation natural-language-processing

Created 2017-03-07

986 commits to master branch, last one 3 months ago

lac baidu

595

3.9k

apache-2.0

105

百度NLP：分词，词性标注，命名实体识别，词重要性

java python chinese-nlp lexical-analysis word-segmentation part-of-speech-tagger named-entity-recognition chinese-word-segmentation

Created 2018-07-02

155 commits to master branch, last one 3 years ago

SymSpell wolfgarbe

298

3.2k

mit

71

SymSpell: 1 million times faster spelling correction & fuzzy search through Symmetric Delete spelling correction algorithm

spelling symspell spellcheck levenshtein spell-check fuzzy-search edit-distance fuzzy-matching text-segmentation word-segmentation damerau-levenshtein spelling-correction levenshtein-distance chinese-text-segmentation chinese-word-segmentation approximate-string-matching

Created 2014-03-25

488 commits to master branch, last one 26 days ago

pythainlp PyThaiNLP

274

988

apache-2.0

46

Thai natural language processing in Python

thai python soundex thai-nlp nlp-library thai-soundex hacktoberfest thai-language text-processing thai-nlp-library word-segmentation computational-linguistics natural-language-processing

Created 2016-06-23

4,785 commits to dev branch, last one 3 days ago

YouTokenToMe VKCOM

103

959

mit

26

Unsupervised text tokenizer focused on computational efficiency

bpe nlp tokenization word-segmentation natural-language-processing

This repository has been archived (exclude archived)

Created 2019-06-06

85 commits to master branch, last one about a year ago

symspellpy mammothb

122

803

mit

16

Python port of SymSpell: 1 million times faster spelling correction & fuzzy search through Symmetric Delete spelling correction algorithm

python spelling symspell spellcheck levenshtein spell-check fuzzy-search edit-distance fuzzy-matching text-segmentation word-segmentation damerau-levenshtein spelling-correction levenshtein-distance chinese-text-segmentation chinese-word-segmentation approximate-string-matching

Created 2018-08-13

292 commits to master branch, last one 5 days ago

ckip-transformers ckiplab

74

698

gpl-3.0

15

CKIP Transformers

ckip transformers language-model word-segmentation part-of-speech-tagging named-entity-recognition

Created 2020-08-17

116 commits to master branch, last one about a year ago

ekphrasis cbaziotis

90

662

mit

18

Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashta...

nlp semeval tokenizer nlp-library tokenization spell-corrector text-processing text-segmentation word-segmentation word-normalization spelling-correction

Created 2017-02-07

77 commits to master branch, last one 2 years ago

VnCoreNLP vncorenlp

146

593

other

31

A Vietnamese natural language processing toolkit (NAACL 2018)

ner nlp java parsing python3 vnmarmot vncorenlp pos-tagger vietnamese pos-tagging rdrsegmenter vietnamese-nlp word-segmenter word-segmentation dependency-parsing vietnamese-tokenizer sentence-segmentation named-entity-recognition natural-language-processing

Created 2017-12-30

46 commits to master branch, last one about a year ago

m3tl JayYip

125

546

apache-2.0

19

BERT for Multitask Learning

cws ner nlp bert transformer part-of-speech encoder-decoder pretrained-models word-segmentation multitask-learning multi-task-learning text-classification named-entity-recognition

Created 2018-11-29

535 commits to master branch, last one about a year ago

Kiwi bab2min

45

433

other

16

Kiwi(지능형 한국어 형태소 분석기)

cpp nlp korean korean-nlp morphology korean-tokenizer word-segmentation korean-text-processing morphological-analysis

Created 2017-02-21

1,117 commits to main branch, last one 2 days ago

AdaSeq modelscope

38

417

apache-2.0

13

AdaSeq: An All-in-One Library for Developing State-of-the-Art Sequence Understanding Models

crf ner nlp bert pytorch chinese-nlp entity-typing multi-modal-ner sequence-labeling word-segmentation relation-extraction token-classification information-extraction named-entity-recognition natural-language-processing natural-language-understanding

Created 2022-12-01

246 commits to master branch, last one about a year ago

nagisa taishi-i

22

391

mit

12

A Japanese tokenizer based on recurrent neural networks

nlp dynet japanese tokenizer nlp-library pos-tagging sequence-labeling word-segmentation natural-language-processing

Created 2018-02-14

194 commits to master branch, last one 5 months ago

jumanpp ku-nlp

44

381

apache-2.0

31

Juman++ (a Morphological Analyzer Toolkit)

cjk nlp juman japanese tokenizer pos-tagger pos-tagging word-segmentation part-of-speech-tagger morphological-analyser morphological-analysis

Created 2016-10-11

1,093 commits to master branch, last one about a year ago

pycantonese jacksonllee

39

360

mit

20

Cantonese Linguistics and NLP

nlp python jyutping cantonese stop-words linguistics pycantonese word-segmentation part-of-speech-tagging computational-linguistics natural-language-processing

Created 2014-12-13

328 commits to main branch, last one 6 months ago

Pytorch-NLU yongzhuo

51

337

apache-2.0

4

Pytorch-NLU，一个中文文本分类、序列标注工具包，支持中文长文本、短文本的多类、多标签分类任务，支持中文命名实体识别、词性标注、分词、抽取式文本摘要等序列标注任务。 Ptorch NLU, a Chinese text classification and sequence annotation toolkit, supports multi class and multi label c...

bert python3 pytorch pos-tagging transformers pretrained-models sequence-labeling word-segmentation text-classification named-entity-recognition chinese-text-segmentation chinese-text-classification

Created 2021-08-29

13 commits to main branch, last one 4 months ago

kiwipiepy bab2min

28

288

other

7

Python API for Kiwi

nlp korean korean-nlp python-library korean-tokenizer word-segmentation morphological-analysis

Created 2019-09-06

670 commits to main branch, last one 2 days ago

bi-lstm-crf jidasheng

48

246

mit

9

A PyTorch implementation of the BI-LSTM-CRF model.

crf ner nlp bilstm pytorch lstm-crf crf-model bilstm-crf bi-lstm-crf pos-tagging sequence-tagging sequence-labeling word-segmentation

Created 2019-12-03

6 commits to master branch, last one 24 days ago

monpa monpa-team

26

245

other

23

MONPA 罔拍是一個提供正體中文斷詞、詞性標註以及命名實體辨識的多任務模型

ner nlp pos bert albert pos-tagging word-segmentation named-entity-recognition chinese-word-segmentation

Created 2019-07-23

54 commits to master branch, last one 2 years ago

fastcws fastcws

8

198

bsd-2-clause

3

轻量级高性能中文分词项目

chinese wordseg wordbreak word-break nlp-chinese word-segment word-segmenter wordsegmentation word-segmentation hidden-markov-model frequency-dictionary

Created 2023-01-27

7 commits to devel branch, last one about a year ago

toiro taishi-i

9

119

apache-2.0

7

A comparison tool of Japanese tokenizers

nlp bert japanese nlp-library word-segmentation natural-language-processing

Created 2020-08-13

74 commits to master branch, last one about a year ago

ckipnlp ckiplab

15

115

gpl-3.0

9

CKIP CoreNLP Toolkits

nlp ckip sentence-parsing word-segmentation sentence-segmentation coreference-resolution part-of-speech-tagging named-entity-recognition

Created 2019-04-01

259 commits to master branch, last one about a year ago

nlp-roadmap Ailln

12

104

mit

4

🗺️ 一个自然语言处理的学习路线图

nlp roadmap word-embedding sequence-labeling word-segmentation natural-language-processing

Created 2019-04-17

42 commits to master branch, last one about a year ago

hanzi-tools peterolson

17

103

unlicense

8

Converts from Chinese characters to pinyin, between simplified and traditional, and does word segmentation.

pinyin word-segmentation chinese-characters simplified-chinese traditional-chinese

Created 2016-11-23

75 commits to master branch, last one 2 years ago

RDRsegmenter datquocnguyen

10

79

other

8

A Fast and Accurate Vietnamese Word Segmenter (LREC 2018)

vietnamese vietnamese-nlp word-segmentation vietnamese-tokenizer

Created 2017-09-19

29 commits to master branch, last one 2 years ago

WordSegmentationTM wolfgarbe

14

77

mit

5

Fast Word Segmentation with Triangular Matrix

symspell spellcheck spell-check spellchecker spell-checker spell-corrector spelling-checker text-segmentation word-segmentation spelling-corrector spelling-correction

Created 2018-04-21

33 commits to master branch, last one 3 years ago