georg-jung / FastBertTokenizer

Fast and memory-efficient library for WordPiece tokenization as it is used by BERT.

Date Created 2023-09-13 (about a year ago)
Commits 195 (last one 11 days ago)
Stargazers 43 (0 this week)
Watchers 3 (0 this week)
Forks 10
License mit
Ranking

RepositoryStats indexes 612,937 repositories, of these georg-jung/FastBertTokenizer is ranked #517,033 (16th percentile) for total stargazers, and #432,024 for total watchers. Github reports the primary language for this repository as C#, for repositories using this language it is ranked #18,619/21,662.

georg-jung/FastBertTokenizer is also tagged with popular topics, for these it's ranked: machine-learning (#7,273/8233),  ai (#3,502/4440),  llm (#2,528/3192),  nlp (#2,191/2474),  natural-language-processing (#1,292/1452),  bert (#523/567)

Other Information

georg-jung/FastBertTokenizer has 6 open pull requests on Github, 0 pull requests have been merged over the lifetime of the repository.

Github issues are enabled, there is 1 open issue and 7 closed issues.

There have been 5 releases, the latest one was published on 2024-04-30 (9 months ago) with the name v1.0.28.

Homepage URL: https://fastberttokenizer.gjung.com/

Star History

Github stargazers over time

454540403535303025252020151510105500Nov '23Nov '23Dec '23Dec '2320242024Feb '24Feb '24Mar '24Mar '24Apr '24Apr '24May '24May '24Jun '24Jun '24Jul '24Jul '24Aug '24Aug '24Sep '24Sep '24Oct '24Oct '24Nov '24Nov '24Dec '24Dec '2420252025Feb '25Feb '25

Watcher History

Github watchers over time, collection started in '23

3333332.52.5222222Jun '24Jun '24Jul '24Jul '24Aug '24Aug '24Sep '24Sep '24Oct '24Oct '24Nov '24Nov '24Dec '24Dec '2420252025Feb '25Feb '25

Recent Commit History

195 commits on the default branch (master) since jan '22

200200180180160160140140120120100100808060604040202000Oct '23Oct '23Nov '23Nov '23Dec '23Dec '2320242024Feb '24Feb '24Mar '24Mar '24Apr '24Apr '24May '24May '24Jun '24Jun '24Jul '24Jul '24Aug '24Aug '24Sep '24Sep '24Oct '24Oct '24Nov '24Nov '24Dec '24Dec '2420252025Feb '25Feb '25

Yearly Commits

Commits to the default branch (master) per year

353530302525202015151010550020242024

Issue History

Total Issues
Open Issues
Closed Issues
887766554433221100Oct '23Oct '23Nov '23Nov '23Dec '23Dec '2320242024Feb '24Feb '24Mar '24Mar '24Apr '24Apr '24May '24May '24Jun '24Jun '24Jul '24Jul '24Aug '24Aug '24Sep '24Sep '24Oct '24Oct '24Nov '24Nov '24Dec '24Dec '2420252025Feb '25Feb '25

Languages

The primary language is C# but there's also others...

C#C#RustRustPowerShellPowerShellPythonPythonDockerfileDockerfile

updated: 2025-02-06 @ 07:46pm, id: 690963846 / R_kgDOKS9Fhg