9 results found Sort:

dqbd
tiktokenizer dqbd
92
720
mit
6
Online playground for OpenAPI tokenizers
nextjs openai chatgpt t3-stack tiktoken tokenizer
Created 2023-03-02
55 commits to master branch, last one 29 days ago
pkoukk
tiktoken-go pkoukk
75
687
mit
6
go version of tiktoken
go gpt-4 golang openai chatgpt tiktoken gpt-35-turbo
Created 2023-03-08
64 commits to main branch, last one 7 months ago
jimmc414
1filellm jimmc414
63
641
mit
9
Specify a github or local repo, github pull request, arXiv or Sci-Hub paper, Youtube transcript or documentation URL on the web and scrape into a text file and clipboard for easier LLM ingestion
doi llm pdf pmid text arxiv ipynb github papers sci-hub tiktoken repository pull-request youtube-transcript-api
Created 2023-03-17
162 commits to main branch, last one 2 days ago
CNSeniorious000
free-chat CNSeniorious000
54
179
mit
4
An elegant LLM chat UI forked from chatgpt-demo of @anse-app. Index site at https://free-chat.asia
aibot astro openai chatgpt solidjs tiktoken openai-api
Created 2023-03-30
367 commits to promplate-demo branch, last one a day ago
saschaschramm
chatgpt saschaschramm
15
148
unknown
10
Analysis of OpenAI's ChatGPT
chat gpt-4 openai chatgpt plugins analysis tiktoken cl100kbase gpt-4-0613 gpt-35-turbo gpt-4-32k-0613 language-model gpt-35-turbo-0301 gpt-35-turbo-1106 chat-markup-language gpt-35-turbo-16k-0613 text-davinci-002-render text-davinci-002-plugins text-davinci-002-render-sha
Created 2022-12-03
64 commits to main branch, last one 10 months ago
openshieldai
openshield openshieldai
3
62
apache-2.0
2
OpenShield is a new generation security layer for AI models
ai llm llama owasp golang models openai python firewall guardian security tiktoken probllama tokenizer openai-api security-tools artificial-intelligence
Created 2024-06-05
783 commits to main branch, last one 10 days ago
tryAGI
Tiktoken tryAGI
3
61
mit
4
This project implements token calculation for OpenAI's gpt-4 and gpt-3.5-turbo model, specifically using `cl100k_base` encoding.
ai gpt4 csharp openai tokens chatgpt encoding p50kbase tiktoken langchain cl100kbase gpt35turbo tiktoken-sharp langchain-dotnet
Created 2023-06-09
143 commits to main branch, last one 5 days ago
ElmiraGhorbani
chatgpt-long-term-memory ElmiraGhorbani
3
57
unknown
2
The ChatGPT Long Term Memory package is a powerful tool designed to empower your projects with the ability to handle a large number of simultaneous users and external sources.
gpt-3 redis memory openai python vector chatbot chatgpt context tiktoken datastore embeddings chatgpt-api llama-index gpt-35-turbo text-retrieval long-term-memory similarity-search text-summarization embedding-similarity
Created 2023-07-19
53 commits to main branch, last one about a year ago
cahya-wirawan
rwkv-tokenizer cahya-wirawan
2
36
apache-2.0
2
A fast RWKV Tokenizer written in Rust
llm rwkv trie tiktoken tokenizer huggingface
Created 2024-05-29
86 commits to main branch, last one 3 months ago