Search Results - RepositoryStats

awesome-semantic-segmentation mrgloom

2.5k

10.6k

unknown

441

:metal: awesome-semantic-segmentation

benchmark evaluation deeplearning semantic-segmentation

Created 2015-10-03

417 commits to master branch, last one 4 years ago

langfuse langfuse

911

9.9k

other

31

🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

llm llmops openai autogen analytics langchain evaluation monitoring playground llama-index open-source self-hosted ycombinator observability llm-evaluation llm-observability prompt-management prompt-engineering large-language-models

Created 2023-05-18

3,908 commits to main branch, last one 19 hours ago

ragas explodinggradients

878

8.6k

apache-2.0

42

Supercharge Your LLM Application Evaluations 🚀

llm llmops evaluation

Created 2023-05-08

791 commits to main branch, last one 5 days ago

promptfoo promptfoo

498

6.0k

mit

20

Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command ...

ci llm rag cicd ci-cd llmops prompts testing llm-eval evaluation pentesting red-teaming llm-evaluation prompt-testing prompt-engineering evaluation-framework vulnerability-scanners llm-evaluation-framework

Created 2023-04-28

4,071 commits to main branch, last one 7 hours ago

opencompass open-compass

529

5.1k

apache-2.0

27

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

llm llama2 llama3 openai chatgpt benchmark evaluation large-language-model

Created 2023-06-15

890 commits to main branch, last one a day ago

govaluate Knetic

512

3.9k

mit

63

Arbitrary expression evaluation for golang

go parsing evaluation expression

This repository has been archived (exclude archived)

Created 2014-12-19

312 commits to master branch, last one 7 days ago

AutoRAG Marker-Inc-Korea

291

3.8k

apache-2.0

30

AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation

qa llm ops rag automl python llm-ops analysis pipeline embeddings evaluation open-source benchmarking optimization llm-evaluation rag-evaluation document-parser retrieval-augmented-generation

Created 2024-01-10

847 commits to main branch, last one 29 days ago

evo MichaelGrupp

763

3.7k

gpl-3.0

48

Python package for the evaluation of odometry and SLAM

ros tum ros2 slam euroc kitti mapping metrics odometry robotics benchmark evaluation trajectory trajectory-analysis trajectory-evaluation

Created 2017-09-13

468 commits to master branch, last one 12 days ago

helicone Helicone

352

3.5k

apache-2.0

20

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

gpt llm llmops openai llm-cost analytics langchain evaluation monitoring playground llama-index open-source ycombinator llm-evaluation agent-monitoring llm-observability prompt-management prompt-engineering large-language-models

Created 2023-01-31

3,607 commits to main branch, last one 7 hours ago

write-you-a-haskell sdiehl

256

3.4k

mit

190

Building a modern functional compiler from first principles. (http://dev.stephendiehl.com/fun/)

book type haskel compiler pdf-book evaluation type-system type-theory type-checking hindley-milner type-inference lambda-calculus functional-language functional-programming intermediate-representation

Created 2015-01-05

299 commits to master branch, last one 7 years ago

Kiln Kiln-AI

230

3.3k

other

33

The easiest tool for fine-tuning LLM models, synthetic data generation, and collaborating on datasets.

ai ml rlhf evals macos ollama openai prompt python windows evaluation fine-tuning collaboration synthetic-data chain-of-thought machine-learning dataset-generation prompt-engineering

Created 2024-07-23

1,481 commits to main branch, last one a day ago

SuperCLUE CLUEbenchmark

104

3.1k

unknown

38

SuperCLUE: 中文通用大模型综合性基准 | A Benchmark for Foundation Models in Chinese

gpt-4 chatgpt chinese evaluation foundation-models

Created 2023-05-02

247 commits to main branch, last one 10 months ago

klipse viebel

148

3.1k

gpl-3.0

53

Klipse is a JavaScript plugin for embedding interactive code snippets in tech blogs.

lua ruby ocaml react prolog python scheme clojure reactjs reasonml brainfuck evaluation javascript common-lisp clojurescript klipse-plugin code-evaluation codemirror-editor interactive-snippets

Created 2015-11-19

1,028 commits to master branch, last one 6 months ago

Automatic_Speech_Recognition zzw922cn

534

2.8k

mit

145

End-to-end Automatic Speech Recognition for Madarian and English in Tensorflow

cnn rnn lstm audio paper phonemes end-to-end evaluation tensorflow deep-learning timit-dataset feature-vector data-preprocessing speech-recognition layer-normalization rnn-encoder-decoder chinese-speech-recognition automatic-speech-recognition

Created 2016-11-13

266 commits to master branch, last one 3 years ago

promptbench microsoft

190

2.6k

mit

18

A unified evaluation framework for large language models

prompt chatgpt benchmark evaluation robustness prompt-engineering adversarial-attacks large-language-models

Created 2023-06-13

259 commits to main branch, last one 6 months ago

ChainForge ianarawjo

206

2.6k

mit

30

An open-source visual programming environment for battle-testing prompts to LLMs.

ai llms llmops evaluation prompt-engineering large-language-models

Created 2023-03-26

396 commits to main branch, last one 5 days ago

lmms-eval EvolvingLMMs-Lab

234

2.3k

other

8

Accelerating the development of large multimodal models (LMMs) with one-click evaluation module - lmms-eval.

agi evaluation multimodal large-language-models

Created 2024-03-07

1,296 commits to main branch, last one 22 hours ago

uptrain uptrain-ai

199

2.3k

apache-2.0

19

UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform ro...

llmops llm-eval llm-test evaluation monitoring openai-evals llm-prompting autoevaluation experimentation machine-learning prompt-engineering jailbreak-detection root-cause-analysis hallucination-detection

Created 2022-11-07

770 commits to main branch, last one 8 months ago

evaluate huggingface

271

2.2k

apache-2.0

44

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

evaluation machine-learning

Created 2022-03-30

954 commits to main branch, last one 2 months ago

VLMEvalKit open-compass

308

2.1k

apache-2.0

12

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

gpt llm vit vqa clip gpt4 qwen llava claude gemini gpt-4v openai chatgpt pytorch evaluation openai-api multi-modal computer-vision large-language-models

Created 2023-12-01

1,260 commits to main branch, last one 20 hours ago

avalanche ContinualAI

305

1.9k

mit

30

Avalanche: an End-to-End Library for Continual Learning based on PyTorch.

library metrics pytorch training framework benchmarks evaluation strategies continualai deep-learning lifelong-learning continual-learning

Created 2020-03-05

3,948 commits to master branch, last one 21 days ago

EvalAI Cloud-CV

850

1.8k

other

52

:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI

ai django docker evalai python codecov angularjs challenge coveralls travis-ci evaluation leaderboard ai-challenges reproducibility machine-learning reproducible-research artificial-intelligence

Created 2016-10-21

2,456 commits to master branch, last one a day ago

lmnr lmnr-ai

103

1.8k

apache-2.0

11

Laminar - open-source all-in-one platform for engineering AI products. Crate data flywheel for you AI app. Traces, Evals, Datasets, Labels. YC S24.

ai rag aiops evals agents llmops analytics rust-lang evaluation monitoring open-source self-hosted llm-workflow observability llm-evaluation developer-tools ai-observability pipeline-builder llm-observability

Created 2024-08-29

408 commits to main branch, last one 16 days ago

AB3DMOT xinshuoweng

406

1.7k

other

50

(IROS 2020, ECCVW 2020) Official Python Implementation for "3D Multi-Object Tracking: A Baseline and New Evaluation Metrics"

kitti 3d-mot 3d-multi kitti-3d robotics tracking real-time evaluation 3d-tracking computer-vision machine-learning 2d-mot-evaluation evaluation-metrics multi-object-tracking 3d-multi-object-tracking

Created 2019-06-19

221 commits to master branch, last one about a year ago

alpaca_eval tatsu-lab

263

1.7k

apache-2.0

9

An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.

nlp rlhf evaluation leaderboard deep-learning foundation-models instruction-following large-language-models

Created 2023-05-25

595 commits to main branch, last one 3 months ago

LLM-eval-survey MLGroupJLU

92

1.5k

unknown

16

The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".

llm llms benchmark evaluation model-assessment large-language-models

Created 2023-07-02

256 commits to main branch, last one 10 months ago

pycm sepandhaghighi

126

1.5k

mit

35

Multi-class confusion matrix library in Python

Created 2018-01-22

3,082 commits to master branch, last one 2 months ago

Awesome-LLM-Long-Context-Modeling Xnhyacinth

47

1.4k

mit

54

📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

llm rag ssm agent blogs papers survey longcot compress benchmark evaluation awsome-list transformer long-term-memory length-extrapolation large-language-models long-context-modeling

Created 2023-09-17

227 commits to main branch, last one a day ago

nlg-eval Maluuba

224

1.4k

other

27

Evaluation code for various unsupervised automated metrics for Natural Language Generation.

nlg nlp bleu cider rouge dialog meteor rouge-l dialogue bleu-score evaluation skip-thoughts machine-translation skip-thought-vectors task-oriented-dialogue natural-language-generation natural-language-processing

Created 2017-06-27

87 commits to master branch, last one about a year ago

lighteval huggingface

212

1.4k

mit

28

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

evaluation huggingface evaluation-metrics evaluation-framework

Created 2024-01-26

351 commits to main branch, last one a day ago