Search Results - RepositoryStats

awesome-semantic-segmentation mrgloom

2.5k

10.5k

unknown

441

:metal: awesome-semantic-segmentation

benchmark evaluation deeplearning semantic-segmentation

Created 2015-10-03

417 commits to master branch, last one 3 years ago

ragas explodinggradients

737

7.2k

apache-2.0

37

Supercharge Your LLM Application Evaluations 🚀

llm llmops evaluation

Created 2023-05-08

670 commits to main branch, last one 19 hours ago

langfuse langfuse

620

6.5k

other

23

🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

Created 2023-05-18

2,966 commits to main branch, last one 13 hours ago

promptfoo promptfoo

373

4.7k

mit

21

Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command ...

ci llm rag cicd ci-cd llmops prompts testing llm-eval evaluation pentesting red-teaming llm-evaluation prompt-testing prompt-engineering evaluation-framework vulnerability-scanners llm-evaluation-framework

Created 2023-04-28

2,765 commits to main branch, last one 12 hours ago

opencompass open-compass

437

4.1k

apache-2.0

26

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

llm llama2 llama3 openai chatgpt benchmark evaluation large-language-model

Created 2023-06-15

762 commits to main branch, last one 22 hours ago

govaluate Knetic

503

3.8k

mit

64

Arbitrary expression evaluation for golang

go parsing evaluation expression

Created 2014-12-19

311 commits to master branch, last one 7 years ago

evo MichaelGrupp

755

3.5k

gpl-3.0

50

Python package for the evaluation of odometry and SLAM

ros tum ros2 slam euroc kitti mapping metrics odometry robotics benchmark evaluation trajectory trajectory-analysis trajectory-evaluation

Created 2017-09-13

448 commits to master branch, last one 15 days ago

write-you-a-haskell sdiehl

257

3.3k

mit

191

Building a modern functional compiler from first principles. (http://dev.stephendiehl.com/fun/)

book type haskel compiler pdf-book evaluation type-system type-theory type-checking hindley-milner type-inference lambda-calculus functional-language functional-programming intermediate-representation

Created 2015-01-05

299 commits to master branch, last one 7 years ago

klipse viebel

148

3.1k

gpl-3.0

55

Klipse is a JavaScript plugin for embedding interactive code snippets in tech blogs.

lua ruby ocaml react prolog python scheme clojure reactjs reasonml brainfuck evaluation javascript common-lisp clojurescript klipse-plugin code-evaluation codemirror-editor interactive-snippets

Created 2015-11-19

1,028 commits to master branch, last one about a month ago

SuperCLUE CLUEbenchmark

97

3k

unknown

39

SuperCLUE: 中文通用大模型综合性基准 | A Benchmark for Foundation Models in Chinese

gpt-4 chatgpt chinese evaluation foundation-models

Created 2023-05-02

247 commits to main branch, last one 5 months ago

Automatic_Speech_Recognition zzw922cn

537

2.8k

mit

146

End-to-end Automatic Speech Recognition for Madarian and English in Tensorflow

cnn rnn lstm audio paper phonemes end-to-end evaluation tensorflow deep-learning timit-dataset feature-vector data-preprocessing speech-recognition layer-normalization rnn-encoder-decoder chinese-speech-recognition automatic-speech-recognition

Created 2016-11-13

266 commits to master branch, last one 3 years ago

promptbench microsoft

182

2.5k

mit

20

A unified evaluation framework for large language models

prompt chatgpt benchmark evaluation robustness prompt-engineering adversarial-attacks large-language-models

Created 2023-06-13

259 commits to main branch, last one 2 months ago

ChainForge ianarawjo

178

2.3k

mit

30

An open-source visual programming environment for battle-testing prompts to LLMs.

ai llms llmops evaluation prompt-engineering large-language-models

Created 2023-03-26

371 commits to main branch, last one 16 days ago

uptrain uptrain-ai

191

2.2k

apache-2.0

20

UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform ro...

llmops llm-eval llm-test evaluation monitoring openai-evals llm-prompting autoevaluation experimentation machine-learning prompt-engineering jailbreak-detection root-cause-analysis hallucination-detection

Created 2022-11-07

770 commits to main branch, last one 3 months ago

evaluate huggingface

258

2.0k

apache-2.0

48

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

evaluation machine-learning

Created 2022-03-30

953 commits to main branch, last one about a month ago

helicone Helicone

213

1.9k

apache-2.0

9

🧊 Open source LLM-Observability Platform for Developers. One-line integration for monitoring, metrics, evals, agent tracing, prompt management, playground, etc. Supports OpenAI SDK, Vercel AI SDK, An...

gpt llm llmops openai llm-cost analytics langchain evaluation monitoring playground llama-index open-source ycombinator llm-evaluation agent-monitoring llm-observability prompt-management prompt-engineering large-language-models

Created 2023-01-31

2,947 commits to main branch, last one 14 hours ago

avalanche ContinualAI

290

1.8k

mit

30

Avalanche: an End-to-End Library for Continual Learning based on PyTorch.

library metrics pytorch training framework benchmarks evaluation strategies continualai deep-learning lifelong-learning continual-learning

Created 2020-03-05

3,929 commits to master branch, last one 16 days ago

EvalAI Cloud-CV

786

1.8k

other

54

:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI

ai django docker evalai python angular7 angularjs challenge evaluation leaderboard ai-challenges reproducibility machine-learning reproducible-research artificial-intelligence

Created 2016-10-21

2,426 commits to master branch, last one 2 months ago

AB3DMOT xinshuoweng

403

1.7k

other

49

(IROS 2020, ECCVW 2020) Official Python Implementation for "3D Multi-Object Tracking: A Baseline and New Evaluation Metrics"

kitti 3d-mot 3d-multi kitti-3d robotics tracking real-time evaluation 3d-tracking computer-vision machine-learning 2d-mot-evaluation evaluation-metrics multi-object-tracking 3d-multi-object-tracking

Created 2019-06-19

221 commits to master branch, last one about a year ago

alpaca_eval tatsu-lab

243

1.5k

apache-2.0

8

An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.

nlp rlhf evaluation leaderboard deep-learning foundation-models instruction-following large-language-models

Created 2023-05-25

589 commits to main branch, last one 4 days ago

pycm sepandhaghighi

124

1.5k

mit

36

Multi-class confusion matrix library in Python

Created 2018-01-22

3,072 commits to master branch, last one about a month ago

LLM-eval-survey MLGroupJLU

90

1.4k

unknown

13

The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".

llm llms benchmark evaluation model-assessment large-language-models

Created 2023-07-02

256 commits to main branch, last one 5 months ago

nlg-eval Maluuba

224

1.3k

other

28

Evaluation code for various unsupervised automated metrics for Natural Language Generation.

nlg nlp bleu cider rouge dialog meteor rouge-l dialogue bleu-score evaluation skip-thoughts machine-translation skip-thought-vectors task-oriented-dialogue natural-language-generation natural-language-processing

Created 2017-06-27

87 commits to master branch, last one 8 months ago

VLMEvalKit open-compass

188

1.3k

apache-2.0

10

Open-source evaluation toolkit of large vision-language models (LVLMs), support 160+ VLMs, 50+ benchmarks

gpt llm vit vqa clip gpt4 qwen llava claude gemini gpt-4v openai chatgpt pytorch evaluation openai-api multi-modal computer-vision large-language-models

Created 2023-12-01

1,006 commits to main branch, last one 24 hours ago

lispy abo-abo

132

1.2k

unknown

27

Short and sweet LISP editing

python scheme clojure emacs-lisp evaluation navigation common-lisp refactoring

Created 2014-01-10

2,707 commits to master branch, last one about a year ago

xai EthicalML

173

1.1k

mit

43

XAI - An eXplainability toolbox for machine learning

ai ml xai bias imbalance evaluation upsampling xai-library downsampling explainability explainable-ai explainable-ml bias-evaluation interpretability machine-learning feature-importance artificial-intelligence machine-learning-explainability

Created 2019-01-11

91 commits to master branch, last one 3 years ago

lmnr lmnr-ai

55

1.1k

apache-2.0

4

Laminar - open-source all-in-one platform for engineering AI products. Traces, Evals, Datasets, Labels. YC S24.

ai rag aiops evals agents llmops analytics rust-lang evaluation monitoring open-source self-hosted llm-workflow observability llm-evaluation developer-tools ai-observability pipeline-builder llm-observability

Created 2024-08-29

186 commits to main branch, last one a day ago

fuzzbench google

269

1.1k

apache-2.0

35

FuzzBench - Fuzzer benchmarking as a service.

fuzzing security evaluation benchmarking benchmark-framework

Created 2020-02-04

1,349 commits to master branch, last one about a month ago

lunary lunary-ai

131

1.1k

apache-2.0

6

The production toolkit for LLMs. Observability, prompt management and evaluations.

ai llm logs openai prompts testing langchain evaluation monitoring self-hosted hacktoberfest observability

Created 2023-05-12

1,339 commits to main branch, last one 2 days ago

torch-fidelity toshas

65

1.0k

other

7

High-fidelity performance metrics for generative models in PyTorch

gan metrics pytorch precision evaluation inception-score reproducibility generative-model reproducible-research perceptual-path-length kernel-inception-distance frechet-inception-distance

Created 2020-04-23

202 commits to master branch, last one 9 months ago