Search Results - RepositoryStats

awesome-semantic-segmentation mrgloom

2.5k

10.6k

unknown

442

:metal: awesome-semantic-segmentation

benchmark evaluation deeplearning semantic-segmentation

Created 2015-10-03

417 commits to master branch, last one 3 years ago

ragas explodinggradients

771

7.6k

apache-2.0

38

Supercharge Your LLM Application Evaluations 🚀

llm llmops evaluation

Created 2023-05-08

705 commits to main branch, last one 2 days ago

langfuse langfuse

667

7.2k

other

28

🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

Created 2023-05-18

3,236 commits to main branch, last one 22 hours ago

promptfoo promptfoo

405

5.0k

mit

21

Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command ...

ci llm rag cicd ci-cd llmops prompts testing llm-eval evaluation pentesting red-teaming llm-evaluation prompt-testing prompt-engineering evaluation-framework vulnerability-scanners llm-evaluation-framework

Created 2023-04-28

3,116 commits to main branch, last one 16 hours ago

opencompass open-compass

457

4.3k

apache-2.0

26

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

llm llama2 llama3 openai chatgpt benchmark evaluation large-language-model

Created 2023-06-15

807 commits to main branch, last one a day ago

govaluate Knetic

507

3.8k

mit

64

Arbitrary expression evaluation for golang

go parsing evaluation expression

Created 2014-12-19

311 commits to master branch, last one 7 years ago

evo MichaelGrupp

760

3.5k

gpl-3.0

49

Python package for the evaluation of odometry and SLAM

ros tum ros2 slam euroc kitti mapping metrics odometry robotics benchmark evaluation trajectory trajectory-analysis trajectory-evaluation

Created 2017-09-13

453 commits to master branch, last one 3 days ago

write-you-a-haskell sdiehl

254

3.4k

mit

191

Building a modern functional compiler from first principles. (http://dev.stephendiehl.com/fun/)

book type haskel compiler pdf-book evaluation type-system type-theory type-checking hindley-milner type-inference lambda-calculus functional-language functional-programming intermediate-representation

Created 2015-01-05

299 commits to master branch, last one 7 years ago

klipse viebel

147

3.1k

gpl-3.0

54

Klipse is a JavaScript plugin for embedding interactive code snippets in tech blogs.

lua ruby ocaml react prolog python scheme clojure reactjs reasonml brainfuck evaluation javascript common-lisp clojurescript klipse-plugin code-evaluation codemirror-editor interactive-snippets

Created 2015-11-19

1,028 commits to master branch, last one 2 months ago

SuperCLUE CLUEbenchmark

97

3.0k

unknown

39

SuperCLUE: 中文通用大模型综合性基准 | A Benchmark for Foundation Models in Chinese

gpt-4 chatgpt chinese evaluation foundation-models

Created 2023-05-02

247 commits to main branch, last one 6 months ago

AutoRAG Marker-Inc-Korea

228

3.0k

apache-2.0

23

AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation

qa llm ops rag automl python llm-ops analysis pipeline embeddings evaluation open-source benchmarking optimization llm-evaluation rag-evaluation document-parser retrieval-augmented-generation

Created 2024-01-10

835 commits to main branch, last one a day ago

Automatic_Speech_Recognition zzw922cn

536

2.8k

mit

146

End-to-end Automatic Speech Recognition for Madarian and English in Tensorflow

cnn rnn lstm audio paper phonemes end-to-end evaluation tensorflow deep-learning timit-dataset feature-vector data-preprocessing speech-recognition layer-normalization rnn-encoder-decoder chinese-speech-recognition automatic-speech-recognition

Created 2016-11-13

266 commits to master branch, last one 3 years ago

promptbench microsoft

184

2.5k

mit

20

A unified evaluation framework for large language models

prompt chatgpt benchmark evaluation robustness prompt-engineering adversarial-attacks large-language-models

Created 2023-06-13

259 commits to main branch, last one 3 months ago

ChainForge ianarawjo

188

2.4k

mit

30

An open-source visual programming environment for battle-testing prompts to LLMs.

ai llms llmops evaluation prompt-engineering large-language-models

Created 2023-03-26

373 commits to main branch, last one a day ago

uptrain uptrain-ai

193

2.2k

apache-2.0

21

UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform ro...

llmops llm-eval llm-test evaluation monitoring openai-evals llm-prompting autoevaluation experimentation machine-learning prompt-engineering jailbreak-detection root-cause-analysis hallucination-detection

Created 2022-11-07

770 commits to main branch, last one 4 months ago

helicone Helicone

250

2.2k

apache-2.0

10

🧊 Open source LLM-Observability Platform for Developers. One-line integration for monitoring, metrics, evals, agent tracing, prompt management, playground, etc. Supports OpenAI SDK, Vercel AI SDK, An...

gpt llm llmops openai llm-cost analytics langchain evaluation monitoring playground llama-index open-source ycombinator llm-evaluation agent-monitoring llm-observability prompt-management prompt-engineering large-language-models

Created 2023-01-31

3,061 commits to main branch, last one 18 hours ago

evaluate huggingface

263

2.1k

apache-2.0

47

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

evaluation machine-learning

Created 2022-03-30

953 commits to main branch, last one 3 months ago

avalanche ContinualAI

295

1.8k

mit

30

Avalanche: an End-to-End Library for Continual Learning based on PyTorch.

library metrics pytorch training framework benchmarks evaluation strategies continualai deep-learning lifelong-learning continual-learning

Created 2020-03-05

3,929 commits to master branch, last one about a month ago

EvalAI Cloud-CV

800

1.8k

other

54

:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI

ai django docker evalai python angular7 angularjs challenge evaluation leaderboard ai-challenges reproducibility machine-learning reproducible-research artificial-intelligence

Created 2016-10-21

2,426 commits to master branch, last one 3 months ago

AB3DMOT xinshuoweng

403

1.7k

other

49

(IROS 2020, ECCVW 2020) Official Python Implementation for "3D Multi-Object Tracking: A Baseline and New Evaluation Metrics"

kitti 3d-mot 3d-multi kitti-3d robotics tracking real-time evaluation 3d-tracking computer-vision machine-learning 2d-mot-evaluation evaluation-metrics multi-object-tracking 3d-multi-object-tracking

Created 2019-06-19

221 commits to master branch, last one about a year ago

alpaca_eval tatsu-lab

245

1.6k

apache-2.0

8

An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.

nlp rlhf evaluation leaderboard deep-learning foundation-models instruction-following large-language-models

Created 2023-05-25

589 commits to main branch, last one about a month ago

VLMEvalKit open-compass

212

1.5k

apache-2.0

11

Open-source evaluation toolkit of large vision-language models (LVLMs), support 160+ VLMs, 50+ benchmarks

gpt llm vit vqa clip gpt4 qwen llava claude gemini gpt-4v openai chatgpt pytorch evaluation openai-api multi-modal computer-vision large-language-models

Created 2023-12-01

1,084 commits to main branch, last one a day ago

pycm sepandhaghighi

124

1.5k

mit

36

Multi-class confusion matrix library in Python

Created 2018-01-22

3,072 commits to master branch, last one 2 months ago

LLM-eval-survey MLGroupJLU

92

1.4k

unknown

14

The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".

llm llms benchmark evaluation model-assessment large-language-models

Created 2023-07-02

256 commits to main branch, last one 6 months ago

nlg-eval Maluuba

224

1.4k

other

28

Evaluation code for various unsupervised automated metrics for Natural Language Generation.

nlg nlp bleu cider rouge dialog meteor rouge-l dialogue bleu-score evaluation skip-thoughts machine-translation skip-thought-vectors task-oriented-dialogue natural-language-generation natural-language-processing

Created 2017-06-27

87 commits to master branch, last one 9 months ago

lmnr lmnr-ai

68

1.3k

apache-2.0

5

Laminar - open-source all-in-one platform for engineering AI products. Crate data flywheel for you AI app. Traces, Evals, Datasets, Labels. YC S24.

ai rag aiops evals agents llmops analytics rust-lang evaluation monitoring open-source self-hosted llm-workflow observability llm-evaluation developer-tools ai-observability pipeline-builder llm-observability

Created 2024-08-29

240 commits to main branch, last one 13 days ago

lispy abo-abo

132

1.2k

unknown

27

Short and sweet LISP editing

python scheme clojure emacs-lisp evaluation navigation common-lisp refactoring

Created 2014-01-10

2,707 commits to master branch, last one about a year ago

xai EthicalML

174

1.1k

mit

44

XAI - An eXplainability toolbox for machine learning

ai ml xai bias imbalance evaluation upsampling xai-library downsampling explainability explainable-ai explainable-ml bias-evaluation interpretability machine-learning feature-importance artificial-intelligence machine-learning-explainability

Created 2019-01-11

91 commits to master branch, last one 3 years ago