10 results found Sort:

Giskard-AI
giskard Giskard-AI
228
3.7k
apache-2.0
29
🐢 Open-Source Evaluation & Testing for LLMs and ML models
llm mlops llmops llm-eval ai-safety ml-safety ai-testing ml-testing ai-red-team ai-security fairness-ai llm-security ml-validation llm-evaluation rag-evaluation red-team-tools responsible-ai trustworthy-ai model-monitoring ethical-artificial-intelligence
Created 2022-03-06
9,731 commits to main branch, last one a day ago
promptfoo
promptfoo promptfoo
230
3.4k
mit
18
Test your prompts, agents, and RAGs. Use LLM evals to improve your app's quality and catch problems. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with comman...
ci llm rag cicd ci-cd llmops prompts testing llm-eval evaluation llm-evaluation prompt-testing prompt-engineering evaluation-framework llm-evaluation-framework
Created 2023-04-28
1,361 commits to main branch, last one 20 hours ago
Arize-ai
phoenix Arize-ai
211
3.0k
other
27
AI Observability & Evaluation
umap mlops ai-roi llmops llm-eval clustering ai-monitoring ml-monitoring ai-observability ml-observability model-monitoring model-observability
Created 2022-11-09
2,531 commits to main branch, last one 13 hours ago
uptrain-ai
uptrain uptrain-ai
174
2.1k
apache-2.0
20
UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform ro...
llmops llm-eval llm-test evaluation monitoring openai-evals llm-prompting autoevaluation experimentation machine-learning prompt-engineering jailbreak-detection root-cause-analysis hallucination-detection
Created 2022-11-07
764 commits to main branch, last one 18 days ago
athina-ai
athina-evals athina-ai
11
151
unknown
5
Python SDK for running evaluations on LLM generated responses
llmops llm-ops llm-eval evaluation llm-evaluation evaluation-metrics evaluation-framework llm-evaluation-toolkit
Created 2023-11-22
455 commits to main branch, last one a day ago
fiddlecube
fiddlecube-sdk fiddlecube
1
73
unknown
1
Generate ideal question-answers for testing RAG
llm-eval llm-training fine-tune-llms synthetic-data
Created 2023-07-04
45 commits to master branch, last one a day ago
Re-Align
just-eval Re-Align
5
67
mit
3
A simple GPT-based evaluation tool for multi-aspect, interpretable assessment of LLMs.
llm gpt4 llm-eval evaluation llm-evaluation llm-evaluation-toolkit
Created 2023-11-19
40 commits to main branch, last one 4 months ago
kuk
rulm-sbs2 kuk
2
54
unknown
3
Бенчмарк сравнивает русские аналоги ChatGPT: Saiga, YandexGPT, Gigachat
llm-eval russian-specific
Created 2023-08-23
112 commits to master branch, last one 9 months ago
parea-ai
parea-sdk-py parea-ai
4
43
apache-2.0
2
Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)
llm llmops metrics llm-eval llm-tools generative-ai llm-evaluation good-first-issue llms-benchmarking prompt-engineering llm-evaluation-toolkit llm-evaluation-framework
Created 2023-07-24
925 commits to main branch, last one 6 days ago
Auto-Playground
ragrank Auto-Playground
8
25
apache-2.0
1
🎯 Your free LLM evaluation toolkit helps you assess the accuracy of facts, how well it understands context, its tone, and more. This helps you see how good your LLM applications are.
llm rag llmops llm-eval evaluation language-model machine-learning prompt-engineering
Created 2024-02-17
278 commits to main branch, last one about a month ago