12 results found Sort:

promptfoo
promptfoo promptfoo
405
5.0k
mit
21
Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command ...
ci llm rag cicd ci-cd llmops prompts testing llm-eval evaluation pentesting red-teaming llm-evaluation prompt-testing prompt-engineering evaluation-framework vulnerability-scanners llm-evaluation-framework
Created 2023-04-28
3,165 commits to main branch, last one 12 hours ago
Arize-ai
phoenix Arize-ai
320
4.3k
other
31
AI Observability & Evaluation
mlops ai-roi llmops datasets llm-eval ai-monitoring aiengineering hacktoberfest ai-observability ml-observability model-observability
Created 2022-11-09
4,036 commits to main branch, last one 13 hours ago
Giskard-AI
giskard Giskard-AI
277
4.1k
apache-2.0
33
🐢 Open-Source Evaluation & Testing for AI & LLM systems
llm mlops llmops llm-eval ai-safety ml-safety ai-testing ml-testing ai-red-team ai-security fairness-ai llm-security ml-validation llm-evaluation rag-evaluation red-team-tools responsible-ai trustworthy-ai evaluation-framework ethical-artificial-intelligence
Created 2022-03-06
10,170 commits to main branch, last one 2 days ago
uptrain-ai
uptrain uptrain-ai
193
2.2k
apache-2.0
21
UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform ro...
llmops llm-eval llm-test evaluation monitoring openai-evals llm-prompting autoevaluation experimentation machine-learning prompt-engineering jailbreak-detection root-cause-analysis hallucination-detection
Created 2022-11-07
770 commits to main branch, last one 4 months ago
iterative
datachain iterative
93
2.1k
apache-2.0
15
ETL, Analytics, Versioning for Unstructured Data
ai cv llm mlops llm-eval embeddings multimodal data-analytics data-wrangling machine-learning
Created 2024-06-25
419 commits to main branch, last one a day ago
athina-ai
athina-evals athina-ai
14
242
unknown
5
Python SDK for running evaluations on LLM generated responses
llmops llm-ops llm-eval evaluation llm-evaluation evaluation-metrics evaluation-framework llm-evaluation-toolkit
Created 2023-11-22
639 commits to main branch, last one 3 days ago
fiddlecube
fiddlecube-sdk fiddlecube
3
123
unknown
1
Generate ideal question-answers for testing RAG
llm-eval llm-training fine-tune-llms synthetic-data
This repository has been archived (exclude archived)
Created 2023-07-04
60 commits to master branch, last one a day ago
Re-Align
just-eval Re-Align
6
76
mit
4
A simple GPT-based evaluation tool for multi-aspect, interpretable assessment of LLMs.
llm gpt4 llm-eval evaluation llm-evaluation llm-evaluation-toolkit
Created 2023-11-19
40 commits to main branch, last one 10 months ago
parea-ai
parea-sdk-py parea-ai
6
75
apache-2.0
2
Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)
llm llmops metrics llm-eval llm-tools generative-ai llm-evaluation good-first-issue llms-benchmarking prompt-engineering llm-evaluation-toolkit llm-evaluation-framework
Created 2023-07-24
1,067 commits to main branch, last one 3 months ago
kuk
rulm-sbs2 kuk
2
57
unknown
3
Бенчмарк сравнивает русские аналоги ChatGPT: Saiga, YandexGPT, Gigachat
llm-eval russian-specific
Created 2023-08-23
112 commits to master branch, last one about a year ago
Auto-Playground
ragrank Auto-Playground
12
29
apache-2.0
1
🎯 Your free LLM evaluation toolkit helps you assess the accuracy of facts, how well it understands context, its tone, and more. This helps you see how good your LLM applications are.
llm rag llmops llm-eval evaluation language-model machine-learning prompt-engineering
Created 2024-02-17
280 commits to main branch, last one 3 months ago
alan-turing-institute
prompto alan-turing-institute
1
25
mit
17
An open source library for asynchronous querying of LLM endpoints
nlp llms hut23 python llm-eval transformer transformers deep-learning llm-evaluation machine-learning large-language-models natural-language-processing
Created 2024-04-03
529 commits to main branch, last one 16 days ago