7 results found Sort:

Agenta-AI
agenta Agenta-AI
273
2.2k
mit
25
The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM Observability all in one place.
llm-tools llm-platform llm-framework llm-as-a-judge llm-evaluation llm-monitoring llm-playground rag-evaluation llmops-platform llm-observability prompt-management prompt-engineering
Created 2023-04-26
12,021 commits to main branch, last one 5 days ago
prometheus-eval
prometheus-eval prometheus-eval
55
879
apache-2.0
3
Evaluate your LLM's response with Prometheus and GPT4 💯
llm gpt4 vllm llmops python litellm evaluation llm-as-a-judge llm-as-evaluator
Created 2024-04-18
205 commits to main branch, last one 2 months ago
metauto-ai
agent-as-a-judge metauto-ai
44
330
mit
3
🤠 Agent-as-a-Judge and DevAI dataset
llms llm-as-a-judge agent-as-a-judge
Created 2024-10-16
20 commits to main branch, last one 4 months ago
IAAR-Shanghai
xFinder IAAR-Shanghai
6
155
other
4
[ICLR 2025] xFinder: Large Language Models as Automated Evaluators for Reliable Evaluation
gpt llm phi qwen regex chatglm dataset xfinder benchmark evaluation judge-model reliability open-compass cc-by-nc-nd-4 lm-evaluation llm-as-a-judge llm-as-evaluator reliable-evaluation key-answer-extraction large-language-models
Created 2024-05-19
41 commits to main branch, last one 12 days ago
martin-wey
CodeUltraFeedback martin-wey
5
69
mit
2
CodeUltraFeedback: aligning large language models to coding preferences
dpo alignment codal-bench llm-as-a-judge code-generation codeultrafeedback large-language-models
Created 2024-01-25
51 commits to main branch, last one 8 months ago
KID-22
LLM-IR-Bias-Fairness-Survey KID-22
3
49
mit
3
This is the repo for the survey of Bias and Fairness in IR with LLMs.
ir llm bias llm4ir llm4rs chatgpt llm4rec fairness llm-as-a-judge llm-as-evaluator recommender-systems information-retrieval large-language-models
Created 2024-03-18
52 commits to main branch, last one 19 hours ago
MJ-Bench
MJ-Bench MJ-Bench
5
44
mit
1
Official implementation for "MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?"
reward-models llm-as-a-judge llm-benchmarking multimodal-judge multimodal-foundation-model
Created 2024-06-11
32 commits to main branch, last one 15 days ago