Search Results - RepositoryStats

414

5.1k

mit

21

Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command ...

ci llm rag cicd ci-cd llmops prompts testing llm-eval evaluation pentesting red-teaming llm-evaluation prompt-testing prompt-engineering evaluation-framework vulnerability-scanners llm-evaluation-framework

Created 2023-04-28

3,223 commits to main branch, last one 11 hours ago

deepeval confident-ai

339

4.2k

apache-2.0

26

The LLM Evaluation Framework

llm-evaluation evaluation-metrics evaluation-framework llm-evaluation-metrics llm-evaluation-framework

Created 2023-08-10

3,980 commits to main branch, last one 10 hours ago

agentic_security msoedov

91

878

apache-2.0

16

Agentic LLM Vulnerability Scanner / AI red teaming kit

llm-fuzzer ai-red-team llm-fuzzing llm-scanner llm-security llm-evaluation llm-guardrails llm-jailbreaks prompt-testing owasp-llm-top-10 llm-vulnerabilities llm-fuzzer-aggregator llm-evaluation-framework

Created 2024-04-11

194 commits to main branch, last one 16 hours ago

MixEval Psycoy

37

230

unknown

1

The official evaluation suite and dynamic data release for MixEval.

mixeval benchmark evaluation llm-inference llm-evaluation benchmark-mixture foundation-models benchmarking-suite evaluation-framework large-language-model large-language-models benchmarking-framework large-multimodal-models llm-evaluation-framework

Created 2024-06-01

120 commits to main branch, last one about a month ago

parea-sdk-py parea-ai

6

75

apache-2.0

2

Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)

llm llmops metrics llm-eval llm-tools generative-ai llm-evaluation good-first-issue llms-benchmarking prompt-engineering llm-evaluation-toolkit llm-evaluation-framework

Created 2023-07-24

1,067 commits to main branch, last one 3 months ago

contextcheck Addepto

6

55

mit

2

MIT-licensed Framework for LLMs, RAGs, Chatbots testing. Configurable via YAML and integrable into CI pipelines for automated testing.

ci llm rag ai-chat ai-testing llm-testing open-source prompt-test rag-testing testing-tools llm-evaluation ai-testing-tool chatbot-testing chatbot-framework testing-framework generative-ai-testing large-language-models summarization-testing llm-evaluation-framework

Created 2024-10-30

134 commits to main branch, last one 22 days ago

KIEval zhuohaoyu

2

35

unknown

3

[ACL'24] A Knowledge-grounded Interactive Evaluation Framework for Large Language Models

llm acl2024 explainable-ai llm-evaluation machine-learning llm-evaluation-metrics llm-evaluation-toolkit llm-evaluation-framework

Created 2024-02-23

19 commits to master branch, last one 5 months ago