Trending repositories for topic llmops
Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you q...
Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. ๐ณDocker-friendly.โกAlways in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, a...
A high-throughput and memory-efficient inference and serving engine for LLMs
ๆฌ้กน็ฎๆจๅจๅไบซๅคงๆจกๅ็ธๅ ณๆๆฏๅ็ไปฅๅๅฎๆ็ป้ช๏ผๅคงๆจกๅๅทฅ็จๅใๅคงๆจกๅๅบ็จ่ฝๅฐ๏ผ
Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
๐ชข Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. ๐YC W23
TensorZero creates a feedback loop for optimizing LLM applications โ turning production data into smarter, faster, and cheaper models.
A collection of production-ready Generative AI Agent templates built for Google Cloud. It accelerates development by providing a holistic, production-ready solution, addressing common challenges (Depl...
The AI-native proxy server for agents. Arch handles the pesky heavy lifting in building agentic apps - routing prompts to agents or specific tools, clarifying user input, unifying access and observabi...
Laminar - open-source all-in-one platform for engineering AI products. Crate data flywheel for you AI app. Traces, Evals, Datasets, Labels. YC S24.
Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command ...
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include: GenAI workflow, RAG, Agent, Unified model management, Evaluation, SF...
Composio equip's your AI agents & LLMs with 100+ high-quality integrations via function calling
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
Open-source observability for your LLM application, based on OpenTelemetry
Community maintained hardware plugin for vLLM on Ascend
Comprehensive resources on Generative AI, including a detailed roadmap, projects, use cases, interview preparation, and coding preparation.
A collection of production-ready Generative AI Agent templates built for Google Cloud. It accelerates development by providing a holistic, production-ready solution, addressing common challenges (Depl...
Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. ๐ณDocker-friendly.โกAlways in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, a...
TensorZero creates a feedback loop for optimizing LLM applications โ turning production data into smarter, faster, and cheaper models.
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Terraform configuration for deploying Dify on Google Cloud with scalability, high availability, and production-level readiness.
Laminar - open-source all-in-one platform for engineering AI products. Crate data flywheel for you AI app. Traces, Evals, Datasets, Labels. YC S24.
Scalable, fast, and disk-friendly vector search in Postgres, the successor of pgvecto.rs.
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
Visual node-edge graph GUI editor for LangGraph and run with local LLM or online API
๐ A curated list of papers & technical articles on AI Quality & Safety
The AI-native proxy server for agents. Arch handles the pesky heavy lifting in building agentic apps - routing prompts to agents or specific tools, clarifying user input, unifying access and observabi...
ๆฌ้กน็ฎๆจๅจๅไบซๅคงๆจกๅ็ธๅ ณๆๆฏๅ็ไปฅๅๅฎๆ็ป้ช๏ผๅคงๆจกๅๅทฅ็จๅใๅคงๆจกๅๅบ็จ่ฝๅฐ๏ผ
A high-throughput and memory-efficient inference and serving engine for LLMs
Langtrace ๐ is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorD...
Fast, streaming indexing, query, and agentic LLM applications in Rust
A database of SDKs, frameworks, libraries, and tools for creating, monitoring, debugging and deploying autonomous AI agents
Sister project to OpenLLMetry, but in Typescript. Open-source observability for your LLM application, based on OpenTelemetry
Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you q...
A high-throughput and memory-efficient inference and serving engine for LLMs
Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. ๐ณDocker-friendly.โกAlways in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, a...
Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
ๆฌ้กน็ฎๆจๅจๅไบซๅคงๆจกๅ็ธๅ ณๆๆฏๅ็ไปฅๅๅฎๆ็ป้ช๏ผๅคงๆจกๅๅทฅ็จๅใๅคงๆจกๅๅบ็จ่ฝๅฐ๏ผ
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
๐ชข Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. ๐YC W23
TensorZero creates a feedback loop for optimizing LLM applications โ turning production data into smarter, faster, and cheaper models.
A collection of production-ready Generative AI Agent templates built for Google Cloud. It accelerates development by providing a holistic, production-ready solution, addressing common challenges (Depl...
Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command ...
Composio equip's your AI agents & LLMs with 100+ high-quality integrations via function calling
The AI-native proxy server for agents. Arch handles the pesky heavy lifting in building agentic apps - routing prompts to agents or specific tools, clarifying user input, unifying access and observabi...
Laminar - open-source all-in-one platform for engineering AI products. Crate data flywheel for you AI app. Traces, Evals, Datasets, Labels. YC S24.
Learn to build your Second Brain AI assistant with LLMs, agents, RAG, fine-tuning, LLMOps and AI systems techniques.
A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced anal...
Evidently is โโan open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
Model Context Protocol (MCP) Server for Langfuse Prompt Management. This server allows you to access and manage your Langfuse prompts through the Model Context Protocol.
A collection of production-ready Generative AI Agent templates built for Google Cloud. It accelerates development by providing a holistic, production-ready solution, addressing common challenges (Depl...
Community maintained hardware plugin for vLLM on Ascend
๐ง ๐ Graph-Based Programmable Neuro-Symbolic LM Framework - a production-first LM framework built with decade old Deep Learning best practices
Learn to build your Second Brain AI assistant with LLMs, agents, RAG, fine-tuning, LLMOps and AI systems techniques.
Scalable, fast, and disk-friendly vector search in Postgres, the successor of pgvecto.rs.
TensorZero creates a feedback loop for optimizing LLM applications โ turning production data into smarter, faster, and cheaper models.
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Laminar - open-source all-in-one platform for engineering AI products. Crate data flywheel for you AI app. Traces, Evals, Datasets, Labels. YC S24.
Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. ๐ณDocker-friendly.โกAlways in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, a...
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).
The AI-native proxy server for agents. Arch handles the pesky heavy lifting in building agentic apps - routing prompts to agents or specific tools, clarifying user input, unifying access and observabi...
A high-throughput and memory-efficient inference and serving engine for LLMs
Visual node-edge graph GUI editor for LangGraph and run with local LLM or online API
Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you q...
Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
ๆฌ้กน็ฎๆจๅจๅไบซๅคงๆจกๅ็ธๅ ณๆๆฏๅ็ไปฅๅๅฎๆ็ป้ช๏ผๅคงๆจกๅๅทฅ็จๅใๅคงๆจกๅๅบ็จ่ฝๅฐ๏ผ
๐ชข Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. ๐YC W23
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
A comprehensive set of LLM benchmark scores and provider prices.
Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you q...
A high-throughput and memory-efficient inference and serving engine for LLMs
Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
ๆฌ้กน็ฎๆจๅจๅไบซๅคงๆจกๅ็ธๅ ณๆๆฏๅ็ไปฅๅๅฎๆ็ป้ช๏ผๅคงๆจกๅๅทฅ็จๅใๅคงๆจกๅๅบ็จ่ฝๅฐ๏ผ
A collection of production-ready Generative AI Agent templates built for Google Cloud. It accelerates development by providing a holistic, production-ready solution, addressing common challenges (Depl...
๐ชข Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. ๐YC W23
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. ๐ณDocker-friendly.โกAlways in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, a...
Composio equip's your AI agents & LLMs with 100+ high-quality integrations via function calling
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
TensorZero creates a feedback loop for optimizing LLM applications โ turning production data into smarter, faster, and cheaper models.
Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced anal...
Learn to build your Second Brain AI assistant with LLMs, agents, RAG, fine-tuning, LLMOps and AI systems techniques.
The AI-native proxy server for agents. Arch handles the pesky heavy lifting in building agentic apps - routing prompts to agents or specific tools, clarifying user input, unifying access and observabi...
Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command ...
A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
The LLM's practical guide: From the fundamentals to deploying advanced LLM and RAG apps to AWS using LLMOps best practices
Evidently is โโan open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
A collection of production-ready Generative AI Agent templates built for Google Cloud. It accelerates development by providing a holistic, production-ready solution, addressing common challenges (Depl...
Model Context Protocol (MCP) Server for Langfuse Prompt Management. This server allows you to access and manage your Langfuse prompts through the Model Context Protocol.
Terraform configuration for deploying Dify on Google Cloud with scalability, high availability, and production-level readiness.
Learn to build your Second Brain AI assistant with LLMs, agents, RAG, fine-tuning, LLMOps and AI systems techniques.
A collection of tricks and tools to speed up transformer models
Community maintained hardware plugin for vLLM on Ascend
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
๐ง ๐ Graph-Based Programmable Neuro-Symbolic LM Framework - a production-first LM framework built with decade old Deep Learning best practices
TensorZero creates a feedback loop for optimizing LLM applications โ turning production data into smarter, faster, and cheaper models.
A comprehensive set of LLM benchmark scores and provider prices.
The AI-native proxy server for agents. Arch handles the pesky heavy lifting in building agentic apps - routing prompts to agents or specific tools, clarifying user input, unifying access and observabi...
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Scalable, fast, and disk-friendly vector search in Postgres, the successor of pgvecto.rs.
Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you q...
End to End Generative AI Industry Projects on LLM Models with Deployment_Awesome LLM Projects
๐ชข Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. ๐YC W23
AIOps modules is a collection of reusable Infrastructure as Code (IaC) modules for Machine Learning (ML), Foundation Models (FM), Large Language Models (LLM) and GenAI development and operations on AW...
Fast, streaming indexing, query, and agentic LLM applications in Rust
Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced anal...
TensorZero creates a feedback loop for optimizing LLM applications โ turning production data into smarter, faster, and cheaper models.
The AI-native proxy server for agents. Arch handles the pesky heavy lifting in building agentic apps - routing prompts to agents or specific tools, clarifying user input, unifying access and observabi...
Laminar - open-source all-in-one platform for engineering AI products. Crate data flywheel for you AI app. Traces, Evals, Datasets, Labels. YC S24.
A collection of production-ready Generative AI Agent templates built for Google Cloud. It accelerates development by providing a holistic, production-ready solution, addressing common challenges (Depl...
A framework for comprehensive diagnosis and optimization of agents using simulated, realistic synthetic interactions
Evaluate your LLM's response with Prometheus and GPT4 ๐ฏ
Learn to build your Second Brain AI assistant with LLMs, agents, RAG, fine-tuning, LLMOps and AI systems techniques.
Dynamiq is an orchestration framework for agentic AI and LLM applications
Scalable, fast, and disk-friendly vector search in Postgres, the successor of pgvecto.rs.
The only fully local production-grade Super SDK that provides a simple, unified, and powerful interface for calling more than 200+ LLMs.
Toolkits to create a human-in-the-loop approval layer to monitor and guide AI agents workflow in real-time.
๐ง ๐ Graph-Based Programmable Neuro-Symbolic LM Framework - a production-first LM framework built with decade old Deep Learning best practices
Visual node-edge graph GUI editor for LangGraph and run with local LLM or online API
Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you q...
A high-throughput and memory-efficient inference and serving engine for LLMs
Composio equip's your AI agents & LLMs with 100+ high-quality integrations via function calling
Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. ๐ณDocker-friendly.โกAlways in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, a...
Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced anal...
Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
ๆฌ้กน็ฎๆจๅจๅไบซๅคงๆจกๅ็ธๅ ณๆๆฏๅ็ไปฅๅๅฎๆ็ป้ช๏ผๅคงๆจกๅๅทฅ็จๅใๅคงๆจกๅๅบ็จ่ฝๅฐ๏ผ
๐ชข Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. ๐YC W23
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Open-source observability for your LLM application, based on OpenTelemetry
Supercharge Your LLM Application Evaluations ๐
RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry
Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command ...
TensorZero creates a feedback loop for optimizing LLM applications โ turning production data into smarter, faster, and cheaper models.
โ๏ธ๐ฆ Build portable, modular & lightweight Fullstack Agents
๐ค ๐๐ฒ๐ฎ๐ฟ๐ป for ๐ณ๐ฟ๐ฒ๐ฒ how to ๐ฏ๐๐ถ๐น๐ฑ an end-to-end ๐ฝ๐ฟ๐ผ๐ฑ๐๐ฐ๐๐ถ๐ผ๐ป-๐ฟ๐ฒ๐ฎ๐ฑ๐ ๐๐๐ & ๐ฅ๐๐ ๐๐๐๐๐ฒ๐บ using ๐๐๐ ๐ข๐ฝ๐ best practices: ~ ๐ด๐ฐ๐ถ๐ณ๐ค๐ฆ ๐ค๐ฐ๐ฅ๐ฆ + 12 ๐ฉ๐ข๐ฏ๐ฅ๐ด-๐ฐ๐ฏ ...
A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
The LLM's practical guide: From the fundamentals to deploying advanced LLM and RAG apps to AWS using LLMOps best practices
BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include: GenAI workflow, RAG, Agent, Unified model management, Evaluation, SF...
Composio equip's your AI agents & LLMs with 100+ high-quality integrations via function calling
Laminar - open-source all-in-one platform for engineering AI products. Crate data flywheel for you AI app. Traces, Evals, Datasets, Labels. YC S24.
Scalable, fast, and disk-friendly vector search in Postgres, the successor of pgvecto.rs.
The ultimate LLM Ops platform - Monitoring, Analytics, Evaluations, Datasets and Prompt Optimization โจ
RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry
Community maintained hardware plugin for vLLM on Ascend
End to End Generative AI Industry Projects on LLM Models with Deployment_Awesome LLM Projects
Visual node-edge graph GUI editor for LangGraph and run with local LLM or online API
Terraform configuration for deploying Dify on Google Cloud with scalability, high availability, and production-level readiness.
Dynamiq is an orchestration framework for agentic AI and LLM applications
๐ง ๐ Graph-Based Programmable Neuro-Symbolic LM Framework - a production-first LM framework built with decade old Deep Learning best practices
Comprehensive resources on Generative AI, including a detailed roadmap, projects, use cases, interview preparation, and coding preparation.
Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. ๐๐ป Integrates with 50+ LLM Providers, ...
Langtrace ๐ is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorD...
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
High-scale LLM gateway, written in Rust. OpenTelemetry-based observability included
Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. ๐ณDocker-friendly.โกAlways in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, a...
๐ค ๐๐ฒ๐ฎ๐ฟ๐ป for ๐ณ๐ฟ๐ฒ๐ฒ how to ๐ฏ๐๐ถ๐น๐ฑ an end-to-end ๐ฝ๐ฟ๐ผ๐ฑ๐๐ฐ๐๐ถ๐ผ๐ป-๐ฟ๐ฒ๐ฎ๐ฑ๐ ๐๐๐ & ๐ฅ๐๐ ๐๐๐๐๐ฒ๐บ using ๐๐๐ ๐ข๐ฝ๐ best practices: ~ ๐ด๐ฐ๐ถ๐ณ๐ค๐ฆ ๐ค๐ฐ๐ฅ๐ฆ + 12 ๐ฉ๐ข๐ฏ๐ฅ๐ด-๐ฐ๐ฏ ...