Promptfoo is a local tool for testing LLM applications with security evaluations and performance comparisons.
AutoAudit is a large language model (LLM) designed for enhancing cybersecurity through advanced AI-driven threat detection and response.
A curated list of tools, datasets, demos, and papers for evaluating large language models (LLMs).
Sample notebooks and prompts for evaluating large language models (LLMs) and generative AI.
The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".
Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
Open-source framework for evaluating and testing AI and LLM systems for performance, bias, and security issues.
Phoenix is an open-source AI observability platform for experimentation, evaluation, and troubleshooting.
A unified toolkit for automatic evaluations of large language models (LLMs).
An open-source project for comparing two LLMs head-to-head with a given prompt, focusing on backend integration.
A comprehensive survey on benchmarks for Multimodal Large Language Models (MLLMs).
A serverless tool for converting various file types to Markdown using Cloudflare Workers and AI.
