Ragas

Ragas belongs in the database because it has become one of the most recognizable open-source names in LLM evaluation. The official site and docs position it as a library for moving from vibe checks to systematic evaluation loops, with support for experiments, metrics, synthetic data generation, online monitoring, and custom evaluation logic. That makes it a core builder tool rather than just a research toy. Teams use it to reason about RAG quality, agent behavior, prompt changes, and model regressions with more discipline than ad hoc spot checks.

It is also a strong entry because the commercial story is still light and the OSS story is real. The public project surfaces emphasize the open-source library and an early-stage hosted platform experience rather than a mature self-serve SaaS price card. For the directory, that means Ragas should be presented honestly as an OSS-first eval framework with optional enterprise and collaboration paths emerging around it, not as a polished fully-packaged platform yet.