Llamator

About

Llamator is a production-oriented service wrapper that facilitates automated red teaming workflows for Large Language Models (LLMs). It provides a robust HTTP API for job submission, state retrieval, and artifact access, alongside an MCP (Streamable HTTP transport) server for seamless agent and tooling integrations. The system handles asynchronous test runs, persists state durably, manages artifacts securely via MinIO, and offers features like API-key protection, OpenAPI schema with authorization, and Prometheus metrics, making it ideal for integrating LLM security testing into existing systems.

Key Features

• 1 GitHub stars
• Asynchronous test runs with durable state persisted in Redis.
• Request persistence with sensitive secret redaction.
• OpenAPI schema (Swagger UI) with API-key authorization and Prometheus metrics.
• Comprehensive artifacts lifecycle management via MinIO, with presigned download URLs.
• Optional API-key protection for both HTTP and MCP interfaces.

Use Cases

• Automating LLM security assessments and red teaming operations.
• Integrating LLAMATOR runs as tools within agent-based systems or CI/CD pipelines.
• Evaluating LLMs for vulnerabilities, misinformation, and hallucinations using predefined attack plans.