How does the two-phase evaluation process work?

In Phase 1, Repo Agents explore each repository to extract structured facts. In Phase 2, Judge Agents use those facts to score the code against specific rubrics for each dimension.

Does this skill require other Beagle plugins to function?

Yes, it depends on the @beagle:llm-artifacts-detection skill to assist Repo Agents in identifying dead code and overengineering patterns.

Which metrics does the LLM Judge skill evaluate?

The skill evaluates five key dimensions: Functionality (30%), Security (25%), Test Quality (20%), Overengineering (15%), and Dead Code (10%).

What is the LLM-as-judge methodology?

It is an evaluation framework where a Large Language Model acts as an evaluator, using specific rubrics and structured criteria to provide consistent, scalable scoring for complex tasks like code implementation.

LLM Code Judge

Name: LLM Code Judge
Author: existential-birds

byexistential-birds

•

Security & Testing

Compares and scores multiple code implementations using a multi-phase LLM-as-judge framework to evaluate functionality, security, and quality.

About

The LLM Judge skill automates complex code comparisons by deploying specialized agents to analyze multiple repositories against a shared specification. It utilizes a sophisticated two-phase architecture: first, parallel agents extract structured facts from each codebase; second, a panel of distinct judge agents applies weighted rubrics to score dimensions such as security, test quality, and overengineering. This skill is indispensable for technical leads and architects who need to verify implementation consistency, evaluate candidate coding challenges, or choose between competing architectural approaches based on objective data.

Key Features

Automated detection of overengineering and dead code via artifact analysis
14 GitHub stars
Parallel agent execution for efficient analysis of multiple repositories simultaneously
Detailed JSON reporting with weighted rankings and a final summary verdict
Two-phase evaluation workflow that separates fact-gathering from subjective judging
Weighted scoring across functionality, security, test quality, and code complexity

Use Cases

Automating the technical evaluation and scoring of candidate coding assignments
Comparing multiple architectural implementations of the same feature specification
Conducting deep-dive security and quality audits across internal codebases

About

Key Features

Automated detection of overengineering and dead code via artifact analysis
14 GitHub stars
Parallel agent execution for efficient analysis of multiple repositories simultaneously
Detailed JSON reporting with weighted rankings and a final summary verdict
Two-phase evaluation workflow that separates fact-gathering from subjective judging
Weighted scoring across functionality, security, test quality, and code complexity

Use Cases

Automating the technical evaluation and scoring of candidate coding assignments
Comparing multiple architectural implementations of the same feature specification
Conducting deep-dive security and quality audits across internal codebases