Can this skill be used in a CI/CD pipeline?

Yes, because it outputs standardized JSON reports and manages its own dependencies and environment, it is ideal for automated validation phases in a development pipeline.

Do I need to manually start my Umbraco instance?

No, the skill includes an auto-start feature that can launch Umbraco from the project directory using 'dotnet run' if it detects that an E2E test requires a running instance.

How do I configure authentication for E2E tests?

You can provide admin credentials and the instance URL through environment variables like UMBRACO_USER_LOGIN and UMBRACO_USER_PASSWORD.

What is the output format of the test results?

The skill generates a file named 'test-report.json' in the project root, providing a timestamp, summary of pass/fail counts, and detailed results for every test executed.

Umbraco Skill Test Runner

Name: Umbraco Skill Test Runner
Author: umbraco

byumbraco

•

Security & Testing

Executes unit, integration, and E2E tests for Umbraco backoffice skills to ensure code quality and functionality.

The Umbraco Skill Test Runner is a specialized tool designed to automate the validation of Umbraco CMS backoffice extensions and skills. It executes a comprehensive suite of tests, including component-level unit tests with Web Test Runner, integration tests against mocked backoffices, and full end-to-end (E2E) tests using Playwright. The tool simplifies the developer experience by automatically managing the Umbraco environment—starting and stopping instances as needed—and generating a standardized JSON report to track test results, durations, and status across the entire project.

Key Features

011 GitHub stars

02Detailed JSON report generation for CI/CD pipelines

03Smart Umbraco instance lifecycle management (Auto-start/stop)

04Automated execution of unit, mocked, and E2E test suites

05Seamless integration with Playwright for browser-based automation

06Pre-configured support for @open-wc/testing component validation

Use Cases

01Automating quality assurance for Umbraco CMS dashboard components

02Validating custom Umbraco backoffice extensions before production deployment

03Integrating comprehensive testing into Umbraco skill development workflows

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add umbraco/umbraco-cms-backoffice-skills umbraco-skill-test-runner

For use in Claude.ai and ChatGPT

Download Skill