How do I visualize false positives identified during evaluation?

You can filter the dataset view for samples with high false positive counts or use the SDK's evaluation patches feature to create a view focused specifically on individual errors.

Which evaluation methods are supported?

The skill supports COCO and Open Images for detections, simple, top-k, and binary for classifications, and ActivityNet for temporal detections in video.

Can I evaluate segmentation masks?

Yes, it supports segmentation evaluation using the simple method, with optional bandwidth parameters to focus specifically on boundary pixel accuracy.

Does this skill handle mAP computation automatically?

Yes, for detection and temporal tasks, you can set the compute_mAP parameter to true to generate mean Average Precision, mean Average Recall, and PR curves.

Do I need to manually install the evaluation plugin?

The skill includes a directive to check for the @voxel51/evaluation plugin and will guide you through the download and enabling process if it is missing.

FiftyOne Model Evaluation

Name: FiftyOne Model Evaluation
Author: voxel51

byvoxel51

•

Data Science & ML

Evaluates computer vision model predictions against ground truth using industry-standard protocols like COCO and Open Images.

The fiftyone-model-evaluation skill streamlines the process of assessing machine learning models within the FiftyOne ecosystem. It provides a standardized workflow to compute critical performance metrics such as mAP, precision, recall, and confusion matrices across various tasks including object detection, classification, segmentation, and regression. By automating plugin management and providing specific directives for evaluation parameters, this skill enables developers to identify model failures, analyze true/false positives, and visualize performance gaps directly within the FiftyOne App or through the Python SDK.

Key Features

01Standardized evaluation protocols including COCO, Open Images, and ActivityNet

02Support for multiple CV tasks: detection, classification, segmentation, and regression

033 GitHub stars

04Automated computation of mAP, precision, recall, and confusion matrices

05Interactive visualization of True Positives, False Positives, and False Negatives

06Seamless integration between FiftyOne App operators and Python SDK views

Use Cases

01Analyzing classification errors through confusion matrices and per-class performance metrics

02Benchmarking object detection models using COCO-style mAP at various IoU thresholds

03Comparing performance between multiple model versions to identify specific failure scenarios

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add voxel51/fiftyone-skills fiftyone-model-evaluation

For use in Claude.ai and ChatGPT

Download Skill