What is the recommended reward scale for PPO trading?

A reward scale of 0.001 is recommended to ensure that reward volatility reflects realistic proxy equity changes, typically resulting in a 5-15% MaxDD range.

Why is my MaxDD showing 80% when my model is profitable?

This is often a 'phantom' metric caused by an aggressive reward scale (like 0.1) treating raw rewards as massive equity shifts; reducing the scale to 0.001 aligns metrics with realistic volatility.

Does this skill help with double-penalization in rewards?

Yes, it includes guidance on reducing drawdown penalty weights when drawdown is already present in the model's observation space, preventing overly conservative behavior.

How does this skill change model gating thresholds?

It provides recalibrated 'Approved' and 'Review' thresholds for Profit Factor, Consistency, and Fitness to match the more conservative metrics produced by the new scaling.

Reward Scaling Calibration

Name: Reward Scaling Calibration
Author: smith6jt-cop

bysmith6jt-cop

0•

Data Science & ML

Calibrates reinforcement learning reward scales to eliminate unrealistic drawdown metrics in algorithmic trading models.

This skill provides a specialized workflow for fine-tuning reward scaling in GPU-native PPO training environments, specifically addressing the 'phantom MaxDD' issue. It helps developers correct scenarios where aggressive scaling causes simulated equity to fluctuate wildly, leading to misleading validation results despite high model performance. By recalibrating the reward scale and adjusting model gating thresholds, this skill ensures that metrics like Maximum Drawdown accurately reflect training stability and reward volatility, allowing for more reliable model selection and evaluation in financial contexts.

Key Features

01Updates ModelGatingConfig thresholds for fitness, PF, and consistency

02Optimizes per-step equity clamping to prevent extreme mathematical swings

03Recalibrates reward scaling to 0.001 for realistic equity simulation

04Fixes unrealistic 35-80% Maximum Drawdown (MaxDD) validation values

05Adjusts reward weights to prevent double-penalizing drawdown observations

060 GitHub stars

Use Cases

01Correcting misleading performance metrics in PPO-based trading bot training

02Optimizing GPU-native reinforcement learning environments for finance

03Fine-tuning model gating configurations for automated strategy selection

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add smith6jt-cop/skills_registry reward-scaling-calibration

For use in Claude.ai and ChatGPT

Download Skill