Is this skill backward compatible with standard bandit models?

Yes. By setting the discount factor to 1.0, the algorithm behaves exactly like standard Thompson Sampling, and new state fields are initialized to support legacy data.

How does this skill handle exploration in shifting environments?

By using effective counts (effective_n), the variance of an unpulled arm increases as its historical data is discounted. This causes the algorithm to naturally re-explore those arms as they become stale.

What is Discounted Thompson Sampling?

It is a variation of the Thompson Sampling algorithm that applies an exponential decay to historical observations, allowing the model to prioritize recent data over stale information in changing environments.

When should I use a discount factor of 0.95 versus 0.99?

Use a discount of 0.99 for slow regime changes occurring over weeks or months, and 0.95 for medium-speed shifts occurring over days. Lower factors like 0.90 are best for highly volatile environments.

Discounted Thompson Sampling for Bandits

Name: Discounted Thompson Sampling for Bandits
Author: smith6jt-cop

bysmith6jt-cop

•

Data Science & ML

Implements adaptive Thompson sampling with exponential discounting to handle non-stationary environments and market regime shifts.

This skill enables Claude to implement and tune Discounted Thompson Sampling (dTS) within reinforcement learning systems, specifically designed for non-stationary environments like financial markets. It provides the mathematical framework and implementation patterns for exponentially decaying past observations, allowing bandit algorithms to adapt quickly to regime changes. By tracking effective counts rather than raw counts, the skill ensures that decision-making is based on recent, relevant data while maintaining the ability to re-explore arms that have become stale, making it ideal for high-volatility trading strategies and dynamic resource allocation.

Key Features

01Diagnostic utility methods for arm statistics and state resets

02Exponential observation discounting for non-stationary reward distributions

03Automatic adaptation to market regime shifts such as bull/bear cycles

04Backward compatibility with standard stationary Thompson Sampling

05Dynamic exploration-exploitation tuning via adjustable discount factors

061 GitHub stars

Use Cases

01Adapting trading algorithms to shifting market regimes and volatility cycles

02Optimizing dynamic resource allocation in environments with changing performance metrics

03Managing exploration-exploitation trade-offs in online systems with evolving data

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add smith6jt-cop/skills_registry discounted-thompson-sampling

For use in Claude.ai and ChatGPT

Download Skill