The procedure presented in this paper has specifically been designed to determine the probability of backtest overfitting (PBO). This is defined as the probability that the strategy with optimal performance IS (in-sample) delivers OOS (out-of-sample) a performance below the median performance of all trials attempted by the researcher. When that probability is high, optimizing IS has a detrimental effect in terms of OOS performance, because the backtest profits from specific features in the IS subset that are not present elsewhere.
|