by Timothy Masters (Author)
About the author
Timothy Masters received a PhD in mathematical statistics with a specialization in numerical computing. Since then he has continuously worked as an independent consultant for government and industry. His early research involved automated feature detection in high-altitude photographs while he developed applications for flood and drought prediction, detection of hidden missile silos, and identification of threatening military vehicles. Later he worked with medical researchers in the development of computer algorithms for distinguishing between benign and malignant cells in needle biopsies. For the last twenty years he has focused primarily on methods for evaluating automated financial market trading systems. He has authored five books on practical applications of predictive modeling: Practical Neural Network Recipes in C++ (Academic Press, 1993); Signal and Image Processing with Neural Networks (Wiley, 1994); Advanced Algorithms for Neural Networks (Wiley, 1995); Neural, Novel, and Hybrid Algorithms for Time Series Prediction (Wiley, 1995); Data Mining Algorithms in C++ (Apress, 2018); Assessing and Improving Prediction and Classification (Apress, 2018); Deep Belief Nets in C++ and CUDA C: Volume 1 (Apress, 2018); and Deep Belief Nets in C++ and CUDA C: Volume 2 (Apress, 2018).
About this book
Build, test, and tune financial, insurance or other market trading systems using C++ algorithms and statistics. You’ve had an idea and have done some preliminary experiments, and it looks promising. Where do you go from here? Well, this book discusses and dissects this case study approach.
Seemingly good backtest performance isn't enough to justify trading real money. You need to perform rigorous statistical tests of the system's validity. Then, if basic tests confirm the quality of your idea, you need to tune your system, not just for best performance, but also for robust behavior in the face of inevitable market changes. Next, you need to quantify its expected future behavior, assessing how bad its real-life performance might actually be, and whether you can live with that. Finally, you need to find its theoretical performance limits so you know if its actual trades conform to this theoretical expectation, enabling you to dump the system if it does not live up to expectations.
This book does not contain any sure-fire, guaranteed-riches trading systems. Those are a dime a dozen... But if you have a trading system, this book will provide you with a set of tools that will help you evaluate the potential value of your system, tweak it to improve its profitability, and monitor its on-going performance to detect deterioration before it fails catastrophically. Any serious market trader would do well to employ the methods described in this book.
What You Will Learn
- See how the 'spaghetti-on-the-wall' approach to trading system development can be done legitimately
- Detect overfitting early in development
- Estimate the probability that your system's backtest results could have been due to just good luck
- Regularize a predictive model so it automatically selects an optimal subset of indicator candidates
- Rapidly find the global optimum for any type of parameterized trading system
- Assess the ruggedness of your trading system against market changes
- Enhance the stationarity and information content of your proprietary indicators
- Nest one layer of walkforward analysis inside another layer to account for selection bias in complex trading systems
- Compute a lower bound on your system's mean future performance
- Bound expected periodic returns to detect on-going system deterioration before it becomes severe
- Estimate the probability of catastrophic drawdown
Experienced C++ programmers, developers, and software engineers. Prior experience with rigorous statistical procedures to evaluate and maximize the quality of systems is recommended as well.
Table of contents
Chapter 1: Introduction 1
The Target Audience, and Overview of Contents 1
What’s in This Book 1
What’s Not in This Book 3
About Trading Systems 4
Market Prices and Returns 5
Two Types of Automated Trading Systems 6
The Agony of Believing the Computer 7
Future Leak Is More Dangerous Than You May Think 7
The Percent Wins Fallacy 8
Chapter 2: Pre-optimization Issues 11
Assessing and Improving Stationarity 11
The STATN Program 13
Improving Location Stationarity by Oscillating 17
Extreme Stationarity Induction 19
Measuring Indicator Information with Entropy 20
Computing the Relative Entropy of an Indicator 22
Entropy Impacts Predictive Model Quality 24
Improving the Entropy of an Indicator 25
Monotonic Tail-Only Cleaning 29
Chapter 3: Optimization Issues 35
Regularizing a Linear Model 35
Overview of the Regularized Model 36
Beta Adjustment with Guaranteed Convergence 40
Differential Case Weighting 41
Rapid Computation with Covariance Updates 42
Outline of the Beta Optimization Process 46
Code for Beta Optimization 48
Descending a Lambda Path 55
Optimizing Lambda with Cross Validation 59
The CD_ MA Program 63
Making a Linear Model Nonlinear 67
Differential Evolution: A Universal Nonlinear Optimizer 69
The DIFF_ EV. CPP Routine for Differential Evolution 75
Chapter 4: Post-optimization Issues 91
Cheap Bias Estimates 91
The StocBias Class 92
Cheap Parameter Relationships 96
Parameter Sensitivity Curves 108
Putting It All Together Trading OEX 112
Chapter 5: Estimating Future Performance I: Unbiased Trade Simulation 121
In-Sample and Out-of-Sample Performance 121
The TrnBias Program to Demonstrate Training Bias 123
Selection Bias 124
Walkforward Analysis 129
Future Leak by Unobvious IS/ OOS Overlap 131
Cross-Validation Analysis 143
Special Precautions for Algorithmic Trading 151
Comparing Cross Validation with Walkforward: XVW 156
Computationally Symmetric Cross Validation 158
What Does This Test Actually Measure? 163
Nested Walkforward Analysis 172
The Nested Walkforward Algorithm 174
A Practical Application of Nested Walkforward 179
An Example Using S& P 100 Components 187
Cross Validation Nested Inside Walkforward 188
Chapter 6: Estimating Future Performance II: Trade Analysis 193
Handling Dynamic Trading Systems 193
Unknown Lookahead to Single Bars, Revisited 194
Profit per Bar? Per Trade? Per Time? 195
Analyzing Completed Trade Returns Is Problematic 196
The PER_ WHAT Program 198
A Lower Bound for Mean Future Returns 209
Brief Digression: Hypothesis Tests 210
So, How Do We Use This Probability? 212
Parametric P-Values 216
Parametric Confidence Intervals 218
Lower Confidence Bounds and Hypothesis Tests 222
Bootstrap Confidence Intervals 222
The Pivot and Percentile Methods 223
The BC a Bootstrap Algorithm 225
The BOOT_ CONF. CPP Subroutines 227
The BOUND_ MEAN Program and Results with SPX 232
Beware of Bootstrapping Ratios 238
Bounding Future Returns 241
Deriving a Lower Bound from Empirical Quantiles 242
Confidence in the Computed Lower Bound 244
What About an Upper Bound on Future Returns? 247
The CONFTEST Program: Overview 248
The CONFTEST Program: Code 251
The BND_ RET Program 257
Bounding Drawdown 262
Intuition Gone Wrong 263
Bootstrapping Drawdown Bounds 265
The DRAWDOWN Program 267
Experiments with the DRAWDOWN Program 277
The CHOOSER_ DD Program 279
Chapter 7: Permutation Tests 283
Overview of Permutation Testing 283
Testing a Fully Specified Trading System 285
Testing the Training Process 286
Walkforward Testing a Trading System Factory 287
Permutation Testing of Predictive Models 289
The Permutation Testing Algorithm 291
Extending the Algorithm for Selection Bias 292
Partitioning Total Return of a Trading System 294
Essential Permutation Algorithms and Code 298
Permuting Simple Market Prices 299
Permuting Multiple Markets with an Offset 301
Example: P-Value and Partitioning 310
Example: Training with Next Bar Returns 312
Example: Permuting Multiple Markets 316
Index 319
Length: 321 pages
Publisher: Apress; 1st ed. edition (December 26, 2018)
Language: English
ISBN-10: 148424172X
ISBN-13: 978-1484241721
PDF version
EPUB version