Deep Learning for MultivariateFinancial Time Series

Abstract：Deep learning is a framework for training and modelling neural networkswhich recently have surpassed all conventional methods in many learningtasks, prominently image and voice recognition.

This thesis uses deep learning algorithms to forecast financial data. Thedeep learning framework is used to train a neural network. The deep neuralnetwork is a DBN coupled to a MLP. It is used to choose stocks to formportfolios. The portfolios have better returns than the median of the stocksforming the list. The stocks forming the S&P 500 are included in the study.The results obtained from the deep neural network are compared to benchmarksfrom a logistic regression network, a multilayer perceptron and a naivebenchmark. The results obtained from the deep neural network are betterand more stable than the benchmarks. The findings support that deep learningmethods will find their way in finance due to their reliability and goodperformance.

Keywords: Back-Propagation Algorithm, Neural networks, Deep Belief Networks,Multilayer Perceptron, Deep Learning, Contrastive Divergence, GreedyLayer-wise Pre-training.

Contents1 Introduction 1
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Literature Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Neural Networks 5
2.1 Single Layer Neural Network . . . . . . . . . . . . . . . . . .6
2.1.1 Artificial Neurons . . . . . . . . . . . . . . . . . . . . . . 6
2.1.2 Activation Function . . . . . . . . . . . . . . . . . . . . .7
2.1.3 Single-Layer Feedforward Networks . . . . . . . . 11
2.1.4 The Perceptron . . . . . . . . . . . . . . . . . . . . . . .12
2.1.5 The Perceptron As a Classifier . . . . . . . . . . . . 12
2.2 Multilayer Neural Networks . . . . . . . . . . . . . . . . . . 15
2.2.1 The Multilayer Perceptron . . . . . . . . . . . . . . . 15
2.2.2 Function Approximation with MLP . . . . . . . . . .16
2.2.3 Regression and Classification . . . . . . . . . . . . . 17
2.2.4 Deep Architectures . . . . . . . . . . . . . . . . . . . . 18
2.3 Deep Belief Networks . . . . . . . . . . . . . . . . . . . . . . 22
2.3.1 Boltzmann Machines . . . . . . . . . . . . . . . . . . . 22
2.3.2 Restricted Boltzmann Machines . . . . . . . . . . .24
2.3.3 Deep Belief Networks . . . . . . . . . . . . . . . . . . .25
2.3.4 Model for Financial Application . . . . . . . . . . . . 27

3 Training Neural Networks 31
3.1 Back-Propagation Algorithm . . . . . . . . . . . . . . . . . . 31
3.1.1 Steepest Descent . . . . . . . . . . . . . . . . . . . . . 31
3.1.2 The Delta Rule . . . . . . . . . . . . . . . . . . . . . . . 32
Case 1 Output Layer . . . . . . . . . . . . . . . . . . . 33
Case 2 Hidden Layer . . . . . . . . . . . . . . . . . . . 33
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.1.3 Forward and Backward Phase . . . . . . . . . . . . .34
Forward Phase . . . . . . . . . . . . . . . . . . . . . . . 34
Backward Phase . . . . . . . . . . . . . . . . . . . . . . 34
3.1.4 Computation of δ for Known Activation Functions . . 35

3.1.5 Choosing Learning Rate . . . . . . . . . . . . . . . . . 36
3.1.6 Stopping Criteria . . . . . . . . . . . . . . . . . . . . . . 36
Early-Stopping . . . . . . . . . . . . . . . . . . . . . . . .37
3.1.7 Heuristics For The Back-Propagation Algorithm . . . .39
3.2 Batch and On-Line Learning . . . . . . . . . . . . . . . . . . 41
3.2.1 Batch Learning . . . . . . . . . . . . . . . . . . . . . . . 42
3.2.2 The Use of Batches . . . . . . . . . . . . . . . . . . . . 42
3.2.3 On-Line Learning . . . . . . . . . . . . . . . . . . . . . .43
3.2.4 Generalization . . . . . . . . . . . . . . . . . . . . . . . .43
3.2.5 Example: Regression with Neural Networks . . . . . . 44
3.3 Training Restricted Boltzmann Machines . . . . . . . . . . . . 47
3.3.1 Contrastive Divergence . . . . . . . . . . . . . . . . . . . . 49
3.4 Training Deep Belief Networks . . . . . . . . . . . . . . . . . . . 53
3.4.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . .58

4 Financial Model 59
4.1 The Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.1.1 Input Data and Financial Model . . . . . . . . . . .60

5 Experiments and Results 63
5.1 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.2 Benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67
5.3.1 Summary of Results . . . . . . . . . . . . . . . . . . 69

6 Discussion 71

Appendices 75

A Appendix 77
A.1 Statistical Physics . . . . . . . . . . . . . . . . . . . . . . . . .77
A.1.1 Logistic Belief Networks . . . . . . . . . . . . . . . .78
A.1.2 Gibbs Sampling . . . . . . . . . . . . . . . . . . . . . .78
A.1.3 Back-Propagation: Regression . . . . . . . . . . . 79
A.2 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

---

文章来源：BigQuant百度网盘