08年第二版-Stochastic Optimization Methods

相似文件 换一批

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

Stochastic Optimization Methods
Univ. Prof. Dr. Kurt Marti
Federal Armed Forces
University Munich
Department of Aerospace Engineering and Technology
85577 Neubiberg/München
Germany
kurt.marti@unibw-muenchen.de
2008 Springer-Verlag Berlin Heidelberg

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏3 回帖

关键词：Optimization Stochastic Stochast Methods Method

Part I Basic Stochastic Optimization Methods
1 Decision/Control Under Stochastic Uncertainty . . . . . . . . . . . 3
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Deterministic Substitute Problems: Basic Formulation . . . . . . . . 5
1.2.1 Minimum or Bounded Expected Costs . . . . . . . . . . . . . . . 6
1.2.2 Minimum or Bounded Maximum Costs
(Worst Case) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2 Deterministic Substitute Problems in Optimal Decision
Under Stochastic Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1 Optimum Design Problems with Random Parameters . . . . . . . . 9
2.1.1 Deterministic Substitute Problems in Optimal
Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.1.2 Deterministic Substitute Problems in Quality
Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2 Basic Properties of Substitute Problems . . . . . . . . . . . . . . . . . . . . 18
2.3 Approximations of Deterministic Substitute Problems
in Optimal Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.3.1 Approximation of the Loss Function . . . . . . . . . . . . . . . . 20
2.3.2 Regression Techniques, Model Fitting, RSM . . . . . . . . . . 22
2.3.3 Taylor Expansion Methods . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.4 Applications to Problems in Quality Engineering . . . . . . . . . . . . 29
2.5 Approximation of Probabilities: Probability Inequalities . . . . . . 30
2.5.1 Bonferroni-Type Inequalities . . . . . . . . . . . . . . . . . . . . . . . . 31
2.5.2 Tschebyscheff-Type Inequalities . . . . . . . . . . . . . . . . . . . . . 32
2.5.3 First Order Reliability Methods (FORM) . . . . . . . . . . . . . 37
Part II Differentiation Methods
3 Differentiation Methods for Probability and Risk Functions 43
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.2 Transformation Method: Differentiation by Using an Integral
Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2.1 Representation of the Derivatives by Surface Integrals . . 51
3.3 The Differentiation of Structural Reliabilities . . . . . . . . . . . . . . . 54
3.4 Extensions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.4.1 More General Response (State) Functions . . . . . . . . . . . . 57
3.5 Computation of Probabilities and its Derivatives
by Asymptotic Expansions of Integral of Laplace Type . . . . . . . 62
3.5.1 Computation of Probabilities of Structural Failure
and Their Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.5.2 Numerical Computation of Derivatives
of the Probability Functions Arising in Chance
Constrained Programming . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.6 Integral Representations of the Probability Function
P(x) and its Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
3.7 Orthogonal Function Series Expansions I: Expansions
in Hermite Functions, Case m = 1 . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.7.1 Integrals over the Basis Functions and the Coefficients
of the Orthogonal Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.7.2 Estimation/Approximation of P(x) and its Derivatives . 82
3.7.3 The Integrated Square Error (ISE) of Deterministic
Approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.8 Orthogonal Function Series Expansions II: Expansions
in Hermite Functions, Case m > 1 . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.9 Orthogonal Function Series Expansions III: Expansions
in Trigonometric, Legendre and Laguerre Series . . . . . . . . . . . . . 91
3.9.1 Expansions in Trigonometric and Legendre Series . . . . . . 92
3.9.2 Expansions in Laguerre Series . . . . . . . . . . . . . . . . . . . . . . . 92
Part III Deterministic Descent Directions
4 Deterministic Descent Directions and Efficient Points . . . . . . 95
4.1 Convex Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
4.1.1 Approximative Convex Optimization Problem . . . . . . . . . 99
4.2 Computation of Descent Directions in Case of Normal
Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
4.2.1 Descent Directions of Convex Programs . . . . . . . . . . . . . . 105
4.2.2 Solution of the Auxiliary Programs . . . . . . . . . . . . . . . . . . 108
4.3 Efficient Solutions (Points) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
4.3.1 Necessary Optimality Conditions Without Gradients . . . 116
4.3.2 Existence of Feasible Descent Directions
in Non-Efficient Solutions of (4.9a), (4.9b) . . . . . . . . . . . . 117
4.4 Descent Directions in Case of Elliptically Contoured
Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
4.5 Construction of Descent Directions by Using Quadratic
Approximations of the Loss Function . . . . . . . . . . . . . . . . . . . . . . 121
Part IV Semi-Stochastic Approximation Methods
5 RSM-Based Stochastic Gradient Procedures . . . . . . . . . . . . . . . 129
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5.2 Gradient Estimation Using the Response Surface
Methodology (RSM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
5.2.1 The Two Phases of RSM . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
5.2.2 The Mean Square Error of the Gradient Estimator . . . . 138
5.3 Estimation of the Mean Square (Mean Functional) Error . . . . . 142
5.3.1 The Argument Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
5.3.2 The Criterial Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
5.4 Convergence Behavior of Hybrid Stochastic Approximation
Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
5.4.1 Asymptotically Correct Response Surface Model . . . . . . 148
5.4.2 Biased Response Surface Model . . . . . . . . . . . . . . . . . . . . . 150
5.5 Convergence Rates of Hybrid Stochastic Approximation
Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
5.5.1 Fixed Rate of Stochastic and Deterministic Steps . . . . . . 158
5.5.2 Lower Bounds for the Mean Square Error . . . . . . . . . . . . 169
5.5.3 Decreasing Rate of Stochastic Steps . . . . . . . . . . . . . . . . . 173

6 Stochastic Approximation Methods with Changing
Error Variances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
6.2 Solution of Optimality Conditions . . . . . . . . . . . . . . . . . . . . . . . . . 178
6.3 General Assumptions and Notations . . . . . . . . . . . . . . . . . . . . . . . 179
6.3.1 Interpretation of the Assumptions . . . . . . . . . . . . . . . . . . . 181
6.3.2 Notations and Abbreviations in this Chapter . . . . . . . . . . 182
6.4 Preliminary Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
6.4.1 Estimation of the Quadratic Error . . . . . . . . . . . . . . . . . . . 183
6.4.2 Consideration of the Weighted Error Sequence . . . . . . . . 185
6.4.3 Further Preliminary Results . . . . . . . . . . . . . . . . . . . . . . . . 188
6.5 General Convergence Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
6.5.1 Convergence with Probability One . . . . . . . . . . . . . . . . . . . 190
6.5.2 Convergence in the Mean . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
6.5.3 Convergence in Distribution . . . . . . . . . . . . . . . . . . . . . . . . 195
6.6 Realization of Search Directions Yn . . . . . . . . . . . . . . . . . . . . . . . . 204
6.6.1 Estimation of G∗ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
6.6.2 Update of the Jacobian . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
6.6.3 Estimation of Error Variances . . . . . . . . . . . . . . . . . . . . . . . 216
6.7 Realization of Adaptive Step Sizes . . . . . . . . . . . . . . . . . . . . . . . . . 220
6.7.1 Optimal Matrix Step Sizes . . . . . . . . . . . . . . . . . . . . . . . . . . 221
6.7.2 Adaptive Scalar Step Size . . . . . . . . . . . . . . . . . . . . . . . . . . 227
6.8 A Special Class of Adaptive Scalar Step Sizes . . . . . . . . . . . . . . . 236
6.8.1 Convergence Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
6.8.2 Examples for the Function Qn(r) . . . . . . . . . . . . . . . . . . . . 241
6.8.3 Optimal Sequence (wn) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
6.8.4 Sequence (Kn) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
Part V Reliability Analysis of Structures/Systems
7 Computation of Probabilities of Survival/Failure
by Means of Piecewise Linearization of the State
Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
7.2 The State Function s∗ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
7.2.1 Characterization of Safe States . . . . . . . . . . . . . . . . . . . . . . 258
7.3 Probability of Safety/Survival . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
7.4 Approximation I of ps, pf : FORM . . . . . . . . . . . . . . . . . . . . . . . . . 262
7.4.1 The Origin of IRν Lies in the Transformed
Safe Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
7.4.2 The Origin of IRν Lies in the Transformed Failure
Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
7.4.3 The Origin of IRν Lies on the Limit State Surface . . . . . . 268
7.4.4 Approximation of Reliability Constraints . . . . . . . . . . . . . 269
7.5 Approximation II of ps, pf : Polyhedral Approximation
of the Safe/Unsafe Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
7.5.1 Polyhedral Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . 273
7.6 Computation of the Boundary Points . . . . . . . . . . . . . . . . . . . . . . 279
7.6.1 State Function s∗ Represented by Problem A . . . . . . . . . 280
7.6.2 State Function s∗ Represented by Problem B . . . . . . . . . 280
7.7 Computation of the Approximate Probability Functions . . . . . . 282
7.7.1 Probability Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
7.7.2 Discretization Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
7.7.3 Convergent Sequences of Discrete Distributions . . . . . . . 293
Part VI Appendix
A Sequences, Series and Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
A.1 Mean Value Theorems for Deterministic Sequences . . . . . . . . . . 301
A.2 Iterative Solution of a Lyapunov Matrix Equation . . . . . . . . . . . 309
B Convergence Theorems for Stochastic Sequences . . . . . . . . . . . 313
B.1 A Convergence Result of Robbins–Siegmund . . . . . . . . . . . . . . . . 313
B.1.1 Consequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
B.2 Convergence in the Mean . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
B.3 The Strong Law of Large Numbers for Dependent Matrix
Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
B.4 A Central Limit Theorem for Dependent Vector Sequences . . . . 319
C Tools from Matrix Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
C.1 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
C.2 The v. Mises-Procedure in Case of Errors . . . . . . . . . . . . . . . . . . . 322
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335

好东西顶起来