Contents 3
1 Introduction 17
2 Groups, Rings, and Fields 19
3 Vector Spaces, Bases, Linear Maps 45
4 Matrices and Linear Maps 107
5 Haar Bases, Haar Wavelets, Hadamard Matrices
6 Direct Sums 155
7 Determinants 181
8 Gaussian Elimination, LU, Cholesky, Echelon Form 219
9 Vector Norms and Matrix Norms 301
10 Iterative Methods for Solving Linear Systems 351
11 The Dual Space and Duality 375
12 Euclidean Spaces 413
13 QR-Decomposition for Arbitrary Matrices 467
14 Hermitian Spaces 489
15 Eigenvectors and Eigenvalues 529
16 Unit Quaternions and Rotations in SO(3) 561
17 Spectral Theorems 583
18 Computing Eigenvalues and Eigenvectors 619
19 Introduction to The Finite Elements Method 649
20 Graphs and Graph Laplacians; Basic Facts 671
21 Spectral Graph Drawing 695
22 Singular Value Decomposition and Polar Form 705
23 Applications of SVD and Pseudo-Inverses 723
24 Basics of Ane Geometry 759
25 Embedding an Ane Space in a Vector Space 805
26 Basics of Projective Geometry 823
27 The Cartan{Dieudonne Theorem 923
28 Isometries of Hermitian Spaces 949
29 The Geometry of Bilinear Forms; Witt's Theorem 963
30 Polynomials, Ideals and PID's 1019
31 Annihilating Polynomials; Primary Decomposition 1055
32 UFD's, Noetherian Rings, Hilbert's Basis Theorem 1085
33 Tensor Algebras 1111
34 Exterior Tensor Powers and Exterior Algebras 1165
35 Introduction to Modules; Modules over a PID 1207
36 Normal Forms; The Rational Canonical Form 1253
37 Topology 1287
38 A Detour On Fractals 1367
39 Dierential Calculus 1375
40 Extrema of Real-Valued Functions 1417
41 Newton's Method and Its Generalizations 1441
42 Quadratic Optimization Problems 1449
43 Schur Complements and Applications 1469
44 Convex Sets, Cones, H-Polyhedra 1477
45 Linear Programs 1489
46 The Simplex Algorithm 1503
47 Linear Programming and Duality 1535
48 Basics of Hilbert Spaces 1571
49 General Results of Optimization Theory 1591
50 Introduction to Nonlinear Optimization 1647
51 Subgradients and Subdierentials 1737
52 Dual Ascent Methods; ADMM 1783
53 Ridge Regression and Lasso Regression 1819
54 Positive Denite Kernels 1837
55 Soft Margin Support Vector Machines 1865
X Appendices 1927
A Total Orthogonal Families in Hilbert Spaces 1929
A.1 Total Orthogonal Families, Fourier Coecients . . . . . . . . . . . . . . . . 1929
A.2 The Hilbert Space `2(K) and the Riesz-Fischer Theorem . . . . . . . . . . . 1937
B Zorn's Lemma; Some Applications 1947
B.1 Statement of Zorn's Lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . 1947
B.2 Proof of the Existence of a Basis in a Vector Space . . . . . . . . . . . . . . 1948
B.3 Existence of Maximal Proper Ideals . . . . . . . . . . . . . . . . . . . . . . 1949
Bibliography 1951