Week

Topic

1

Floating point arithmetic. Approximating sequences. Order of convergence. Summation order and floating point non-associativity. Finite difference approximation of derivative and catastrophic loss of precision. Condition number.

2

Linear combinations. Vector spaces and subspaces. Bases. Dimension. Orthogonal matrices. Matrix subspaces. Fundamental theorem of linear algebra. Rank-nullity.

3

Vector and matrix norms. Singular value decomposition theorem & proof. Link to Karhunen-Loève. Rank-1 expansions. Operator approximation.

4

Linear statement of principal applied mathematics problems: coordinate changes (linear systems), reduced-order models (least squares), operator invariants (eigenproblems). Solution by the SVD. Pseudo-inverse. Midterm 1.

5

Additional operator representations: $QR$, $LU$, $L{L}^{\ast }$, $QT{Q}^{\ast }$. Computational complexity.

6

Projection: Householder, Givens. Hessenberg form. $QR$ least-squares solution. Stability

7

Interpolation in the monomial basis. Newton form (matrix triangularization). Lagrange form, including barycentric. Continuum limits. Taylor series. Residuals. Finite difference calculus.

8

Interpolation in $B$-spline basis. Bezier curves and surfaces. Computational geometry. Simplicia. Midterm 2.

9

Approximation in monomial basis. Continuum norms. Stationarity conditions & extrema. $L_1,L_2,L_{\infty }$ approximants (momentum, energy, constraints of physical systems).

10

Interpolation in orthogonal bases: trigonometric, Legendre, Laguerre, Hermite, Bessel, spherical harmonics. Fast decompositions.

11

Linear operator approximation 1: quadrature ($\int d{x}^n$). Newton-Cotes. Moments. Gauss. Convergence. Stability.

12

Linear operator approximation 2: differentiation ($d/d{x}^n,\nabla ,{\nabla }^2$), linear ODE (${\sum }_k{a}_{k}d/d{x}^k$). Convergence. Stability.

13

Non-linear operator approximation 1: $f:ℝ\to ℝ,f\left(x\right)=0$. 0-degree approximant (bisection), 1-degree approximants (secant, Newton). Convergence, fixed points.

14

Non-linear operator approximation 2: Non-linear operator composition $f=l_n\circ \dots \circ l_1$, Quasi-linear approximants $𝑨=𝝈\circ {𝑳}_n\circ \dots 𝝈\circ {𝑳}_1$. Artificial neural networks.

15

Non-linear operator approximation 3: $f:{ℝ}^n\to ℝ,f\left(x\right)=0$. 0,1,2-degree approximants. Convexity, steepest descent, sampled descent directions (stochastic gradient). Recurrent, generative, adversarial neural networks.

Week

Topic

1

Mixed-operator approximation 1: Sum of linear operators, linear integro-differential equations

2

Mixed-operator approximation 2: Sum of linear with non-linear operators, general ODEs (${\sum }_k{a}_{k}d/d{x}^k+f$, $y^{\text{'}}=f\left(y\right)$). Linear multistep methods. Stability. Convergence

3

Mixed-operator approximation 3: Runge-Kutta methods

4

Mixed-operator approximation 4: Integro-differential equations. Fredholm alternative.

5

Mixed-operator approximation 5: Fractional calculus approximation. Midterm 1

6

Operator invariants. Eigenproblems. Spectral expansion. Characteristic polynomial. Cayley-Hamilton.

7

Iterative coordinate transforms: Jacobi, Gauss-Seidel, SOR

8

Iterative reduced order methods: Krylov spaces, Arnoldi & Lanczos factorizations. Conjugate gradient, GMRES.

9

Iterative operator invariant methods: $QR$, Lanczos, Arnoldi. SVD computation

10

Random variables, probability distributions, sampling, random number generators

11

Operator sampling: random choice (Glimm's method, Monte Carlo integration)

12

Stochastic calculus: stochastic processes, Ito, Stratonovich

13

SDE: Euler-Maruyama, Milstein, Runge-Kutta

14

Discrete representations: graphs, graph Laplacian

15

Differentiation & integration on graphs, graph diffusion equation.