Math 341 - Numerical Linear Algebra for Deep Neural Networks

Class: TuTh 10:00AM - 11:15PM in Math & Psychology 401 (1/26-5/12/2026), Instructor: Bedrich Sousedik

This course provides an introduction to linear algebra and related numerical aspects with an emphasis on the applications to deep neural networks. The topics include matrix factorizations (LU, QR, SVD), computation of eigenvalues, least-squares, special matrices including their use in probability and statistics, optimization algorithms such as Newton’s method, (stochastic) gradient descent and ADAM, and, finally, how these techniques are used to learn from data are discussed.

Prerequisites: You must complete MATH 152, MATH 221, and CMSC 201 with a grade of ‘C’ or better before you can enroll in this class.

The detailed class syllabus (pdf) is here. However, look below to see the actual progress of the class.

How to get started with Matlab at UMBC document is here, and some basic software tutorials are here.

1/27-29 No class (campus closed due to snow).

2/3 Introduction. Review of matrices and vectors.

2/5 Matrix-matrix multiplication, matrix inverse, A=CR factorization. Hw 1 (due 2/17).

2/10 LU factorization, permutation matrices.

2/12 Cholesky factorization, block matrices, Schur complament. Rectangular systems. Hw 2 (due 2/19).

2/17 Rectangular systems: Echelon form, basis of Ax=0, four fundamental subspaces.

2/19 Four fundamental subspaces (orthogonal complementarity), Orthogonal matrices. Hw 3 (due 3/3).

2/24 Gram-Schmidt algorithm. QR factorization.

2/26 Orthogonal matrices: rotators and reflectors. QR factorization revisited. Hw 4 (due 3/5).

3/3 The (lineasr) least squares problems: geometric perspective, normal equations, linearization of some models, use of QR.

3/5 Computing in presence of rounding errors: condition number of a matrix.

3/10 SVD. Hw 5 (due 3/26).

3/12 Exam.

3/15-23 Spring break.

3/24 No class (out of town).

3/26 ... eigenvalues ... ?

3/26

3/31

4/2

4/7

4/9

4/14

4/16

4/21

4/23

4/28

4/30

5/5

5/7

5/12