Random Matrix Methods for Statistics and Machine Learning

Mastermath, Spring 2027

Level: M2
Credits: 6 EC
Lectures: Monday, 10:00–12:45

1. Prerequisites

Students are expected to have a solid background in the following subjects.

Linear algebra and matrix analysis

  • Eigenvalues, eigenvectors and matrix diagonalisation

  • Spectral theorem for real symmetric and Hermitian matrices

  • Singular value decomposition

  • Positive definite and positive semidefinite matrices

  • Matrix norms, trace, determinant and matrix inversion

  • Basic properties of covariance matrices

Probability and statistics

  • Random variables, random vectors and their distributions

  • Expectation, variance and covariance

  • Independence and conditional expectation

  • Gaussian random variables and multivariate Gaussian distributions

  • Laws of large numbers and central limit theorem

  • Basic statistical estimation, including sample covariance matrices

Analysis

  • Limits, continuity and convergence of sequences and functions

  • Differentiation, integration and Taylor expansions

  • Familiarity with mathematical proofs and elementary asymptotic arguments

Basic complex analysis, particularly complex differentiation and integration, is useful but not required. Measure-theoretic probability is helpful but not essential.

Students who have completed Mastermath High-Dimensional Probability or a comparable advanced probability or statistics course will have useful additional background. There are no formal dependencies on other Mastermath courses.

As preparation, students may consult Chapter 1 and Section 2.1 of Couillet and Liao (2022).

2. Aim of the course

High-dimensional data and overparameterised models are increasingly common in statistics and machine learning. In these settings, classical fixed-dimensional asymptotic theory may provide misleading predictions. Random matrix theory offers mathematical tools for understanding the spectral behaviour of large random matrices when dimension and sample size grow simultaneously.

This course introduces the mathematical foundations of random matrix methods and their applications to high-dimensional statistics and machine learning.

The main topics are:

  • Empirical spectral distributions, resolvents and Stieltjes transforms

  • Semicircle law and Marchenko–Pastur law

  • Marchenko–Pastur equations and deterministic equivalents

  • Spectral properties of large sample covariance and correlation matrices

  • Eigenvalue and eigenvector behaviour in high dimensions

  • Spiked covariance models and phase transitions

  • High-dimensional PCA, covariance estimation and signal detection

  • Random kernel matrices and large-scale kernel methods

  • Selected applications to neural networks, clustering and community detection

The emphasis is on mathematical understanding, interpretation of asymptotic results and their consequences for statistical and machine learning procedures.

Learning outcomes

Upon successful completion of the course, students will be able to:

  1. Explain the fundamental differences between classical and high-dimensional asymptotic regimes.

  2. Apply spectral tools, including Stieltjes transforms and resolvents, to random matrix models.

  3. Derive or use limiting spectral laws for important classes of random matrices.

  4. Analyse eigenvalue and eigenvector behaviour in high-dimensional covariance models.

  5. Explain phase transitions in spiked models and their consequences for PCA and signal detection.

  6. Apply random matrix insights to selected problems in statistics and machine learning.

Teaching format

The course consists of weekly meetings combining theoretical lectures with supervised problem-solving sessions.

Each meeting includes approximately 90 minutes of lectures, 60 minutes of interactive exercises and a 15-minute break.

3. Rules about exercises and examination

Exercises

Ungraded exercise sheets will be provided regularly throughout the course. The exercises will include theoretical problems, mathematical derivations and applications of random matrix methods.

Students are strongly encouraged to work on these exercises independently before the corresponding problem-solving sessions. Selected exercises will be discussed during the weekly meetings, and solutions or solution outlines will be provided.

There are no compulsory homework assignments, and exercises do not contribute to the final grade.

Students may use AI tools to support their learning and to explore possible solution approaches. However, they are encouraged to work through mathematical arguments independently, as the examination assesses individual understanding without AI assistance.

Examination

The final grade is determined entirely by an individual, supervised, written examination (100%).

The examination will assess:

  • Understanding of the main concepts and theoretical results

  • Ability to carry out mathematical derivations and proofs

  • Application of random matrix methods to statistical and machine learning problems

  • Interpretation of high-dimensional spectral phenomena

A representative set of practice exercises and a mock examination will be provided to help students prepare for the examination.

Retake

The retake will also consist of an individual, supervised, written examination covering the same syllabus and having the same assessment structure as the regular examination.

The examination and retake each account for 100% of the grade for the respective attempt. There are no homework grades to carry over.

4. Lecture notes and literature

Primary textbook

Couillet, R. and Liao, Z. (2022). Random Matrix Methods for Machine Learning. Cambridge University Press.

Selected material from the following chapters will be covered:

  • Chapter 1: Introduction

  • Chapter 2: Random Matrix Theory

  • Chapter 3: Statistical Inference in Linear Models

  • Chapter 4: Kernel Methods

  • Chapter 5: Large Neural Networks (selected material)

  • Chapter 7: Community Detection on Graphs (selected material)

The course will not cover all sections of these chapters. Detailed reading instructions will be provided during the semester.

Additional references

Bai, Z. D. and Silverstein, J. W. (2010). Spectral Analysis of Large Dimensional Random Matrices. Springer.

Yao, J., Zheng, S. and Bai, Z. D. (2015). Large Sample Covariance Matrices and High-Dimensional Data Analysis. Cambridge University Press.

Additional lecture notes, exercises and supplementary references will be made available on the ELO platform.

5. Lecturers

Lecturer: Dr. Nestor Parolya
Affiliation: TU Delft/Applied Mathematics
Email: n.parolya@tudelft.nl

Additional information regarding office hours and communication will be provided at the beginning of the course.