uvagedl.github.io
UvA - An Introduction to Group Equivariant Deep Learning
Welcome to the public page for the mini-course on Group Equivariant Deep Learning. The course is still under development, in the upcoming months I will regularly update them. The most important piece of material in this course are the lecture notes.
Erik Bekkers
Amsterdam Machine Learning Lab (AMLab)
University of Amsterdam
Autumn School on Scientific Machine Learning and Dynamical Systems
For participants of the Autumn School on Scientific Machine Learning and Dynamical Systems, see
Useful libraries:
- escnn: Complete library for E(n) equivariant networks for data on grids in 2D and 3D!
- e3nn: Steerable/tensor field methods for 3D point clouds. Central is the Clebsch-Gordan tensor product and related operations.
- e2cnn: My own stripped down version of regular group convolutions with code for point-wise SO(2) Fourier transforms (switching between regular and irreducible representations).
- ponita: Also my own code: latest code for 3D point clouds using regular group convolutions over position-orientation space. As expressive as steerable methods but much faster! And easy to use, check it out! (Benchmarked on interatomic energy and force prediction, n-body interacting particle systems, and denoising diffusion models for generating 3D molacular representations.
CODE: Tutorials and other deep learning topics
- See Deep Learning 2 Course Website for our GEDL tutorials, but also for materials on other interesting topics such as Bayesian Deep Learning, Deep Probabilstic Models, Causality, Dynamical Systems, etc. Please check them out as well!
Video lectures
Youtube playlist:
Lecture slide deck:
- Lecture 1: Regular group convolutional neural networks
- Lecture 1.1 - Introduction
- Lecture 1.2 - Group theory: Groups, group product, inverse, action, representation, affine groups G = Rd ⋊ H
- Lecture 1.3 - Regular group convolutions: Template matching viewpoing
- Lecture 1.4 - SE(2) Equivariant NN Example: histopathology
- Lecture 1.5 - A brief history of G-CNNs
- Lecture 1.6 - Group Theory: Transitive action, homogeneous space, quotient space
- Lecture 1.7 - Group convolutions are all you need! (Equivariant linear layers between feature maps are group convolutions)
- Lecture 2: Steeragble group convolutional neural networks
- Lecture 2.1 - Steerable kernels/basis functions (Definition and SO(2) example)
- Lecture 2.2 - Revisiting regular G-convs with steerable kernels (Motivating the Fourier transform)
- Lecture 2.3 - Group Theory: Irreducible representations and Fourier transform
- Lecture 2.4 - Group Theory: Induced representations and feature fields
- Lecture 2.5 - Steerable group convolutions, and how to use them
- Lecture 2.6 - Activation functions for steerable G-CNNs
- Lecture 2.7 - Derivation of Harmonic networks from regular g-convs
- Lecture 3: Equivariant graph neural networks
- Lecture 3.1 - Motivation for SE(3) equivariant graph NNs
- Lecture 3.2 - Equivariant message passing as non-linear convolution
- Lecture 3.3 - Tensor products as conditional linear layers (and MLPs)
- Lecture 3.4 - Group Theory: SO(3) irreps (Wigner-D matrices), Clebsch-Gordan TP
- Lecture 3.5 - Literature: 3D Steerable (graph) convolutions
- Lecture 3.6 - Literature: Regular (as opposed to steerable) equivariant graph NNs
- Lecture 3.7 - Literature: Gauge equivariant graph NNs
Lecture notes
- Lecture notes (work in progress…)
Other materials
Slides and lecture recording (summer school):
- Slides and corresponding link to recorded talk
- Slides talk at Lorentz Center
Colab assignments (to be updated…):
- [Library demos] Please check out these amazing libraries: e2cnn and e3nn
- Colab Assigment 1 (SE(2) gconvs): Thanks to Gabriele Cesa for providing these! These assignments were used in the Geometric-Deep Learning Course
- Colab Assigment 2 (SE(3) steerable gconvs