uvagedl

UvA - An Introduction to Group Equivariant Deep Learning

Welcome to the public page for the mini-course on Group Equivariant Deep Learning. The course is still under development, in the upcoming months I will regularly update them. The most important piece of material in this course are the lecture notes.

Erik Bekkers
Amsterdam Machine Learning Lab (AMLab)
University of Amsterdam

---

Autumn School on Scientific Machine Learning and Dynamical Systems

For participants of the Autumn School on Scientific Machine Learning and Dynamical Systems, see

• Tutorial on group convolutions with vector field data.

Useful libraries:

• escnn: Complete library for E(n) equivariant networks for data on grids in 2D and 3D!
• e3nn: Steerable/tensor field methods for 3D point clouds. Central is the Clebsch-Gordan tensor product and related operations.
• e2cnn: My own stripped down version of regular group convolutions with code for point-wise SO(2) Fourier transforms (switching between regular and irreducible representations).
• ponita: Also my own code: latest code for 3D point clouds using regular group convolutions over position-orientation space. As expressive as steerable methods but much faster! And easy to use, check it out! (Benchmarked on interatomic energy and force prediction, n-body interacting particle systems, and denoising diffusion models for generating 3D molacular representations.

---

CODE: Tutorials and other deep learning topics

• See Deep Learning 2 Course Website for our GEDL tutorials, but also for materials on other interesting topics such as Bayesian Deep Learning, Deep Probabilstic Models, Causality, Dynamical Systems, etc. Please check them out as well!

Video lectures

Youtube playlist:

• Youtube playlist

Lecture slide deck:

• Lecture 1: Regular group convolutional neural networks
  • Lecture 1.1 - Introduction
  • Lecture 1.2 - Group theory: Groups, group product, inverse, action, representation, affine groups G = Rd ⋊ H
  • Lecture 1.3 - Regular group convolutions: Template matching viewpoing
  • Lecture 1.4 - SE(2) Equivariant NN Example: histopathology
  • Lecture 1.5 - A brief history of G-CNNs
  • Lecture 1.6 - Group Theory: Transitive action, homogeneous space, quotient space
  • Lecture 1.7 - Group convolutions are all you need! (Equivariant linear layers between feature maps are group convolutions)
• Lecture 2: Steeragble group convolutional neural networks
  • Lecture 2.1 - Steerable kernels/basis functions (Definition and SO(2) example)
  • Lecture 2.2 - Revisiting regular G-convs with steerable kernels (Motivating the Fourier transform)
  • Lecture 2.3 - Group Theory: Irreducible representations and Fourier transform
  • Lecture 2.4 - Group Theory: Induced representations and feature fields
  • Lecture 2.5 - Steerable group convolutions, and how to use them
  • Lecture 2.6 - Activation functions for steerable G-CNNs
  • Lecture 2.7 - Derivation of Harmonic networks from regular g-convs
• Lecture 3: Equivariant graph neural networks
  • Lecture 3.1 - Motivation for SE(3) equivariant graph NNs
  • Lecture 3.2 - Equivariant message passing as non-linear convolution
  • Lecture 3.3 - Tensor products as conditional linear layers (and MLPs)
  • Lecture 3.4 - Group Theory: SO(3) irreps (Wigner-D matrices), Clebsch-Gordan TP
  • Lecture 3.5 - Literature: 3D Steerable (graph) convolutions
  • Lecture 3.6 - Literature: Regular (as opposed to steerable) equivariant graph NNs
  • Lecture 3.7 - Literature: Gauge equivariant graph NNs

Lecture notes

• Lecture notes (work in progress…)

The lecture notes that I wrote when preparing the course are a bit sloppy and contain various typos (my apologies for still not having found the time to fix it). Instead I highly recommend the UvA Master AI thesis by Lars Veefkind. It contains all the essentials for learning about group convolutions, and includes representation theory (Peter-Weyl theorem, Schurr’s Lemma etc.) presented in an accessible and manner!

• Thesis by Lars Veefkind on A Probabilistic Approach to Learning the Degree of Equivariance in Steerable CNNs.

Other materials

Slides and lecture recording (summer school):

• Slides and corresponding link to recorded talk
• Slides talk at Lorentz Center

Colab assignments (to be updated…):

• [Library demos] Please check out these amazing libraries: e2cnn and e3nn
• Colab Assigment 1 (SE(2) gconvs): Thanks to Gabriele Cesa for providing these! These assignments were used in the Geometric-Deep Learning Course
• Colab Assigment 2 (SE(3) steerable gconvs