Mean field sequence: an introduction
This article introduces the concept of Mean Field Theory (MFT) in the context of neural network interpretability, drawing parallels to statistical physics. It discusses the mathematical framework, its applications, and experimental results, particularly highlighting its relevance for understanding complex neural network behaviors.
Episodes
Chapters
01Introduction to Mean Field Theory
Introduces adaptive mean field theory as an approach to interpreting neural network internals and outlines the goals of the series.
1:22Original02Mean Field Theory Overview
Explains that adaptive mean field theory models infinite-width networks by treating neurons as interacting particles to reveal emergent features.
2:23Original03FAQ on MFT
Addresses common questions about MFT, arguing that MFT generalizes beyond Gaussian process/NTK limits and applies to SGD.
5:31Original04Background–Foreground Self-Consistency
Describes self-consistency in physics-like mean-field settings where the background and foreground influence each other through a fixed-point relationship.
2:06Original05Neural Nets as Mean Field Systems
Argues that neural nets, due to high connectivity, can be described by a mean-field loop in which foreground neurons interact with a self-consistent background.
3:40Original06Self-Consistency Equations
States that the background field satisfies a fixed-point equation due to mutual dependence of components and background.
2:07Original07Toy 2-Layer Self-Consistency
Demonstrates a wide two-layer toy example illustrating how a background and independently trained foreground neurons align, showing self-consistency.
2:20Original