Neural Message Passing for Quantum Chemistry

Abstract

Message Passing Neural Networks (MPNNs) are a unified framework for existing neural network models that learn graph-based representations for molecular properties, achieving state-of-the-art results on chemical prediction benchmarks.

Introduction

While deep learning has seen success in language, audio, and image processing, its application to chemistry is nascent, necessitating the development of models with appropriate inductive biases, such as those operating on graph-structured data like molecules.

Message Passing Neural Networks Framework and QM9 Dataset

The MPNN framework unifies existing graph-based neural models, and its effectiveness is demonstrated on the QM9 dataset for predicting quantum mechanical properties of organic molecules, achieving chemical accuracy on most targets.

Message Passing Neural Networks

Message Passing Neural Networks (MPNNs) update node hidden states through a message passing phase and then compute a graph-level representation using a readout function, with learned message, update, and readout functions.

MPNN Variants in Literature

Several existing models like Convolutional Networks for Learning Molecular Fingerprints, Gated Graph Neural Networks, Interaction Networks, Molecular Graph Convolutions, Deep Tensor Neural Networks, and Laplacian Based Methods can be described within the MPNN framework.

Laplacian Based Methods and Moving Forward

Laplacian-based methods generalize convolutions to graphs, and while effective, computational time is a concern, prompting research into modifications like passing messages on subsets of the graph.

QM9 Dataset Details

The QM9 dataset contains 134k organic molecules with computed DFT properties, providing a benchmark for evaluating MPNNs on tasks related to atomic binding, molecular vibrations, electron states, and electron spatial distribution.

MPNN Variants and Training

Various MPNN variants were explored, including different message functions, virtual graph elements, readout functions, and a multi-tower architecture to improve scalability and performance, trained on the QM9 dataset using SGD with the ADAM optimizer.

Results and State-of-the-Art Performance

MPNNs achieved chemical accuracy on 11 out of 13 QM9 targets, outperforming previous state-of-the-art methods, with improvements seen when spatial information and explicit hydrogens were included, and through ensembling.

Towers and Additional Experiments

The multi-tower MPNN architecture improved generalization and training time, outperforming a baseline GG-NN model, and while the pair message function performed worse than the edge network, further research into attention mechanisms is suggested.

Conclusions and Future Work

MPNNs possess a useful inductive bias for molecular property prediction, highlighting the importance of long-range interactions and scalability, with future work focusing on generalization to larger graphs and incorporating attention mechanisms.

Acknowledgements and References

The authors acknowledge helpful discussions and list references for various models and techniques used in their research on neural message passing for quantum chemistry.

Graph Laplacian Transformation

The neural message passing framework extends graph Laplacian methods by applying a nonlinearity after a weighted sum of node features, representing a layer-wise update.

Layer-wise Propagation Rule

The Kipf & Welling (2016) model uses a layer-wise propagation rule that approximates graph Laplacian methods by averaging neighbor information, updated by a trainable weight matrix and a nonlinearity.

Atomization Energies

Four types of atomization energies are defined: U0 (0 Kelvin, fixed volume), U (room temperature, fixed volume), H (room temperature, fixed pressure), and G (room temperature, fixed pressure), all representing the energy to break a molecule into atoms.

Molecular Vibrations

The highest fundamental vibrational frequency indicates molecular rigidity, while the Zero Point Vibrational Energy represents the minimum vibrational energy a molecule possesses even at absolute zero.

Electronic Orbital Energies

HOMO and LUMO energies define the highest occupied and lowest unoccupied electron states, respectively, with their difference, the electron energy gap, determining the minimum energy for electronic excitation.

Electron Distribution Properties

Electronic Spatial Extent quantifies the spread of the electron cloud, and the Norm of the dipole moment reflects the anisotropy of charge distribution, influencing material properties.

Polarizability and Performance Metrics

Static polarizability measures a molecule's response to an electric field, and Table 5 presents mean absolute errors for various chemical properties across different targets and models.

Model Performance Comparisons

Tables 6-10 compare different message passing neural network architectures, training set sizes, and input featureizations, demonstrating the importance of capturing long-range interactions and the effectiveness of the edge network.