Neural Message Passing for Quantum Chemistry

Host: We're diving into a fascinating look at how artificial intelligence is tackling chemistry and drug discovery. The authors of this paper saw incredible potential for using supervised machine learning to predict the properties of different molecules. Guest: That sounds highly useful for finding new materials or medicines, but how exactly does a neural network look at a molecule? Host: It essentially looks at them as graphs, where atoms are nodes connected by bonds. The authors noticed several existing models were already doing this by using what they call a message passing algorithm. Guest: What does it mean for a neural network to pass messages inside a molecule? Host: It means the different parts of the molecule share information with their neighbors, gathering and aggregating that data to understand the entire molecular structure. This paper takes all those similar, existing approaches and unites them into a single, common framework. Guest: What do they call this new unified framework? Host: They call them Message Passing Neural Networks, or MPNNs. After building this framework, they decided to test its limits on a standard chemical prediction benchmark. Guest: Did this unified approach actually perform better than the older models? Host: It really did, achieving state-of-the-art results for predicting molecular properties. Guest: If the results were that strong, what is the next step for this kind of research? Host: They basically argue that their MPNNs did so well, future research will actually need harder datasets featuring larger molecules or much more precise real-world data just to keep pushing the limits of the technology.

Host: We're starting at the very beginning of the research to see exactly why chemistry needs an AI upgrade. Over the past decade, deep neural networks have revolutionized how we process text, audio, and images, but applying them to predict the properties of molecules is still in its infancy. Guest: Why is chemistry lagging behind? What have researchers been doing instead of deep learning? Host: Until now, most machine learning in chemistry has relied heavily on manual feature engineering. It's actually very similar to how image processing worked right before convolutional neural networks became the standard. Guest: What is pushing the field to finally make that leap toward deeper neural networks now? Host: It all comes down to a massive surge in new data. Thanks to large-scale quantum simulations and high-throughput experiments, we are generating chemical data at a rate that classical techniques just can't effectively process. Guest: If we have all this data, can we just plug it into the same AI models used for recognizing faces or translating text? Host: Not quite, because molecules require models with a specific "inductive bias," meaning the architecture needs to naturally fit the physical shape of the data. For chemical systems, that points to neural networks designed to operate on graph-structured data. Guest: That makes sense, since a molecule is essentially a graph of atoms connected by bonds, right? Host: Precisely, and if we can build graph-based models that respect those natural atomic symmetries, they could eventually help automate incredibly complex searches in drug discovery and materials science.

Host: Let's look at how we can predict the quantum properties of molecules using a unified approach called Message Passing Neural Networks. This framework gathers the best parts of several existing graph-based models into one general system. Guest: So by treating a molecule as a graph of atoms and bonds, this framework learns its own features instead of relying on hand-crafted chemical rules? Host: Exactly, which makes it much easier to understand the relationships between different models and invent new variations. To prove how well this works, the researchers tested it on a major benchmark dataset called QM9. Guest: What exactly is inside that dataset, and what are we trying to predict? Host: QM9 contains about 130,000 small organic molecules, and each one has 13 quantum mechanical properties. Normally, those properties are calculated using an expensive, time-consuming simulation method called DFT, but the MPNN is trained to predict them directly. Guest: Does the neural network need to know the exact 3D shape of the molecule to make those predictions? Host: It can use the complete 3D spatial geometry, like atomic distances and bond angles. However, the researchers also tested it using only the basic graph topology, meaning the model had to figure out the 3D conformation implicitly. Guest: How do they know if the model's predictions are actually good enough to be useful to chemists? Host: They measure the results against a strict standard called "chemical accuracy," which is a highly precise target error established by the chemistry community. Guest: Did this new MPNN framework manage to hit that rigorous target? Host: It did, setting a new state of the art by reaching chemical accuracy on 11 of the 13 targets using spatial data, and even hitting 5 targets using just the basic graph structure. Guest: That's a huge step forward for processing molecular data. Host: It really is. The authors hope this kind of rigorous empirical work makes MPNNs the default tool for chemistry, much like how convolutional neural networks eventually took over computer vision.

Host: We are diving into a powerful framework that unifies how we analyze graph-structured data. It is called the Message Passing Neural Network, or MPNN, and it actually organizes at least eight existing models under one mathematical umbrella. Guest: How does this framework actually process a graph? Host: It works through two main phases during its forward pass: a message passing phase, and a readout phase. During the first phase, nodes essentially exchange information with each other for a set number of time steps. Guest: How exactly are they communicating during those steps? Host: Each node gathers information from its direct neighbors using a learned message function. Then, a vertex update function uses those incoming messages to update the node's own internal hidden state. Guest: So every node is continually updating its state based on its surroundings? Host: Exactly, and once those time steps finish, the network moves to the readout phase. Here, a readout function takes all those final individual node states and computes a single feature vector to represent the whole graph. Guest: If the system looks at the nodes in a different order, does that change the final readout? Host: It is mathematically required that the order does not matter. The readout function must be invariant to permutations, which ensures the model always recognizes structurally identical graphs regardless of how the nodes are listed. Guest: That makes sense, so the network is just learning the best functions for the messages, the updates, and the readout? Host: Right, all three are learned, differentiable functions. You can even extend this to learn hidden states for the edges connecting the nodes, though the authors note only one previous model had actually tried that.

Host: Let's look at how the research community has actually built out these message passing networks in practice. While they all follow the same basic blueprint of sending messages and updating nodes, different papers use very different math for those specific steps. Guest: So they're all using the same overall template, but swapping out the internal functions. What does one of those early variations look like? Host: A prime example is a 2015 model by Duvenaud and colleagues, which simply groups neighboring node and edge features together to form a message. However, the math they used adds up all the connected nodes and connected edges entirely separately. Guest: Oh, so it essentially unlinks them. Doesn't that mean the network loses track of which specific edge connects to which specific node? Host: Exactly, it completely misses the correlations between edge states and node states. To try different approaches, later models like Gated Graph Neural Networks swapped out the update function for a Gated Recurrent Unit, reusing the exact same weights at every time step. Guest: Treating it like a recurrent neural network to save on memory makes a lot of sense. Do any of these models actually update the edges themselves, or is it always just the nodes? Host: That brings us to Molecular Graph Convolutions by Kearnes and colleagues. They introduced a really unique twist where the edge representations are actively updated during the message passing phase, rather than just acting as static pathways. Guest: That’s a clever way to capture more complex relationships. Are there other major tweaks researchers have tried? Host: Plenty, like Interaction Networks which allow you to feed external, outside influences into the node updates. On the flip side, Deep Tensor Neural Networks simplify the math significantly by just adding the new message directly to the current node state. Guest: It really highlights how flexible this whole framework is, letting researchers scale the complexity up or down depending on what they are trying to model.

Host: Let's explore how advanced mathematical models are being pushed to solve some massive computational bottlenecks in chemistry. To start, researchers have developed Laplacian-based methods, which essentially take the convolution operations we normally use on images and adapt them to work on graphs. Guest: Images have a neat grid of pixels, so how do you apply those same operations to a messy graph of data? Host: They use the graph's mathematical structure, specifically using something called a graph Laplacian and an adjacency matrix, to define how messages pass between connected nodes. It's incredibly powerful, but calculating this takes a huge amount of computing time. Guest: So how does this connect to chemistry? Are they using these graph models to simulate molecules? Host: Exactly, because calculating molecular properties using exact quantum physics is just too difficult. Scientists usually rely on approximations like Density Functional Theory, or DFT, but it scales terribly as the number of electrons increases. Guest: How slow is it to run those traditional DFT calculations? Host: Computing a single molecule with just nine heavy atoms takes about an hour on a standard processor. If you increase that to a seventeen-atom molecule, the computation time jumps to eight hours. Guest: Wow, if you're trying to screen thousands of molecules for a new material, that's completely unworkable. Couldn't earlier machine learning models speed that up? Host: They tried, but older machine learning approaches relied on hand-engineered features that had a hard time generalizing to new chemical compositions. The advantage of these newer message-passing networks is that they learn the molecule's symmetries and structures directly without relying on those rigid, hand-crafted rules. Guest: That makes sense. What data do they use to actually train and test these newer networks? Host: They use a public dataset called QM9, which contains about 134,000 small molecules made of basic elements like carbon, oxygen, nitrogen, and fluorine. It provides a massive, standardized testing ground to see if these models can truly outperform traditional physics calculations.

Host: Let's look closely at the specific chemical properties our models need to predict, and the architectural tweaks required to handle them. The dataset they use, called QM9, features properties that fall into four main buckets. Guest: What kind of buckets are we talking about? Host: The first is binding energy, which measures how much energy it takes to break the molecule apart. The other three cover the molecule's fundamental vibrations, its electron energy states, and the physical spatial spread of those electrons. Guest: That is a wide range of complex quantum chemistry. How do they set up the neural network to process all that? Host: They use a graph neural network baseline, but they make an interesting choice by treating undirected chemical bonds as directed edges. Essentially, every bond becomes a two-way street with separate incoming and outgoing messages. Guest: If a normal chemical bond is just a shared connection, why force it to have a direction in the model? Host: It gives the model separate, dedicated channels for incoming and outgoing data. This actually doubles the size of the message channel, allowing the network to capture much more detail. Guest: Got it. And what exactly goes into these messages passing between the atoms? Host: They tried a few different mathematical rules, starting with simple matrix multiplication. But they also designed an "Edge Network" function that can process complex, continuous inputs, like the exact spatial distance between two atoms. Guest: Does the message only depend on the atom sending the information, or does the receiving atom matter too? Host: That was another variation they tested, called a "Pair Message." In that setup, the information passed along the bond is shaped by the hidden states of both the sending and the receiving atoms. Guest: What if two atoms are on opposite ends of a large molecule and don't share a direct bond? Host: To solve that, they introduce "virtual edges" as a preprocessing step. By adding these artificial connections between unconnected atoms, they create shortcuts so information can travel long distances across the molecule instantly.

Host: Let's explore some structural variations and training strategies that make these networks more efficient and expressive. One fascinating tweak the researchers tested was adding a latent "master" node to the graph. Guest: What is a master node? Does it control the other nodes in the network? Host: Not exactly; it acts more like a global scratchpad. It connects to every single input node with a special edge, allowing each node to read and write to it during every step of message passing, which helps information travel across long distances instantly. Guest: That sounds incredibly efficient. How does the network gather up all that global information once the message passing is actually done? Host: They experimented with a readout function called a set2set model, which handles sets of data beautifully. It processes all the node states and computes a final graph-level embedding that doesn't care about the order of the nodes, making it much more expressive than just summing the states together. Guest: With complex readouts and all those nodes passing messages, doesn't the sheer volume of math become a bottleneck? Host: It definitely can, requiring a massive number of multiplications, which is why they introduced a "Multiple Towers" approach for scalability. Instead of processing one large, dense embedding per node, they break the embedding into several smaller copies, run message passing on each tower separately, and then mix them back together using a neural network. Guest: So splitting the data into these separate towers actually saves processing time? Host: Exactly, because running the calculations on several smaller dimensions takes much less time than on one giant one, yielding up to a two-fold speedup in practice. This speedup is crucial because how you represent the physical inputs can be very computationally expensive. Guest: What kind of physical inputs make it so expensive? Host: A great example is hydrogen. If you choose to represent every single hydrogen atom as its own explicit node in the graph, rather than just a simple count feature, it can slow down the training time by a factor of ten. Guest: A ten-times slowdown just for mapping hydrogen? If adding nodes is that costly, how detailed do they get with the connections between the atoms? Host: They tested three levels of detail for the edges. They could use simple discrete bond types, a system of distance bins that estimates the space between atoms that aren't directly bonded, or even a raw 5-dimensional vector that calculates the exact Euclidean distance between pairs.

Host: Let's look at how these models were actually tuned and see the final performance numbers they achieved. To start, they trained on the QM-9 dataset, which has over 130,000 molecules, using a randomized search to find the best hyperparameters. Guest: How did they split up all those molecules to make sure they were testing the models fairly? Host: They set aside 10,000 molecules for a test set and another 10,000 for validation, leaving the remaining 110,000 for training. The ultimate goal was to see if the model's error could drop below established targets for "chemical accuracy." Guest: So, how did they measure whether the model actually hit that chemical accuracy mark? Host: They calculated an error ratio by dividing the model's error by the chemical accuracy target, meaning any score under 1 is a success. Impressively, their new network hit that mark on 11 out of 13 targets, setting a new state-of-the-art on all 13. Guest: That's a massive leap in performance! What specific choices gave them such an edge over older baselines? Host: A big factor was the input representation, specifically including spatial distance and explicitly treating hydrogen atoms as their own nodes in the graph. They also discovered that training a separate model for each target was highly effective. Guest: Really? I would have assumed training one master model to predict all 13 targets at once would work better. Host: You'd think so, but training individual models consistently outperformed joint training, sometimes by up to 40 percent. They pushed those results even further by ensembling, which just means averaging the predictions of their five best models. Guest: Did they test how the network held up if they took away some of that detailed input, like the spatial distances between atoms? Host: They did, and found they could compensate for missing spatial data by changing the output layer or using a multi-tower network structure. These tweaks helped capture long-range interactions between nodes, keeping performance remarkably high.

Host: As we look at some final experimental tweaks, we also reach the ultimate takeaways for this model. The researchers found that using towers works well because it acts like an ensemble of models, but trying to combine those towers with the edge network actually made training too difficult. Guest: Did they experiment with any other settings, like changing how the network learns over time? Host: They did try untying the network's weights so they would be different at each time step, but it actually performed worse. It turned out the best approach was keeping those weights tied and just increasing the hidden dimension size, while firmly sticking with the edge network function over other message types. Guest: So with all these optimized settings, what is the grand conclusion of the study? Host: The main conclusion is that Message Passing Neural Networks are uniquely suited for predicting molecular properties. They perform exceptionally well and completely eliminate the need for humans to manually hand-craft complex chemical features. Guest: That sounds like a huge success, but are there any remaining hurdles for future research? Host: The biggest challenge is generalizing to molecules that are much larger than the ones they trained on. This is especially tricky when using spatial information, because adding more atoms completely changes the distribution of distances between them. Guest: Plus, since they treat it as a fully connected graph where everything interacts, wouldn't a bigger molecule just create way too many messages? Host: Exactly, the number of incoming messages grows with the number of atoms, which can easily overwhelm the network. To solve this in the future, the authors suggest adding an attention mechanism, which would allow the model to selectively focus on only the most important incoming messages.

Host: As we wrap up our journey through this research, it's a great time to look at the community and foundational work that made it possible. The authors take a moment here to acknowledge some key contributors, including well-known AI researchers like Alex Graves and Lukasz Kaiser for their helpful discussions. Guest: It's always interesting to see who helped shape a paper behind the scenes. Did anyone else get a specific shoutout? Host: Yes, they specifically thank Adrian Roitberg for pointing out an issue with how they used "partial charges" in an earlier draft. It’s a nice glimpse into how scientific feedback directly catches errors and improves a model. Guest: That makes sense, since getting those chemistry details exactly right is crucial. What stands out in their reference list? Host: The bibliography is a perfect reflection of this paper's interdisciplinary nature, blending deep learning milestones with foundational quantum chemistry. You literally see modern computer science papers sitting right next to mid-century physics. Guest: So they have classic quantum mechanics paired up with modern neural network architecture? Host: Exactly. For instance, they cite the famous 1964 Hohenberg-Kohn paper on electron gas right alongside the 2014 paper that introduced the Adam optimizer for machine learning. Guest: That really puts it in perspective, showing exactly how they brought these two massive, distinct fields together to predict molecular properties.

Host: As we step past the references and into the appendix, the authors take a moment to connect their framework to some other well-known graph architectures. Specifically, they point out that a popular group of models called Laplacian-based models can actually be viewed as just another type of Message Passing Neural Network. Guest: What exactly is a Laplacian-based model trying to do in the first place? Host: They were originally designed to take the concept of a convolution—which is a common mathematical operation used in image recognition—and generalize it so it works on any standard graph structure. Guest: So they're adapting an image processing technique for graphs, but how does that translate into message passing? Host: The authors show that the math behind these generalized convolutions is essentially just passing messages between nodes. However, there is a key difference in how these specific models are tuned. Guest: What sets them apart from the other message passing networks? Host: It all comes down to simplicity and scale. When you translate those Laplacian models into the MPNN framework, you find that their message functions are actually very basic. Guest: I imagine keeping the math simple makes the model run a lot faster? Host: Exactly, and that efficiency is crucial for their intended use cases. While you might need complex message functions to decode the dense structure of a tiny molecule, these Laplacian models are typically deployed on massive datasets, like sprawling social networks.

Host: Let's explore the mathematical engine that transforms features across a molecular network using something called the Graph Laplacian. This framework for quantum chemistry actually builds directly on Kipf and Welling's foundational 2016 work on graph convolutions. Guest: I'm curious about the math here, so how exactly is this Graph Laplacian defined for our molecules? Host: It essentially captures the structure of the molecule by taking an identity matrix and subtracting a normalized version of the graph's connections. That normalization uses the adjacency matrix, which maps the bonds, and the degree matrix, which counts how many bonds each atom has. Guest: Okay, so the Laplacian maps out the normalized structure, but how do we use it to actually update the data or features on each atom? Host: We take the eigenvectors of that Laplacian matrix, combine them with learned parameter matrices, and apply a non-linear function like ReLU. This operation takes our input vector of node features and mathematically transforms it into a new output vector. Guest: That sounds like a massive equation, doing matrix multiplication across every single node and feature dimension. Host: It is quite heavy, but the text offers a brilliant simplification by relabeling the input vector as our current hidden state, and the output vector as our next hidden state. When you frame it that way, this massive equation becomes easily recognizable as a standard message passing update. Guest: Ah, so all this complex linear algebra is really just a rigorous way of defining how atoms send information to their neighbors? Host: Exactly. The Laplacian matrix defines the specific message function between nodes, and the non-linearity acts as the update function, proving this graph transformation is just a special case of a Message Passing Neural Network.

Host: We're going to dive into the math behind how information actually moves through a graph neural network layer by layer. Specifically, we'll look at the famous propagation rule proposed by Kipf and Welling in 2016. Guest: I've seen that equation before, and it looks pretty intimidating with all the matrices and fractional exponents. What exactly is it doing? Host: Let's break it down into pieces, starting with the goal, which is updating the features of every node in the graph. To do that, the model essentially calculates a weighted average of each node's neighbors. Guest: Okay, so how does it know who the neighbors are? Is that where the adjacency matrix comes in? Host: Exactly, that is the A in the formula, but they also add an identity matrix to it. This creates a self-loop, ensuring a node's own current features are included when calculating that new average for the next layer. Guest: That makes sense, you don't want a node to forget its own information. But what about the D to the negative one-half parts? Host: D is the degree matrix, which simply counts how many connections each node has. Multiplying by those fractional powers normalizes the adjacency matrix, so highly connected nodes don't drown out the signals of nodes with fewer connections. Guest: Ah, so it balances out the messages based on the number of neighbors. And where does the actual learning happen? Host: That happens with W, the layer-specific trainable weight matrix. The network learns these weights to transform the incoming features before passing them through a non-linear activation function, represented by the Greek letter sigma. Guest: So when the text mentions message and update functions, it's really just pulling normalized data from neighbors, transforming it with learned weights, and applying that activation function? Host: You nailed it. Despite the complex matrix notation, at its core, this propagation rule simply computes a smart, normalized weighted average of a node's local neighborhood at every single step.

Host: Let's dive into the energetic forces that hold molecules together and what it takes to break them apart. In molecular modeling, a key concept we look at is atomization energy, which is essentially the amount of energy required to completely dismantle a molecule into its individual constituent atoms. Guest: That sounds fairly straightforward, but is there just one standard measurement for that energy? Host: Actually, there are four distinct types measured in electron volts, or eV, and they depend on the surrounding conditions. The first is U0, which is the energy needed to break the molecule apart at absolute zero, assuming the volume stays perfectly fixed. Guest: Absolute zero is pretty extreme. What if the molecule is just sitting at normal room temperature? Host: That brings us to the second type, U, which is simply the atomization energy at room temperature, still keeping that volume fixed. However, in physical laboratories, holding a reacting system at a strict fixed volume isn't always practical. Guest: Right, usually things are just exposed to standard air pressure in a room, so how do we measure it then? Host: For that, we use the third metric, H, known as the enthalpy of atomization. It also looks at room temperature, but it assumes fixed pressure instead of a fixed volume. Guest: Okay, so we have fixed volume at absolute zero, fixed volume at room temperature, and fixed pressure at room temperature. What is the fourth type? Host: The final one is G, or the free energy of atomization. This accounts for systems held at both a fixed temperature and fixed pressure while breaking apart, which incorporates the full effects of temperature into the math. Guest: So having these four distinct metrics lets scientists see exactly how stable a molecule is under various physical conditions. Host: Exactly. Mapping out these specific energies gives us a clear, detailed understanding of a molecule's true stability and its overall energetic landscape.

Host: Let's zoom in on how molecules are constantly in motion, even when they seem perfectly still. Following our look at atomization energies, we are now exploring fundamental molecular vibrations, starting with something called the highest fundamental vibrational frequency. Guest: What exactly does that highest frequency tell us about a molecule? Host: It is often written as omega-one and measured in inverse centimeters, and it represents the specific vibration mode that requires the most energy for the molecule to oscillate naturally. This measurement is a great indicator of how rigid a molecule is and how strong its interatomic bonds are. Guest: That makes sense, sort of like how a stiffer, tighter guitar string requires more energy and vibrates differently. Do these vibrations ever completely stop if you freeze the molecule? Host: You might think so, but surprisingly no, which brings us to Zero Point Vibrational Energy, or ZPVE. Even at absolute zero, the rules of quantum mechanical uncertainty dictate that atoms must keep vibrating. Guest: So a molecule always has a baseline level of energy left over, no matter how cold it gets? Host: Exactly, and that residual energy is the zero-point vibrational energy present in the molecule's ground state. Once we figure out the allowed vibrational modes for a molecule, we can calculate this energy in electron volts, which is absolutely vital for understanding how molecules behave at extremely low temperatures.

Host: We're shifting our focus now to how electrons behave and the specific energy levels they occupy inside a molecule. To understand this, we need to look at two key acronyms: HOMO and LUMO. Guest: I'm guessing those acronyms describe where the electrons are actually located? Host: Exactly. Quantum mechanics tells us electrons live in discrete energy states, and because of the Pauli exclusion principle, they can't all cram into the exact same state. At absolute zero temperature, they simply fill these available states by stacking up from the lowest energy level upwards. Guest: So do HOMO and LUMO describe the very top of that stack? Host: Spot on. HOMO stands for Highest Occupied Molecular Orbital, which is the top energy state that actually contains electrons. LUMO stands for Lowest Unoccupied Molecular Orbital, meaning the very next rung on the ladder that remains completely empty. Guest: That makes sense, so there must be an energy difference between the highest filled rung and the first empty rung. Host: We call that difference the electron energy gap. It represents the absolute minimum energy transition required to excite an electron, pushing it from the occupied HOMO up into the empty LUMO. Guest: Why is knowing that minimum required energy jump so fundamental for a molecule? Host: Because it dictates how the molecule interacts with light in the real world. That specific energy gap determines the longest possible wavelength of light the molecule is capable of absorbing.

Host: We are going to explore how electrons are physically arranged in space around a molecule. One key metric scientists use is the Electronic Spatial Extent, which simply tells us how spread out a molecule's electron cloud is. Guest: It makes sense that electron clouds can be larger or smaller, but how do you actually calculate the size of something so fuzzy? Host: We look at the second moment of the charge distribution, which is typically measured in units called Bohr squared. It mathematically factors in both the density of the electrons and their distance from the center, giving us a precise measurement of that overall spread. Guest: Okay, so that gives us the total size of the cloud, but what if the electrons are mostly bunched up on one side? Host: To measure that, we use a complementing metric called the norm of the dipole moment, measured in units called Debyes. This captures the anisotropy, or the directional unevenness, of the molecule's electric charge. Guest: If a molecule has a really uneven charge, does that mean it creates its own electric field? Host: Exactly, that dipole moment approximates the electric field you would measure if you were standing far away from the molecule. The strength of this resulting field actually drives a lot of real-world material properties. Guest: Can you give an example of a material property that is driven by that uneven charge? Host: Water is a perfect example, because its uneven charge leads to strong hydrogen bonding. That large dipole moment significantly impacts how water molecules interact with each other, which is exactly why water has such high surface tension.

The text also introduces the Norm of the static polarizability α (Bohr³). Polarizability measures a molecule's capacity to spontaneously develop a dipole moment in response to an external electric field. This property is directly related to the extent to which Van der Waals interactions influence the dynamics of the medium. Finally, the document presents Table 5, which lists mean absolute errors for chemical accuracy across various targets, including mu, alpha, HOMO, LUMO, gap, R2, ZPVE, U0, U, H, G, Cv, and Omega. It also provides DFT errors and chemical accuracy values, which are used to calculate the mean absolute errors of the models by multiplying the error ratio by the chemical accuracy. These values, along with estimates of mean absolute error for DFT calculations against ground truth, were provided by Faber et al. (2017).

Table 6 presents a comparison of the performance of the best architecture, an edge network with a set2set output, on different sized training sets. The results are surprising in their data efficiency; for instance, on targets R2 and Omega, the models achieve comparable or better performance with 11k samples than the best baseline with 110k samples. Table 7 compares models trained without spatial information, specifically exploring the impact of virtual edges, a master node, and a set2set output. The findings suggest that capturing long-range interactions is crucial for model performance. Table 8 compares a GG-NN model with towers and a set2set output against a baseline GG-NN with a set2set output, using distance bins. The towers model consistently outperforms the baseline on most targets, both in individual and joint training regimes. Table 9 contrasts the edge network with a pair message network in a joint training setup, showing that the edge network generally performs better. Lastly, Table 10 evaluates MPNNs with different input featureizations, including models with no distance information, distance information, and distance plus exponential hydrogen information, using Set2Set output and GRU update functions.