A simple neural network module for relational reasoning

Host: We are diving into a fascinating development in artificial intelligence where machines learn to understand the connections between different objects. This comes from a key research paper introducing a simple neural network module built specifically for what is called relational reasoning. Guest: What exactly does relational reasoning mean for an AI? Host: It is the ability to look at various pieces of data and figure out how they interact or relate to one another. While humans do this effortlessly, standard neural networks have historically struggled with it. Guest: So this new module is designed to fix that specific blind spot? Host: Exactly, and it was developed by a team of researchers including Adam Santoro, David Raposo, and Peter Lillicrap. They created a dedicated architectural piece just to handle these complex relationships. Guest: Where was this research team working? Host: The team is based at DeepMind in London, which is one of the world's leading artificial intelligence research laboratories. Guest: Coming from DeepMind, it sounds like a major push to give neural networks a more human-like way of solving problems. Host: That is precisely the ambition behind the project. By building this specialized, simple module, they aim to give AI the structural tools to actually reason about its environment.

Host: Let's start by looking at the big picture of how AI understands the connections between different objects. Relational reasoning is a huge part of intelligent behavior, but historically, neural networks have really struggled to learn it. Guest: When you say relational reasoning, do you just mean the AI figuring out how different things interact or relate to each other? Host: Exactly, like understanding that a red sphere is located behind a blue cube, rather than just identifying the two shapes in isolation. To solve this gap, the researchers propose using Relation Networks, or RNs, as a simple plug-and-play module for existing AI architectures. Guest: If it's just a plug-and-play module, how did they prove it actually makes a difference? Host: They tested these RN-augmented networks on three complex tasks, starting with visual question answering on a highly challenging dataset called CLEVR. Adding the module allowed the network to achieve state-of-the-art, super-human performance on those visual relational problems. Guest: Beating human performance on visual questions is a massive claim. Did it work for non-visual tasks too? Host: It did, as they also successfully tested it on text-based question answering using a suite called bAbI, as well as on complex reasoning about dynamic physical systems. They even used a specific dataset called Sort-of-CLEVR to prove that standard, powerful convolutional networks simply cannot handle relational questions on their own. Guest: But once those standard networks got the RN module plugged in, I assume they could suddenly solve them? Host: Spot on. The research shows that simply equipping a deep learning architecture with an RN module gives it the power to implicitly discover entities and successfully reason about their relationships.

Host: Let's begin by looking at a core part of intelligence: the ability to understand how different things relate to each other. For humans, this is intuitive, like a child instantly figuring out which two trees in a park are furthest apart to set up a race. Guest: We naturally compare things all the time, but I'm guessing this kind of relational reasoning is much harder for artificial intelligence? Host: Exactly. Historically, the older "symbolic" AI was great at the logic of relations, but it struggled to connect those abstract symbols to messy real-world inputs. Guest: And what about modern deep learning? Doesn't that handle messy real-world data pretty well? Host: It does, but standard deep learning networks often stumble when they have to figure out complex relationships without massive amounts of specific training data. Even very powerful standard neural networks can fail at surprisingly simple relational tasks. Guest: So, is there a way to give modern neural networks that same logical reasoning ability? Host: That’s where the authors introduce something called "Relation Networks," or RNs. They are simple, plug-and-play modules designed specifically to handle flexible relational reasoning within neural networks. Guest: "Plug-and-play" sounds incredibly useful. Does that mean you can just attach them to existing AI models? Host: Yes, you can pair them with standard networks that process images or text, and the RN actually helps those systems recognize distinct objects to reason about. When tested on a highly complex visual question-answering dataset called CLEVR, models using RNs actually achieved super-human performance. Guest: Wow, super-human performance on visual questions is huge. Does it work for other types of problems too? Host: It really does. The researchers also used RNs to successfully solve text-based logic questions, and even make inferences about complex physical systems and motion data.

Host: Let's explore an architecture built from the ground up to understand how things connect, known as a Relation Network. Just like a Convolutional Neural Network is inherently wired to process spatial data like images, a Relation Network has the ability to compute relations baked right into its basic design. Guest: How exactly do you bake relational reasoning into a network's structure? Host: It all comes down to a specific composite function. The network takes an input set of objects, creates every possible pair from those objects, and passes each pair through a shared mini-network to evaluate them. Guest: When you say every possible pair, does it already know which objects are actually related to each other? Host: No, it operates completely blind to that at first. It uses an all-to-all approach, meaning it considers every single combination and learns to infer for itself whether a relationship exists and what it implies. Guest: If you have a lot of objects, checking every single combination sounds like it would get way too heavy to compute. Host: It definitely would be for a standard neural network, which would try to learn unique parameters for every single pair combination. But Relation Networks are highly data efficient because they apply the exact same function to evaluate every single pair. Guest: Oh, so it just learns one really robust rule for checking pairs and reuses it over and over. What happens after it checks them all? Host: It simply adds all the individual pair results together into one final sum. This is a crucial step because summation is order-invariant, meaning it ensures the network treats the input as a true set where the original order of the objects doesn't change the final outcome.

Host: Let's look at how these relational networks are actually put to the test across different types of problems, starting with visual question answering. To do this, the researchers focused heavily on a dataset called CLEVR. Guest: What exactly does a visual question answering task look like, and why did they choose this CLEVR dataset in particular? Host: In visual QA, an AI has to answer questions based on an image, but typical datasets have a major flaw. They often contain linguistic biases that let the AI guess the answer just from the question's phrasing, without really looking at the picture. Guest: So the AI was basically cheating using word patterns instead of actually understanding the scene. How does CLEVR prevent that? Host: CLEVR strips away all that real-world messiness by using clean, computer-generated images of simple 3D objects like spheres and cubes. The questions are specifically designed to force the AI to reason about relationships, like asking if a cube is made of the same material as a cylinder. Guest: How did older, standard AI models handle those kinds of relational questions before this new network came along? Host: Surprisingly poorly. Even top-tier models equipped with complex attention mechanisms completely bombed on questions that required comparing objects, performing only slightly better than a simple baseline model that just blindly guessed based on the question type. Guest: That makes it the perfect proving ground for a network built specifically to understand relationships. How exactly did the researchers feed these CLEVR images into their new model? Host: They trained their models using two separate formats to test different kinds of inputs. One format used standard 2D image pixels, while the other used a state description matrix, which essentially lists every object's exact coordinates, color, shape, and material.

Host: Let's look at a custom dataset the researchers built just to put their neural network to the test. They wanted to prove their Relation Network architecture was better at relational reasoning than standard models, so they created something called Sort-of-CLEVR. Guest: What makes this specific dataset so good at testing reasoning? Host: The main feature is that it strictly separates relational questions from non-relational ones. A non-relational question might ask, "What is the shape of the gray object?", where the model only needs to look at one specific thing. Guest: And a relational question would force the model to look at how multiple things interact? Host: Exactly, like asking, "What is the shape of the object farthest from the gray object?" To answer that, the model has to locate the gray object, compare distances to all other objects, and then identify the furthest one. Guest: That sounds like it could be really complicated for a computer to figure out from a regular photograph. Host: That's why they made the visuals artificially simple, using just six basic 2D shapes per image, with clear, unambiguous colors. They even took it a step further and converted all the text questions into fixed-length binary code. Guest: Why binary code instead of just asking the question in plain English? Host: It removes a huge variable. If a model fails an English question, you wouldn't know if it struggled with the grammar or if it failed at the actual visual reasoning. Guest: Ah, that makes sense. By stripping away complex language and complex images, they isolate the actual reasoning skills. Host: Precisely. By generating ten relational and ten non-relational questions for each of these simplified images, they created a perfectly controlled environment for their test.

Host: We are turning our attention now to how we measure a model's ability to reason through text, specifically looking at a dataset called bAbI. It is a pure text-based question-answering dataset that breaks reasoning down into 20 distinct tasks. Guest: What kind of reasoning tasks are we talking about? Host: Things like deduction, induction, or simple counting. For example, it might give the model two facts like Sandra picked up the football and Sandra went to the office, and then ask where the football is. Guest: So the model has to connect those two facts to answer that it is at the office. How hard is it to get a passing grade on these? Host: To officially succeed on a task, a model's performance has to surpass 95 percent accuracy. Memory-augmented neural networks actually do very well here, especially when they are trained on all 20 tasks simultaneously. Guest: Training on all of them at once sounds tough. How do the different models stack up? Host: Under that joint training approach, using ten thousand examples per task, Memory Networks pass 14 out of 20. The DNC passes 18, and the Sparse DNC leads with 19 out of 20. Guest: Did any of the models manage to pass all 20 tasks? Host: The creators of a model called EntNet did report a perfect 20 out of 20, but there is a catch. They did not achieve that through joint training like the others, and when EntNet is forced to train on all tasks jointly, it actually drops to 16 out of 20.

Host: Let's explore how models understand physical rules by observing moving objects. The researchers used a physics engine to simulate a tabletop environment with ten colored balls bouncing around. Guest: Sounds like a digital game of billiards. Are they all just bouncing off each other and the walls independently? Host: Some are, but here is the twist: randomly selected pairs of balls are secretly linked together by invisible springs or rigid constraints, meaning they can't move independently. Guest: Oh, that makes it tricky. How exactly is the AI observing this scene? Is it watching a normal video? Host: It's a bit more stripped down than that. The AI receives a data matrix that lists each ball's color and its exact spatial coordinates across sixteen sequential time steps. Guest: So it gets a precise numerical track of where everything is over time. What is it supposed to figure out from those coordinates? Host: It has two distinct tasks. The first is to infer which specific balls share those invisible connections, and the second is to count the total number of connected groups, which the researchers call "systems." Guest: I imagine it has to look at the distance between balls, right? Like if two balls always stay the exact same distance apart, they must be linked. Host: Exactly, it has to reason about their relative positions and velocities over time. And while the first task asks for those connections explicitly, the second task is much harder because the model has to map out all those connections implicitly just to output a final count of the groups.

Host: Let's look at how these architectures are actually put together in practice. The core issue here is that Relation Networks expect discrete objects, but our data usually comes as unstructured pixels or text. Guest: Right, a camera just sees a grid of colors, not a list of separate items. So how do they translate a raw image into objects for the network? Host: They use a standard Convolutional Neural Network to scan the image. The CNN outputs a grid of feature maps, and they simply treat every individual cell in that final grid as a separate object. Guest: Wait, so a single grid cell might just be a patch of blank wall or a piece of a shadow. Does the model know what a real physical object actually is? Host: It does not need to be explicitly told. They tag each cell with its spatial coordinates, and the network figures out during training whether that cell represents a background, a texture, or a physical shape. Guest: That makes sense. But if it is analyzing all these random patches, how does it know which ones to focus on to answer a specific question? Host: That is where the question itself comes in. They use an LSTM to process the words of the question into a mathematical summary, and they inject that summary directly into every object pair before the network compares them. Guest: Oh, so the question acts as a guide, telling the network which object relationships are relevant and which ones to ignore? Host: Exactly. And this flexibility extends to natural language tasks too. If you give the model a text passage instead of an image, it processes each complete sentence through an LSTM and treats the entire sentence as a distinct object. Guest: That is incredibly elegant, and it seems much simpler than the massive visual and language models that use complex attention layers. Host: That is the exact point the authors emphasize. By letting the network define its own objects from simple building blocks like grid cells or sentences, the overall architecture remains remarkably streamlined and simple to configure.

Host: Let's look at how the model actually performed when put to the test. Working straight from raw pixels and text, it achieved a state-of-the-art 95.5 percent accuracy on the CLEVR dataset. Guest: That sounds impressive, but how does that score compare to other systems, or even to a real person? Host: It actually surpassed human performance on this task, and beat the previous best model by a massive 27 percent. It did especially well in the "compare attribute" and "count" categories, where other AI models usually struggle. Guest: I'm guessing it crushed those specific categories because they require such heavy relational reasoning? Host: Exactly. And because the researchers used relatively simple components for the vision and language processing, it proves the true hurdle in this task was always the relational reasoning, not just seeing or reading. Guest: Wait, the text mentions another recent study that reported a slightly higher overall score of 96.9 percent—how does that factor in? Host: Good catch, but that competing model relied on privileged training information. They fed their system extra supervisory signals from the underlying computer programs used to generate the test questions. Guest: Ah, so it's not a fair comparison if our model only learned from the raw images and questions without those extra hints. Host: Precisely. When that competing model was tested without those extra signals, this approach greatly outperformed it, proving Relational Networks can reach super-human levels under much more natural conditions.

Host: Let's see how flexible our model is when we completely change the type of data it looks at. To test if the Relation Network, or RN, is robust to different input formats, the researchers trained it on a modified version of the CLEVR dataset. Guest: What did they change about it? I thought CLEVR was all about analyzing images of 3D shapes. Host: It usually is, but here they used a state description matrix instead of raw pixels. This means the input was basically a structured table listing out the exact features of every object, like its shape, color, and coordinates. Guest: Oh, so the model is just reading a list of object traits rather than actually looking at a picture. How well did it handle that shift? Host: It handled it beautifully, achieving an accuracy of 96.4 percent. This high score proves a major point about the overall generality of the RN module. Guest: Since it scored so high without pictures, does that mean the network doesn't actually need visual data to understand relationships? Host: Exactly, it shows the RN is agnostic to the kind of input it receives. As long as the network receives some clear representation of object features, it can successfully learn and reason about how those objects relate to one another. Guest: That makes sense, so you could probably use this architecture for problems that have absolutely nothing to do with computer vision. Host: That is the ultimate takeaway here. Because Relation Networks aren't restricted to visual problems, they can be applied to very different contexts and tasks wherever relational reasoning is needed.

Host: Let's look at exactly what makes visual reasoning datasets so challenging for standard AI models. Previously, some researchers thought the main hurdle in the CLEVR dataset was just parsing the complex text questions, but the authors here suspected the real issue was actually relational reasoning. Guest: So they think the models are struggling to figure out how objects interact with each other, rather than just getting confused by the language? Host: Exactly. To prove this, they used a custom dataset called "Sort-of-CLEVR" which explicitly separates questions into relational and non-relational categories. The original CLEVR dataset doesn't do that, which made testing this hypothesis difficult. Guest: How did the different models perform once they could actually separate those two types of questions? Host: The results were stark. A standard setup—a Convolutional Neural Network augmented with a basic neural network called an MLP—scored over 94 percent on the non-relational questions. But on the relational questions, it plateaued at just 63 percent. Guest: That is a massive drop. Did adding their specialized Relation Network fix the problem? Host: It did. When they augmented the CNN with their Relation Network, or RN, the model achieved over 94 percent accuracy on both types of questions. This strongly indicates that models lacking a dedicated relational module are almost entirely incapable of solving tasks that require simple reasoning. Guest: What kind of simple relation completely trips up that standard model? Host: Questions asking what is "closest to" or "furthest from" something else were particularly brutal, with the standard model only succeeding about 52 percent of the time. To answer that, the model has to gauge the distance between every single pair of objects, and then compare all those distances. Guest: Oh, I see. It has to calculate every possible combination, which probably gets overwhelming really fast. Host: Precisely. It creates a combinatoric explosion of difficulty. Plus, depending on the image, those relevant distances might be tiny or huge, making the task incredibly tough unless you have a module specifically built to compare relations.

Host: Let's see how our model holds up against a famous set of twenty reasoning tests known as the bAbI dataset. It successfully passed 18 of those 20 tasks. Guest: Eighteen out of twenty sounds great, but were any of those tasks particularly challenging? Host: Definitely, especially the basic induction task where our model achieved a tiny 2.1 percent error rate. That same task proved incredibly difficult for rival architectures like the Sparse DNC and EntNet, which failed more than half the time. Guest: What about the two tasks it didn't pass? Host: Those required piecing together two or three supporting facts, but the model didn't fail catastrophically. To pass a task, a model needs 95 percent accuracy, and ours only missed that cutoff by about 3 and 11 percent on those two. Guest: That is really close, but how do we know these results are reliable and not just a lucky run? Host: That is a crucial point, because the researchers got these numbers using a single run, or single seed, chosen from a validation set. They didn't run the model multiple times to cherry-pick the best possible hyperparameter settings. Guest: Do other teams actually do that to get better scores? Host: They do, and previous models like the Sparse DNC used multiple runs, showing massive performance swings where they might pass three more or three fewer tasks just by chance. Guest: So hitting 18 out of 20 on a single, unmanipulated run proves this new model is highly stable.

Host: Let's look at how AI models handle objects in motion, specifically systems governed by physical rules. The researchers tested their model, called a Relational Network or RN, on tasks where it had to reason about the movement of balls along a surface. Guest: Were these balls just moving randomly, or were they interacting with each other in some specific way? Host: Some of them were linked together by invisible forces, and the network had to figure out those hidden connections just by watching the balls move. In this connection inference task, the RN correctly identified the hidden links 93 percent of the time. Guest: That is incredibly accurate for just observing motion. Did they give it any other challenges? Host: They also gave it a counting task where the model had to report the total number of connected systems, and it achieved 95 percent accuracy there. For comparison, a standard neural network with a similar size, known as an MLP, couldn't do better than random guessing on either task. Guest: So the standard model totally failed to grasp the physical relationships, but the RN succeeded. Did this ability to see connections apply to anything besides simulated balls? Host: It actually did, and this is a really fascinating result. They applied the RN to completely unseen motion capture data of a human walking. Guest: Wait, it went straight from analyzing moving balls to analyzing human movement? Host: Exactly. Even though it was only trained on those simple moving objects, the network was able to successfully predict the invisible connections between the different body joints of the walking human. Guest: That is amazing that learning the basic physical rules of connected shapes transferred directly to mapping out a moving human body.

Host: As we wrap up our journey through this research, it is clear that adding a dedicated Relation Network, or RN, completely changes the game for artificial intelligence systems. By simply plugging this module into standard deep learning architectures, the authors saw massive performance boosts on tasks requiring complex reasoning. Guest: Just how massive are we talking? Did they share the final numbers on their testing? Host: They did, and on a visual reasoning test called CLEVR, adding the RN skyrocketed the system's accuracy from 68.5% to 95.5%. That actually surpassed human performance on the same test, and they also solved 18 out of 20 tasks on a separate text-based benchmark called bAbI. Guest: That is a massive leap for the visual test. Why does adding this one specific module make such a difference? Host: It essentially comes down to separating processing from reasoning. Standard networks like CNNs are fantastic at processing local visual details, but they aren't naturally built to reason about how different objects relate to one another. Guest: So the RN takes over the relationship math, letting the visual network just focus on extracting the visual features? Host: Exactly, it frees the visual network to specialize. And fascinatingly, adding the RN actually forced the upstream visual network to output much clearer, object-like representations, even though the system was never explicitly told what an object should look like. Guest: That is incredible that it learned to structure that data entirely on its own. Where do the researchers see this technology being used next? Host: They suggest applying it to anything that benefits from understanding complex structures, like modeling social networks or improving reinforcement learning agents. They also noted a need to improve the network's computational efficiency going forward. Guest: I can see why, since comparing every single pair of objects to find relationships would demand a lot of computing power. Host: Precisely, the math scales up quadratically. To fix that, they propose using attention mechanisms to filter out completely unrelated objects, ensuring this powerful tool remains practical for complex, real-world environments.

Host: Before we dive into the main research, it is always good to recognize the broader community behind the work. The authors open by expressing their gratitude to several individuals who helped shape this project. Guest: Who exactly are they thanking here? Host: They specifically highlight colleagues from the DeepMind team, naming researchers like Murray Shanahan, Ari Morcos, and Scott Reed. Guest: DeepMind is a major artificial intelligence lab, but did these specific people actually write the paper? Host: They are not the primary authors, which is exactly why they are being recognized in this section rather than on the byline. Instead, the authors are thanking them for providing critical feedback and engaging in deep discussions. Guest: That makes perfect sense, since bouncing complex ideas off other experts is a huge part of the scientific process. Host: Exactly, and getting that outside perspective from peers like Daan Wierstra and Alex Lerchner helps the authors refine their ideas and ensure the final research is robust.

Host: We are going to dive into the extra background details the researchers included to place their model in the wider landscape of AI. Because the Relation Network is so versatile, it actually touches on everything from computer vision to natural language understanding. Guest: How does this network compare to older methods that also tried to get AI to understand relationships? Host: The biggest difference is how little hand-holding the Relation Network needs. Older approaches relied on strict, rule-based logic or manually designed graphs, but this network figures things out from relatively unstructured inputs. Guest: So it can just take raw outputs from standard image or text processors without needing the data perfectly organized first? Host: Exactly, and that flexibility is a huge advantage for tasks like Visual Question Answering, where the AI has to answer questions about a scene. Other recent models could do this, but they required a lot of heavy lifting. Guest: What kind of heavy lifting did those other models need? Host: To handle complex, relationship-based questions on datasets like CLEVR, parallel models either needed highly customized modules or direct access to the ground-truth code used to generate the images. Guest: That sounds like they needed a cheat sheet to find the right answer. Host: Precisely, whereas the Relation Network is conceptually much simpler and doesn't need those cheat sheets. Even without them, it matches or beats those heavily supervised systems. Guest: The text mentioned text-based question answering at the very end, so does it simplify that process too? Host: Yes, while many recent neural networks for text rely on complex, dedicated memory structures, this simple Relation Network module shows it can step right into that arena and perform just as effectively.

Host: Let's look under the hood at how this visual reasoning model was actually trained from raw pixels. To build it, the team used 70,000 image scenes and nearly 700,000 questions from the CLEVR dataset. Guest: That is a massive amount of data, but did they just feed those images straight in, or was there some prep work first? Host: They used a common trick called data augmentation to help the model learn better. The images were resized to 128 by 128 pixels, padded out slightly, and then randomly cropped and rotated just a tiny bit. Guest: I see, so those random little shifts force the model to really understand the shapes instead of just memorizing exactly where certain pixels are. Host: Exactly. To crunch all that data, they used ten distributed computers that continuously synced up to a central server, training for about 1.4 million iterations before the model stopped improving. Guest: Did all that heavy computing pay off in the end? Host: It did, hitting an impressive 96.8% accuracy on their validation set. But the most surprising takeaway was actually about the physical size of the neural network. Guest: Surprising how? Usually in AI, we hear that bigger models with way more parameters perform better. Host: You would think so, but here, smaller models actually won out. For processing the questions, a network with 128 hidden units beat out larger versions that had 256 or 512 units. Guest: Oh wow, did that smaller-is-better rule hold true for the image-processing part of the model, too? Host: It sure did. The convolutional layers performed best using just 24 kernels, rather than larger setups like 32 or 64. By keeping the network lean and heavily restricting it with techniques like a 50% dropout rate, they prevented the model from memorizing the training data and forced it to actually learn visual reasoning.

Failure cases. Although our model gets most answers correct, a closer examination of the failure cases help us to identify limitations of our architecture. In Table 2, we show some examples of CLEVR questions that our model fails to answer correctly, along with the ground-truth answers. Based on our observations, we hypothesize that our architecture fails especially when objects are heavily occluded, or whenever a high precision object position representation is required. We also observe that many failure cases for our model are also challenging for humans.

CCLEVR from state descriptions. The model that we train on the state description version of CLEVR is similar to the model trained on the pixel version of CLEVR, but without the vision processing module. We used a 256 unit LSTM for question processing and word-lookup embeddings of size 32. For the RN we used a four-layer MLP with 512 units per layer, with ReLU non-linearities for go. A three-layer MLP consisting of 512, 1024 (with 2% dropout) and 29 units with ReLU non-linearities was used for fe. To train the model we used 10 distributed workers that synchronously updated a central parameter server. Each worker learned with mini-batches of size 64, using the Adam optimizer and a learning rate of 1e-4.

D Sort-of-CLEVR. The Sort-of-CLEVR dataset contains 10000 images of size 75 x 75, 200 of which were withheld for validation. There were 20 questions generated per image (10 relational and 10 non-relational). Non-relational questions are split into three categories: (i) query shape, e.g. "What is the shape of the red object?"; (ii) query horizontal position, e.g. "Is the red object on the left or right of the image?"; (iii) query vertical position, e.g. “Is the red object on the top or bottom of the image?”. These questions are non-relational because one can answer them by reasoning about the attributes (e.g. position, shape) of a single entity which is identified by its unique color (e.g. red). Relational questions are split into three categories: (i) closest-to, e.g. “What is the shape of the object that is closest to the green object?”; (ii) furthest-from, e.g. “What is the shape of the object that is furthest from the green object?"; (iii) count, e.g. “How many objects have the shape of the green object?". We consider these relational because answering them requires reasoning about the attributes of one or more objects that are defined relative to the attributes of a reference object. This reference object is uniquely identified by its color. Questions were encoded as binary strings of length 11, where the first 6 bits identified the color of the object to which the question referred, as a one-hot vector, and the last 5 bits identified the question type and subtype.

In this task our model used: four convolutional layers with 32, 64, 128 and 256 kernels, ReLU non-linearities, and batch normalization; the questions, which were encoded as fixed-length binary strings, were treated as question embeddings and passed directly to the RN alongside the object pairs; a four-layer MLP consisting of 2000 units per layer with ReLU non-linearities was used for gө; and a four-layer MLP consisting of 2000, 1000, 500, and 100 units with ReLU non-linearities used for f. An additional final linear layer produced logits for a softmax over the possible answers. The softmax output was optimized with a cross-entropy loss function using the Adam optimizer with a learning rate of le¯¹ and mini-batches of size 64. We also trained a comparable MLP based model (CNN+MLP model) on the Sort-of-CLEVR task, to explore the extent to which a standard model can learn to answer relational questions. We used the same CNN and LSTM, trained end-to-end, as described above. However, this time we replaced the RN with an MLP with the same number of layers and number of units per layer. Note that there are more parameters in this model because the input layer of the MLP connects to the full CNN image embedding.

EbAbI model for language understanding. For the bAbI task, each of the 20 sentences in the support set was processed through a 32 unit LSTM to produce an object. For the RN, ge was a four-layer MLP consisting of 256 units per layer. For fo, we used a three-layer MLP consisting of 256, 512, and 159 units, where the final layer was a linear layer that produced logits for a softmax over the answer vocabulary. A separate LSTM with 32 units was used to process the question. The softmax output was optimized with a cross-entropy loss function using the Adam optimizer with a learning rate of 2e-4.

F Dynamic physical system reasoning. For the connection inference task the targets were binary vectors representing the existence (or non-existence) of a connection between each ball pair. For a total of 10 objects, the targets were 102 length vectors. For the counting task, the targets were one-hot vectors (of length 10) indicating the number of systems of connected balls. It is important to point out that in the first task the supervision signal provided by the targets explicitly informs about the relations that need to be computed. In the second task, the supervision signal (counts of systems) do not provide explicit information about the kind of relations that need to be computed. Therefore, the models that solve the counting task must successfully infer the relations implicitly. Inputs to the RN were state descriptions. Each row of a state description matrix provided information about a particular object (i.e. ball), including its coordinate position and color. Since the system was dynamic, and hence evolved through time, each row contained object property descriptions for 16 consecutive time-frames. For example, a row could be comprised of 33 floats: 16 for the object's a coordinate position across 16 frames, 16 for the object's y coordinate position across 16 frames, and 1 for the object's color. The RN treated each row in this state description matrix as an object. Thus, it had to infer an object description contained information of the object's properties evolving through time. For the connection inference task, the RN's ge was a four-layer MLP consisting of three layers with 1000 units and one layer with 500 units. For fo, we used a three-layer MLP consisting of 500, 100, and 100 units, where the final layer was a linear layer that produced logits corresponding to the existence/absence of a connection between each ball pair. The output was optimized with a cross-entropy loss function using the Adam optimizer with a learning rate of le¯4 and a batch size of 50. The same model was used for the counting task, but this time the output layer of the RN was a linear layer with 10 units. For baseline comparisons we replaced the RNs with MLPs with comparable number of parameters. Please see the supplementary videos: https://www.youtube.com/channel/UCIAnkrNn45DOMeYwtVpmbUQ