Gemini Co-Lead on World Models, RL's Next Domains & Continual Learning

Host: In this interview, Oriol Vinyals, the co-lead of Google's Gemini, talks about their unique approach to AI, which focuses heavily on building what they call "world models." Guest: What exactly is a world model, and how is it different from the language models we are already used to? Host: While traditional models learn by reading massive amounts of text, a world model tries to understand the physical world by combining that text with images, video, and spatial planning. Guest: So it is learning from what it sees in the world, not just what it reads? Host: Exactly, because as Oriol points out, researchers have already successfully squeezed almost all of human written knowledge into current AI models. Guest: That makes sense, but since there is so much video on the internet, haven't they already started training on that too? Host: They have, and models like Gemini Omni are amazing at mixing text and video, but Oriol argues we haven't yet seen a true "GPT moment" for purely visual data. Guest: What would that ultimate breakthrough for video and images actually look like? Host: It would mean training an AI on vast amounts of raw images and video without relying on text to guide it, and still having it deeply understand real-world physics, like cause and effect. Guest: Wow, so it would learn how the world works just by watching it, the way a person might observe their surroundings. Host: Precisely, and cracking that pure visual understanding is what many researchers at Google see as the real path forward for artificial intelligence.

Host: Let's look at one of the biggest hurdles in developing AI world models, which is teaching an AI the rules of physics, like gravity, using just video data. Guest: That sounds tough, but couldn't an AI just learn how gravity works naturally by watching thousands of videos of things falling? Host: You'd think so, but it's actually incredibly tricky for a model to link those raw visual patterns to actual concepts without explicit language telling it what is happening. Guest: Ah, so the AI needs a text label that literally says "this is gravity" to truly understand what it is looking at? Host: Exactly, which forces developers right now to manually create datasets where videos are explicitly paired with detailed text descriptions. Guest: But wouldn't that severely limit the training data, since most videos on the internet don't have perfect, frame-by-frame text descriptions? Host: Spot on, and that data bottleneck is exactly why the amount of usable material is much less than the total media out there. That brings us to a major focus of current research, called representation learning. Guest: What makes representation learning the solution to this data bottleneck? Host: It aims to help the AI extract pure concepts directly from raw, unlabeled visuals, completely bypassing the need for human language labels. Guest: Is that something developers are already doing at a large scale? Host: Not quite yet, as it's still heavily in the research phase and very hard to scale up. But if researchers can finally unlock it, it would be a massive leap forward for these world models.

Host: We're looking at how AI is learning the rules of the physical universe, like gravity, just by watching endless hours of video. This creates what researchers call a "world model." Guest: That sounds amazing, but how do developers actually test that? If the AI also processes text, couldn't it just memorize Wikipedia's definition of gravity without really understanding how it works visually? Host: Exactly, that's a huge challenge researchers call evaluation, or "evals." One idea to solve this is testing them indirectly, like asking the AI to predict or induce movement in a complex visual system to prove it truly gets the physics. Guest: That makes sense. The text then shifts gears to talk about AI agents taking actions, like the consumer agents in Google's Spark—how does that connect? Host: It’s the next logical step; once a model understands an environment, you want it to act in it, like navigating a digital computer screen. The big breakthrough in capabilities there comes from optimizing the core AI model and the software system built around it at the exact same time. Guest: Are they building these agents to do highly specific tasks, like organizing your day, or are they meant to handle literally anything? Host: They usually start specialized because it's easier to make them useful and controllable right away. But as we saw with the Transformer architecture—which started strictly for language translation and now does almost everything—these specialized tools inevitably evolve into general-purpose systems.

Host: Right now, developers spend a lot of time hand-coding complex structures around AI models, like setting up sub-agents and delegating tasks. But we are moving toward a future where the AI itself will just write these systems on the fly. Guest: Wait, so instead of a human building a multi-agent system, the AI just instantly creates its own custom team for whatever task I give it? Host: Exactly, it figures out the most efficient way to tackle the problem, including deciding exactly how long it needs to reason before answering. Eventually, this dynamic scaffolding will wash out our current hard-coded setups. Guest: If these agents are running on their own for hundreds of steps, how do they stay on track without breaking down? Host: That relies on massive breakthroughs in context windows, allowing the underlying model to natively process huge amounts of information without relying on clever prompting tricks. Guest: That makes sense for a single session, but how does the AI actually remember things across multiple past interactions? Host: It operates similarly to human memory or a computer's cache, splitting into working memory for the immediate task and episodic memory for long-term storage. Guest: For that episodic memory, is it basically just writing its own notes and saving them into a digital file system? Host: Yes, the agent can write its thoughts into directories and folders, acting as an external, readable knowledge base it can retrieve from later. Guest: Why use external files instead of just baking those memories directly into the core AI model's brain? Host: Because from a practical standpoint, it is incredibly difficult to serve a single AI model at scale if you have to constantly update its core weights with every individual user's personal memories.

Host: Let's look at how AI models might become highly personalized to individual users in the future. Right now, creating a completely unique set of model weights for every single person would be incredibly difficult to support on the hardware side. Guest: If giving everyone a custom-weighted model is too hard, how are we personalizing things right now? Host: Instead of changing the core model, we use shared models that connect to a user's personal knowledge base. You can also add external layers of knowledge to enhance the model's capabilities without having to touch the underlying weights at all. Guest: I see, but what about models that actually update and learn continuously as you interact with them? Host: That concept is called continual learning, and it is a massive topic of debate right now. Some researchers believe that achieving true continual learning requires entirely new research bets, rather than just scaling up our current core models. Guest: So how does a major lab like Google balance exploring completely new ideas versus just making their current models bigger? Host: It is a delicate trade-off between protecting long-term, experimental research and staying right at the cutting edge of frontier models. If researchers get too disconnected from fast-moving core models like Gemini, they lose touch with what the technology can actually do. Guest: I imagine that takes a very specific type of organizational structure to pull off. Host: Exactly, and that is why unifying Google Brain and DeepMind was so crucial. They established Gemini as the central focus to keep pushing the frontier, which actually provides the stability needed to place riskier bets on future innovations.

Host: We're looking at how AI models learn and adapt today, particularly through reinforcement learning, or RL. Lately, we've seen a massive surge in AI using RL to solve complex math and coding problems. Guest: The speaker in the text mentions this exponential growth, but seems to wonder where RL will take off next. Why is it so difficult to apply this success to other domains? Host: It essentially comes down to how data is generated. When an AI learns to play a board game like Go, every move creates a totally unique board state, which basically gives the system infinite training data for free. Guest: Oh, I see. But with large language models, we don't have that infinite engine, so they are limited by the actual text data that exists. Host: Exactly. Because we don't have that clear source of infinite complexity for language, researchers are shifting their focus toward what they call "meta capabilities" to measure true intelligence. Guest: What exactly counts as a meta capability? Host: It's less about being a gold medalist in math, and more about the fundamental traits of intelligence itself. Things like in-context learning—which is how efficiently a model learns from a new experience—or how perfectly it can follow instructions. Guest: But since language models have read almost the whole internet, how do you test if they are actually adapting rather than just recalling something they already memorized? Host: A really clever test is to use complex games. You can give the model the instruction manual for a deep strategy game, like Civilization, and see if it can play just by reading the rules. Guest: So the true test isn't just whether it can make a valid move, but whether it actually figures out the strategy on the fly? Host: Spot on. The ultimate goal is to see if the AI can understand the rules and then actively improve its gameplay in an entirely unfamiliar scenario, without relying on prior training data.

Host: Right now, one of the biggest questions in AI is about generalization: if you train a model to be incredibly good at a specific subject like math or coding, does it actually get smarter at everything else? Guest: That’s an interesting thought, but how would getting better at math make an AI better at unrelated, everyday tasks? Host: It turns out, it induces a kind of deep, general reasoning. The speaker points out that models trained heavily on math and coding can suddenly reason through complex, real-world problems, like figuring out US moving logistics and taxes, even though they weren't explicitly trained for that. Guest: If it works that well, why do researchers focus so much on just math, coding, and games? Why not just train them on everything from the start? Host: Because those domains are strictly verifiable, meaning a computer can instantly check if an answer is right or wrong during reinforcement learning. For most things in life, like writing a great story or designing a fun game, you can't just write a simple true-or-false verifier. Guest: So how do they plan to train AI on those messy, creative tasks if they can't automatically score them? Host: By leaning into the fact that evaluating a solution is almost always easier than creating one from scratch. The hope is that as AI models get smarter, they can act as judges to evaluate each other's work on those subjective tasks, opening up entirely new training domains. Guest: That opens up a lot of possibilities. For people actually building AI startups right now, does this mean they should be training their own custom models to compete? Host: Not necessarily. The speaker suggests that the smartest move for most founders is to focus intensely on building great "evals" and gathering unique data. Creating rigorous evaluation tests for your specific app is incredibly valuable, whether you build your own model or not.

Host: When building new AI products, developers face a tough choice about whether to train their own models or build on top of existing giants. Guest: It seems like building on top of the big models is risky, since those base capabilities are constantly upgrading. Host: Exactly, which is why if you just tweak or "post-train" a model's weights, you end up on a treadmill having to redo your work every few months to stay ahead. Guest: So how do smaller companies actually protect their products from being wiped out by the next major model release? Host: The speaker suggests focusing on building a highly complex, specialized knowledge base for your application. It's much more efficient than training model weights, and it creates a lasting defensive moat around your product. Guest: That makes sense for today, but as these base models get smarter, what new capabilities are researchers trying to unlock next? Host: There is a very clear research path for "meta-learning," which is teaching models how to learn. But the frontier experts are least sure how to cross is achieving true scientific innovation. Guest: Wait, I thought AI was already innovating, given how well it can write code or solve complex math problems. Host: It definitely has a superhuman ability to process massive amounts of mechanistic data. However, generating truly novel, outstanding ideas with scientific "taste" is something models haven't mastered yet. Guest: Why is it so hard to train an AI to have that kind of creative intuition? Host: It comes down to measurement, because if true innovation is hard to clearly evaluate, it's nearly impossible to train a model to optimize or "hill climb" its way toward it.

Host: It is incredibly common to assume that to make an artificial intelligence smarter in a general sense, you have to train it on a massive, broad variety of data. Guest: That sounds logical. The more diverse the information, the more well-rounded the AI, right? Host: That was the running theory, but one top researcher recently admitted changing their mind after seeing that training models on narrow, incredibly difficult subjects actually unlocks broader generalization. Guest: Narrow subjects? What kind of difficult data are we talking about here? Host: Specifically, math and coding. It turns out that forcing a model to master the strict logic of those specific fields makes it vastly more capable across the board. Guest: That is fascinating. With leaps like that, are we basically hitting AGI, or artificial general intelligence, already? Host: It is funny you ask, because the researcher points out that if you showed today's models to an engineer just seven years ago, they absolutely would have declared it AGI. Guest: So the goalposts just keep moving further away as the technology actually improves? Host: Exactly. The researcher notes true AGI is still missing the ability to genuinely learn from experience, and we are also starting to face hard physical limits in energy and hardware. Guest: I imagine running up against hardware limits really forces these companies to rethink their long-term strategies. Host: It does, which is exactly why you see a giant like Alphabet selling some of its computing power to competitors like Anthropic. It creates immediate revenue to strategically reinvest back into their own research and balance out those physical bottlenecks.

Host: The interview wraps up by highlighting a massive advantage Google has in the AI race: they build both their own cutting-edge models and the custom computer chips to run them. Guest: Why is building their own chips such a big deal, rather than just buying them from a hardware company like NVIDIA? Host: It comes down to timing, since designing and manufacturing hardware takes months or even years of lead time. By having their AI researchers and hardware engineers under the same roof, they can anticipate exactly what their future models will need. Guest: That makes sense, so they are essentially designing the perfect engine for a car they haven't even finished building yet. Host: Exactly, and the speaker shares a fascinating memory from around 2013 with AI legends like Jeff Hinton and Ilya Sutskever. They were literally sitting in a room trying to guess if their future servers would need more standard processors or specialized GPUs. Guest: Trying to predict the hardware needs for deep learning back in 2013 must have been incredibly difficult. Host: It was a huge gamble back then, but having that tight feedback loop between the people dreaming up the AI and the people building the data centers is still a massive strategic advantage today. Guest: And at the very end of the interview, the guest signs off with some surprisingly simple advice for the listeners, right? Host: Yes, his parting wisdom is to just get your hands dirty and play with these models. No matter what industry you're in or what your day job is, his point is that you need to experience the technology firsthand because it's only going to get better.