Research Published …

DeepMind published a series of papers about large language models (LLMs) last year, including an analysis of Gopher, our large language model. Language modelling technology, which is also currently being developed by several other labs and companies, promises to strengthen many applications, from search engines to a new wave of chatbot-like conversational assistants and beyond.…

Research Published …

In the last few years, a focus in language modelling has been on improving performance through increasing the number of parameters in transformer-based models. This approach has led to impressive results and state-of-the-art performance across many natural language processing tasks. We also pursued this line of research at DeepMind and recently showcased Gopher, a 280-billion…

Working toward greater generalisability in artificial intelligence Today, conference season is kicking off with The Tenth International Conference on Learning Representations (ICLR 2022), running virtually from 25-29 April, 2022. Participants from around the world are gathering to share their cutting-edge work in representational learning, from advancing the state of the art in artificial intelligence to…

For our first ever 'Five minutes with' we caught up with Kevin Millikin, a software engineer on the DevTools team. He’s in Salt Lake City this week to present at PyCon US, the largest annual gathering for those using and developing the open-source Python programming language. At DeepMind... I build bespoke software tools for our…

One key aspect of intelligence is the ability to quickly learn how to perform a new task when given a brief instruction. For instance, a child may recognise real animals at the zoo after seeing a few pictures of the animals in a book, despite differences between the two. But for a typical visual model…

Many recent successes in language models (LMs) have been achieved within a ‘static paradigm’, where the focus is on improving performance on the benchmarks that are created without considering the temporal aspect of data. For instance, answering questions on events that the model could learn about during training, or evaluating on text sub-sampled from the…

Life at DeepMind Published …

To train agents to interact well with humans, we need to be able to measure progress. But human interaction is complex and measuring progress is difficult. In this work we developed a method, called the Standardised Test Suite (STS), for evaluating agents in temporally extended, multi-modal interactions. We examined interactions that consist of human participants…