distill.pubadded August 2026
Distillby Benjamin Sanchez-Lengeling; Emily Reif; Adam Pearce; Alexander B Wiltschko2 Sept 2021
Neural networks have been adapted to leverage the structure and properties of graphs. We explore the components needed for building a graph neural network - and motivate the design choices behind...
Distillby Ameya Daigavane; Balaraman Ravindran; Gaurav Aggarwal2 Sept 2021
Understanding the building blocks and design choices of graph neural networks. This article is one of two Distill publications about graph neural networks.
Distillby Editorial Team2 Jul 2021
Over the past five years, Distill has supported authors in publishing artifacts that push beyond the traditional expectations of scientific papers.
Distillby Ettore Randazzo; Alexander Mordvintsev; Eyvind Niklasson; Michael Levin6 May 2021
A robustness investigation. This article makes strong use of colors in figures and demos. Click here to adjust the color palette.
Distillby Michael Petrov; Chelsea Voss; Ludwig Schubert; Nick Cammarata; Gabriel Goh; Chris Olah8 Apr 2021
Open up any ImageNet conv net and look at the weights in the last layer. You’ll find a uniform spatial pattern to them, dramatically unlike anything we see elsewhere in the network.
Distillby Chelsea Voss; Gabriel Goh; Nick Cammarata; Michael Petrov; Ludwig Schubert; Chris Olah5 Apr 2021
If we think of interpretability as a kind of “anatomy of neural networks,” most of the circuits thread has involved studying tiny little veins – looking at the small-scale, at individual neurons...
Distillby Gabriel Goh; Nick Cammarata; Chelsea Voss; Shan Carter; Michael Petrov; Ludwig Schubert; Alec Radford; Chris Olah4 Mar 2021
Acknowledgments We are deeply grateful to Sandhini Agarwal, Daniela Amodei, Dario Amodei, Tom Brown, Jeff Clune, Steve Dowling, Gretchen Krueger, Brice Menard, Reiichiro Nakano, Aditya Ramesh...
Distillby Eyvind Niklasson; Alexander Mordvintsev; Ettore Randazzo; Michael Levin11 Feb 2021
Neural Cellular Automata Model of Pattern Formation Neural Cellular Automata (NCA In this work, we apply NCA to the task of texture synthesis.
Distillby Chelsea Voss; Nick Cammarata; Gabriel Goh; Michael Petrov; Ludwig Schubert; Ben Egan; Swee Kiat Lim; Chris Olah4 Feb 2021
The problem of understanding a neural network is a little bit like reverse engineering a large compiled binary of a computer program.
Distillby Nick Cammarata; Gabriel Goh; Shan Carter; Chelsea Voss; Ludwig Schubert; Chris Olah30 Jan 2021
We reverse engineer a non-trivial learned algorithm from the weights of a neural network and use its core ideas to craft an artificial artificial neural network from scratch that reimplements it.
Distillby Ludwig Schubert; Chelsea Voss; Nick Cammarata; Gabriel Goh; Chris Olah27 Jan 2021
A family of early-vision neurons reacting to directional transitions from high to low spatial frequency.
Distillby Chris Olah; Nick Cammarata; Chelsea Voss; Ludwig Schubert; Gabriel Goh8 Dec 2020
Convolutional neural networks contain a hidden world of symmetries within themselves. This symmetry is a powerful tool in understanding the features and circuits inside neural networks.
Distillby Jacob Hilton; Nick Cammarata; Shan Carter; Gabriel Goh; Chris Olah17 Nov 2020
In this article, we apply interpretability techniques to a reinforcement learning (RL) model trained to play the video game CoinRun .
Distillby Fred Hohman; Matthew Conlen; Jeffrey Heer; Duen Horng Chau11 Sept 2020
Examining the design of interactive articles by synthesizing theory from disciplines such as education, journalism, and visualization. Computing has changed how people communicate.
Distillby Ettore Randazzo; Alexander Mordvintsev; Eyvind Niklasson; Michael Levin; Sam Greydanus27 Aug 2020
Achieving Distributed Coordination with Neural Cellular Automata Growing Neural Cellular Automata Our question is closely related to another unsolved problem in developmental and regenerative...
Distillby Alexander Mordvintsev; Ettore Randazzo; Eyvind Niklasson; Michael Levin; Sam Greydanus27 Aug 2020
How can we construct robust, general-purpose self-organising systems? Self-organisation is omnipresent on all scales of biological life.
Distillby Nick Cammarata; Gabriel Goh; Shan Carter; Ludwig Schubert; Michael Petrov; Chris Olah17 Jun 2020
Every vision model we’ve explored in detail contains neurons which detect curves. Curve detectors in vision models have been hinted at in the literature as far back as 2013 (see figures in Zeiler &...
Distillby Apoorv Agnihotri; Nipun Batra5 May 2020
Breaking Bayesian Optimization into small, sizeable chunks. Many modern machine learning algorithms have a large number of hyperparameters.
Distillby Chris Olah; Nick Cammarata; Ludwig Schubert; Gabriel Goh; Michael Petrov; Shan Carter1 Apr 2020
A guided tour of the first five layers of InceptionV1, taxonomized into “neuron groups.” The first few articles of the Circuits project will be focused on early vision in InceptionV1 Over the...
Distillby Mingwei Li; Zhenge Zhao; Carlos Scheidegger16 Mar 2020
The Grand Tour Deep neural networks often achieve best-in-class performance in supervised learning contests such as the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) To understand a...
19%