What does a brain really see?
Many animals rely on vision to move through the world, find food, and interact with others. How does the brain transform the light striking the retina into visual perception? In our lab, we use computational tools and cellular-resolution brain maps to understand how these transformations happen at the level of single neurons.
The view from a fly’s cockpit
To understand how a brain encodes visual perception, we first ask what visual signals are available to it. Our favorite animal to study is the fruit fly (Drosophila melanogaster), an amazing flyer with signature red compound eyes, each made up of hundreds of facet lenses. What a fly can see depends on the geometry of its eyes. Using X-ray scans, we build an eyemap — the viewing direction of every facet of the compound eye (left) — and plot these directions (3D vectors) on the same kind of projection a cartographer uses to flatten the globe (right, color-matched).
Vision evolved to guide behavior, so what an animal sees also depends on how it moves. With the eyemap, we can simulate visual input in a virtual environment. This animation shows a virtual fly moving through a naturalistic landscape, with the scene rendered as its compound eyes actually sample it — coarse, nearly panoramic, and uneven across the visual field — rather than as a camera would.
Map the world into the brain
Behind each eye facet sits a bundle of photoreceptors, which convert light into neural signals and pass them on to columnar neurons in the optic lobe (the fly brain’s visual center). This columnar arrangement preserves neighbor relationships, so each layered region, from the retina down to the lobula plate (many intermediate layers are not shown here), holds a map of visual space.
But how can we use this map to study the properties of individual neurons? Here lies another privilege of working with fruit flies: electron microscopy (EM) connectomes, which contain every neuron and synapse in the brain. The image below shows a sample of visual neurons originating from one optic lobe. Combining the eyemap with the connectome lets us tie every point in the visual world to a location in the brain.
Predict a(ny) neuron’s function
A neuron’s wiring is not arbitrary: the shape of its dendrites, where its axon goes, and the neurons it talks to all constrain what it can compute. We apply novel computational methods to large-scale neuroanatomical data to make quantitative, testable predictions about neuronal function. Using the eyemap, we can place a neuron’s inputs in visual space and predict its receptive field, such as its local motion preference. Take VS1, a large neuron that receives inputs in the lobula plate and sends its axon into the central brain. From the eyemap and its inputs, we can map which direction of motion it prefers at each point in space (arrows in the figure): back-to-front above the fly and downward in front of it — the pattern a fly would see as it lifts its head (a pitch rotation).
With access to both the visual input to the compound eyes and every neuron in the brain, we can build biologically constrained network models that simulate how the brain turns the images captured by the eyes into signals that drive behavior, and how that behavior in turn shapes the next view — closing the loop from stimulus to brain to behavior.