Natural vision

Decoding Scenes from V1 During Natural Vision

Reconstructing static natural scenes from marmoset V1 activity during free viewing.

Authors Tejasvi Kothapalli¹, Jacob Yates¹

Affiliation ¹ UC Berkeley Redwood Center for Theoretical Neuroscience

Published Aug 15, 2026

Tags Decoding, Denoising, Natural Scenes, 1/f², Free-viewing, V1, Fovea

Abstract

To our knowledge, decoding a full static natural scene from V1 activity during natural vision has never been done. Here, we present an attempt to reconstruct a static scene from V1 activity while a marmoset freely views it. We record neural activity from marmoset foveal V1. Due to retinotopy and the recording location, all recorded cells respond to a ~0.5° × 0.5° region of visual space near the center of gaze. At every fixation, ridge regression predicts this gaze-contingent patch from the population’s neural activity. Eye position places each decoded patch in scene coordinates; patches accumulated across fixations and sessions are then combined with a 1/f prior to reconstruct the full scene. The interactive visualization below shows both patch-level decoding fidelity and the resulting full-scene reconstruction, for each of the two animals. Every reconstruction is fully held out: the decoder was never trained on the scene it reconstructs.

Animal

Each animal has its own sessions, decoder and reconstructions.

Scenes

Loading scenes… Select a scene, then scroll down to watch its reconstruction unfold.

Selected scene

Loading…

Total —

Current — Presentation — — cells

Click a point in the original scene to view its patch-decoding fidelity.
←→ step through fixations · hold to accelerate

Static natural scene Hull 0.35° Decoded patch Eye position

Click to inspect a patch

34.1° 19.2°

Decoded patches

1/f reconstruction Contrast restored

Fixation0 / 0 Viewing time0:00 / 0:00 Speed

Loading scene data…

Acknowledgments

Thanks Eero Simoncelli for inspiring me to do this project at CSHL and for incredibly helpful conversations about how to implement this.

Thanks Bruno Olshausen for suggesting a 1/f prior, which was crucial for cleaning up the reconstruction.

Thanks Jacob Yates for the infinite wisdom and support, as always.

Thanks to my best friend Ashwin Nathan for giving me access to Codex.