Skip to yearly menu bar Skip to main content


Poster

MonkeySee: Space-time-resolved reconstructions of natural images from macaque multi-unit activity

Lynn Le · Paolo Papale · Katja Seeliger · Antonio Lozano · Thirza Dado · Feng Wang · Pieter Roelfsema · Marcel A. J. van Gerven · Yağmur Güçlütürk · Umut Güçlü

[ ] [ Project Page ]
Thu 12 Dec 11 a.m. PST — 2 p.m. PST

Abstract:

In this paper, we reconstruct naturalistic images directly from macaque brain signals using a convolutional neural network (CNN) based decoder. We investigate the ability of this CNN-based decoding technique to differentiate among neuronal populations from areas V1, V4, and IT, revealing distinct readout characteristics for each. This research marks a progression from low-level to high-level brain signals, thereby enriching the existing framework for utilizing CNN-based decoders to decode brain activity. Our results demonstrate high-precision reconstructions of naturalistic images, highlighting the efficiency of CNN-based decoders in advancing our knowledge of how the brain's representations translate into pixels. Additionally, we present a novel space-time-resolved decoding technique, demonstrating how temporal resolution in decoding can advance our understanding of neural representations. Moreover, we introduce a Learned Receptive Field (LRF) layer that sheds light on the CNN-based model's data processing during training, enhancing understanding of its structure and interpretive capacity.

Live content is unavailable. Log in and register to view live content