Tag: reveals

A cubic millimetre is a tiny volume — less than a teardrop. But a cubic millimetre of mouse brain is densely packed with tens of thousands of neurons and other cells in a staggeringly complex architectural weave.

Reconstructing such elaborate arrangements requires monumental effort, but the researchers affiliated with the Machine Intelligence from Cortical Networks (MICrONS) programme pulled it off. It took US$100 million and years of effort by more than 100 scientists, coordinated by 3 groups that had never collaborated before. There were weeks of all-nighters and a painstaking global proofreading effort that continues even now — for a volume that represents just 0.2% of the typical mouse brain. Despite the hurdles, the core of the project — conceived and funded by the US Intelligence Advanced Research Projects Activity (IARPA) — is complete.

Human brain mapping

The resulting package includes a high-resolution 3D electron microscopy reconstruction of the cells and organelles in two separate volumes of the mouse visual cortex, coupled with fluorescent imaging of neuronal activity from the same volumes. Even the coordinators of the MICrONS project, who describe IARPA’s assembly of the consortium as a ‘shotgun wedding’ of parallel research efforts, were pleasantly surprised by the outcome. “It formed this contiguous team, and we’ve been working extremely well together,” says Andreas Tolias, a systems neuroscientist who led the functional imaging effort at Baylor College of Medicine in Houston, Texas. “It’s impressive.”

The MICrONS project is a milestone in the field of ‘connectomics’, which aims to unravel the synaptic-scale organization of the brain and chart the circuits that coordinate the organ’s many functions. The data from these first two volumes are already providing the neuroscience community with a valuable resource. But this work is also bringing scientists into strange and challenging new territory. “The main casualty of this information is understanding,” says Jeff Lichtman, a connectomics pioneer at Harvard University in Cambridge, Massachusetts. “The more we know, the harder it is to turn this into a simple, easy-to-understand model of how the brain works.”

Short circuits

There are many ways to look at the brain, but for connectivity researchers, electron microscopy has proved especially powerful.

In 1986, scientists at the University of Cambridge, UK, used serial-section electron microscopy to generate a complete map of the nervous system for the roundworm Caenorhabditis elegans¹. That connectome was a landmark achievement in the history of biology. It required the arduous manual annotation and reconstruction of some 8,000 2D images, but yielded a Rosetta Stone for understanding the nervous system of this simple, but important, animal model.

The rise of digital twins

No comparable resource exists for more complex animals, but early forays into the rodent connectome have given hints of what such a map could reveal. Lichtman recalls the assembly he and his colleagues produced in 2015 from a 1,500-cubic-micron section of mouse neocortex — roughly one-millionth of the volume used in the MICrONS project². “Most people were just shocked to see the density of wires all pushed together in any little part of brain,” he says.

Similarly, Moritz Helmstaedter, a connectomics researcher at the Max Planck Institute for Brain Research in Frankfurt, Germany, says that his team’s efforts³ in reconstructing a densely packed region of the mouse somatosensory cortex, which processes sensations related to touch, in 2019 challenged existing dogma — especially the assumption that neurons in the cortex are randomly wired. “We explicitly proved that wrong,” Helmstaedter says. “We found this extreme precision.” These and other studies have collectively helped to cement the importance of electron-microscopy-based circuit maps as a complement to techniques such as light microscopy and molecular methods.

Bigger and better

IARPA’s motivation for the MICrONS project was grounded in artificial intelligence. The goal was to generate a detailed connectomic map at the cubic-millimetre-scale, which could then be ‘reverse-engineered’ to identify architectural principles that might guide the development of biologically informed artificial neural networks.

Tolias, neuroscientist Sebastian Seung at Princeton University in New Jersey, and neurobiologist Clay Reid at the Allen Institute for Brain Science in Seattle, Washington, had all applied independently for funding to contribute to separate elements of this programme. But IARPA’s programme officers elected to combine the 3 teams into a single consortium — including a broader network of collaborators — issuing $100 million in 2016 to support a 5-year effort.

The MICrONS team selected two areas from the mouse visual cortex: the aforementioned cubic millimetre, and a much smaller volume that served as a pilot for the workflow. These were chosen so the team could investigate the interactions between disparate regions in the visual pathway, explains Tolias, who oversaw the brain-activity-imaging aspect of the work at Baylor. To achieve that, the researchers genetically engineered a mouse to express a calcium-sensitive ‘reporter gene’, which produces a fluorescent signal whenever a neuron or population of neurons fires. His team then assembled video footage of diverse realistic scenes, which the animal watched with each eye independently for two hours while a microscope tracked neuronal activity.

Probing fine-scale connections in the brain

The mouse was then shipped to Seattle for preparation and imaging of the relevant brain volumes — and the pressure kicked up another notch. Nuno da Costa, a neuroanatomist and associate investigator at the Allen Institute, says he and Tolias compressed their groups’ schedules to accommodate the final, time-consuming stage of digital reconstruction and analysis conducted by Seung’s group. “We really pushed ourselves to deliver — to fail as early as possible so we can course-correct in time,” da Costa says. This meant a race against the clock to excise the tissue, carve it into ultra-thin slices and then image the stained slices with a fleet of 5 electron microscopes. “We invested in this approach where we could buy very old machines, and really automate them to make them super-fast,” says da Costa. The researchers could thus maximize throughput and had backups should a microscope fail.

For phase one of the project, which involved reconstructing the smaller cortical volume, sectioning of the tissue came down to the heroic efforts of Agnes Bodor, a neuroscientist at the Allen Institute, who spent more than a month hand-collecting several thousand 40-nanometre-thick sections of tissue using a diamond-bladed instrument known as a microtome, da Costa says. That manual effort was untenable for the larger volume in phase two of the project, so the Allen team adopted an automated approach. Over 12 days of round-the-clock, supervised work, the team generated almost 28,000 sections containing more than 200,000 cells⁴. It took six months to image all those sections, yielding some 2 petabytes of data.

The Allen and Baylor teams also collaborated to link the fluorescently imaged cells with their counterparts in the reconstructed connectomic volume.

Throughout this process, the Allen team relayed its data sets to the team at Princeton University. Serial-section electron microscopy is a well-established technique, but assembly of the reconstructed volume entails considerable computational work. Images must be precisely aligned with one another while accounting for any preparation- or imaging-associated deformations, and then they are subjected to ‘segmentation’ to identify and annotate neurons, non-neuronal cells such as glia, organelles and other structures. “The revolutionary technology in MICrONS was image alignment,” Seung says. This part is crucial, because a misstep in the positioning of a single slice can derail the remainder of the reconstruction process. Manual curation would be entirely impractical at the cubic-millimetre scale. But through its work in phase one, the team developed a reconstruction workflow that could be scaled up for the larger brain volume, and continuing advances in deep-learning methods made it possible to automate key alignment steps.

To check the work, Sven Dorkenwald, who was a graduate student in Seung’s laboratory and is now a research fellow at the Allen Institute, developed a proofreading framework to refine the team’s reconstructions and ensure their biological fidelity. This approach, which verified the paths of neuronal processes through the connectome, carved the volumes into ‘supervoxels’ — 3D shapes that define segmented cellular or subcellular features, which can be rearranged to improve connectomic accuracy — and Dorkenwald says the final MICrONS data set had 112 billion of them. The system is analogous to the online encyclopedia Wikipedia in some ways, allowing many users to contribute edits in parallel while also logging the history of changes. But even crowdsourced proofreading is slow going — Dorkenwald estimates that each axon (the neuronal projections that transmit signals to other cells) in the MICrONS data set takes up to 50 hours to proofread.

Charting new territory

The MICrONS team published a summary⁵ of its phase one results in 2022. Much of its other early findings still await publication, including a detailed description of the work from phase two — although this is currently available as a preprint article⁴. But there are already some important demonstrations of what connectomics at this scale can deliver.

FlyWire: online community for whole-brain connectomics

One MICrONS preprint, for example, describes what is perhaps the most comprehensive circuit map so far for a cortical column⁶, a layered arrangement of neurons that is thought to be the fundamental organizational unit of the cerebral cortex. The team’s reconstruction yielded a detailed census of all the different cell types residing in the column and revealed previously unknown patterns in how various subtypes of neuron connect with one another. “Inhibitory cells have this remarkable specificity towards some excitatory cell types, even when these excitatory cells are mixed together in the same layer,” says da Costa. Such insights could lead to more precise classification of the cells that boost or suppress circuit activity and reveal the underlying rules that guide the wiring of those circuits.

Crucially, says Tolias, the MICrONS project was about more than the connectome: “It was large-scale, functional imaging of the same mouse.” Much of his team’s work has focused on translating calcium reporter-based activity measurements into next-generation computational models. In 2023, the researchers posted a preprint that describes the creation of a deep-learning-based ‘digital twin’ on the basis of experimentally measured cortical responses to visual stimuli⁷. The predictions generated by this ‘twin’ can then be tested, further refining the model and enhancing its accuracy.

One surprising and valuable product of the MICrONS effort involves fruit flies. Early in the project, Seung’s team began exploring serial-section electron-microscopy data from the Drosophila melanogaster brain produced by researchers at the Howard Hughes Medical Institute’s Janelia Research Campus in Ashburn, Virginia⁸. “I realized that because we had developed this image-alignment technology, we had a chance to do something that people thought was impossible,” says Seung. His team — including Dorkenwald — used the Janelia data as a proving ground for the algorithms that had been developed for MICrONS. The result was the first complete assembly of the fruit-fly brain connectome — around 130,000 neurons in total⁹.

Given that the wiring of the nervous system is generally conserved across fruit flies, Dorkenwald is enthusiastic about how these data — which are publicly accessible at http://flywire.ai — could enable future experiments. “You can do functional imaging on a fly, and because you can find the same neurons over in the connectome, you will be able to do these functional-structure analyses,” he says.

The mouse connectome will not be so simple, because connectivity varies from individual to individual. But the MICrONS data are nevertheless valuable for the neuroscience community, says Helmstaedter, who was not part of the MICrONS project. “It’s great data, and it’s inspiring people just to go look at it and see it,” he says. There’s also the power of demonstrating what is possible, and how it could be done better. “You’ve got to do something brute force first to find out where you can make it easier the next round,” says Kristen Harris, a neuroscientist at the University of Texas at Austin. “And the act of doing it — just getting the job done — is just spectacular.”

Terra incognita

Even as analysis of the MICrONS data set proceeds, its limitations are already becoming clear. For one thing, volumes from other distinct cortical regions will be needed to identify features that are broadly observed throughout the brain versus those features that are distinct to the visual cortex. And many axons from this first cubic millimetre will inevitably connect to points unknown, Lichtman notes, limiting researchers’ ability to fully understand the structure and function of the circuits within it.

Scaling up will be even harder. Lichtman estimates that a whole-brain electron-microscopy reconstruction would produce roughly an exabyte of data, which is equivalent to a billion gigabytes and is 1,000 times greater than the petabytes of data produced by the MICrONS project. “This may be a ‘Mars shot’ — it’s really much harder than going to the Moon,” he says.

Still, the race is under way. One major effort is BRAIN CONNECTS, a project backed by the US National Institutes of Health with $150 million in funding, which is coordinated by multiple researchers, including Seung, da Costa and Lichtman. “We’re not delivering the whole mouse brain yet, but testing if it’s possible,” da Costa says. “Mitigating all the risks, bringing the cost down, and seeing if we can actually prepare a whole-mouse-brain or whole-hemisphere sample.”

In parallel, Lichtman is working with a team at Google Research in Mountain View, California, led by computer scientist Viren Jain — who collaborated with MICrONS and is also part of the BRAIN CONNECTS leadership team — to map sizable volumes of the human cortex using electron microscopy. They’ve already released data from their first cubic millimetre and have plans to begin charting other regions from people with various neurological conditions¹⁰.

NatureTech hub

These efforts will require improved tools. The serial-section electron-microscopy strategy that MICrONS used is too labour-intensive to use at larger scales and yields relatively low-quality data that are hard to analyse. But alternatives are emerging. For example, ‘block-face’ electron-microscopy methods, in which the sample is imaged as a solid volume and then gradually shaved away with a high-intensity ion-beam, require less work in terms of image alignment and can be applied to thick sections of tissue that are easier to manage. These methods can be combined with cutting-edge multi-beam scanning electron microscopes, which image specimens using up to 91 electron beams simultaneously, thus accelerating data collection. “That’s one of the leading contenders for scale up to a whole mouse brain,” says Seung, who will be working with Lichtman on this strategy.

Further automation and more artificial-intelligence tools will also be assets. Helmstaedter and his colleagues have been looking into ways to simplify image assembly with an automated segmentation algorithm called RoboEM, which traces neural processes with minimal human intervention and can potentially eliminate a lot of the current proofreading burden¹¹. Still, higher-quality sample preparation and imaging are probably the true key to efficiency at scale, Helmstaedter says. “The better your data, the less you have to worry about automation.”

However they are generated, making sense of these connectome maps will take more than fancy technology. Tolias thinks “it will be almost impossible” to replicate the coupling of structure and activity produced by MICrONS at the whole-brain scale. But it’s also unclear whether that will be necessary and to what extent functional information can be inferred through a better understanding of brain structure and organization.

For Lichtman, the connectome’s value will ultimately transcend conventional hypothesis-driven science. A connectome “forces you to see things you weren’t looking for, and yet they’re staring you in the face”, he says. “I think if we do a whole mouse brain, there will be just an infinite number of ‘wow, really?’ discoveries.”