How does intelligence emerge from the natural elements of the brain?
Melika Payvand, research fellow in the group of Giacomo Indiveri at the Faculty of Science/ETHZ dual Institute of Neuroinformatics, is fascinated by the abilities of the human brain. But how do these 86 billion neurons (or on average 1.3 kg) create intelligence and memory? And can we transfer these principles to build intelligent devices? Together with a team of eager scientists from the University of Grenoble Alpes and Toshiba’s Corporate Research and Development Center, she developed a ground-breaking new adaptive hardware architecture published in their Nature Communications article Self-organization of an inhomogeneous memristive hardware for sequence learning.
Here, she explains their approach: “Computing, as we know it, is done using digital computers that reduce the dynamic range of computation to rails (power and ground) to gain noise immunity and perform robust computation. However, the brain has a substrate that is far from perfect, and neurons and synapses are highly noisy and variable.
Micro-organisms have evolved over the course of millions of years resulting in the intelligent animals that we are, interacting with the real world while performing "computation" using these noisy elements. How has this intelligence emerged? The answers to these questions are mostly unknown. But there is neuroscientific evidence that neuronal systems in the brain dynamically store and organize incoming information into a web of memory representations which is essential for the generation of complex behaviors. This memory formation is as a result of a combination of learning rules at different time scales.
Inspired by these mechanisms, we have designed and demonstrated an adaptive hardware architecture, “MEMSORN”. MEMSORN takes advantage of the spatial and temporal noise of the physics of the substrate to self-organize its structure to the incoming input and learn a sequence. Specifically, we take advantage of the inherent noise of the resistive memory, an emerging memory technology, to learn the structure of a sequence without any supervision. The unsupervised learning rules are directly derived from the statistical characteristics of the available hardware, and thus learning emerges from the substrate itself. Therefore, this work represents a fundamental step toward the design of future brain-inspired intelligent devices and applications. Specifically, these findings have applications in adapting the edge devices to the unique users. For example, adapting wearable devices to each patient, or the smart home devices to the users’ accent, habits, or preferences.
I am very grateful for the opportunity to work with such inspiring colleagues, namely my co-author Filippo Moro at CEA-LETI, Thomas Dalgaty and Elisa Vianello also from Grenoble, Kumiko Nomura and Yoshifumi Nishi at Toshiba as well as Giacomo!”