Deep physical neural networks using physics-aware training

Deep neural networks have become ubiquitous in today’s data-driven world, but their energy requirements increasingly limit their scalability and broader use. Here, we propose the construction of deep physical neural networks that are made from layers of controllable physical systems, which can learn hierarchical representations of input data analogous to deep neural networks. To train these physical neural networks, we introduce a hybrid in situ-in silico algorithm physics-aware training. This training method has favorable scaling properties as it uses backpropagation, the de-facto training method for deep neural networks. To demonstrate the universality of our approach, we train diverse physical neural networks based on optics, mechanics, and electronics to experimentally perform audio and image classification tasks. Our approach broadens the possibility of using novel physical systems for deep learning and potentially enables them to perform machine learning faster and more energy-efficiently than conventional electronic processors.