-
Photonic Neural Networks and Optics-informed Deep Learning Fundamentals
Authors:
A. Tsakyridis,
M. Moralis-Pegios,
G. Giamougiannis,
M. Kirtas,
N. Passalis,
A. Tefas,
N. Pleros
Abstract:
The recent explosive compute growth, mainly fueled by the boost of AI and DNNs, is currently instigating the demand for a novel computing paradigm that can overcome the insurmountable barriers imposed by conventional electronic computing architectures. PNNs implemented on silicon integration platforms stand out as a promising candidate to endow NN hardware, offering the potential for energy effici…
▽ More
The recent explosive compute growth, mainly fueled by the boost of AI and DNNs, is currently instigating the demand for a novel computing paradigm that can overcome the insurmountable barriers imposed by conventional electronic computing architectures. PNNs implemented on silicon integration platforms stand out as a promising candidate to endow NN hardware, offering the potential for energy efficient and ultra-fast computations through the utilization of the unique primitives of photonics i.e. energy efficiency, THz bandwidth and low-latency. Thus far, several demonstrations have revealed the huge potential of PNNs in performing both linear and non-linear NN operations at unparalleled speed and energy consumption metrics. Transforming this potential into a tangible reality for DL applications requires, however, a deep understanding of the basic PNN principles, requirements and challenges across all constituent architectural, technological and training aspects. In this tutorial, we, initially, review the principles of DNNs along with their fundamental building blocks, analyzing also the key mathematical operations needed for their computation in a photonic hardware. Then, we investigate, through an intuitive mathematical analysis, the interdependence of bit precision and energy efficiency in analog photonic circuitry, discussing the opportunities and challenges of PNNs. Followingly, a performance overview of PNN architectures, weight technologies and activation functions is presented, summarizing their impact in speed, scalability and power consumption. Finally, we provide an holistic overview of the optics-informed NN training framework that incorporates the physical properties of photonic building blocks into the training process in order to improve the NN classification accuracy and effectively elevate neuromorphic photonic hardware into high-performance DL computational settings.
△ Less
Submitted 22 November, 2023;
originally announced December 2023.
-
Non-negative isomorphic neural networks for photonic neuromorphic accelerators
Authors:
Manos Kirtas,
Nikolaos Passalis,
Nikolaos Pleros,
Anastasios Tefas
Abstract:
Neuromorphic photonic accelerators are becoming increasingly popular, since they can significantly improve computation speed and energy efficiency, leading to femtojoule per MAC efficiency. However, deploying existing DL models on such platforms is not trivial, since a great range of photonic neural network architectures relies on incoherent setups and power addition operational schemes that canno…
▽ More
Neuromorphic photonic accelerators are becoming increasingly popular, since they can significantly improve computation speed and energy efficiency, leading to femtojoule per MAC efficiency. However, deploying existing DL models on such platforms is not trivial, since a great range of photonic neural network architectures relies on incoherent setups and power addition operational schemes that cannot natively represent negative quantities. This results in additional hardware complexity that increases cost and reduces energy efficiency. To overcome this, we can train non-negative neural networks and potentially exploit the full range of incoherent neuromorphic photonic capabilities. However, existing approaches cannot achieve the same level of accuracy as their regular counterparts, due to training difficulties, as also recent evidence suggests. To this end, we introduce a methodology to obtain the non-negative isomorphic equivalents of regular neural networks that meet requirements of neuromorphic hardware, overcoming the aforementioned limitations. Furthermore, we also introduce a sign-preserving optimization approach that enables training of such isomorphic networks in a non-negative manner.
△ Less
Submitted 2 October, 2023;
originally announced October 2023.
-
Perfect Linear Optics using Silicon Photonics
Authors:
Miltiadis Moralis-Pegios,
George Giamougiannis,
Apostolos Tsakyridis,
David Lazovsky,
Nikos Pleros
Abstract:
In recent years, there has been growing interest in using photonic technology to perform the underlying linear algebra operations required by different applications, including neuromorphic photonics, quantum computing and microwave processing, mainly aiming at taking advantage of the silicon photonics' (SiPho) credentials to support high-speed and energy-efficient operations. Map**, however, a t…
▽ More
In recent years, there has been growing interest in using photonic technology to perform the underlying linear algebra operations required by different applications, including neuromorphic photonics, quantum computing and microwave processing, mainly aiming at taking advantage of the silicon photonics' (SiPho) credentials to support high-speed and energy-efficient operations. Map**, however, a targeted matrix with absolute accuracy into the optical domain remains a huge challenge in linear optics, since state-of-the-art linear optical circuit architectures are highly sensitive to fabrication imperfections. This leads to reduced fidelity metrics that degrade faster as the insertion losses of the constituent optical matrix node or the matrix dimensions increase. In this work, we present for the first time a novel coherent SiPho crossbar (Xbar) that can support on-chip fidelity restoration while implementing linear algebra operations, realizing the first experimental demonstration of perfect on-chip arbitrary linear optical transformations. We demonstrate the experimental implementation of 10000 arbitrary linear transformations in the photonic domain achieving a record high fidelity of 99.997%+-0.002, limited mainly by the statistical error enforced by the measurement equipment. Our work represents the first integrated universal linear optical circuit that provides almost unity and loss-independent fidelity in the realization of arbitrary matrices, opening new avenues for exploring the use of light in resolving universal computational tasks.
△ Less
Submitted 30 June, 2023;
originally announced June 2023.
-
Coherent photonic crossbar as a universal linear operator
Authors:
George Giamougiannis,
Apostolos Tsakyridis,
Yang** Ma,
Angelina Totovic,
David Lazovsky,
Nikos Pleros
Abstract:
Linear optics aim at realizing any real- and/or complex-valued matrix operator via optical elements, addressing a broad field of applications in the areas of quantum photonics, microwave photonics and optical neural networks. The transfer of linear operators into photonic experimental layouts typically relies on Singular Value Decomposition (SVD) techniques combining meshes of cascaded 2x2 Mach Ze…
▽ More
Linear optics aim at realizing any real- and/or complex-valued matrix operator via optical elements, addressing a broad field of applications in the areas of quantum photonics, microwave photonics and optical neural networks. The transfer of linear operators into photonic experimental layouts typically relies on Singular Value Decomposition (SVD) techniques combining meshes of cascaded 2x2 Mach Zehnder Interferometers (MZIs), with the main challenges being the precision in the experimental representation of the targeted matrix, referred to as fidelity, and the overall insertion loss. We demonstrate a novel interferometric coherent photonic crossbar architecture (Xbar) that demarcates from state-of-the-art SVD implementations and can realize any linear operator, supporting full restoration of the loss-induced fidelity. Its novel interferometric design allows for the direct map** of each matrix element to a single, designated Xbar node, bringing down the number of programming steps to only one. We present the theoretical foundations of the Xbar, proving that its insertion losses scale linearly with the node losses as opposed to the exponential scaling witnessed by the SVD counterparts. This leads to a matrix design with significantly lower overall insertion losses compared to SVD-based schemes when utilizing state-of-the-art silicon photonic fabrication metrics, allowing for alternative node technologies with lower energy consumption and higher operational speed credentials to be employed. Finally, we validate that our Xbar architecture is the first linear operator that supports fidelity restoration, outperforming SVD schemes in loss- and phase-error fidelity performance and forming a significantly more robust layout to loss and phase deviations.
△ Less
Submitted 24 August, 2022;
originally announced August 2022.
-
Energy-Proportional Data Center Network Architecture Through OS, Switch and Laser Co-design
Authors:
Haiyang Han,
Nikos Terzenidis,
Dimitris Syrivelis,
Arash F. Beldachi,
George T. Kanellos,
Yigit Demir,
Jie Gu,
Srikanth Kandula,
Nikos Pleros,
Fabián Bustamante,
Nikos Hardavellas
Abstract:
Optical interconnects are already the dominant technology in large-scale data center networks. However, the high optical loss of many optical components coupled with the low efficiency of laser sources result in high aggregate power requirements for the thousands of optical transceivers used by these networks. As optical interconnects stay always on even as traffic demands ebb and flow, most of th…
▽ More
Optical interconnects are already the dominant technology in large-scale data center networks. However, the high optical loss of many optical components coupled with the low efficiency of laser sources result in high aggregate power requirements for the thousands of optical transceivers used by these networks. As optical interconnects stay always on even as traffic demands ebb and flow, most of this power is wasted. We present LC/DC, a data center network system architecture in which the operating system, the switch, and the optical components are co-designed to achieve energy proportionality.
LC/DC capitalizes on the path divergence of data center networks to turn on and off redundant paths according to traffic demand, while maintaining full connectivity. Turning off redundant paths allows the optical transceivers and their electronic drivers to power down and save energy. Maintaining full connectivity hides the laser turn-on delay. At the node layer, intercepting send requests within the OS allows for the NIC's laser turn-on delay to be fully overlapped with TCP/IP packet processing, and thus egress links can remain powered off until needed with zero performance penalty.
We demonstrate the feasibility of LC/DC by i) implementing the necessary modifications in the Linux kernel and device drivers, ii) implementing a 10Gbit/s FPGA switch, and iii) performing physical experiments with optical devices and circuit simulations. Our results on university data center traces and models of Facebook and Microsoft data center traffic show that LC/DC saves on average 60% of the optical transceivers power (68% max) at the cost of 6% higher packet delay.
△ Less
Submitted 20 December, 2021; v1 submitted 3 December, 2021;
originally announced December 2021.
