-
Supporting Better Insights of Data Science Pipelines with Fine-grained Provenance
Authors:
Adriane Chapman,
Luca Lauro,
Paolo Missier,
Riccardo Torlone
Abstract:
Successful data-driven science requires complex data engineering pipelines to clean, transform, and alter data in preparation for machine learning, and robust results can only be achieved when each step in the pipeline can be justified, and its effect on the data explained. In this framework, our aim is to provide data scientists with facilities to gain an in-depth understanding of how each step i…
▽ More
Successful data-driven science requires complex data engineering pipelines to clean, transform, and alter data in preparation for machine learning, and robust results can only be achieved when each step in the pipeline can be justified, and its effect on the data explained. In this framework, our aim is to provide data scientists with facilities to gain an in-depth understanding of how each step in the pipeline affects the data, from the raw input to training sets ready to be used for learning. Starting from an extensible set of data preparation operators commonly used within a data science setting, in this work we present a provenance management infrastructure for generating, storing, and querying very granular accounts of data transformations, at the level of individual elements within datasets whenever possible. Then, from the formal definition of a core set of data science preprocessing operators, we derive a provenance semantics embodied by a collection of templates expressed in PROV, a standard model for data provenance. Using those templates as a reference, our provenance generation algorithm generalises to any operator with observable input/output pairs. We provide a prototype implementation of an application-level provenance capture library to produce, in a semi-automatic way, complete provenance documents that account for the entire pipeline. We report on the ability of our implementations to capture provenance in real ML benchmark pipelines and over TCP-DI synthetic data. We finally show how the collected provenance can be used to answer a suite of provenance benchmark queries that underpin some common pipeline inspection questions, as expressed on the Data Science Stack Exchange.
△ Less
Submitted 27 October, 2023;
originally announced October 2023.
-
Deep learning empowered synthetic dimension dynamics: morphing of light into topological modes
Authors:
Shiqi Xia,
Sihong Lei,
Daohong Song,
Luigi Di Lauro,
Imtiaz Alamgir,
Liqin Tang,
**gjun Xu,
Roberto Morandotti,
Hrvoje Buljan,
Zhigang Chen
Abstract:
Synthetic dimensions (SDs) opened the door for exploring previously inaccessible phenomena in high-dimensional synthetic space. However, construction of synthetic lattices with desired coupling properties is a challenging and unintuitive task, largely limiting the exploration and current application of SD dynamics. Here, we overcome this challenge by using deep learning artificial neural networks…
▽ More
Synthetic dimensions (SDs) opened the door for exploring previously inaccessible phenomena in high-dimensional synthetic space. However, construction of synthetic lattices with desired coupling properties is a challenging and unintuitive task, largely limiting the exploration and current application of SD dynamics. Here, we overcome this challenge by using deep learning artificial neural networks (ANNs) to validly design the dynamics in SDs. We use ANNs to construct a lattice in real space that has a predesigned spectrum of mode eigenvalues. By employing judiciously chosen perturbations (wiggling of waveguides), we show experimentally and theoretically resonant mode coupling and tailored dynamics in SDs, which leads to effective transport or confinement of a complex beam profile. As an enlightening example, we demonstrate morphing of light into a topologically protected edge mode in ANN-designed Su-Schrieffer-Heeger photonic lattices. Such ANN-assisted construction of SDs advances towards utopian networks, opening new avenues in fundamental research beyond geometric limitations. Our findings may offer a flexible and efficient solution for mode lasing, optical switching, and communication technologies.
△ Less
Submitted 28 April, 2023;
originally announced April 2023.
-
Thermo-optical pulsing in a microresonator filtered fiber-laser: a route towards all-optical control and synchronization
Authors:
Maxwell Rowley,
Benjamin Wetzel,
Luigi Di Lauro,
Juan S. Totero Gongora,
Hualong Bao,
Jonathan Silver,
Leonardo Del Bino,
Pascal Del' Haye,
Marco Peccianti,
Alessia Pasquazi
Abstract:
We report on 'slow' pulsing dynamics in a silica resonator-based laser system: by nesting a high-Q rod-resonator inside an amplifying fiber cavity, we demonstrate that trains of microsecond pulses can be generated with repetition rates in the hundreds of kilohertz. We show that such pulses are produced with a period equivalent to several hundreds of laser cavity roundtrips via the interaction betw…
▽ More
We report on 'slow' pulsing dynamics in a silica resonator-based laser system: by nesting a high-Q rod-resonator inside an amplifying fiber cavity, we demonstrate that trains of microsecond pulses can be generated with repetition rates in the hundreds of kilohertz. We show that such pulses are produced with a period equivalent to several hundreds of laser cavity roundtrips via the interaction between the gain dynamics in the fiber cavity and the thermo-optical effects in the high-Q resonator. Experiments reveal that the pulsing properties can be controlled by adjusting the amplifying fiber cavity parameters. Our results, confirmed by numerical simulations, provide useful insights on the dynamical onset of complex self-organization phenomena in resonator-based laser systems where thermo-optical effects play an active role. In addition, we show how the thermal state of the resonator can be probed and even modified by an external, counter-propagating optical field, thus hinting towards novel approaches for all-optical control and sensing applications.
△ Less
Submitted 21 March, 2019;
originally announced March 2019.
-
Laser Cavity-Soliton Micro-Combs
Authors:
Hualong Bao,
Andrew Cooper,
Maxwell Rowley,
Luigi Di Lauro,
Juan Sebastian Totero Gongora,
Sai T. Chu,
Brent E. Little,
Gian-Luca Oppo,
Roberto Morandotti,
David J. Moss,
Benjamin Wetzel,
Marco Peccianti,
Alessia Pasquazi
Abstract:
The field of micro-cavity based frequency combs, or 'micro-combs'[1,2], has recently witnessed many fundamental breakthroughs[3-19] enabled by the discovery of temporal cavity-solitons, self-localised waves sustained by a background of radiation usually containing 95% of the total power[20]. Simple methods for their efficient generation and control are currently researched to finally establish mic…
▽ More
The field of micro-cavity based frequency combs, or 'micro-combs'[1,2], has recently witnessed many fundamental breakthroughs[3-19] enabled by the discovery of temporal cavity-solitons, self-localised waves sustained by a background of radiation usually containing 95% of the total power[20]. Simple methods for their efficient generation and control are currently researched to finally establish micro-combs as out-of-the-lab widespread tools[21]. Here we demonstrate micro-comb laser cavity-solitons, an intrinsically highly-efficient, background free class of solitary waves. Laser cavity-solitons have underpinned key breakthroughs in semiconductor lasers[22,23] and photonic memories[24-26]. By merging their properties with the physics of both micro-resonators[1,2] and multi-mode systems[27], we provide a new paradigm for the generation and control of self-localised pulses in micro-cavities. We demonstrate 50 nm wide soliton combs induced with average powers one order of magnitude lower than those typically required by state-of-the-art approaches[26]. Furthermore, we can tune the repetition-rate to well over a megahertz with no-active feedback.
△ Less
Submitted 28 February, 2019; v1 submitted 26 February, 2019;
originally announced February 2019.
