-
VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation
Authors:
Changhan Wang,
Morgane Rivière,
Ann Lee,
Anne Wu,
Chaitanya Talnikar,
Daniel Haziza,
Mary Williamson,
Juan Pino,
Emmanuel Dupoux
Abstract:
We introduce VoxPopuli, a large-scale multilingual corpus providing 100K hours of unlabelled speech data in 23 languages. It is the largest open data to date for unsupervised representation learning as well as semi-supervised learning. VoxPopuli also contains 1.8K hours of transcribed speeches in 16 languages and their aligned oral interpretations into 5 other languages totaling 5.1K hours. We pro…
▽ More
We introduce VoxPopuli, a large-scale multilingual corpus providing 100K hours of unlabelled speech data in 23 languages. It is the largest open data to date for unsupervised representation learning as well as semi-supervised learning. VoxPopuli also contains 1.8K hours of transcribed speeches in 16 languages and their aligned oral interpretations into 5 other languages totaling 5.1K hours. We provide speech recognition baselines and validate the versatility of VoxPopuli unlabelled data in semi-supervised learning under challenging out-of-domain settings. We will release the corpus at https://github.com/facebookresearch/voxpopuli under an open license.
△ Less
Submitted 27 July, 2021; v1 submitted 2 January, 2021;
originally announced January 2021.
-
Adjoint-based trailing edge shape optimization of a transonic turbine vane using large eddy simulations
Authors:
Chaitanya Talnikar,
Qiqi Wang
Abstract:
The shape of the trailing edge of a gas turbine nozzle guide vane has a significant effect on the downstream stagnation pressure loss and heat transfer over the surface of the vane. Traditionally, adjoint-based design optimization methods for turbomachinery components have used low-fidelity simulations like Reynolds averaged Navier-Stokes. To reliably capture the complex flow phenomena involved in…
▽ More
The shape of the trailing edge of a gas turbine nozzle guide vane has a significant effect on the downstream stagnation pressure loss and heat transfer over the surface of the vane. Traditionally, adjoint-based design optimization methods for turbomachinery components have used low-fidelity simulations like Reynolds averaged Navier-Stokes. To reliably capture the complex flow phenomena involved in turbulent flow over a turbine vane, high-fidelity simulations like large eddy simulation (LES) are required. In this paper, an adjoint-based trailing edge shape optimization using LES is performed to reduce pressure loss and heat transfer over the surface of the vane. The chaotic dynamics of turbulence limits the effectiveness of the adjoint method for long-time averaged objective functions computed from LES. A viscosity stabilized unsteady adjoint method is used to obtain gradients of the design objective function with reasonable accuracy. A gradient utilizing Bayesian optimization is used to robustly handle noise in the objective function and gradient evaluations. The trailing edge shape is parameterized using a linear combination of $5$ convex designs. Results from the optimization, performed on the supercomputer Mira, are compared with optimal designs generated using derivative-free design optimization of the same problem.
△ Less
Submitted 25 November, 2020; v1 submitted 12 November, 2020;
originally announced November 2020.
-
Joint Masked CPC and CTC Training for ASR
Authors:
Chaitanya Talnikar,
Tatiana Likhomanenko,
Ronan Collobert,
Gabriel Synnaeve
Abstract:
Self-supervised learning (SSL) has shown promise in learning representations of audio that are useful for automatic speech recognition (ASR). But, training SSL models like wav2vec~2.0 requires a two-stage pipeline. In this paper we demonstrate a single-stage training of ASR models that can utilize both unlabeled and labeled data. During training, we alternately minimize two losses: an unsupervised…
▽ More
Self-supervised learning (SSL) has shown promise in learning representations of audio that are useful for automatic speech recognition (ASR). But, training SSL models like wav2vec~2.0 requires a two-stage pipeline. In this paper we demonstrate a single-stage training of ASR models that can utilize both unlabeled and labeled data. During training, we alternately minimize two losses: an unsupervised masked Contrastive Predictive Coding (CPC) loss and the supervised audio-to-text alignment loss Connectionist Temporal Classification (CTC). We show that this joint training method directly optimizes performance for the downstream ASR task using unsupervised data while achieving similar word error rates to wav2vec~2.0 on the Librispeech 100-hour dataset. Finally, we postulate that solving the contrastive task is a regularization for the supervised CTC loss.
△ Less
Submitted 13 February, 2021; v1 submitted 30 October, 2020;
originally announced November 2020.
-
Linear Range in Gradient Descent
Authors:
Angxiu Ni,
Chaitanya Talnikar
Abstract:
This paper defines linear range as the range of parameter perturbations which lead to approximately linear perturbations in the states of a network. We compute linear range from the difference between actual perturbations in states and the tangent solution. Linear range is a new criterion for estimating the effectivenss of gradients and thus having many possible applications. In particular, we pro…
▽ More
This paper defines linear range as the range of parameter perturbations which lead to approximately linear perturbations in the states of a network. We compute linear range from the difference between actual perturbations in states and the tangent solution. Linear range is a new criterion for estimating the effectivenss of gradients and thus having many possible applications. In particular, we propose that the optimal learning rate at the initial stages of training is such that parameter changes on all minibatches are within linear range. We demonstrate our algorithm on two shallow neural networks and a ResNet.
△ Less
Submitted 23 May, 2019; v1 submitted 11 May, 2019;
originally announced May 2019.
-
Feasibility analysis of ensemble sensitivity computation in turbulent flows
Authors:
Nisha Chandramoorthy,
Pablo Fernandez,
Chaitanya Talnikar,
Qiqi Wang
Abstract:
In chaotic systems, such as turbulent flows, the solutions to tangent and adjoint equations exhibit an unbounded growth in their norms. This behavior renders the instantaneous tangent and adjoint solutions unusable for sensitivity analysis. The Lea-Allen-Haine ensemble sensitivity (ES) estimates provide a way of computing meaningful sensitivities in chaotic systems by utilizing tangent/adjoint sol…
▽ More
In chaotic systems, such as turbulent flows, the solutions to tangent and adjoint equations exhibit an unbounded growth in their norms. This behavior renders the instantaneous tangent and adjoint solutions unusable for sensitivity analysis. The Lea-Allen-Haine ensemble sensitivity (ES) estimates provide a way of computing meaningful sensitivities in chaotic systems by utilizing tangent/adjoint solutions over short trajectories. In this paper, we analyze the feasibility of ES computations under optimistic mathematical assumptions on the flow dynamics. Furthermore, we estimate upper bounds on the rate of convergence of the ES method in numerical simulations of turbulent flow. Even at the optimistic upper bound, the ES method is computationally intractable in each of the numerical examples considered.
△ Less
Submitted 13 July, 2019; v1 submitted 19 November, 2018;
originally announced November 2018.
-
Adjoint sensitivity analysis on chaotic dynamical systems by Non-Intrusive Least Squares Adjoint Shadowing (NILSAS)
Authors:
Angxiu Ni,
Chaitanya Talnikar
Abstract:
We develop the NILSAS algorithm, which performs adjoint sensitivity analysis of chaotic systems via computing the adjoint shadowing direction. NILSAS constrains its minimization to the adjoint unstable subspace, and can be implemented with little modification to existing adjoint solvers. The computational cost of NILSAS is independent of the number of parameters. We demonstrate NILSAS on the Loren…
▽ More
We develop the NILSAS algorithm, which performs adjoint sensitivity analysis of chaotic systems via computing the adjoint shadowing direction. NILSAS constrains its minimization to the adjoint unstable subspace, and can be implemented with little modification to existing adjoint solvers. The computational cost of NILSAS is independent of the number of parameters. We demonstrate NILSAS on the Lorenz 63 system and a weakly turbulent three-dimensional flow over a cylinder.
△ Less
Submitted 8 January, 2019; v1 submitted 25 January, 2018;
originally announced January 2018.
-
Sensitivity analysis on chaotic dynamical systems by Finite Difference Non-Intrusive Least Squares Shadowing (FD-NILSS)
Authors:
Angxiu Ni,
Qiqi Wang,
Pablo Fernandez,
Chaitanya Talnikar
Abstract:
We present the Finite Difference Non-Intrusive Least Squares Shadowing (FD-NILSS) algorithm for computing sensitivities of long-time averaged quantities in chaotic dynamical systems. FD-NILSS does not require tangent solvers, and can be implemented with little modification to existing numerical simulation software. We also give a formula for solving the least-squares problem in FD-NILSS, which can…
▽ More
We present the Finite Difference Non-Intrusive Least Squares Shadowing (FD-NILSS) algorithm for computing sensitivities of long-time averaged quantities in chaotic dynamical systems. FD-NILSS does not require tangent solvers, and can be implemented with little modification to existing numerical simulation software. We also give a formula for solving the least-squares problem in FD-NILSS, which can be applied in NILSS as well. Finally, we apply FD-NILSS for sensitivity analysis of a chaotic flow over a 3-D cylinder at Reynolds number 525, where FD-NILSS computes accurate sensitivities and the computational cost is in the same order as the numerical simulation.
△ Less
Submitted 23 June, 2019; v1 submitted 17 November, 2017;
originally announced November 2017.
-
Unsteady adjoint of pressure loss for a fundamental transonic turbine vane
Authors:
Chaitanya Talnikar,
Qiqi Wang,
Gregory M. Laskowski
Abstract:
High fidelity simulations, e.g., large eddy simulation are often needed for accurately predicting pressure losses due to wake mixing in turbomachinery applications. An unsteady adjoint of such high fidelity simulations is useful for design optimization in these aerodynamic applications. In this paper we present unsteady adjoint solutions using a large eddy simulation model for a vane from VKI usin…
▽ More
High fidelity simulations, e.g., large eddy simulation are often needed for accurately predicting pressure losses due to wake mixing in turbomachinery applications. An unsteady adjoint of such high fidelity simulations is useful for design optimization in these aerodynamic applications. In this paper we present unsteady adjoint solutions using a large eddy simulation model for a vane from VKI using aerothermal objectives. The unsteady adjoint method is effective in capturing the gradient for a short time interval aerothermal objective, whereas the method provides diverging gradients for long time-averaged thermal objectives. As the boundary layer on the suction side near the trailing edge of the vane is turbulent, it poses a challenge for the adjoint solver. The chaotic dynamics cause the adjoint solution to diverge exponentially from the trailing edge region when solved backwards in time. This results in the corruption of the sensitivities obtained from the adjoint solutions. An energy analysis of the unsteady compressible Navier-Stokes adjoint equations indicates that adding artificial viscosity to the adjoint equations can potentially dissipate the adjoint energy while potentially maintain the accuracy of the adjoint sensitivities. Analyzing the growth term of the adjoint energy provides a metric for identifying the regions in the flow where the adjoint term is diverging. Results for the vane from simulations performed on the Titan supercomputer are demonstrated.
△ Less
Submitted 21 November, 2015;
originally announced November 2015.
-
Parallel optimization for large eddy simulations
Authors:
Chaitanya Talnikar,
Patrick Blonigan,
Julien Bodart,
Qiqi Wang
Abstract:
We developed a parallel Bayesian optimization algorithm for large eddy simulations. These simulations challenge optimization methods because they take hours or days to compute, and their objective function contains noise as turbulent statistics that are averaged over a finite time. Surrogate based optimization methods, including Bayesian optimization, have shown promise for noisy and expensive obj…
▽ More
We developed a parallel Bayesian optimization algorithm for large eddy simulations. These simulations challenge optimization methods because they take hours or days to compute, and their objective function contains noise as turbulent statistics that are averaged over a finite time. Surrogate based optimization methods, including Bayesian optimization, have shown promise for noisy and expensive objective functions. Here we adapt Bayesian optimization to minimize drag in a turbulent channel flow and to design the trailing edge of a turbine blade to reduce turbulent heat transfer and pressure loss. Our optimization simultaneously runs several simulations, each parallelized to thousands of cores, in order to utilize additional concurrency offered by today's supercomputers.
△ Less
Submitted 3 November, 2014; v1 submitted 31 October, 2014;
originally announced October 2014.
