-
Exploring Domain-Specific Enhancements for a Neural Foley Synthesizer
Authors:
Ashwin Pillay,
Sage Betko,
Ari Liloia,
Hao Chen,
Ankit Shah
Abstract:
Foley sound synthesis refers to the creation of authentic, diegetic sound effects for media, such as film or radio. In this study, we construct a neural Foley synthesizer capable of generating mono-audio clips across seven predefined categories. Our approach introduces multiple enhancements to existing models in the text-to-audio domain, with the goal of enriching the diversity and acoustic charac…
▽ More
Foley sound synthesis refers to the creation of authentic, diegetic sound effects for media, such as film or radio. In this study, we construct a neural Foley synthesizer capable of generating mono-audio clips across seven predefined categories. Our approach introduces multiple enhancements to existing models in the text-to-audio domain, with the goal of enriching the diversity and acoustic characteristics of the generated foleys. Notably, we utilize a pre-trained encoder that retains acoustical and musical attributes in intermediate embeddings, implement class-conditioning to enhance differentiability among foley classes in their intermediate representations, and devise an innovative transformer-based architecture for optimizing self-attention computations on very large inputs without compromising valuable information. Subsequent to implementation, we present intermediate outcomes that surpass the baseline, discuss practical challenges encountered in achieving optimal results, and outline potential pathways for further research.
△ Less
Submitted 8 September, 2023;
originally announced September 2023.
-
Deep learning for COVID-19 topic modelling via Twitter: Alpha, Delta and Omicron
Authors:
Janhavi Lande,
Arti Pillay,
Rohitash Chandra
Abstract:
Topic modelling with innovative deep learning methods has gained interest for a wide range of applications that includes COVID-19. Topic modelling can provide, psychological, social and cultural insights for understanding human behaviour in extreme events such as the COVID-19 pandemic. In this paper, we use prominent deep learning-based language models for COVID-19 topic modelling taking into acco…
▽ More
Topic modelling with innovative deep learning methods has gained interest for a wide range of applications that includes COVID-19. Topic modelling can provide, psychological, social and cultural insights for understanding human behaviour in extreme events such as the COVID-19 pandemic. In this paper, we use prominent deep learning-based language models for COVID-19 topic modelling taking into account data from emergence (Alpha) to the Omicron variant. We apply topic modeling to review the public behaviour across the first, second and third waves based on Twitter dataset from India. Our results show that the topics extracted for the subsequent waves had certain overlap** themes such as covers governance, vaccination, and pandemic management while novel issues aroused in political, social and economic situation during COVID-19 pandemic. We also found a strong correlation of the major topics qualitatively to news media prevalent at the respective time period. Hence, our framework has the potential to capture major issues arising during different phases of the COVID-19 pandemic which can be extended to other countries and regions.
△ Less
Submitted 28 February, 2023;
originally announced March 2023.
-
Hybrid Genetic Optimisation for Quantum Feature Map Design
Authors:
Rowan Pellow-Jarman,
Anban Pillay,
Ilya Sinayskiy,
Francesco Petruccione
Abstract:
Kernel methods are an important class of techniques in machine learning. To be effective, good feature maps are crucial for map** non-linearly separable input data into a higher dimensional (feature) space, thus allowing the data to be linearly separable in feature space. Previous work has shown that quantum feature map design can be automated for a given dataset using NSGA-II, a genetic algorit…
▽ More
Kernel methods are an important class of techniques in machine learning. To be effective, good feature maps are crucial for map** non-linearly separable input data into a higher dimensional (feature) space, thus allowing the data to be linearly separable in feature space. Previous work has shown that quantum feature map design can be automated for a given dataset using NSGA-II, a genetic algorithm, while both minimizing circuit size and maximizing classification accuracy. However, the evaluation of the accuracy achieved by a candidate feature map is costly. In this work, we demonstrate the suitability of kernel-target alignment as a substitute for accuracy in genetic algorithm-based quantum feature map design. Kernel-target alignment is faster to evaluate than accuracy and doesn't require some data points to be reserved for its evaluation. To further accelerate the evaluation of genetic fitness, we provide a method to approximate kernel-target alignment. To improve kernel-target alignment and root mean squared error, the final trainable parameters of the generated circuits are further trained using COBYLA to determine whether a hybrid approach applying conventional circuit parameter training can easily complement the genetic structure optimization approach. A total of eight new approaches are compared to the original across nine varied binary classification problems from the UCI machine learning repository, showing that kernel-target alignment and its approximation produce feature map circuits enabling comparable accuracy to the previous work but with larger margins on training data (in excess of 20\% larger) that improve further with circuit parameter training.
△ Less
Submitted 6 February, 2023;
originally announced February 2023.
-
Tighter Bounds on the Expressivity of Transformer Encoders
Authors:
David Chiang,
Peter Cholak,
Anand Pillay
Abstract:
Characterizing neural networks in terms of better-understood formal systems has the potential to yield new insights into the power and limitations of these networks. Doing so for transformers remains an active area of research. Bhattamishra and others have shown that transformer encoders are at least as expressive as a certain kind of counter machine, while Merrill and Sabharwal have shown that fi…
▽ More
Characterizing neural networks in terms of better-understood formal systems has the potential to yield new insights into the power and limitations of these networks. Doing so for transformers remains an active area of research. Bhattamishra and others have shown that transformer encoders are at least as expressive as a certain kind of counter machine, while Merrill and Sabharwal have shown that fixed-precision transformer encoders recognize only languages in uniform $TC^0$. We connect and strengthen these results by identifying a variant of first-order logic with counting quantifiers that is simultaneously an upper bound for fixed-precision transformer encoders and a lower bound for transformer encoders. This brings us much closer than before to an exact characterization of the languages that transformer encoders recognize.
△ Less
Submitted 13 November, 2023; v1 submitted 25 January, 2023;
originally announced January 2023.
-
An Audio Synthesis Framework Derived from Industrial Process Control
Authors:
Ashwin Pillay
Abstract:
Since its conception, digital synthesis has significantly influenced the advancement of music, leading to new genres and production styles. Through existing synthesis techniques, one can recreate naturally occurring sounds as well as generate innovative artificial timbres. However, research in audio technology continues to pursue new methods of synthesizing sounds, kee** the transformation of mu…
▽ More
Since its conception, digital synthesis has significantly influenced the advancement of music, leading to new genres and production styles. Through existing synthesis techniques, one can recreate naturally occurring sounds as well as generate innovative artificial timbres. However, research in audio technology continues to pursue new methods of synthesizing sounds, kee** the transformation of music constant. This research attempts to formulate the framework of a new synthesis technique by redefining the popular Proportional-Integral-Derivative (PID) algorithm used in feedback-based process control. The framework is then implemented as a Python application to study the available control parameters and their effect on the synthesized output. Further, applications of this technique as an audio signal and LFO generator, including its potentiality as an alternative to FM and Wavetable synthesis techniques, are studied in detail. The research concludes by highlighting some of the imperfections in the current framework and the possible research directions to be considered to address them.
△ Less
Submitted 21 September, 2021;
originally announced September 2021.
-
An Audio Envelope Generator Derived from Industrial Process Control
Authors:
Ashwin Pillay
Abstract:
Audio envelopes serve a crucial role in ensuring the versatility of synthesizers in producing timbres. To this end, the Attack, Decay, Release and Sustain (ADSR) envelope generator and its derivatives have been established as a mainstay in modern music. However, there may be merit in exploring alternate techniques to produce envelopes that could not only resemble ADSR but also be used to create no…
▽ More
Audio envelopes serve a crucial role in ensuring the versatility of synthesizers in producing timbres. To this end, the Attack, Decay, Release and Sustain (ADSR) envelope generator and its derivatives have been established as a mainstay in modern music. However, there may be merit in exploring alternate techniques to produce envelopes that could not only resemble ADSR but also be used to create novel timbres. Consequently, an attempt is made in this research to formulate the framework of a new envelope generator by redefining the Proportional-Integral-Derivative (PID) algorithm used in feedback-based process control. Additionally, a detailed analysis is made on the modes of operation and the nature of envelopes thus generated to establish it as a potential harbinger of distinctive styles of music.
△ Less
Submitted 26 June, 2021;
originally announced June 2021.
-
Multi-experiment parameter identifiability of ODEs and model theory
Authors:
Alexey Ovchinnikov,
Anand Pillay,
Gleb Pogudin,
Thomas Scanlon
Abstract:
Structural identifiability is a property of an ODE model with parameters that allows for the parameters to be determined from continuous noise-free data. This is a natural prerequisite for practical identifiability. Conducting multiple independent experiments could make more parameters or functions of parameters identifiable, which is a desirable property to have. How many experiments are sufficie…
▽ More
Structural identifiability is a property of an ODE model with parameters that allows for the parameters to be determined from continuous noise-free data. This is a natural prerequisite for practical identifiability. Conducting multiple independent experiments could make more parameters or functions of parameters identifiable, which is a desirable property to have. How many experiments are sufficient? In the present paper, we provide an algorithm to determine the exact number of experiments for multi-experiment local identifiability and obtain an upper bound that is off at most by one for the number of experiments for multi-experiment global identifiability.
Interestingly, the main theoretical ingredient of the algorithm has been discovered and proved using model theory (in the sense of mathematical logic). We hope that this unexpected connection will stimulate interactions between applied algebra and model theory, and we provide a short introduction to model theory in the context of parameter identifiability. As another related application of model theory in this area, we construct a nonlinear ODE system with one output such that single-experiment and multiple-experiment identifiability are different for the system. This contrasts with recent results about single-output linear systems.
We also present a Monte Carlo randomized version of the algorithm with a polynomial arithmetic complexity. Implementation of the algorithm is provided and its performance is demonstrated on several examples. The source code is available at https://github.com/pogudingleb/ExperimentsBound.
△ Less
Submitted 17 August, 2021; v1 submitted 21 November, 2020;
originally announced November 2020.
-
Computing all identifiable functions of parameters for ODE models
Authors:
Alexey Ovchinnikov,
Anand Pillay,
Gleb Pogudin,
Thomas Scanlon
Abstract:
Parameter identifiability is a structural property of an ODE model for recovering the values of parameters from the data (i.e., from the input and output variables). This property is a prerequisite for meaningful parameter identification in practice. In the presence of nonidentifiability, it is important to find all functions of the parameters that are identifiable. The existing algorithms check w…
▽ More
Parameter identifiability is a structural property of an ODE model for recovering the values of parameters from the data (i.e., from the input and output variables). This property is a prerequisite for meaningful parameter identification in practice. In the presence of nonidentifiability, it is important to find all functions of the parameters that are identifiable. The existing algorithms check whether a given function of parameters is identifiable or, under the solvability condition, find all identifiable functions. However, this solvability condition is not always satisfied, which presents a challenge. Our first main result is an algorithm that computes all identifiable functions without any additional assumptions, which is the first such algorithm as far as we know. Our second main result concerns the identifiability from multiple experiments (with generically different inputs and initial conditions among the experiments). For this problem, we prove that the set of functions identifiable from multiple experiments is what would actually be computed by input-output equation-based algorithms (whether or not the solvability condition is fulfilled), which was not known before. We give an algorithm that not only finds these functions but also provides an upper bound for the number of experiments to be performed to identify these functions. We provide an implementation of the presented algorithms.
△ Less
Submitted 3 June, 2021; v1 submitted 16 April, 2020;
originally announced April 2020.
-
The prospects for mathematical logic in the twenty-first century
Authors:
Samuel R. Buss,
Alexander S. Kechris,
Anand Pillay,
Richard A. Shore
Abstract:
The four authors present their speculations about the future developments of mathematical logic in the twenty-first century. The areas of recursion theory, proof theory and logic for computer science, model theory, and set theory are discussed independently.
The four authors present their speculations about the future developments of mathematical logic in the twenty-first century. The areas of recursion theory, proof theory and logic for computer science, model theory, and set theory are discussed independently.
△ Less
Submitted 3 May, 2002;
originally announced May 2002.
