-
Better Synthetic Data by Retrieving and Transforming Existing Datasets
Authors:
Saumya Gandhi,
Ritu Gala,
Vijay Viswanathan,
Tongshuang Wu,
Graham Neubig
Abstract:
Despite recent advances in large language models, building dependable and deployable NLP models typically requires abundant, high-quality training data. However, task-specific data is not available for many use cases, and manually curating task-specific data is labor-intensive. Recent work has studied prompt-driven synthetic data generation using large language models, but these generated datasets…
▽ More
Despite recent advances in large language models, building dependable and deployable NLP models typically requires abundant, high-quality training data. However, task-specific data is not available for many use cases, and manually curating task-specific data is labor-intensive. Recent work has studied prompt-driven synthetic data generation using large language models, but these generated datasets tend to lack complexity and diversity. To address these limitations, we introduce a method, DataTune, to make better use of existing, publicly available datasets to improve automatic dataset generation. DataTune performs dataset transformation, enabling the repurposing of publicly available datasets into a format that is directly aligned with the specific requirements of target tasks. On a diverse set of language-based tasks from the BIG-Bench benchmark, we find that finetuning language models via DataTune improves over a few-shot prompting baseline by 49% and improves over existing methods that use synthetic or retrieved training data by 34%. We find that dataset transformation significantly increases the diversity and difficulty of generated data on many tasks. We integrate DataTune into an open-source repository to make this method accessible to the community: https://github.com/neulab/prompt2model.
△ Less
Submitted 26 April, 2024; v1 submitted 22 April, 2024;
originally announced April 2024.
-
Extending the Forward Forward Algorithm
Authors:
Saumya Gandhi,
Ritu Gala,
Jonah Kornberg,
Advaith Sridhar
Abstract:
The Forward Forward algorithm, proposed by Geoffrey Hinton in November 2022, is a novel method for training neural networks as an alternative to backpropagation. In this project, we replicate Hinton's experiments on the MNIST dataset, and subsequently extend the scope of the method with two significant contributions. First, we establish a baseline performance for the Forward Forward network on the…
▽ More
The Forward Forward algorithm, proposed by Geoffrey Hinton in November 2022, is a novel method for training neural networks as an alternative to backpropagation. In this project, we replicate Hinton's experiments on the MNIST dataset, and subsequently extend the scope of the method with two significant contributions. First, we establish a baseline performance for the Forward Forward network on the IMDb movie reviews dataset. As far as we know, our results on this sentiment analysis task marks the first instance of the algorithm's extension beyond computer vision. Second, we introduce a novel pyramidal optimization strategy for the loss threshold - a hyperparameter specific to the Forward Forward method. Our pyramidal approach shows that a good thresholding strategy causes a difference of up to 8% in test error. Lastly, we perform visualizations of the trained parameters and derived several significant insights, such as a notably larger (10-20x) mean and variance in the weights acquired by the Forward Forward network.
Repository: https://github.com/Ads-cmu/ForwardForward
△ Less
Submitted 14 July, 2023; v1 submitted 9 July, 2023;
originally announced July 2023.
-
Quadrature-averaged homodyne detection for cavity parameter estimation
Authors:
Giada R. La Gala,
Arvind S. Kumar,
Rick Leijssen,
Ewold Verhagen,
Juha T. Muhonen
Abstract:
Balanced homodyne interferometry is a well-known detection technique that allows for sensitive characterization of light fields. Conventionally a homodyne interferometer is operated by locking the relative phase of a reference beam to the signal beam by means of an active feedback loop. A less often used method is to perform a slow continuous modulation of the reference beam arm length that corres…
▽ More
Balanced homodyne interferometry is a well-known detection technique that allows for sensitive characterization of light fields. Conventionally a homodyne interferometer is operated by locking the relative phase of a reference beam to the signal beam by means of an active feedback loop. A less often used method is to perform a slow continuous modulation of the reference beam arm length that corresponds to averaging all relative phases during the measurement.Here we show theoretically and experimentally that this quadrature averaging can be advantageous in estimating the parameters of a resonant optical cavity. We demonstrate that the averaging turns the transduction function, from cavity frequency fluctuations into the interferometer signal, into a simple function of the laser detuning that, notably, does not depend on the parameters of possible non-resonant channels present in the system. The method needs no active feedback and gives results that are easy to interpret. Moreover, the phase-averaged measurement allows to characterize the absolute magnitude of a cavity frequency modulation.
△ Less
Submitted 13 September, 2022;
originally announced September 2022.
-
LEGEND-1000 Preconceptual Design Report
Authors:
LEGEND Collaboration,
N. Abgrall,
I. Abt,
M. Agostini,
A. Alexander,
C. Andreoiu,
G. R. Araujo,
F. T. Avignone III,
W. Bae,
A. Bakalyarov,
M. Balata,
M. Bantel,
I. Barabanov,
A. S. Barabash,
P. S. Barbeau,
C. J. Barton,
P. J. Barton,
L. Baudis,
C. Bauer,
E. Bernieri,
L. Bezrukov,
K. H. Bhimani,
V. Biancacci,
E. Blalock,
A. Bolozdynya
, et al. (239 additional authors not shown)
Abstract:
We propose the construction of LEGEND-1000, the ton-scale Large Enriched Germanium Experiment for Neutrinoless $ββ$ Decay. This international experiment is designed to answer one of the highest priority questions in fundamental physics. It consists of 1000 kg of Ge detectors enriched to more than 90% in the $^{76}$Ge isotope operated in a liquid argon active shield at a deep underground laboratory…
▽ More
We propose the construction of LEGEND-1000, the ton-scale Large Enriched Germanium Experiment for Neutrinoless $ββ$ Decay. This international experiment is designed to answer one of the highest priority questions in fundamental physics. It consists of 1000 kg of Ge detectors enriched to more than 90% in the $^{76}$Ge isotope operated in a liquid argon active shield at a deep underground laboratory. By combining the lowest background levels with the best energy resolution in the field, LEGEND-1000 will perform a quasi-background-free search and can make an unambiguous discovery of neutrinoless double-beta decay with just a handful of counts at the decay $Q$ value. The experiment is designed to probe this decay with a 99.7%-CL discovery sensitivity in the $^{76}$Ge half-life of $1.3\times10^{28}$ years, corresponding to an effective Majorana mass upper limit in the range of 9-21 meV, to cover the inverted-ordering neutrino mass scale with 10 yr of live time.
△ Less
Submitted 23 July, 2021;
originally announced July 2021.
-
Real-Time Cognitive Evaluation of Online Learners through Automatically Generated Questions
Authors:
Ritu Gala,
Revathi Vijayaraghavan,
Valmik Nikam,
Arvind Kiwelekar
Abstract:
With the increased adoption of E-learning platforms, kee** online learners engaged throughout a lesson is challenging. One approach to tackle this challenge is to probe learn-ers periodically by asking questions. The paper presents an approach to generate questions from a given video lecture automatically. The generated questions are aimed to evaluate learners' lower-level cognitive abilities. T…
▽ More
With the increased adoption of E-learning platforms, kee** online learners engaged throughout a lesson is challenging. One approach to tackle this challenge is to probe learn-ers periodically by asking questions. The paper presents an approach to generate questions from a given video lecture automatically. The generated questions are aimed to evaluate learners' lower-level cognitive abilities. The approach automatically extracts text from video lectures to generates wh-kinds of questions. When learners respond with an answer, the proposed approach further evaluates the response and provides feedback. Besides enhancing learner's engagement, this approach's main benefits are that it frees instructors from design-ing questions to check the comprehension of a topic. Thus, instructors can spend this time productively on other activities.
△ Less
Submitted 6 June, 2021;
originally announced June 2021.
-
Blockchain-Based Approach to Foster Student Engagement on Campus
Authors:
Ritu Gala,
Eshita Shukla,
Nidhee Kamble,
Revathi Vijayaraghavan,
Dhiren Patel
Abstract:
On-campus activities like positions of responsibility in campus amenities and participation in research, benefit the students as well as the university, while also making students financially self-sufficient to a certain extent. However, this student participation is stymied by lack of awareness and motivation. Significant impetus to innovation and student participation can be provided by incentiv…
▽ More
On-campus activities like positions of responsibility in campus amenities and participation in research, benefit the students as well as the university, while also making students financially self-sufficient to a certain extent. However, this student participation is stymied by lack of awareness and motivation. Significant impetus to innovation and student participation can be provided by incentivization of these activities. In this paper, we propose a system to create a blockchain-based economy, to incentivize students with empirical benefits or monetary awards calculated using objective algorithms. The incentivization algorithms have been designed for three promising use cases: research work, positions of responsibility in universities, and crowdfunding. The demonstrated implementation of this system utilises VJTI Chain, an already established Proof of Authority blockchain in VJTI Mumbai, India. This creates a circular economy within the university which encourages students to earn more rewards by reinforcing positive feedback.
△ Less
Submitted 26 May, 2021;
originally announced May 2021.
-
Mixture Representation Learning with Coupled Autoencoders
Authors:
Yeganeh M. Marghi,
Rohan Gala,
Uygar Sümbül
Abstract:
Jointly identifying a mixture of discrete and continuous factors of variability without supervision is a key problem in unraveling complex phenomena. Variational inference has emerged as a promising method to learn interpretable mixture representations. However, posterior approximation in high-dimensional latent spaces, particularly for discrete factors remains challenging. Here, we propose an uns…
▽ More
Jointly identifying a mixture of discrete and continuous factors of variability without supervision is a key problem in unraveling complex phenomena. Variational inference has emerged as a promising method to learn interpretable mixture representations. However, posterior approximation in high-dimensional latent spaces, particularly for discrete factors remains challenging. Here, we propose an unsupervised variational framework using multiple interacting networks called cpl-mixVAE that scales well to high-dimensional discrete settings. In this framework, the mixture representation of each network is regularized by imposing a consensus constraint on the discrete factor. We justify the use of this framework by providing both theoretical and experimental results. Finally, we use the proposed method to jointly uncover discrete and continuous factors of variability describing gene expression in a single-cell transcriptomic dataset profiling more than a hundred cortical neuron types.
△ Less
Submitted 12 April, 2021; v1 submitted 20 July, 2020;
originally announced July 2020.
-
A coupled autoencoder approach for multi-modal analysis of cell types
Authors:
Rohan Gala,
Nathan Gouwens,
Zizhen Yao,
Agata Budzillo,
Osnat Penn,
Bosiljka Tasic,
Gabe Murphy,
Hongkui Zeng,
Uygar Sümbül
Abstract:
Recent developments in high throughput profiling of individual neurons have spurred data driven exploration of the idea that there exist natural grou**s of neurons referred to as cell types. The promise of this idea is that the immense complexity of brain circuits can be reduced, and effectively studied by means of interactions between cell types. While clustering of neuron populations based on…
▽ More
Recent developments in high throughput profiling of individual neurons have spurred data driven exploration of the idea that there exist natural grou**s of neurons referred to as cell types. The promise of this idea is that the immense complexity of brain circuits can be reduced, and effectively studied by means of interactions between cell types. While clustering of neuron populations based on a particular data modality can be used to define cell types, such definitions are often inconsistent across different characterization modalities. We pose this issue of cross-modal alignment as an optimization problem and develop an approach based on coupled training of autoencoders as a framework for such analyses. We apply this framework to a Patch-seq dataset consisting of transcriptomic and electrophysiological profiles for the same set of neurons to study consistency of representations across modalities, and evaluate cross-modal data prediction ability. We explore the problem where only a subset of neurons is characterized with more than one modality, and demonstrate that representations learned by coupled autoencoders can be used to identify types sampled only by a single modality.
△ Less
Submitted 5 November, 2019;
originally announced November 2019.
-
State preparation and tomography of a nanomechanical resonator with fast light pulses
Authors:
Juha T. Muhonen,
Giada R. La Gala,
Rick Leijssen,
Ewold Verhagen
Abstract:
Pulsed optomechanical measurements enable squeezing, non-classical state creation and backaction-free sensing. We demonstrate pulsed measurement of a cryogenic nanomechanical resonator with record precision close to the quantum regime. We use these to prepare thermally squeezed and purified conditional mechanical states, and to perform full state tomography. These demonstrations exploit large phot…
▽ More
Pulsed optomechanical measurements enable squeezing, non-classical state creation and backaction-free sensing. We demonstrate pulsed measurement of a cryogenic nanomechanical resonator with record precision close to the quantum regime. We use these to prepare thermally squeezed and purified conditional mechanical states, and to perform full state tomography. These demonstrations exploit large photon-phonon coupling in a nanophotonic cavity to reach a single-pulse imprecision of 9 times the mechanical zero-point amplitude $x_\mathrm{zpf}$. We study the effect of other mechanical modes which limit the conditional state width to 58 $x_\mathrm{zpf}$, and show how decoherence causes the state to grow in time.
△ Less
Submitted 23 December, 2018;
originally announced December 2018.