Showing 1–50 of 123 results for author: Cooper, E

An Initial Investigation of Language Adaptation for TTS Systems under Low-resource Scenarios

Authors: Cheng Gong, Erica Cooper, Xin Wang, Chunyu Qiang, Mengzhe Geng, Dan Wells, Longbiao Wang, Jianwu Dang, Marc Tessier, Aidan Pine, Korin Richmond, Junichi Yamagishi

Abstract: Self-supervised learning (SSL) representations from massively multilingual models offer a promising solution for low-resource language speech tasks. Despite advancements, language adaptation in TTS systems remains an open problem. This paper explores the language adaptation capability of ZMM-TTS, a recent SSL-based multilingual TTS system proposed in our previous work. We conducted experiments on… ▽ More Self-supervised learning (SSL) representations from massively multilingual models offer a promising solution for low-resource language speech tasks. Despite advancements, language adaptation in TTS systems remains an open problem. This paper explores the language adaptation capability of ZMM-TTS, a recent SSL-based multilingual TTS system proposed in our previous work. We conducted experiments on 12 languages using limited data with various fine-tuning configurations. We demonstrate that the similarity in phonetics between the pre-training and target languages, as well as the language category, affects the target language's adaptation performance. Additionally, we find that the fine-tuning dataset size and number of speakers influence adaptability. Surprisingly, we also observed that using paired data for fine-tuning is not always optimal compared to audio-only data. Beyond speech intelligibility, our analysis covers speaker similarity, language identification, and predicted MOS. △ Less

Submitted 13 June, 2024; originally announced June 2024.

Comments: Accepted to Interspeech 2024

Generating Speakers by Prompting Listener Impressions for Pre-trained Multi-Speaker Text-to-Speech Systems

Authors: Zhengyang Chen, Xuechen Liu, Erica Cooper, Junichi Yamagishi, Yanmin Qian

Abstract: This paper proposes a speech synthesis system that allows users to specify and control the acoustic characteristics of a speaker by means of prompts describing the speaker's traits of synthesized speech. Unlike previous approaches, our method utilizes listener impressions to construct prompts, which are easier to collect and align more naturally with everyday descriptions of speaker traits. We ado… ▽ More This paper proposes a speech synthesis system that allows users to specify and control the acoustic characteristics of a speaker by means of prompts describing the speaker's traits of synthesized speech. Unlike previous approaches, our method utilizes listener impressions to construct prompts, which are easier to collect and align more naturally with everyday descriptions of speaker traits. We adopt the Low-rank Adaptation (LoRA) technique to swiftly tailor a pre-trained language model to our needs, facilitating the extraction of speaker-related traits from the prompt text. Besides, different from other prompt-driven text-to-speech (TTS) systems, we separate the prompt-to-speaker module from the multi-speaker TTS system, enhancing system flexibility and compatibility with various pre-trained multi-speaker TTS systems. Moreover, for the prompt-to-speaker characteristic module, we also compared the discriminative method and flow-matching based generative method and we found that combining both methods can help the system simultaneously capture speaker-related information from prompts better and generate speech with higher fidelity. △ Less

Submitted 13 June, 2024; originally announced June 2024.

Comments: Accepted for presentation at Interspeech 2024 (with more analysis in the final Appendix part)

Spoof Diarization: "What Spoofed When" in Partially Spoofed Audio

Authors: Lin Zhang, Xin Wang, Erica Cooper, Mireia Diez, Federico Landini, Nicholas Evans, Junichi Yamagishi

Abstract: This paper defines Spoof Diarization as a novel task in the Partial Spoof (PS) scenario. It aims to determine what spoofed when, which includes not only locating spoof regions but also clustering them according to different spoofing methods. As a pioneering study in spoof diarization, we focus on defining the task, establishing evaluation metrics, and proposing a benchmark model, namely the Counte… ▽ More This paper defines Spoof Diarization as a novel task in the Partial Spoof (PS) scenario. It aims to determine what spoofed when, which includes not only locating spoof regions but also clustering them according to different spoofing methods. As a pioneering study in spoof diarization, we focus on defining the task, establishing evaluation metrics, and proposing a benchmark model, namely the Countermeasure-Condition Clustering (3C) model. Utilizing this model, we first explore how to effectively train countermeasures to support spoof diarization using three labeling schemes. We then utilize spoof localization predictions to enhance the diarization performance. This first study reveals the high complexity of the task, even in restricted scenarios where only a single speaker per audio file and an oracle number of spoofing methods are considered. Our code is available at https://github.com/nii-yamagishilab/PartialSpoof. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Comments: Accepted to Interspeech 2024

Absorption Troughs of Lyman Alpha Emitters in HETDEX

Authors: Laurel H. Weiss, Dustin Davis, Karl Gebhardt, Simon Gazagnes, Mahan Mirza Khanlari, Erin Mentuch Cooper, John Chisholm, Danielle Berg, William P. Bowman, Chris Byrohl, Robin Ciardullo, Maximilian Fabricius, Daniel Farrow, Caryl Gronwall, Gary J. Hill, Lindsay R. House, Donghui Jeong, Hasti Khoraminezhad, Wolfram Kollatschny, Eiichiro Komatsu, Maja Lujan Niemeyer, Shun Saito, Donald P. Schneider, Gregory R. Zeimann

Abstract: The Hobby-Eberly Telescope Dark Energy Experiment (HETDEX) is designed to detect and measure the redshifts of more than one million Ly$α$ emitting galaxies (LAEs) between $1.88 < z < 3.52$. In addition to its cosmological measurements, these data enable studies of Ly$α$ spectral profiles and the underlying radiative transfer. Using the roughly half a million LAEs in the HETDEX Data Release 3, we s… ▽ More The Hobby-Eberly Telescope Dark Energy Experiment (HETDEX) is designed to detect and measure the redshifts of more than one million Ly$α$ emitting galaxies (LAEs) between $1.88 < z < 3.52$. In addition to its cosmological measurements, these data enable studies of Ly$α$ spectral profiles and the underlying radiative transfer. Using the roughly half a million LAEs in the HETDEX Data Release 3, we stack various subsets to obtain the typical Ly$α$ profile for the $z \sim 2-3$ epoch and to understand their physical properties. We find clear absorption wings around Ly$α$ emission, which extend $\sim 2000$ km $\mathrm{s}^{-1}$ both redward and blueward of the central line. Using far-UV spectra of nearby ($0.002 < z < 0.182$) LAEs in the CLASSY treasury and optical/near-IR spectra of $2.8 < z < 6.7$ LAEs in the MUSE-Wide survey, we observe absorption profiles in both redshift regimes. Dividing the sample by volume density shows that the troughs increase in higher density regions. This trend suggests that the depth of the absorption is dependent on the local density of objects near the LAE, a geometry that is similar to damped Lyman-$α$ systems. Simple simulations of Ly$α$ radiative transfer can produce similar troughs due to absorption of light from background sources by HI gas surrounding the LAEs. △ Less

Submitted 4 January, 2024; originally announced January 2024.

Comments: 15 pages, 10 figures, accepted for publication in The Astrophysical Journal

Uncertainty as a Predictor: Leveraging Self-Supervised Learning for Zero-Shot MOS Prediction

Authors: Aditya Ravuri, Erica Cooper, Junichi Yamagishi

Abstract: Predicting audio quality in voice synthesis and conversion systems is a critical yet challenging task, especially when traditional methods like Mean Opinion Scores (MOS) are cumbersome to collect at scale. This paper addresses the gap in efficient audio quality prediction, especially in low-resource settings where extensive MOS data from large-scale listening tests may be unavailable. We demonstra… ▽ More Predicting audio quality in voice synthesis and conversion systems is a critical yet challenging task, especially when traditional methods like Mean Opinion Scores (MOS) are cumbersome to collect at scale. This paper addresses the gap in efficient audio quality prediction, especially in low-resource settings where extensive MOS data from large-scale listening tests may be unavailable. We demonstrate that uncertainty measures derived from out-of-the-box pretrained self-supervised learning (SSL) models, such as wav2vec, correlate with MOS scores. These findings are based on data from the 2022 and 2023 VoiceMOS challenges. We explore the extent of this correlation across different models and language contexts, revealing insights into how inherent uncertainties in SSL models can serve as effective proxies for audio quality assessment. In particular, we show that the contrastive wav2vec models are the most performant in all settings. △ Less

Submitted 25 December, 2023; originally announced December 2023.

Comments: 5 pages, 3 figures, sasb draft

ZMM-TTS: Zero-shot Multilingual and Multispeaker Speech Synthesis Conditioned on Self-supervised Discrete Speech Representations

Authors: Cheng Gong, Xin Wang, Erica Cooper, Dan Wells, Longbiao Wang, Jianwu Dang, Korin Richmond, Junichi Yamagishi

Abstract: Neural text-to-speech (TTS) has achieved human-like synthetic speech for single-speaker, single-language synthesis. Multilingual TTS systems are limited to resource-rich languages due to the lack of large paired text and studio-quality audio data. In most cases, TTS systems are built using a single speaker's voice. However, there is growing interest in develo** systems that can synthesize voices… ▽ More Neural text-to-speech (TTS) has achieved human-like synthetic speech for single-speaker, single-language synthesis. Multilingual TTS systems are limited to resource-rich languages due to the lack of large paired text and studio-quality audio data. In most cases, TTS systems are built using a single speaker's voice. However, there is growing interest in develo** systems that can synthesize voices for new speakers using only a few seconds of their speech. This paper presents ZMM-TTS, a multilingual and multispeaker framework utilizing quantized latent speech representations from a large-scale, pre-trained, self-supervised model. Our paper is the first to incorporate the representations from text-based and speech-based self-supervised learning models into multilingual speech synthesis tasks. We conducted comprehensive subjective and objective evaluations through a series of experiments. Our model has been proven effective in terms of speech naturalness and similarity for both seen and unseen speakers in six high-resource languages. We also tested the efficiency of our method on two hypothetical low-resource languages. The results are promising, indicating that our proposed approach can synthesize audio that is intelligible and has a high degree of similarity to the target speaker's voice, even without any training data for the new, unseen language. △ Less

Submitted 21 December, 2023; originally announced December 2023.

Comments: 13 pages, 5 figures

Speaker-Text Retrieval via Contrastive Learning

Authors: Xuechen Liu, Xin Wang, Erica Cooper, Xiaoxiao Miao, Junichi Yamagishi

Abstract: In this study, we introduce a novel cross-modal retrieval task involving speaker descriptions and their corresponding audio samples. Utilizing pre-trained speaker and text encoders, we present a simple learning framework based on contrastive learning. Additionally, we explore the impact of incorporating speaker labels into the training process. Our findings establish the effectiveness of linking s… ▽ More In this study, we introduce a novel cross-modal retrieval task involving speaker descriptions and their corresponding audio samples. Utilizing pre-trained speaker and text encoders, we present a simple learning framework based on contrastive learning. Additionally, we explore the impact of incorporating speaker labels into the training process. Our findings establish the effectiveness of linking speaker and text information for the task for both English and Japanese languages, across diverse data configurations. Additional visual analysis unveils potential nuanced associations between speaker clustering and retrieval performance. △ Less

Submitted 10 December, 2023; originally announced December 2023.

Comments: Submitted to IEEE Signal Processing Letters

The Pre-explosion Environments and The Progenitor of SN 2023ixf from the Hobby Eberly Telescope Dark Energy Experiment (HETDEX)

Authors: Chenxu Liu, Xinlei Chen, Xinzhong Er, Gregory R. Zeimann, Jozsef Vinko, J. Craig Wheeler, Erin Mentuch Cooper, Dustin Davis, Daniel J. Farrow, Karl Gebhardt, Helong Guo, Gary J. Hill, Lindsay House, Wolfram Kollatschny, Fanchuan Kong, Brajesh Kumar, Xiangkun Liu, Sarah Tuttle, Michael Endl, Parker Duke, William D. Cochran, **ghua Zhang, Xiaowei Liu

Abstract: Supernova (SN) 2023ixf was discovered on May 19th, 2023. The host galaxy, M101, was observed by the Hobby Eberly Telescope Dark Energy Experiment (HETDEX) collaboration over the period April 30, 2020 -- July 10, 2020, using the Visible Integral-field Replicable Unit Spectrograph (VIRUS; $3470\lesssimλ\lesssim5540$ Å) on the 10-m Hobby-Eberly Telescope (HET). The fiber filling factor within $\pm$ 3… ▽ More Supernova (SN) 2023ixf was discovered on May 19th, 2023. The host galaxy, M101, was observed by the Hobby Eberly Telescope Dark Energy Experiment (HETDEX) collaboration over the period April 30, 2020 -- July 10, 2020, using the Visible Integral-field Replicable Unit Spectrograph (VIRUS; $3470\lesssimλ\lesssim5540$ Å) on the 10-m Hobby-Eberly Telescope (HET). The fiber filling factor within $\pm$ 30 arcsec of SN 2023ixf is 80% with a spatial resolution of 1 arcsec. The r<5.5 arcsec surroundings are 100% covered. This allows us to analyze the spatially resolved pre-explosion local environments of SN 2023ixf with nebular emission lines. The 2-dimensional (2D) maps of the extinction and the star-formation rate (SFR) surface density ($Σ_{\rm SFR}$) show weak increasing trends in the radial distributions within the r<5.5 arcsec regions, suggesting lower values of extinction and SFR in the vicinity of the progenitor of SN 2023ixf. The median extinction and that of the surface density of SFR within r<3 arcsec are $E(B-V)=0.06\pm0.14$, and $Σ_{\rm SFR}=10^{-5.44\pm0.66}~\rm M_{\odot}\cdot yr^{-1}\cdot arcsec^{-2}$. There is no significant change in extinction before and after the explosion. The gas metallicity does not change significantly with the separation from SN 2023ixf. The metal-rich branch of the $R_{23}$ calculations indicates that the gas metallicity around SN 2023ixf is similar to the solar metallicity ($\sim Z_{\odot}$). The archival deep images from the Canada-France-Hawaii Telescope Legacy Survey (CFHTLS) show a clear detection of the progenitor of SN 2023ixf in the $z$-band at $22.778\pm0.063$ mag, but non-detections in the remaining four bands of CFHTLS ($u,g,r,i$). The results suggest a massive progenitor of $\approx$ 22 $M_\odot$. △ Less

Submitted 17 November, 2023; originally announced November 2023.

Comments: 11 pages, 5 figures, Accepted by ApJL

Partial Rank Similarity Minimization Method for Quality MOS Prediction of Unseen Speech Synthesis Systems in Zero-Shot and Semi-supervised setting

Authors: Hemant Yadav, Erica Cooper, Junichi Yamagishi, Sunayana Sitaram, Rajiv Ratn Shah

Abstract: This paper introduces a novel objective function for quality mean opinion score (MOS) prediction of unseen speech synthesis systems. The proposed function measures the similarity of relative positions of predicted MOS values, in a mini-batch, rather than the actual MOS values. That is the partial rank similarity is measured (PRS) rather than the individual MOS values as with the L1 loss. Our exper… ▽ More This paper introduces a novel objective function for quality mean opinion score (MOS) prediction of unseen speech synthesis systems. The proposed function measures the similarity of relative positions of predicted MOS values, in a mini-batch, rather than the actual MOS values. That is the partial rank similarity is measured (PRS) rather than the individual MOS values as with the L1 loss. Our experiments on out-of-domain speech synthesis systems demonstrate that the PRS outperforms L1 loss in zero-shot and semi-supervised settings, exhibiting stronger correlation with ground truth. These findings highlight the importance of considering rank order, as done by PRS, when training MOS prediction models. We also argue that mean squared error and linear correlation coefficient metrics may be unreliable for evaluating MOS prediction models. In conclusion, PRS-trained models provide a robust framework for evaluating speech quality and offer insights for develo** high-quality speech synthesis systems. Code and models are available at github.com/nii-yamagishilab/partial_rank_similarity/ △ Less

Submitted 8 October, 2023; originally announced October 2023.

Comments: Accepted to ASRU 2023

The VoiceMOS Challenge 2023: Zero-shot Subjective Speech Quality Prediction for Multiple Domains

Authors: Erica Cooper, Wen-Chin Huang, Yu Tsao, Hsin-Min Wang, Tomoki Toda, Junichi Yamagishi

Abstract: We present the second edition of the VoiceMOS Challenge, a scientific event that aims to promote the study of automatic prediction of the mean opinion score (MOS) of synthesized and processed speech. This year, we emphasize real-world and challenging zero-shot out-of-domain MOS prediction with three tracks for three different voice evaluation scenarios. Ten teams from industry and academia in seve… ▽ More We present the second edition of the VoiceMOS Challenge, a scientific event that aims to promote the study of automatic prediction of the mean opinion score (MOS) of synthesized and processed speech. This year, we emphasize real-world and challenging zero-shot out-of-domain MOS prediction with three tracks for three different voice evaluation scenarios. Ten teams from industry and academia in seven different countries participated. Surprisingly, we found that the two sub-tracks of French text-to-speech synthesis had large differences in their predictability, and that singing voice-converted samples were not as difficult to predict as we had expected. Use of diverse datasets and listener information during training appeared to be successful approaches. △ Less

Submitted 6 October, 2023; v1 submitted 4 October, 2023; originally announced October 2023.

Comments: Accepted to ASRU 2023

DDSP-based Neural Waveform Synthesis of Polyphonic Guitar Performance from String-wise MIDI Input

Authors: Nicolas Jonason, Xin Wang, Erica Cooper, Lauri Juvela, Bob L. T. Sturm, Junichi Yamagishi

Abstract: We explore the use of neural synthesis for acoustic guitar from string-wise MIDI input. We propose four different systems and compare them with both objective metrics and subjective evaluation against natural audio and a sample-based baseline. We iteratively develop these four systems by making various considerations on the architecture and intermediate tasks, such as predicting pitch and loudness… ▽ More We explore the use of neural synthesis for acoustic guitar from string-wise MIDI input. We propose four different systems and compare them with both objective metrics and subjective evaluation against natural audio and a sample-based baseline. We iteratively develop these four systems by making various considerations on the architecture and intermediate tasks, such as predicting pitch and loudness control features. We find that formulating the control feature prediction task as a classification task rather than a regression task yields better results. Furthermore, we find that our simplest proposed system, which directly predicts synthesis parameters from MIDI input performs the best out of the four proposed systems. Audio examples are available at https://erl-j.github.io/neural-guitar-web-supplement. △ Less

Submitted 14 September, 2023; originally announced September 2023.

SynVox2: Towards a privacy-friendly VoxCeleb2 dataset

Authors: Xiaoxiao Miao, Xin Wang, Erica Cooper, Junichi Yamagishi, Nicholas Evans, Massimiliano Todisco, Jean-François Bonastre, Mickael Rouvier

Abstract: The success of deep learning in speaker recognition relies heavily on the use of large datasets. However, the data-hungry nature of deep learning methods has already being questioned on account the ethical, privacy, and legal concerns that arise when using large-scale datasets of natural speech collected from real human speakers. For example, the widely-used VoxCeleb2 dataset for speaker recogniti… ▽ More The success of deep learning in speaker recognition relies heavily on the use of large datasets. However, the data-hungry nature of deep learning methods has already being questioned on account the ethical, privacy, and legal concerns that arise when using large-scale datasets of natural speech collected from real human speakers. For example, the widely-used VoxCeleb2 dataset for speaker recognition is no longer accessible from the official website. To mitigate these concerns, this work presents an initiative to generate a privacy-friendly synthetic VoxCeleb2 dataset that ensures the quality of the generated speech in terms of privacy, utility, and fairness. We also discuss the challenges of using synthetic data for the downstream task of speaker verification. △ Less

Submitted 12 September, 2023; originally announced September 2023.

Comments: conference

Utilisation of open intent recognition models for customer support intent detection

Authors: Rasheed Mohammad, Oliver Favell, Shariq Shah, Emmett Cooper, Edlira Vakaj

Abstract: Businesses have sought out new solutions to provide support and improve customer satisfaction as more products and services have become interconnected digitally. There is an inherent need for businesses to provide or outsource fast, efficient and knowledgeable support to remain competitive. Support solutions are also advancing with technologies, including use of social media, Artificial Intelligen… ▽ More Businesses have sought out new solutions to provide support and improve customer satisfaction as more products and services have become interconnected digitally. There is an inherent need for businesses to provide or outsource fast, efficient and knowledgeable support to remain competitive. Support solutions are also advancing with technologies, including use of social media, Artificial Intelligence (AI), Machine Learning (ML) and remote device connectivity to better support customers. Customer support operators are trained to utilise these technologies to provide better customer outreach and support for clients in remote areas. Interconnectivity of products and support systems provide businesses with potential international clients to expand their product market and business scale. This paper reports the possible AI applications in customer support, done in collaboration with the Knowledge Transfer Partnership (KTP) program between Birmingham City University and a company that handles customer service systems for businesses outsourcing customer support across a wide variety of business sectors. This study explored several approaches to accurately predict customers' intent using both labelled and unlabelled textual data. While some approaches showed promise in specific datasets, the search for a single, universally applicable approach continues. The development of separate pipelines for intent detection and discovery has led to improved accuracy rates in detecting known intents, while further work is required to improve the accuracy of intent discovery for unknown intents. △ Less

Submitted 31 July, 2023; originally announced July 2023.

Comments: 9 pages, 3 figures, conference

HETDEX Public Source Catalog 1 -- Stacking 50K Lyman Alpha Emitters

Authors: Dustin Davis, Karl Gebhardt, Erin Mentuch Cooper, William P. Bowman, Barbara Garcia Castanheira, John Chisholm, Robin Ciardullo, Maximilian Fabricius, Daniel J. Farrow, Steven L. Finkelstein, Caryl Gronwall, Eric Gawiser, Gary J. Hill, Ulrich Hopp, Lindsay R. House, Donghui Jeong, Wolfram Kollatschny, Eiichiro Komatsu, Chenxu Liu, Maja Lujan Niemeyer, Alberto Saldana-Lopez, Shun Saito, Donald P. Schneider, Jan Snigula, Sarah Tuttle , et al. (3 additional authors not shown)

Abstract: We describe the ensemble properties of the $1.9 < z < 3.5$ Lyman Alpha Emitters (LAEs) found in the HETDEX survey's first public data release, HETDEX Public Source Catalog 1 (Mentuch Cooper et al. 2023). Stacking the low-resolution ($R \sim$ 800) spectra greatly increases the signal-to-noise ratio, revealing spectral features otherwise hidden by noise, and we show that the stacked spectrum is repr… ▽ More We describe the ensemble properties of the $1.9 < z < 3.5$ Lyman Alpha Emitters (LAEs) found in the HETDEX survey's first public data release, HETDEX Public Source Catalog 1 (Mentuch Cooper et al. 2023). Stacking the low-resolution ($R \sim$ 800) spectra greatly increases the signal-to-noise ratio, revealing spectral features otherwise hidden by noise, and we show that the stacked spectrum is representative of an average member of the set. The flux limited, Ly$α$ signal-to-noise ratio restricted stack of 50K HETDEX LAEs shows the ensemble biweight ``average" $z \sim 2.6$ LAE to be a blue (UV continuum slope $\sim -2.4$ and E(B-V) $< 0.1$), moderately bright (M$_{\text{UV}} \sim -19.7$) star forming galaxy with strong Ly$α$ emission (log $L_{Lyα}$ $\sim$ 42.8 and $W_λ$(Ly$α$) $\sim$ 114Å), and potentially significant leakage of ionizing radiation. The restframe UV light is dominated by a young, metal poor stellar population with an average age 5-15 Myr and metallicity of 0.2-0.3 Z$_{\odot}$. △ Less

Submitted 6 July, 2023; originally announced July 2023.

Comments: 17 pages, 11 figures, 2 data files (ApJ Accepted)

Exploring Isolated Musical Notes as Pre-training Data for Predominant Instrument Recognition in Polyphonic Music

Authors: Lifan Zhong, Erica Cooper, Junichi Yamagishi, Nobuaki Minematsu

Abstract: With the growing amount of musical data available, automatic instrument recognition, one of the essential problems in Music Information Retrieval (MIR), is drawing more and more attention. While automatic recognition of single instruments has been well-studied, it remains challenging for polyphonic, multi-instrument musical recordings. This work presents our efforts toward building a robust end-to… ▽ More With the growing amount of musical data available, automatic instrument recognition, one of the essential problems in Music Information Retrieval (MIR), is drawing more and more attention. While automatic recognition of single instruments has been well-studied, it remains challenging for polyphonic, multi-instrument musical recordings. This work presents our efforts toward building a robust end-to-end instrument recognition system for polyphonic multi-instrument music. We train our model using a pre-training and fine-tuning approach: we use a large amount of monophonic musical data for pre-training and subsequently fine-tune the model for the polyphonic ensemble. In pre-training, we apply data augmentation techniques to alleviate the domain gap between monophonic musical data and real-world music. We evaluate our method on the IRMAS testing data, a polyphonic musical dataset comprising professionally-produced commercial music recordings. Experimental results show that our best model achieves a micro F1-score of 0.674 and an LRAP of 0.814, meaning 10.9% and 8.9% relative improvement compared with the previous state-of-the-art end-to-end approach. Also, we are able to build a lightweight model, achieving competitive performance with only 519K trainable parameters. △ Less

Submitted 15 June, 2023; originally announced June 2023.

Comments: Submitted to APSIPA 2023

Speaker anonymization using orthogonal Householder neural network

Authors: Xiaoxiao Miao, Xin Wang, Erica Cooper, Junichi Yamagishi, Natalia Tomashenko

Abstract: Speaker anonymization aims to conceal a speaker's identity while preserving content information in speech. Current mainstream neural-network speaker anonymization systems disentangle speech into prosody-related, content, and speaker representations. The speaker representation is then anonymized by a selection-based speaker anonymizer that uses a mean vector over a set of randomly selected speaker… ▽ More Speaker anonymization aims to conceal a speaker's identity while preserving content information in speech. Current mainstream neural-network speaker anonymization systems disentangle speech into prosody-related, content, and speaker representations. The speaker representation is then anonymized by a selection-based speaker anonymizer that uses a mean vector over a set of randomly selected speaker vectors from an external pool of English speakers. However, the resulting anonymized vectors are subject to severe privacy leakage against powerful attackers, reduction in speaker diversity, and language mismatch problems for unseen-language speaker anonymization. To generate diverse, language-neutral speaker vectors, this paper proposes an anonymizer based on an orthogonal Householder neural network (OHNN). Specifically, the OHNN acts like a rotation to transform the original speaker vectors into anonymized speaker vectors, which are constrained to follow the distribution over the original speaker vector space. A basic classification loss is introduced to ensure that anonymized speaker vectors from different speakers have unique speaker identities. To further protect speaker identities, an improved classification loss and similarity loss are used to push original-anonymized sample pairs away from each other. Experiments on VoicePrivacy Challenge datasets in English and the \textit{AISHELL-3} dataset in Mandarin demonstrate the proposed anonymizer's effectiveness. △ Less

Submitted 12 September, 2023; v1 submitted 30 May, 2023; originally announced May 2023.

Comments: Accepted by IEEE/ACM Transactions on Audio, Speech, and Language Processing

Range-Based Equal Error Rate for Spoof Localization

Authors: Lin Zhang, Xin Wang, Erica Cooper, Nicholas Evans, Junichi Yamagishi

Abstract: Spoof localization, also called segment-level detection, is a crucial task that aims to locate spoofs in partially spoofed audio. The equal error rate (EER) is widely used to measure performance for such biometric scenarios. Although EER is the only threshold-free metric, it is usually calculated in a point-based way that uses scores and references with a pre-defined temporal resolution and counts… ▽ More Spoof localization, also called segment-level detection, is a crucial task that aims to locate spoofs in partially spoofed audio. The equal error rate (EER) is widely used to measure performance for such biometric scenarios. Although EER is the only threshold-free metric, it is usually calculated in a point-based way that uses scores and references with a pre-defined temporal resolution and counts the number of misclassified segments. Such point-based measurement overly relies on this resolution and may not accurately measure misclassified ranges. To properly measure misclassified ranges and better evaluate spoof localization performance, we upgrade point-based EER to range-based EER. Then, we adapt the binary search algorithm for calculating range-based EER and compare it with the classical point-based EER. Our analyses suggest utilizing either range-based EER, or point-based EER with a proper temporal resolution can fairly and properly evaluate the performance of spoof localization. △ Less

Submitted 28 May, 2023; originally announced May 2023.

Comments: Accepted to Interspeech 2023

Incentivizing honest performative predictions with proper scoring rules

Authors: Caspar Oesterheld, Johannes Treutlein, Emery Cooper, Rubi Hudson

Abstract: Proper scoring rules incentivize experts to accurately report beliefs, assuming predictions cannot influence outcomes. We relax this assumption and investigate incentives when predictions are performative, i.e., when they can influence the outcome of the prediction, such as when making public predictions about the stock market. We say a prediction is a fixed point if it accurately reflects the exp… ▽ More Proper scoring rules incentivize experts to accurately report beliefs, assuming predictions cannot influence outcomes. We relax this assumption and investigate incentives when predictions are performative, i.e., when they can influence the outcome of the prediction, such as when making public predictions about the stock market. We say a prediction is a fixed point if it accurately reflects the expert's beliefs after that prediction has been made. We show that in this setting, reports maximizing expected score generally do not reflect an expert's beliefs, and we give bounds on the inaccuracy of such reports. We show that, for binary predictions, if the influence of the expert's prediction on outcomes is bounded, it is possible to define scoring rules under which optimal reports are arbitrarily close to fixed points. However, this is impossible for predictions over more than two outcomes. We also perform numerical simulations in a toy setting, showing that our bounds are tight in some situations and that prediction error is often substantial (greater than 5-10%). Lastly, we discuss alternative notions of optimality, including performative stability, and show that they incentivize reporting fixed points. △ Less

Submitted 30 May, 2023; v1 submitted 27 May, 2023; originally announced May 2023.

Comments: Accepted for the 39th Conference on Uncertainty in Artificial Intelligence (UAI 2023)

Improving Generalization Ability of Countermeasures for New Mismatch Scenario by Combining Multiple Advanced Regularization Terms

Authors: Chang Zeng, Xin Wang, Xiaoxiao Miao, Erica Cooper, Junichi Yamagishi

Abstract: The ability of countermeasure models to generalize from seen speech synthesis methods to unseen ones has been investigated in the ASVspoof challenge. However, a new mismatch scenario in which fake audio may be generated from real audio with unseen genres has not been studied thoroughly. To this end, we first use five different vocoders to create a new dataset called CN-Spoof based on the CN-Celeb1… ▽ More The ability of countermeasure models to generalize from seen speech synthesis methods to unseen ones has been investigated in the ASVspoof challenge. However, a new mismatch scenario in which fake audio may be generated from real audio with unseen genres has not been studied thoroughly. To this end, we first use five different vocoders to create a new dataset called CN-Spoof based on the CN-Celeb1\&2 datasets. Then, we design two auxiliary objectives for regularization via meta-optimization and a genre alignment module, respectively, and combine them with the main anti-spoofing objective using learnable weights for multiple loss terms. The results on our cross-genre evaluation dataset for anti-spoofing show that the proposed method significantly improved the generalization ability of the countermeasures compared with the baseline system in the genre mismatch scenario. △ Less

Submitted 18 May, 2023; originally announced May 2023.

Comments: Accepted by interspeech2023

Investigating Range-Equalizing Bias in Mean Opinion Score Ratings of Synthesized Speech

Authors: Erica Cooper, Junichi Yamagishi

Abstract: Mean Opinion Score (MOS) is a popular measure for evaluating synthesized speech. However, the scores obtained in MOS tests are heavily dependent upon many contextual factors. One such factor is the overall range of quality of the samples presented in the test -- listeners tend to try to use the entire range of scoring options available to them regardless of this, a phenomenon which is known as ran… ▽ More Mean Opinion Score (MOS) is a popular measure for evaluating synthesized speech. However, the scores obtained in MOS tests are heavily dependent upon many contextual factors. One such factor is the overall range of quality of the samples presented in the test -- listeners tend to try to use the entire range of scoring options available to them regardless of this, a phenomenon which is known as range-equalizing bias. In this paper, we systematically investigate the effects of range-equalizing bias on MOS tests for synthesized speech by conducting a series of listening tests in which we progressively "zoom in" on a smaller number of systems in the higher-quality range. This allows us to better understand and quantify the effects of range-equalizing bias in MOS tests. △ Less

Submitted 6 October, 2023; v1 submitted 17 May, 2023; originally announced May 2023.

Comments: Proceedings of Interspeech 2023. DOI: 10.21437/Interspeech.2023-1076

Using Dark Energy Explorers and Machine Learning to Enhance the Hobby-Eberly Telescope Dark Energy Experiment

Authors: Lindsay R. House, Karl Gebhardt, Keely Finkelstein, Erin Mentuch Cooper, Dustin Davis, Robin Ciardullo, Daniel J Farrow, Steven L. Finkelstein, Caryl Gronwall, Donghui Jeong, L. Clifton Johnson, Chenxu Liu, Benjamin P. Thomas, Gregory Zeimann

Abstract: We present analysis using a citizen science campaign to improve the cosmological measures from the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX). The goal of HETDEX is to measure the Hubble expansion rate, $H(z)$, and angular diameter distance, $D_A(z)$, at $z =$ 2.4, each to percent-level accuracy. This accuracy is determined primarily from the total number of detected Lyman-$α$ emitters… ▽ More We present analysis using a citizen science campaign to improve the cosmological measures from the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX). The goal of HETDEX is to measure the Hubble expansion rate, $H(z)$, and angular diameter distance, $D_A(z)$, at $z =$ 2.4, each to percent-level accuracy. This accuracy is determined primarily from the total number of detected Lyman-$α$ emitters (LAEs), the false positive rate due to noise, and the contamination due to [O II] emitting galaxies. This paper presents the citizen science project, Dark Energy Explorers, with the goal of increasing the number of LAEs, decreasing the number of false positives due to noise and the [O II] galaxies. Initial analysis shows that citizen science is an efficient and effective tool for classification most accurately done by the human eye, especially in combination with unsupervised machine learning. Three aspects from the citizen science campaign that have the most impact are 1) identifying individual problems with detections, 2) providing a clean sample with 100% visual identification above a signal-to-noise cut, and 3) providing labels for machine learning efforts. Since the end of 2022, Dark Energy Explorers has collected over three and a half million classifications by 11,000 volunteers in over 85 different countries around the world. By incorporating the results of the Dark Energy Explorers we expect to improve the accuracy on the $D_A(z)$ and $H(z)$ parameters at $z =$ 2.4 by 10 - 30%. While the primary goal is to improve on HETDEX, Dark Energy Explorers has already proven to be a uniquely powerful tool for science advancement and increasing accessibility to science worldwide. △ Less

Submitted 14 April, 2023; originally announced April 2023.

Comments: 14 pages, 6 figures, accepted for publication in The Astrophysical Journal

Introducing the Texas Euclid Survey for Lyman Alpha (TESLA) Survey: Initial Study Correlating Galaxy Properties to Lyman-Alpha Emission

Authors: Oscar A. Chavez Ortiz, Steven L. Finkelstein, Dustin Davis, Gene Leung, Erin Mentuch Cooper, Micaela Bagley, Rebecca Larson, Caitlin M. Casey, Adam P. McCarron, Karl Gebhardt, Yuchen Guo, Chenxu Liu, Isaac Laseter, Jason Rhodes, Ralf Bender, Max Fabricius, Ariel G. Sanchez, Claudia Scarlata, Peter Capak, David Sanders, Istvan Szapudi, Eric Baxter, Conor McPartland, John R. Weaver, Sune Toft , et al. (2 additional authors not shown)

Abstract: We present the Texas Euclid Survey for Lyman-Alpha (TESLA), a spectroscopic survey in the 10 square degree of the Euclid North Ecliptic Pole (NEP) field. Using TESLA, we study how the physical properties of Lyman-alpha emitters (LAEs) correlate with Lyman-alpha emission to understand the escape of Lyman alpha from galaxies at redshifts 2 -- 3.5. We present an analysis of 43 LAEs performed in the N… ▽ More We present the Texas Euclid Survey for Lyman-Alpha (TESLA), a spectroscopic survey in the 10 square degree of the Euclid North Ecliptic Pole (NEP) field. Using TESLA, we study how the physical properties of Lyman-alpha emitters (LAEs) correlate with Lyman-alpha emission to understand the escape of Lyman alpha from galaxies at redshifts 2 -- 3.5. We present an analysis of 43 LAEs performed in the NEP field using early data from the TESLA survey. We use Subaru Hyper Suprime-Cam imaging in the grizy-bands, Spitzer/IRAC channels 1 and 2 from the Hawaii 20 square degree (H20) survey and spectra acquired by the Visible Integral-Field Replicable Unit Spectrograph (VIRUS) on the Hobby-Eberly Telescope. We perform spectral energy distribution (SED) fitting to compute the galaxy properties of 43 LAEs, and study correlations between stellar mass, star formation rate (SFR), and dust, to the Lyman-alpha rest-frame equivalent widths (EW). We uncover marginal (1 sigma significance) correlations between stellar mass and Lyman-alpha EW, and star formation rate (SFR) and Lyman-alpha EW, with a Spearman correlation coefficient of -0.$34_{-.14}^{+.17}$ and -0.$37_{-.14}^{+.16}$ respectively. We show that the Lyman-alpha distribution of the 43 LAEs is consistent with being drawn from an exponential distribution with an e-folding scale of 150 Angstrom. Once complete the TESLA survey will enable the study of ~ thousands of LAEs to explore correlations between galaxy properties and Lyman-alpha EW. The large sample size will allow the construction of a predictive model for the Lyman-alpha EW as a function of SED-derived galaxy properties, which could be used to improve Lyman-alpha based constraints on reionization. △ Less

Submitted 6 April, 2023; originally announced April 2023.

The Stellar Mass - Black Hole Mass Relation at $z\sim2$ Down to $\mathcal{M}_\mathrm{BH}\sim10^7 M_\odot$ Determined by HETDEX

Authors: Yechi Zhang, Masami Ouchi, Karl Gebhardt, Chenxu Liu, Yuichi Harikane, Erin Mentuch Cooper, Dustin Davis, Daniel J. Farrow, Eric Gawiser, Gary J. Hill, Wolfram Kollatschny, Yoshiaki Ono, Donald P. Schneider, Steven L. Finkelstein, Caryl Gronwall, Shardha Jogee, Mirko Krumpe

Abstract: We investigate the stellar mass - black hole mass ($\mathcal{M}_*-\mathcal{M}_\mathrm{BH}$) relation with type 1 AGN down to $\mathcal{M}_\mathrm{BH}=10^7 M_\odot$, corresponding to a $\simeq -21$ absolute magnitude in rest-frame ultraviolet (UV), at $z = 2-2.5$. Exploiting the deep and large-area spectroscopic survey of the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX), we identify 66 ty… ▽ More We investigate the stellar mass - black hole mass ($\mathcal{M}_*-\mathcal{M}_\mathrm{BH}$) relation with type 1 AGN down to $\mathcal{M}_\mathrm{BH}=10^7 M_\odot$, corresponding to a $\simeq -21$ absolute magnitude in rest-frame ultraviolet (UV), at $z = 2-2.5$. Exploiting the deep and large-area spectroscopic survey of the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX), we identify 66 type 1 AGN with $\mathcal{M}_\mathrm{BH}$ ranging from $10^7$ to $10^{10} M_\odot$ that are measured with single-epoch virial method using C{\sc iv} emission lines detected in the HETDEX spectra. $\mathcal{M}_*$ of the host galaxies are estimated from optical to near-infrared photometric data taken with Spitzer, WISE, and ground-based 4-8m class telescopes by CIGALE SED fitting. We further assess the validity of SED fitting in two cases by host-nuclear decomposition performed through surface brightness profile fitting on spatially-resolved host galaxies with JWST/NIRCam CEERS data. We obtain the $\mathcal{M}_*-\mathcal{M}_\mathrm{BH}$ relation covering the unexplored low-mass ranges of $\mathcal{M}_\mathrm{BH}~\sim~10^7-10^8~M_\odot$, and conduct forward modelling to fully account for the selection biases and observational uncertainties. The intrinsic $\mathcal{M}_*-\mathcal{M}_\mathrm{BH}$ relation at $z\sim 2$ has a moderate positive offset of $0.52\pm0.14$~dex from the local relation, suggestive of more efficient black hole growth at higher redshift even in the low-mass regime of $\mathcal{M}_\mathrm{BH}~\sim~10^7-10^8~M_\odot$. Our $\mathcal{M}_*-\mathcal{M}_\mathrm{BH}$ relation is inconsistent with the $\mathcal{M}_\mathrm{BH}$ suppression at the low-$\mathcal{M}_*$ regime predicted by recent hydrodynamic simulations at a $98\%$ confidence level, suggesting that feedback in the low-mass systems may be weaker than those produced in hydrodynamic simulations. △ Less

Submitted 6 March, 2023; originally announced March 2023.

Comments: 16 pages, 8 figures, accepted for publication in ApJ

Identifying Active Galactic Nuclei at $z\sim3$ from the HETDEX Survey Using Machine Learning

Authors: Valentina Tardugno Poleo, Steven Finkelstein, Gene C. K. Leung, Erin Mentuch Cooper, Karl Gebhardt, Daniel Farrow, Eric Gawiser, Gregory Zeimann, Donald Schneider, Leah Morabito, Daniel Mock, Chenxu Liu

Abstract: We used data from the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX) to study the incidence of AGN in continuum-selected galaxies at $z\sim3$. From optical and infrared imaging in the 24 deg$^{2}$ Spitzer HETDEX Exploratory Large Area (SHELA) survey, we constructed a sample of photometric-redshift selected $z\sim3$ galaxies. We extracted HETDEX spectra at the position of 716 of these sourc… ▽ More We used data from the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX) to study the incidence of AGN in continuum-selected galaxies at $z\sim3$. From optical and infrared imaging in the 24 deg$^{2}$ Spitzer HETDEX Exploratory Large Area (SHELA) survey, we constructed a sample of photometric-redshift selected $z\sim3$ galaxies. We extracted HETDEX spectra at the position of 716 of these sources and used machine learning methods to identify those which exhibited AGN-like features. The dimensionality of the spectra was reduced using an autoencoder, and the latent space was visualized through t-distributed stochastic neighbor embedding (t-SNE). Gaussian mixture models were employed to cluster the encoded data and a labeled dataset was used to label each cluster as either AGN, stars, high-redshift galaxies, or low-redshift galaxies. Our photometric redshift (photo-z) sample was labeled with an estimated $92\%$ overall accuracy, an AGN accuracy of $83\%$, and an AGN contamination of $5\%$. The number of identified AGN was used to measure an AGN fraction for different magnitude bins. The UV absolute magnitude where the AGN fraction reaches $50\%$ is $M_{UV} = -23.8$. When combined with results in the literature, our measurements of AGN fraction imply that the bright end of the galaxy luminosity function exhibits a power-law rather than exponential decline, with a relatively shallow faint-end slope for the $z\sim3$ AGN luminosity function. △ Less

Submitted 21 February, 2023; originally announced February 2023.

The Marriage of Effects and Rewrites

Authors: Ezra e. k. Cooper

Abstract: In the research on computational effects, defined algebraically, effect symbols are often expected to obey certain equations. If we orient these equations, we get a rewrite system, which may be an effective way of transforming or optimizing the effects in a program. In order to do so, we need to establish strong normalization, or termination, of the rewrite system. Here we define a framework for c… ▽ More In the research on computational effects, defined algebraically, effect symbols are often expected to obey certain equations. If we orient these equations, we get a rewrite system, which may be an effective way of transforming or optimizing the effects in a program. In order to do so, we need to establish strong normalization, or termination, of the rewrite system. Here we define a framework for carrying out such proofs, and extend the well-known Recursive Path Ordering of Dershowitz to show termination of some effect systems. △ Less

Submitted 5 February, 2023; originally announced February 2023.

Comments: 15 pages, 2 figures. Submitted to FSCD 2023

ACM Class: F.4.2

Cosmological-Scale Lyman-alpha Forest Absorption Around Galaxies and AGN Probed with the HETDEX and SDSS Spectroscopic Data

Authors: Dongsheng Sun, Ken Mawatari, Masami Ouchi, Yoshiaki Ono, Hidenobu Yajima, Yechi Zhang, Makito Abe, William P. Bowman, Erin Mentuch Cooper, Dustin Davis, Daniel J. Farrow, Karl Gebhardt, Gary J. Hill, Chenxu Liu, Donald P. Schneider

Abstract: We present cosmological-scale 3-dimensional (3D) neutral hydrogen ({\sc Hi}) tomographic maps at $z=2-3$ over a total of 837 deg$^2$ in two blank fields that are developed with Ly$α$ forest absorptions of 14,736 background Sloan Digital Sky Survey (SDSS) quasars at $z$=2.08-3.67. Using the tomographic maps, we investigate the large-scale ($\gtrsim 10$ $h^{-1}$cMpc) average {\sc Hi} radial profiles… ▽ More We present cosmological-scale 3-dimensional (3D) neutral hydrogen ({\sc Hi}) tomographic maps at $z=2-3$ over a total of 837 deg$^2$ in two blank fields that are developed with Ly$α$ forest absorptions of 14,736 background Sloan Digital Sky Survey (SDSS) quasars at $z$=2.08-3.67. Using the tomographic maps, we investigate the large-scale ($\gtrsim 10$ $h^{-1}$cMpc) average {\sc Hi} radial profiles and two-direction profiles of the line-of-sight (LoS) and transverse (Trans) directions around galaxies and AGN at $z=2-3$ identified by the Hobby-Eberly Telescope Dark Energy eXperiment (HETDEX) and SDSS surveys, respectively. The peak of the {\sc Hi} radial profile around galaxies is lower than the one around AGN, suggesting that the dark-matter halos of galaxies are less massive on average than those of AGN. The LoS profile of AGN is narrower than the Trans profile, indicating the Kaiser effect. There exist weak absorption outskirts at $\gtrsim 30$ $h^{-1}$cMpc beyond {\sc Hi} structures of galaxies and AGN found in the LoS profiles that can be explained by the {\sc Hi} gas at $\gtrsim 30$ $h^{-1}$cMpc falls toward the source positions. Our findings indicate that the {\sc Hi} radial profile of AGN has transitions from proximity zones ($\lesssim$ a few $h^{-1}$cMpc) to the {\sc Hi} structures ($\sim 1-30$ $h^{-1}$cMpc) and the weak absorption outskirts ($\gtrsim 30$ $h^{-1}$cMpc). Although there is no significant dependence of AGN types (type-1 vs. type-2) on the {\sc Hi} profiles, the peaks of the radial profiles anti-correlate with AGN luminosities, suggesting that AGN's ionization effects are stronger than the gas mass differences. △ Less

Submitted 25 April, 2023; v1 submitted 12 January, 2023; originally announced January 2023.

Comments: 26 pages, 32 figures, accepted to ApJ

HETDEX Public Source Catalog 1: 220K Sources Including Over 50K Lyman Alpha Emitters from an Untargeted Wide-area Spectroscopic Survey

Authors: Erin Mentuch Cooper, Karl Gebhardt, Dustin Davis, Daniel J. Farrow, Chenxu Liu, Gregory Zeimann, Robin Ciardullo, John J. Feldmeier, Niv Drory, Donghui Jeong, Barbara Benda, William P. Bowman, Michael Boylan-Kolchin, Oscar A. Chavez Ortiz, Maya H. Debski, Mona Dentler, Maximilian Fabricius, Rameen Farooq, Steven L. Finkelstein, Eric Gawiser, Caryl Gronwall, Gary J. Hill, Ulrich Hopp, Lindsay R. House, Steven Janowiecki , et al. (21 additional authors not shown)

Abstract: We present the first publicly released catalog of sources obtained from the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX). HETDEX is an integral field spectroscopic survey designed to measure the Hubble expansion parameter and angular diameter distance at 1.88<z<3.52 by using the spatial distribution of more than a million Ly-alpha-emitting galaxies over a total target area of 540 deg^2.… ▽ More We present the first publicly released catalog of sources obtained from the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX). HETDEX is an integral field spectroscopic survey designed to measure the Hubble expansion parameter and angular diameter distance at 1.88<z<3.52 by using the spatial distribution of more than a million Ly-alpha-emitting galaxies over a total target area of 540 deg^2. The catalog comes from contiguous fiber spectra coverage of 25 deg^2 of sky from January 2017 through June 2020, where object detection is performed through two complementary detection methods: one designed to search for line emission and the other a search for continuum emission. The HETDEX public release catalog is dominated by emission-line galaxies and includes 51,863 Lyα-emitting galaxy (LAE) identifications and 123,891 OII-emitting galaxies at z<0.5. Also included in the catalog are 37,916 stars, 5274 low-redshift (z<0.5) galaxies without emission lines, and 4976 active galactic nuclei. The catalog provides sky coordinates, redshifts, line identifications, classification information, line fluxes, OII and Ly-alpha line luminosities where applicable, and spectra for all identified sources processed by the HETDEX detection pipeline. Extensive testing demonstrates that HETDEX redshifts agree to within deltaz < 0.02, 96.1% of the time to those in external spectroscopic catalogs. We measure the photometric counterpart fraction in deep ancillary Hyper Suprime-Cam imaging and find that only 55.5% of the LAE sample has an r-band continuum counterpart down to a limiting magnitude of r~26.2 mag (AB) indicating that an LAE search of similar sensitivity with photometric pre-selection would miss nearly half of the HETDEX LAE catalog sample. Data access and details about the catalog can be found online at http://hetdex.org/. △ Less

Submitted 4 January, 2023; originally announced January 2023.

Comments: 38 pages, 20 figures. Data access and details about the catalog can be found online at http://hetdex.org/. A copy of the catalogs presented in this work (Version 3.2) is available to download at Zenodo doi:10.5281/zenodo.7448504

The HETDEX Survey: Emission Line Exploration and Source Classification

Authors: Dustin Davis, Karl Gebhardt, Erin Mentuch Cooper, Robin Ciardullo, Maximilian Fabricius, Daniel J. Farrow, John J. Feldmeier, Steven L. Finkelstein, Eric Gawiser, Caryl Gronwall, Gary J. Hill, Ulrich Hopp, Lindsay R. House, Donghui Jeong, Wolfram Kollatschny, Eiichiro Komatsu, Martin Landriau, Chenxu Liu, Shun Saito, Sarah Tuttle, Isak G. B. Wold, Gregory R. Zeimann, Yechi Zhang

Abstract: The Hobby-Eberly Telescope Dark Energy Experiment (HETDEX) is an untargeted spectroscopic survey that aims to measure the expansion rate of the Universe at $z \sim 2.4$ to 1% precision for both $H(z)$ and $D_A(z)$. HETDEX is in the process of map** in excess of one million Lyman Alpha emitting (LAE) galaxies and a similar number of lower-z galaxies as a tracer of the large-scale structure. The s… ▽ More The Hobby-Eberly Telescope Dark Energy Experiment (HETDEX) is an untargeted spectroscopic survey that aims to measure the expansion rate of the Universe at $z \sim 2.4$ to 1% precision for both $H(z)$ and $D_A(z)$. HETDEX is in the process of map** in excess of one million Lyman Alpha emitting (LAE) galaxies and a similar number of lower-z galaxies as a tracer of the large-scale structure. The success of the measurement is predicated on the post-observation separation of galaxies with Ly$α$ emission from the lower-$z$ interlo** galaxies, primarily [OII], with low contamination and high recovery rates. The Emission Line eXplorer (ELiXer) is the principal classification tool for HETDEX, providing a tunable balance between contamination and completeness as dictated by science needs. By combining multiple selection criteria, ELiXer improves upon the 20 Angstrom rest-frame equivalent width cut commonly used to distinguish LAEs from lower-$z$ [OII] emitting galaxies. Despite a spectral resolving power, R $\sim800$, that cannot resolve the [OII] doublet, we demonstrate the ability to distinguish LAEs from foreground galaxies with 98.1% accuracy. We estimate a contamination rate of Ly$α$ by [OII] of 1.2% and a Ly$α$ recovery rate of 99.1% using the default ELiXer configuration. These rates meet the HETDEX science requirements. △ Less

Submitted 4 January, 2023; originally announced January 2023.

Comments: 38 pages, 11 figures

Engineering Graph States of Atomic Ensembles by Photon-Mediated Entanglement

Authors: Eric S. Cooper, Philipp Kunkel, Avikar Periwal, Monika Schleier-Smith

Abstract: Graph states are versatile resources for quantum computation and quantum-enhanced measurement. Their generation illustrates a high level of control over entanglement. We report on the generation of continuous-variable graph states of atomic spin ensembles, which form the nodes of the graph. The edges represent the entanglement structure, which we program by combining global photon-mediated interac… ▽ More Graph states are versatile resources for quantum computation and quantum-enhanced measurement. Their generation illustrates a high level of control over entanglement. We report on the generation of continuous-variable graph states of atomic spin ensembles, which form the nodes of the graph. The edges represent the entanglement structure, which we program by combining global photon-mediated interactions in an optical cavity with local spin rotations. By tuning the entanglement between two subsystems, we either localize correlations within each subsystem or enable Einstein-Podolsky-Rosen steering. We further engineer a four-mode square graph state, highlighting the flexibility of our approach. Our method is scalable to larger and more complex graphs, laying groundwork for measurement-based quantum computation and advanced protocols in quantum metrology. △ Less

Submitted 31 August, 2023; v1 submitted 22 December, 2022; originally announced December 2022.

Searching for Supernovae in HETDEX Data Release 3

Authors: J. Vinko, B. P. Thomas, J. C. Wheeler, A. Y. Q. Ho, E. Mentuch Cooper, K. Gebhardt, R. Ciardullo, D. J. Farrow, G. J. Hill, Z. Jager, W. Kollatschny, C. Liu, E. Regos, K. Sarneczky

Abstract: We have extracted 636 spectra taken at the positions of 583 transient sources from the third Data Release of the Hobby-Eberly Telescope Dark Energy eXperiment (HETDEX). The transients were discovered by the Zwicky Transient Facility (ZTF) during 2018 - 2022. The HETDEX spectra are useful to classify a large number of objects found by photometric surveys for free. We attempt to explore and classify… ▽ More We have extracted 636 spectra taken at the positions of 583 transient sources from the third Data Release of the Hobby-Eberly Telescope Dark Energy eXperiment (HETDEX). The transients were discovered by the Zwicky Transient Facility (ZTF) during 2018 - 2022. The HETDEX spectra are useful to classify a large number of objects found by photometric surveys for free. We attempt to explore and classify the spectra by utilizing machine learning (ML) and template matching techniques. We have identified two transient sources, ZTF20aatpoos = AT2020fiz and ZTF19abdkelq as supernova candidates. We classify AT2020fiz as a Type IIP supernova observed ~10 days after explosion, and we propose ZTF19abdkelq as a likely Type Ia SN caught ~40 days after maximum light. ZTF photometry of these two sources are consistent with their classification as supernovae. Beside these two objects, we have confirmed several ZTF transients as variable AGNs based on their spectral appearance, and also determined the host galaxy types for several other ZTF transients. △ Less

Submitted 16 December, 2022; originally announced December 2022.

Comments: submitted to ApJ

Can Knowledge of End-to-End Text-to-Speech Models Improve Neural MIDI-to-Audio Synthesis Systems?

Authors: Xuan Shi, Erica Cooper, Xin Wang, Junichi Yamagishi, Shrikanth Narayanan

Abstract: With the similarity between music and speech synthesis from symbolic input and the rapid development of text-to-speech (TTS) techniques, it is worthwhile to explore ways to improve the MIDI-to-audio performance by borrowing from TTS techniques. In this study, we analyze the shortcomings of a TTS-based MIDI-to-audio system and improve it in terms of feature computation, model selection, and trainin… ▽ More With the similarity between music and speech synthesis from symbolic input and the rapid development of text-to-speech (TTS) techniques, it is worthwhile to explore ways to improve the MIDI-to-audio performance by borrowing from TTS techniques. In this study, we analyze the shortcomings of a TTS-based MIDI-to-audio system and improve it in terms of feature computation, model selection, and training strategy, aiming to synthesize highly natural-sounding audio. Moreover, we conducted an extensive model evaluation through listening tests, pitch measurement, and spectrogram analysis. This work demonstrates not only synthesis of highly natural music but offers a thorough analytical approach and useful outcomes for the community. Our code, pre-trained models, supplementary materials, and audio samples are open sourced at https://github.com/nii-yamagishilab/midi-to-audio. △ Less

Submitted 20 March, 2023; v1 submitted 24 November, 2022; originally announced November 2022.

Comments: Accepted by ICASSP 2023

The Active Galactic Nuclei in the Hobby-Eberly Telescope Dark Energy Experiment Survey (HETDEX) III. A red quasar with extremely high equivalent widths showing powerful outflows

Authors: Chenxu Liu, Karl Gebhardt, Wolfram Kollatschny, Robin Ciardullo, Erin Mentuch Cooper, Dustin Davis, Daniel J. Farrow, Steven L. Finkelstein, Eric Gawiser, Caryl Gronwall, Gary J. Hill, Lindsay House, Donald P. Schneider, Tanya Urrutia, Gregory R. Zeimann

Abstract: We report an Active Galactic Nucleus (AGN) with extremely high equivalent width (EW), EW(LyA+NV,rest)>921 AA in the rest-frame, at z~2.24 in the Hobby-Eberly Telescope Dark Energy Experiment Survey (HETDEX) as a representative case of the high EW AGN population. The continuum level is a non-detection in the HETDEX spectrum, thus the measured EW is a lower limit. The source is detected with signifi… ▽ More We report an Active Galactic Nucleus (AGN) with extremely high equivalent width (EW), EW(LyA+NV,rest)>921 AA in the rest-frame, at z~2.24 in the Hobby-Eberly Telescope Dark Energy Experiment Survey (HETDEX) as a representative case of the high EW AGN population. The continuum level is a non-detection in the HETDEX spectrum, thus the measured EW is a lower limit. The source is detected with significant emission lines (>7sigma) at LyA+NV, CIV, and moderate emission line (~4sigma) at HeII within the wavelength coverage of HETDEX (3500 AA - 5500 AA). The r-band magnitude is 24.57 from the Hyper Suprime-Cam-HETDEX joint survey with a detection limit of r=25.12 at 5sigma. The LyA emission line spans a clearly resolved region of ~10 arcsec (85 kpc) in diameter. The LyA line profile is strongly double peaked. The spectral decomposed blue gas and red gas Ly$α$ emission are separated by ~1.2 arcsec (10.1 kpc) with a line-of-sight velocity offset of ~1100 km/s. This source is probably an obscured AGN with powerful winds. △ Less

Submitted 23 October, 2022; originally announced October 2022.

Comments: 10 pages, 5 figures, 1 table, accepted for publication in ApJ

A Search for Lensed Lyman-Alpha Emitters within the Early HETDEX Data Set

Authors: Isaac H. Laseter, Steven L. Finkelstein, Micaela J. Bagley, Dustin M. Davis, Karl Gebhardt, Caryl Gronwall, Robin Ciardullo, Gregory R. Zeimann, Erin Mentuch Cooper, Daniel Farrow

Abstract: The Hobby-Eberly Telescope Dark Energy Experiment (HETDEX) is a large-volume spectroscopic survey without pre-selection of sources, searching ~ 540 deg^2 for Lyman-alpha emitting galaxies (LAEs) at 1.9 < z < 3.5. Taking advantage of such a wide-volume survey, we perform a pilot study using early HETDEX data to search for lensed Lyman-alpha emitters. After performing a proof-of-concept using a prev… ▽ More The Hobby-Eberly Telescope Dark Energy Experiment (HETDEX) is a large-volume spectroscopic survey without pre-selection of sources, searching ~ 540 deg^2 for Lyman-alpha emitting galaxies (LAEs) at 1.9 < z < 3.5. Taking advantage of such a wide-volume survey, we perform a pilot study using early HETDEX data to search for lensed Lyman-alpha emitters. After performing a proof-of-concept using a previously known lensed LAE covered by HETDEX, we perform a search for previously unknown lensed LAEs in the HETDEX spectroscopic sample. We present a catalog of 26 potential LAEs lensed by foreground, red, non-star-forming galaxies at z ~ 0.4 - 0.7. We estimate the magnification for each candidate system, finding 12 candidates to be within the strong lensing regime (magnification $μ$ > 2). Follow-up observations of these potential lensed LAEs have the potential to confirm their lensed nature and explore these distant galaxies in more detail. △ Less

Submitted 25 October, 2022; v1 submitted 13 October, 2022; originally announced October 2022.

Comments: 28 pages, 31 figures, 2 tables, accepted for publication in ApJ

Modeling Quantum Enhanced Sensing on a Quantum Computer

Authors: Cindy Tran, Tanaporn Na Narong, Eric S. Cooper

Abstract: Quantum computers allow for direct simulation of the quantum interference and entanglement used in modern interferometry experiments with applications ranging from biological sensing to gravitational wave detection. Inspired by recent developments in quantum sensing at the Laser Interferometer Gravitational-wave Observatory (LIGO), here we present two quantum circuit models that demonstrate the ro… ▽ More Quantum computers allow for direct simulation of the quantum interference and entanglement used in modern interferometry experiments with applications ranging from biological sensing to gravitational wave detection. Inspired by recent developments in quantum sensing at the Laser Interferometer Gravitational-wave Observatory (LIGO), here we present two quantum circuit models that demonstrate the role of quantum mechanics and entanglement in modern precision sensors. We implemented these quantum circuits on IBM quantum processors, using a single qubit to represent independent photons traveling through the LIGO interferometer and two entangled qubits to illustrate the improved sensitivity that LIGO has achieved by using non-classical states of light. The one-qubit interferometer illustrates how projection noise in the measurement of independent photons corresponds to phase sensitivity at the standard quantum limit. In the presence of technical noise on a real quantum computer, this interferometer achieves the sensitivity of 11\% above the standard quantum limit. The two-qubit interferometer demonstrates how entanglement circumvents the limits imposed by the quantum shot noise, achieving the phase sensitivity 17\% below the standard quantum limit. These experiments illustrate the role that quantum mechanics plays in setting new records for precision measurements on platforms like LIGO. The experiments are broadly accessible, remotely executable activities that are well suited for introducing undergraduate students to quantum computation, error propagation, and quantum sensing on real quantum hardware. △ Less

Submitted 16 September, 2022; originally announced September 2022.

Comments: 11 pages, 8 figures

Joint Speaker Encoder and Neural Back-end Model for Fully End-to-End Automatic Speaker Verification with Multiple Enrollment Utterances

Authors: Chang Zeng, Xiaoxiao Miao, Xin Wang, Erica Cooper, Junichi Yamagishi

Abstract: Conventional automatic speaker verification systems can usually be decomposed into a front-end model such as time delay neural network (TDNN) for extracting speaker embeddings and a back-end model such as statistics-based probabilistic linear discriminant analysis (PLDA) or neural network-based neural PLDA (NPLDA) for similarity scoring. However, the sequential optimization of the front-end and ba… ▽ More Conventional automatic speaker verification systems can usually be decomposed into a front-end model such as time delay neural network (TDNN) for extracting speaker embeddings and a back-end model such as statistics-based probabilistic linear discriminant analysis (PLDA) or neural network-based neural PLDA (NPLDA) for similarity scoring. However, the sequential optimization of the front-end and back-end models may lead to a local minimum, which theoretically prevents the whole system from achieving the best optimization. Although some methods have been proposed for jointly optimizing the two models, such as the generalized end-to-end (GE2E) model and NPLDA E2E model, all of these methods are designed for use with a single enrollment utterance. In this paper, we propose a new E2E joint method for speaker verification especially designed for the practical case of multiple enrollment utterances. In order to leverage the intra-relationship among multiple enrollment utterances, our model comes equipped with frame-level and utterance-level attention mechanisms. We also utilize several data augmentation techniques, including conventional noise augmentation using MUSAN and RIRs datasets and a unique speaker embedding-level mixup strategy for better optimization. △ Less

Submitted 1 September, 2022; originally announced September 2022.

Comments: Submitted to TASLP

Stellar Populations of Lyman-alpha Emitting Galaxies in the HETDEX Survey I: An Analysis of LAEs in the GOODS-N Field

Authors: Adam P. McCarron, Steven L. Finkelstein, Oscar A. Chavez Ortiz, Dustin Davis, Erin Mentuch Cooper, Intae Jung, Delaney R. White, Gene C. K. Leung, Karl Gebhardt, Viviana Acquaviva, William P. Bowman, Robin Ciardullo, Eric Gawiser, Caryl Gronwall, Gary J. Hill, Wolfram Kollatschny, Martin Landriau, Chenxu Liu, Daniel N. Mock, Ariel G. Sanchez

Abstract: We present the results of a stellar-population analysis of Lyman-alpha emitting galaxies (LAES) in GOODS-N at 1.9 < z < 3.5 spectroscopically identified by the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX). We provide a method for connecting emission-line detections from the blind spectroscopic survey to imaging counterparts, a crucial tool needed as HETDEX builds a massive database of ~1… ▽ More We present the results of a stellar-population analysis of Lyman-alpha emitting galaxies (LAES) in GOODS-N at 1.9 < z < 3.5 spectroscopically identified by the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX). We provide a method for connecting emission-line detections from the blind spectroscopic survey to imaging counterparts, a crucial tool needed as HETDEX builds a massive database of ~1 million Lyman-alpha detections. Using photometric data spanning as many as 11 filters covering 0.4-4.5 microns from the Hubble and Spitzer Space Telescopes, we study the objects' global properties and explore which properties impact the strength of Lyman-alpha emission. We measure a median stellar mass of 0.8 (^+2.9_-0.5) x 10^9 Msol and conclude that the physical properties of HETDEX spectroscopically-selected LAEs are comparable to LAEs selected by previous deep narrow band studies. We find that stellar mass and star formation rate correlate strongly with the Lyman-alpha equivalent width. We then use a known sample of z>7 LAEs to perform a proto-study of predicting Lyman-alpha emission from galaxies in the Epoch of Reionization, finding agreement at the 1-sigma level between prediction and observation for the majority of strong emitters. △ Less

Submitted 2 August, 2022; originally announced August 2022.

Comments: 29 pages, 22 figures, Accepted to the Astrophysical Journal

The Active Galactic Nuclei in the Hobby-Eberly Telescope Dark Energy Experiment Survey (HETDEX) II. Luminosity Function

Authors: Chenxu Liu, Karl Gebhardt, Erin Mentuch Cooper, Yechi Zhang, Donald P. Schneider, Robin Ciardullo, Dustin Davis, Daniel J. Farrow, Steven L. Finkelstein, Caryl Gronwall, Gary J. Hill, Lindsay House, Donghui Jeong, Wolfram Kollatschny, Maja Lujan Niemeyer, Sarah Tuttle

Abstract: We present the LyA emission line luminosity function (LF) of the Active Galactic Nuclei (AGN) in the first release of the Hobby-Eberly Telescope Dark Energy Experiment Survey (HETDEX) AGN catalog (Liu et al. 2022, Paper I). The AGN are selected either by emission-line pairs characteristic of AGN or by single broad emission line, free of any photometric pre-selections (magnitude/color/morphology).… ▽ More We present the LyA emission line luminosity function (LF) of the Active Galactic Nuclei (AGN) in the first release of the Hobby-Eberly Telescope Dark Energy Experiment Survey (HETDEX) AGN catalog (Liu et al. 2022, Paper I). The AGN are selected either by emission-line pairs characteristic of AGN or by single broad emission line, free of any photometric pre-selections (magnitude/color/morphology). The sample consists of 2,346 AGN spanning 1.88<z<3.53, covering an effective area of 30.61 deg^2. Approximately 2.6 of the HETDEX AGN are not detected at $>5σ$ confidence at r~26 in the deepest $r$-band images we have searched. The LyA line luminosity ranges from ~10^42.3 to ~10^45.9 erg s^-1. Our LyA LF shows a turnover luminosity with opposite slopes on the bright end and the faint end: The space density is highest at L_LyA^*=10^43.4 erg s^-1. We explore the evolution of the AGN LF over a broader redshift range (0.8<z<3); constructing the rest-frame ultraviolet (UV) LF with the 1450 AA monochromatic luminosity of the power-law component of the continuum ($\rm M_{1450}$) from M_1450~-18 to ~-27.5. We divide the sample into three redshift bins (z~1.5, 2.1, and 2.6). In all three redshift bins, our UV LFs indicate that the space density of AGN is highest at the turnover luminosity M_1450^* with opposite slopes on the bright end and the faint end. The M_1450 LFs in the three redshift bins can be well-fit with a luminosity-evolution-density-evolution (LEDE) model: the turnover luminosity (M_1450^*) increases and the turnover density (Phi^*) decreases with increasing redshift. △ Less

Submitted 24 July, 2022; originally announced July 2022.

Comments: 19 pages, 13 figures, 4 tables, accepted for publication in ApJ

Lyα Halos around [O III]-Selected Galaxies in HETDEX

Authors: Maja Lujan Niemeyer, William P. Bowman, Robin Ciardullo, Max Gronke, Eiichiro Komatsu, Maximilian Fabricius, Daniel J. Farrow, Steven L. Finkelstein, Karl Gebhardt, Caryl Gronwall, Gary J. Hill, Chenxu Liu, Erin Mentuch Cooper, Donald P. Schneider, Sarah Tuttle, Gregory R. Zeimann

Abstract: We present extended Lyman-α (Lyα) emission out to 800 kpc of 1034 [O III]-selected galaxies at redshifts 1.9<z<2.35 using the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX). The locations and redshifts of the galaxies are taken from the 3D-HST survey. The median-stacked surface brightness profile of Lyα emission of the [O III]-selected galaxies agrees well with that of 968 bright Lyα-emitt… ▽ More We present extended Lyman-α (Lyα) emission out to 800 kpc of 1034 [O III]-selected galaxies at redshifts 1.9<z<2.35 using the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX). The locations and redshifts of the galaxies are taken from the 3D-HST survey. The median-stacked surface brightness profile of Lyα emission of the [O III]-selected galaxies agrees well with that of 968 bright Lyα-emitting galaxies (LAEs) at r>40 kpc from the galaxy centers. The surface brightness in the inner parts (r<10 kpc) around the [O III]-selected galaxies, however, is ten times fainter than that of the LAEs. Our results are consistent with the notion that photons dominating the outer regions of the Lyα halos are not produced in the central galaxies but originate outside of them. △ Less

Submitted 22 July, 2022; originally announced July 2022.

Comments: 7 pages, 3 figures, accepted by Astrophysical Journal Letters

The Active Galactic Nuclei in the Hobby-Eberly Telescope Dark Energy Experiment Survey (HETDEX) I. Sample selection

Authors: Chenxu Liu, Karl Gebhardt, Erin Mentuch Cooper, Dustin Davis, Donald P. Schneider, Robin Ciardullo, Daniel J. Farrow, Steven L. Finkelstein, Caryl Gronwall, Yuchen Guo, Gary J. Hill, Lindsay House, Donghui Jeong, Shardha Jogee, Wolfram Kollatschny, Mirko Krumpe, Martin Landriau, Oscar A Chavez Ortiz, Yechi Zhang

Abstract: We present the first Active Galactic Nuclei (AGN) catalog in the Hobby-Eberly Telescope Dark Energy Experiment Survey (HETDEX) observed between January 2017 and June 2020. HETDEX is an ongoing spectroscopic survey with no pre-selection based on magnitudes, colors or morphologies, enabling us to select AGN based on their spectral features. Both luminous quasars and low-luminosity Seyferts are found… ▽ More We present the first Active Galactic Nuclei (AGN) catalog in the Hobby-Eberly Telescope Dark Energy Experiment Survey (HETDEX) observed between January 2017 and June 2020. HETDEX is an ongoing spectroscopic survey with no pre-selection based on magnitudes, colors or morphologies, enabling us to select AGN based on their spectral features. Both luminous quasars and low-luminosity Seyferts are found in our catalog. AGN candidates are selected with at least two significant AGN emission lines, such as the LyA and CIV line pair, or with single broad emission lines (FWHM > 1000 km/s). Each source is further confirmed by visual inspections. This catalog contains 5,322 AGN, covering an effective sky coverage of 30.61 deg^2. A total of 3,733 of these AGN have secure redshifts, and we provide redshift estimates for the remaining 1,589 single broad-line AGN with no cross matched spectral redshifts from SDSS DR14Q. The redshift range of the AGN catalog is 0.25 < z < 4.32, with a median of z = 2.1. The bolometric luminosity range is 10^9-10^14 Lsun with a median of 10^12 Lsun. The median r-band magnitude of the AGN is 21.6 mag, with 34% of the AGN have r > 22.5, and 2.6% reaching the detection limit at r ~ 26 mag of the deepest imaging surveys we searched. We also provide a composite spectrum of the AGN sample covering 700 AA - 4400 AA. △ Less

Submitted 29 April, 2022; v1 submitted 28 April, 2022; originally announced April 2022.

Comments: Accepted for publication in The Astrophysical Journal Supplement Series, 34 pages, 19 figures, 3 tables

The PartialSpoof Database and Countermeasures for the Detection of Short Fake Speech Segments Embedded in an Utterance

Authors: Lin Zhang, Xin Wang, Erica Cooper, Nicholas Evans, Junichi Yamagishi

Abstract: Automatic speaker verification is susceptible to various manipulations and spoofing, such as text-to-speech synthesis, voice conversion, replay, tampering, adversarial attacks, and so on. We consider a new spoofing scenario called "Partial Spoof" (PS) in which synthesized or transformed speech segments are embedded into a bona fide utterance. While existing countermeasures (CMs) can detect fully s… ▽ More Automatic speaker verification is susceptible to various manipulations and spoofing, such as text-to-speech synthesis, voice conversion, replay, tampering, adversarial attacks, and so on. We consider a new spoofing scenario called "Partial Spoof" (PS) in which synthesized or transformed speech segments are embedded into a bona fide utterance. While existing countermeasures (CMs) can detect fully spoofed utterances, there is a need for their adaptation or extension to the PS scenario. We propose various improvements to construct a significantly more accurate CM that can detect and locate short-generated spoofed speech segments at finer temporal resolutions. First, we introduce newly developed self-supervised pre-trained models as enhanced feature extractors. Second, we extend our PartialSpoof database by adding segment labels for various temporal resolutions. Since the short spoofed speech segments to be embedded by attackers are of variable length, six different temporal resolutions are considered, ranging from as short as 20 ms to as large as 640 ms. Third, we propose a new CM that enables the simultaneous use of the segment-level labels at different temporal resolutions as well as utterance-level labels to execute utterance- and segment-level detection at the same time. We also show that the proposed CM is capable of detecting spoofing at the utterance level with low error rates in the PS scenario as well as in a related logical access (LA) scenario. The equal error rates of utterance-level detection on the PartialSpoof database and ASVspoof 2019 LA database were 0.77 and 0.90%, respectively. △ Less

Submitted 30 January, 2023; v1 submitted 11 April, 2022; originally announced April 2022.

Comments: Published in IEEE/ACM Transactions on Audio, Speech, and Language Processing (DOI: 10.1109/TASLP.2022.3233236)

Journal ref: IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 813-825, 2023

Analyzing Language-Independent Speaker Anonymization Framework under Unseen Conditions

Authors: Xiaoxiao Miao, Xin Wang, Erica Cooper, Junichi Yamagishi, Natalia Tomashenko

Abstract: In our previous work, we proposed a language-independent speaker anonymization system based on self-supervised learning models. Although the system can anonymize speech data of any language, the anonymization was imperfect, and the speech content of the anonymized speech was distorted. This limitation is more severe when the input speech is from a domain unseen in the training data. This study ana… ▽ More In our previous work, we proposed a language-independent speaker anonymization system based on self-supervised learning models. Although the system can anonymize speech data of any language, the anonymization was imperfect, and the speech content of the anonymized speech was distorted. This limitation is more severe when the input speech is from a domain unseen in the training data. This study analyzed the bottleneck of the anonymization system under unseen conditions. It was found that the domain (e.g., language and channel) mismatch between the training and test data affected the neural waveform vocoder and anonymized speaker vectors, which limited the performance of the whole system. Increasing the training data diversity for the vocoder was found to be helpful to reduce its implicit language and channel dependency. Furthermore, a simple correlation-alignment-based domain adaption strategy was found to be significantly effective to alleviate the mismatch on the anonymized speaker vectors. Audio samples and source code are available online. △ Less

Submitted 28 March, 2022; originally announced March 2022.

Comments: Submit to Interspeech2022

The VoiceMOS Challenge 2022

Authors: Wen-Chin Huang, Erica Cooper, Yu Tsao, Hsin-Min Wang, Tomoki Toda, Junichi Yamagishi

Abstract: We present the first edition of the VoiceMOS Challenge, a scientific event that aims to promote the study of automatic prediction of the mean opinion score (MOS) of synthetic speech. This challenge drew 22 participating teams from academia and industry who tried a variety of approaches to tackle the problem of predicting human ratings of synthesized speech. The listening test data for the main tra… ▽ More We present the first edition of the VoiceMOS Challenge, a scientific event that aims to promote the study of automatic prediction of the mean opinion score (MOS) of synthetic speech. This challenge drew 22 participating teams from academia and industry who tried a variety of approaches to tackle the problem of predicting human ratings of synthesized speech. The listening test data for the main track of the challenge consisted of samples from 187 different text-to-speech and voice conversion systems spanning over a decade of research, and the out-of-domain track consisted of data from more recent systems rated in a separate listening test. Results of the challenge show the effectiveness of fine-tuning self-supervised speech models for the MOS prediction task, as well as the difficulty of predicting MOS ratings for unseen speakers and listeners, and for unseen systems in the out-of-domain setting. △ Less

Submitted 3 July, 2022; v1 submitted 21 March, 2022; originally announced March 2022.

Comments: Accepted to Interspeech 2022

Surface Brightness Profile of Lyman-$α$ Halos out to 320 kpc in HETDEX

Authors: Maja Lujan Niemeyer, Eiichiro Komatsu, Chris Byrohl, Dustin Davis, Maximilian Fabricius, Karl Gebhardt, Gary J. Hill, Lutz Wisotzki, William P. Bowman, Robin Ciardullo, Daniel J. Farrow, Steven L. Finkelstein, Eric Gawiser, Caryl Gronwall, Donghui Jeong, Martin Landriau, Chenxu Liu, Erin Mentuch Cooper, Masami Ouchi, Donald P. Schneider, Gregory R. Zeimann

Abstract: We present the median-stacked Lyman-$α$ surface brightness profile of 968 spectroscopically selected Lyman-$α$ emitting galaxies (LAEs) at redshifts $1.9<z<3.5$ in the early data of the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX). The selected LAEs are high-confidence Lyman-$α$ detections with large signal-to-noise ratios observed with good seeing conditions (point-spread-function full-… ▽ More We present the median-stacked Lyman-$α$ surface brightness profile of 968 spectroscopically selected Lyman-$α$ emitting galaxies (LAEs) at redshifts $1.9<z<3.5$ in the early data of the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX). The selected LAEs are high-confidence Lyman-$α$ detections with large signal-to-noise ratios observed with good seeing conditions (point-spread-function full-width-at-half-maximum $<1.4"$), excluding active galactic nuclei (AGN). The Lyman-$α$ luminosities of the LAEs are $10^{42.4}-10^{43}\, \mathrm{erg}\, \mathrm{s}^{-1}$. We detect faint emission in the median-stacked radial profiles at the level of $(3.6\pm 1.3)\times 10^{-20}\,\mathrm{erg}\,\mathrm{s}^{-1}\,\mathrm{cm}^{-2}\,\mathrm{arcsec}^{-2}$ from the surrounding Lyman-$α$ halos out to $r\simeq 160$ kpc (physical). The shape of the median-stacked radial profile is consistent at $r<80\,\mathrm{kpc}$ with that of much fainter LAEs at $3<z<4$ observed with the Multi Unit Spectroscopic Explorer (MUSE), indicating that the median-stacked Lyman-$α$ profiles have similar shapes at redshifts $2<z<4$ and across a factor of $10$ in Lyman-$α$ luminosity. While we agree with the results from the MUSE sample at $r<80\,\mathrm{kpc}$, we extend the profile over a factor of two in radius. At $r>80\,\mathrm{kpc}$, our profile is flatter than the MUSE model. The measured profile agrees at most radii with that of galaxies in the Byrohl et al. (2021) cosmological radiative transfer simulation at $z=3$. This suggests that the surface brightness of a Lyman-$α$ halo at $r\lesssim 100$ kpc is dominated by resonant scattering of Lyman-$α$ photons from star-forming regions in the central galaxy, whereas at $r > 100$ kpc it is dominated by photons from galaxies in surrounding dark matter halos. △ Less

Submitted 9 March, 2022; originally announced March 2022.

Comments: 15 pages, 7 figures, accepted by The Astrophysical Journal

Language-Independent Speaker Anonymization Approach using Self-Supervised Pre-Trained Models

Authors: Xiaoxiao Miao, Xin Wang, Erica Cooper, Junichi Yamagishi, Natalia Tomashenko

Abstract: Speaker anonymization aims to protect the privacy of speakers while preserving spoken linguistic information from speech. Current mainstream neural network speaker anonymization systems are complicated, containing an F0 extractor, speaker encoder, automatic speech recognition acoustic model (ASR AM), speech synthesis acoustic model and speech waveform generation model. Moreover, as an ASR AM is la… ▽ More Speaker anonymization aims to protect the privacy of speakers while preserving spoken linguistic information from speech. Current mainstream neural network speaker anonymization systems are complicated, containing an F0 extractor, speaker encoder, automatic speech recognition acoustic model (ASR AM), speech synthesis acoustic model and speech waveform generation model. Moreover, as an ASR AM is language-dependent, trained on English data, it is hard to adapt it into another language. In this paper, we propose a simpler self-supervised learning (SSL)-based method for language-independent speaker anonymization without any explicit language-dependent model, which can be easily used for other languages. Extensive experiments were conducted on the VoicePrivacy Challenge 2020 datasets in English and AISHELL-3 datasets in Mandarin to demonstrate the effectiveness of our proposed SSL-based language-independent speaker anonymization method. △ Less

Submitted 27 April, 2022; v1 submitted 26 February, 2022; originally announced February 2022.

Magnetic order, disorder, and excitations under pressure in the Mott insulator Sr$_2$IrO$_4$

Authors: Xiang Li, S. E. Cooper, A. Krishnadas, A. de la Torre, R. S. Perry, F. Baumberger, D. M. Silevitch, D. Hsieh, T. F. Rosenbaum, Yejun Feng

Abstract: Protected by the interplay of on-site Coulomb interactions and spin-orbit coupling, Sr$_2$IrO$_4$ at high pressure is a rare example of a Mott insulator with a paramagnetic ground state. Here, using optical Raman scattering, we measure both the phonon and magnon evolution in Sr$_2$IrO$_4$ under pressure, and identify three different magnetically-ordered phases, culminating in a spin-disordered sta… ▽ More Protected by the interplay of on-site Coulomb interactions and spin-orbit coupling, Sr$_2$IrO$_4$ at high pressure is a rare example of a Mott insulator with a paramagnetic ground state. Here, using optical Raman scattering, we measure both the phonon and magnon evolution in Sr$_2$IrO$_4$ under pressure, and identify three different magnetically-ordered phases, culminating in a spin-disordered state beyond 18 GPa. A strong first-order structural phase transition drives the magnetic evolution at $\sim$10 GPa with reduced structural anisotropy in the IrO$_6$ cages, leading to increasingly isotropic exchange interactions between the Heisenberg spins and a spin-flip transition to $c$-axis-aligned antiferromagnetic order. In the disordered phase of Heisenberg $J_\mathrm{eff}=1/2$ pseudospins, the spin excitations are quasi-elastic and continuous to 10 meV, potentially hosting a gapless quantum spin liquid in Sr$_2$IrO$_4$. △ Less

Submitted 22 October, 2021; originally announced October 2021.

Journal ref: Physical Review B 114, L201111 (2021)

LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speech

Authors: Wen-Chin Huang, Erica Cooper, Junichi Yamagishi, Tomoki Toda

Abstract: An effective approach to automatically predict the subjective rating for synthetic speech is to train on a listening test dataset with human-annotated scores. Although each speech sample in the dataset is rated by several listeners, most previous works only used the mean score as the training target. In this work, we present LDNet, a unified framework for mean opinion score (MOS) prediction that p… ▽ More An effective approach to automatically predict the subjective rating for synthetic speech is to train on a listening test dataset with human-annotated scores. Although each speech sample in the dataset is rated by several listeners, most previous works only used the mean score as the training target. In this work, we present LDNet, a unified framework for mean opinion score (MOS) prediction that predicts the listener-wise perceived quality given the input speech and the listener identity. We reflect recent advances in LD modeling, including design choices of the model architecture, and propose two inference methods that provide more stable results and efficient computation. We conduct systematic experiments on the voice conversion challenge (VCC) 2018 benchmark and a newly collected large-scale MOS dataset, providing an in-depth analysis of the proposed framework. Results show that the mean listener inference method is a better way to utilize the mean scores, whose effectiveness is more obvious when having more ratings per sample. △ Less

Submitted 18 October, 2021; originally announced October 2021.

Comments: Submitted to ICASSP 2022. Code available at: https://github.com/unilight/LDNet

The Hobby-Eberly Telescope Dark Energy Experiment (HETDEX) Survey Design, Reductions, and Detections

Authors: Karl Gebhardt, Erin Mentuch Cooper, Robin Ciardullo, Viviana Acquaviva, Ralf Bender, William P. Bowman, Barbara G. Castanheira, Gavin Dalton, Dustin Davis, Roelof S. de Jong, D. L. DePoy, Yaswant Devarakonda, Sun Dongsheng, Niv Drory, Maximilian Fabricius, Daniel J. Farrow, John Feldmeier, Steven L. Finkelstein, Cynthia S. Froning, Eric Gawiser, Caryl Gronwall, Laura Herold, Gary J. Hill, Ulrich Hopp, Lindsay R. House , et al. (38 additional authors not shown)

Abstract: We describe the survey design, calibration, commissioning, and emission-line detection algorithms for the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX). The goal of HETDEX is to measure the redshifts of over a million Ly$α$ emitting galaxies between 1.88<z<3.52, in a 540 deg^2 area encompassing a co-moving volume of 10.9 Gpc^3. No pre-selection of targets is involved; instead the HETDEX m… ▽ More We describe the survey design, calibration, commissioning, and emission-line detection algorithms for the Hobby-Eberly Telescope Dark Energy Experiment (HETDEX). The goal of HETDEX is to measure the redshifts of over a million Ly$α$ emitting galaxies between 1.88<z<3.52, in a 540 deg^2 area encompassing a co-moving volume of 10.9 Gpc^3. No pre-selection of targets is involved; instead the HETDEX measurements are accomplished via a spectroscopic survey using a suite of wide-field integral field units distributed over the focal plane of the telescope. This survey measures the Hubble expansion parameter and angular diameter distance, with a final expected accuracy of better than 1%. We detail the project's observational strategy, reduction pipeline, source detection, and catalog generation, and present initial results for science verification in the COSMOS, Extended Groth Strip, and GOODS-N fields. We demonstrate that our data reach the required specifications in throughput, astrometric accuracy, flux limit, and object detection, with the end products being a catalog of emission-line sources, their object classifications, and flux-calibrated spectra. △ Less

Submitted 7 October, 2021; originally announced October 2021.

Comments: 51 pages, accepted for publication in the Astrophysical Journal

The HETDEX Instrumentation: Hobby-Eberly Telescope Wide Field Upgrade and VIRUS

Authors: Gary J. Hill, Hanshin Lee, Phillip J. MacQueen, Andreas Kelz, Niv Drory, Brian L. Vattiat, John M. Good, Jason Ramsey, Herman Kriel, Trent Peterson, D. L. DePoy, Karl Gebhardt, J. L. Marshall, Sarah E. Tuttle, Svend M. Bauer, Taylor S. Chonis, Maximilian H. Fabricius, Cynthia Froning, Marco Haeuser, Briana L. Indahl, Thomas Jahn, Martin Landriau, Ron Leck, Francesco Montesano, Travis Prochaska , et al. (24 additional authors not shown)

Abstract: The Hobby-Eberly Telescope (HET) Dark Energy Experiment (HETDEX) is undertaking a blind wide-field low-resolution spectroscopic survey of 540 square degrees of sky to identify and derive redshifts for a million Lyman-alpha emitting galaxies (LAEs) in the redshift range 1.9 < z < 3.5. The ultimate goal is to measure the expansion rate of the Universe at this epoch, to sharply constrain cosmological… ▽ More The Hobby-Eberly Telescope (HET) Dark Energy Experiment (HETDEX) is undertaking a blind wide-field low-resolution spectroscopic survey of 540 square degrees of sky to identify and derive redshifts for a million Lyman-alpha emitting galaxies (LAEs) in the redshift range 1.9 < z < 3.5. The ultimate goal is to measure the expansion rate of the Universe at this epoch, to sharply constrain cosmological parameters and thus the nature of dark energy. A major multi-year wide field upgrade (WFU) of the HET was completed in 2016 that substantially increased the field of view to 22 arcminutes diameter and the pupil to 10 meters, by replacing the optical corrector, tracker, and prime focus instrument package and by develo** a new telescope control system. The new, wide-field HET now feeds the Visible Integral-field Replicable Unit Spectrograph (VIRUS), a new low-resolution integral field spectrograph (LRS2), and the Habitable Zone Planet Finder (HPF), a precision near-infrared radial velocity spectrograph. VIRUS consists of 156 identical spectrographs fed by almost 35,000 fibers in 78 integral field units arrayed at the focus of the upgraded HET. VIRUS operates in a bandpass of 3500-5500 Angstroms with resolving power R~800. VIRUS is the first example of large scale replication applied to instrumentation in optical astronomy to achieve spectroscopic surveys of very large areas of sky. This paper presents technical details of the HET WFU and VIRUS, as flowed-down from the HETDEX science requirements, along with experience from commissioning this major telescope upgrade and the innovative instrumentation suite for HETDEX. △ Less

Submitted 7 December, 2021; v1 submitted 7 October, 2021; originally announced October 2021.

Comments: 65 pages, 25 figures, published in the Astronomical Journal; replaced with final published version

Journal ref: AJ 162 298 (2021)

Generalization Ability of MOS Prediction Networks

Authors: Erica Cooper, Wen-Chin Huang, Tomoki Toda, Junichi Yamagishi

Abstract: Automatic methods to predict listener opinions of synthesized speech remain elusive since listeners, systems being evaluated, characteristics of the speech, and even the instructions given and the rating scale all vary from test to test. While automatic predictors for metrics such as mean opinion score (MOS) can achieve high prediction accuracy on samples from the same test, they typically fail to… ▽ More Automatic methods to predict listener opinions of synthesized speech remain elusive since listeners, systems being evaluated, characteristics of the speech, and even the instructions given and the rating scale all vary from test to test. While automatic predictors for metrics such as mean opinion score (MOS) can achieve high prediction accuracy on samples from the same test, they typically fail to generalize well to new listening test contexts. In this paper, using a variety of networks for MOS prediction including MOSNet and self-supervised speech models such as wav2vec2, we investigate their performance on data from different listening tests in both zero-shot and fine-tuned settings. We find that wav2vec2 models fine-tuned for MOS prediction have good generalization capability to out-of-domain data even for the most challenging case of utterance-level predictions in the zero-shot setting, and that fine-tuning to in-domain data can improve predictions. We also observe that unseen systems are especially challenging for MOS prediction models. △ Less

Submitted 14 February, 2022; v1 submitted 6 October, 2021; originally announced October 2021.

Comments: \c{opyright} 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

On the Interplay Between Sparsity, Naturalness, Intelligibility, and Prosody in Speech Synthesis

Authors: Cheng-I Jeff Lai, Erica Cooper, Yang Zhang, Shiyu Chang, Kaizhi Qian, Yi-Lun Liao, Yung-Sung Chuang, Alexander H. Liu, Junichi Yamagishi, David Cox, James Glass

Abstract: Are end-to-end text-to-speech (TTS) models over-parametrized? To what extent can these models be pruned, and what happens to their synthesis capabilities? This work serves as a starting point to explore pruning both spectrogram prediction networks and vocoders. We thoroughly investigate the tradeoffs between sparsity and its subsequent effects on synthetic speech. Additionally, we explored several… ▽ More Are end-to-end text-to-speech (TTS) models over-parametrized? To what extent can these models be pruned, and what happens to their synthesis capabilities? This work serves as a starting point to explore pruning both spectrogram prediction networks and vocoders. We thoroughly investigate the tradeoffs between sparsity and its subsequent effects on synthetic speech. Additionally, we explored several aspects of TTS pruning: amount of finetuning data versus sparsity, TTS-Augmentation to utilize unspoken text, and combining knowledge distillation and pruning. Our findings suggest that not only are end-to-end TTS models highly prunable, but also, perhaps surprisingly, pruned TTS models can produce synthetic speech with equal or higher naturalness and intelligibility, with similar prosody. All of our experiments are conducted on publicly available models, and findings in this work are backed by large-scale subjective tests and objective measures. Code and 200 pruned models are made available to facilitate future research on efficiency in TTS. △ Less

Submitted 27 October, 2021; v1 submitted 3 October, 2021; originally announced October 2021.