-
End-to-End Joint Target and Non-Target Speakers ASR
Authors:
Ryo Masumura,
Naoki Makishima,
Taiga Yamane,
Yoshihiko Yamazaki,
Saki Mizuno,
Mana Ihori,
Mihiro Uchida,
Keita Suzuki,
Hiroshi Sato,
Tomohiro Tanaka,
Akihiko Takashima,
Satoshi Suzuki,
Takafumi Moriya,
Nobukatsu Hojo,
Atsushi Ando
Abstract:
This paper proposes a novel automatic speech recognition (ASR) system that can transcribe individual speaker's speech while identifying whether they are target or non-target speakers from multi-talker overlapped speech. Target-speaker ASR systems are a promising way to only transcribe a target speaker's speech by enrolling the target speaker's information. However, in conversational ASR applicatio…
▽ More
This paper proposes a novel automatic speech recognition (ASR) system that can transcribe individual speaker's speech while identifying whether they are target or non-target speakers from multi-talker overlapped speech. Target-speaker ASR systems are a promising way to only transcribe a target speaker's speech by enrolling the target speaker's information. However, in conversational ASR applications, transcribing both the target speaker's speech and non-target speakers' ones is often required to understand interactive information. To naturally consider both target and non-target speakers in a single ASR model, our idea is to extend autoregressive modeling-based multi-talker ASR systems to utilize the enrollment speech of the target speaker. Our proposed ASR is performed by recursively generating both textual tokens and tokens that represent target or non-target speakers. Our experiments demonstrate the effectiveness of our proposed method.
△ Less
Submitted 4 June, 2023;
originally announced June 2023.
-
Downstream Task Agnostic Speech Enhancement with Self-Supervised Representation Loss
Authors:
Hiroshi Sato,
Ryo Masumura,
Tsubasa Ochiai,
Marc Delcroix,
Takafumi Moriya,
Takanori Ashihara,
Kentaro Shinayama,
Saki Mizuno,
Mana Ihori,
Tomohiro Tanaka,
Nobukatsu Hojo
Abstract:
Self-supervised learning (SSL) is the latest breakthrough in speech processing, especially for label-scarce downstream tasks by leveraging massive unlabeled audio data. The noise robustness of the SSL is one of the important challenges to expanding its application. We can use speech enhancement (SE) to tackle this issue. However, the mismatch between the SE model and SSL models potentially limits…
▽ More
Self-supervised learning (SSL) is the latest breakthrough in speech processing, especially for label-scarce downstream tasks by leveraging massive unlabeled audio data. The noise robustness of the SSL is one of the important challenges to expanding its application. We can use speech enhancement (SE) to tackle this issue. However, the mismatch between the SE model and SSL models potentially limits its effect. In this work, we propose a new SE training criterion that minimizes the distance between clean and enhanced signals in the feature representation of the SSL model to alleviate the mismatch. We expect that the loss in the SSL domain could guide SE training to preserve or enhance various levels of characteristics of the speech signals that may be required for high-level downstream tasks. Experiments show that our proposal improves the performance of an SE and SSL pipeline on five downstream tasks with noisy input while maintaining the SE performance.
△ Less
Submitted 24 May, 2023;
originally announced May 2023.
-
A 3-approximation list scheduling algorithm for a single-machine scheduling problem with a non-renewable resource and total weighted completion time criterion
Authors:
Susumu Hashimoto,
Shinji Mizuno
Abstract:
This paper studies a single-machine scheduling problem with a non-renewable resource (NR-SSP) and total weighted completion time criterion. The non-renewable resource is consumed when the machine starts processing a job. We consider the case where each job's weight in the objective function is proportional to its resource consumption amount. The problem is known to be NP-hard in this case. We prop…
▽ More
This paper studies a single-machine scheduling problem with a non-renewable resource (NR-SSP) and total weighted completion time criterion. The non-renewable resource is consumed when the machine starts processing a job. We consider the case where each job's weight in the objective function is proportional to its resource consumption amount. The problem is known to be NP-hard in this case. We propose a 3-approximation list scheduling algorithm for this problem. Besides, we show that the approximation ratio 3 is tight for the algorithm.
△ Less
Submitted 10 May, 2021;
originally announced May 2021.
-
On a reduction of the weighted induced bipartite subgraph problem to the weighted independent set problem
Authors:
Yotaro Takazawa,
Shinji Mizuno
Abstract:
We study the weighted induced bipartite subgraph problem (WIBSP). The goal of WIBSP is, given a graph and nonnegative weights for the nodes, to find a set W of nodes with the maximum total weight such that a subgraph induced by W is bipartite. WIBSP is also referred as to the graph bipartization problem or the odd cycle transversal problem. In this paper, we show that WIBSP can be reduced to the w…
▽ More
We study the weighted induced bipartite subgraph problem (WIBSP). The goal of WIBSP is, given a graph and nonnegative weights for the nodes, to find a set W of nodes with the maximum total weight such that a subgraph induced by W is bipartite. WIBSP is also referred as to the graph bipartization problem or the odd cycle transversal problem. In this paper, we show that WIBSP can be reduced to the weighted independent set problem (WISP) where the number of nodes becomes twice and the maximum degree increase by 1. WISP is a well-studied combinatorial optimization problem. Thus, by using the reduction and results about WISP, we can obtain nontrivial approximation and exact algorithms for WIBSP.
△ Less
Submitted 25 July, 2018;
originally announced July 2018.
