MM-KWS: Multi-modal Prompts for Multilingual User-defined Keyword Spotting
Authors:
Zhiqi Ai,
Zhiyong Chen,
Shugong Xu
Abstract:
In this paper, we propose MM-KWS, a novel approach to user-defined keyword spotting leveraging multi-modal enrollments of text and speech templates. Unlike previous methods that focus solely on either text or speech features, MM-KWS extracts phoneme, text, and speech embeddings from both modalities. These embeddings are then compared with the query speech embedding to detect the target keywords. T…
▽ More
In this paper, we propose MM-KWS, a novel approach to user-defined keyword spotting leveraging multi-modal enrollments of text and speech templates. Unlike previous methods that focus solely on either text or speech features, MM-KWS extracts phoneme, text, and speech embeddings from both modalities. These embeddings are then compared with the query speech embedding to detect the target keywords. To ensure the applicability of MM-KWS across diverse languages, we utilize a feature extractor incorporating several multilingual pre-trained models. Subsequently, we validate its effectiveness on Mandarin and English tasks. In addition, we have integrated advanced data augmentation tools for hard case mining to enhance MM-KWS in distinguishing confusable words. Experimental results on the LibriPhrase and WenetPhrase datasets demonstrate that MM-KWS outperforms prior methods significantly.
△ Less
Submitted 11 June, 2024;
originally announced June 2024.
Low-Power Wireless Wearable ECG Monitoring Chestbelt Based on Ferroelectric Microprocessor
Authors:
Zhendong Ai,
Zihan Wang,
Wei Cui
Abstract:
Since cadiovascular disease (CVD) posts a heavy threat to people's health, long-term electrocardiogram (ECG) monitoring is of great value for the improvement of treatment. To realize remote long-term ECG monitoring, a low-power wireless wearable ECG monitoring device is proposed in this paper. The ECG monitoring device, abbreviated as ECGM, is designed based on ferroelectric microprocessor which p…
▽ More
Since cadiovascular disease (CVD) posts a heavy threat to people's health, long-term electrocardiogram (ECG) monitoring is of great value for the improvement of treatment. To realize remote long-term ECG monitoring, a low-power wireless wearable ECG monitoring device is proposed in this paper. The ECG monitoring device, abbreviated as ECGM, is designed based on ferroelectric microprocessor which provides ultra-low power consumption and contains four parts-MCU, BLE, Sensors and Power. The MCU part means circuit of MSP430FR2433, the core of ECGM. The BLE part is the CC2640R2F module applied for wireless transmission of the collected bio-signal data. And the sensors part includes several sensors like BMD101 used for monitoring bio-signals and motion of the wearer, while the Power part consists of battery circuit, charging circuit and 3.3V/1.8V/4.4V power supply circuit. The ECGM first collects ECG signals from the fabric electrodes adhered to wearers' chest, preprocesses the signals to eliminate the injected noise, and then transmit the output data to wearers' hand-held mobile phones through Bluetooth low energy (BLE). The wearers are enabled to acquire ECGs and other physiological parameters on their phones as well as some corresponding suggestions. The novelty of the system lies in the combination of low-power ECG sensor chip with ferroelectric microprocessor, thus achieving ultra-low power consumption and high signal quality.
△ Less
Submitted 6 November, 2020;
originally announced December 2020.
