Neural network-based virtual microphone estimation with virtual microphone and beamformer-level multi-task loss
Authors:
Hanako Segawa,
Tsubasa Ochiai,
Marc Delcroix,
Tomohiro Nakatani,
Rintaro Ikeshita,
Shoko Araki,
Takeshi Yamada,
Shoji Makino
Abstract:
Array processing performance depends on the number of microphones available. Virtual microphone estimation (VME) has been proposed to increase the number of microphone signals artificially. Neural network-based VME (NN-VME) trains an NN with a VM-level loss to predict a signal at a microphone location that is available during training but not at inference. However, this training objective may not…
▽ More
Array processing performance depends on the number of microphones available. Virtual microphone estimation (VME) has been proposed to increase the number of microphone signals artificially. Neural network-based VME (NN-VME) trains an NN with a VM-level loss to predict a signal at a microphone location that is available during training but not at inference. However, this training objective may not be optimal for a specific array processing back-end, such as beamforming. An alternative approach is to use a training objective considering the array-processing back-end, such as a loss on the beamformer output. This approach may generate signals optimal for beamforming but not physically grounded. To combine the advantages of both approaches, this paper proposes a multi-task loss for NN-VME that combines both VM-level and beamformer-level losses. We evaluate the proposed multi-task NN-VME on multi-talker underdetermined conditions and show that it achieves a 33.1 % relative WER improvement compared to using only real microphones and 10.8 % compared to using a prior NN-VME approach.
△ Less
Submitted 20 November, 2023;
originally announced November 2023.
Single Microhole per Pixel in CMOS Image Sensor with Enhanced Optical Sensitivity in Near-Infrared
Authors:
E. Ponizovskaya Devine,
Wayesh Qarony,
Ahasan Ahamed,
Ahmed S Mayet,
Soroush Ghandiparsi,
Cesar Bartolo-Perez,
Aly F Elrefaie,
Toshishige Yamada,
Shih-Yuan Wang,
M. Saif Islam
Abstract:
Silicon photodiode based CMOS sensors with backside-illumination for 300 to 1000 nm wavelength range were studied. We showed that a single hole in the photodiode increases the optical efficiency of the pixel. In near-infrared wavelengths, the enhancement allows 70% absorption in a 3 microns thick Si. It is 4x better than for the flat pixel. We compared different shapes and sizes of single holes an…
▽ More
Silicon photodiode based CMOS sensors with backside-illumination for 300 to 1000 nm wavelength range were studied. We showed that a single hole in the photodiode increases the optical efficiency of the pixel. In near-infrared wavelengths, the enhancement allows 70% absorption in a 3 microns thick Si. It is 4x better than for the flat pixel. We compared different shapes and sizes of single holes and holes arrays. We have shown that a certain size and shape in single holes pronounce better optical efficiency enhancement. The crosstalk was successfully reduced with trenches between pixels. We optimized the trenches to achieve minimal pixel separation for 1.12 microns pixel.
△ Less
Submitted 20 November, 2020;
originally announced November 2020.
