-
MR4MR: Mixed Reality for Melody Reincarnation
Authors:
Atsuya Kobayashi,
Ryogo Ishino,
Ryuku Nobusue,
Takumi Inoue,
Keisuke Okazaki,
Shoma Sawa,
Nao Tokui
Abstract:
There is a long history of an effort made to explore musical elements with the entities and spaces around us, such as musique concrète and ambient music. In the context of computer music and digital art, interactive experiences that concentrate on the surrounding objects and physical spaces have also been designed. In recent years, with the development and popularization of devices, an increasing…
▽ More
There is a long history of an effort made to explore musical elements with the entities and spaces around us, such as musique concrète and ambient music. In the context of computer music and digital art, interactive experiences that concentrate on the surrounding objects and physical spaces have also been designed. In recent years, with the development and popularization of devices, an increasing number of works have been designed in Extended Reality to create such musical experiences. In this paper, we describe MR4MR, a sound installation work that allows users to experience melodies produced from interactions with their surrounding space in the context of Mixed Reality (MR). Using HoloLens, an MR head-mounted display, users can bump virtual objects that emit sound against real objects in their surroundings. Then, by continuously creating a melody following the sound made by the object and re-generating randomly and gradually changing melody using music generation machine learning models, users can feel their ambient melody "reincarnating".
△ Less
Submitted 14 September, 2022;
originally announced September 2022.
-
ExKaldi-RT: A Real-Time Automatic Speech Recognition Extension Toolkit of Kaldi
Authors:
Yu Wang,
Chee Siang Leow,
Akio Kobayashi,
Takehito Utsuro,
Hiromitsu Nishizaki
Abstract:
This paper describes the ExKaldi-RT online automatic speech recognition (ASR) toolkit that is implemented based on the Kaldi ASR toolkit and Python language. ExKaldi-RT provides tools for building online recognition pipelines. While similar tools are available built on Kaldi, a key feature of ExKaldi-RT that it works on Python, which has an easy-to-use interface that allows online ASR system devel…
▽ More
This paper describes the ExKaldi-RT online automatic speech recognition (ASR) toolkit that is implemented based on the Kaldi ASR toolkit and Python language. ExKaldi-RT provides tools for building online recognition pipelines. While similar tools are available built on Kaldi, a key feature of ExKaldi-RT that it works on Python, which has an easy-to-use interface that allows online ASR system developers to develop original research, such as by applying neural network-based signal processing and by decoding model trained with deep learning frameworks. We performed benchmark experiments on the minimum LibriSpeech corpus, and it showed that ExKaldi-RT could achieve competitive ASR performance in real-time recognition.
△ Less
Submitted 8 August, 2021; v1 submitted 3 April, 2021;
originally announced April 2021.
-
ExSampling: a system for the real-time ensemble performance of field-recorded environmental sounds
Authors:
Atsuya Kobayashi,
Reo Anzai,
Nao Tokui
Abstract:
We propose ExSampling: an integrated system of recording application and Deep Learning environment for a real-time music performance of environmental sounds sampled by field recording. Automated sound map** to Ableton Live tracks by Deep Learning enables field recording to be applied to real-time performance, and create interactions among sound recorders, composers and performers.
We propose ExSampling: an integrated system of recording application and Deep Learning environment for a real-time music performance of environmental sounds sampled by field recording. Automated sound map** to Ableton Live tracks by Deep Learning enables field recording to be applied to real-time performance, and create interactions among sound recorders, composers and performers.
△ Less
Submitted 17 June, 2020;
originally announced June 2020.
