Showing 1–1 of 1 results for author: Narita, G

GANStrument: Adversarial Instrument Sound Synthesis with Pitch-invariant Instance Conditioning

Authors: Gaku Narita, Junichi Shimizu, Taketo Akama

Abstract: We propose GANStrument, a generative adversarial model for instrument sound synthesis. Given a one-shot sound as input, it is able to generate pitched instrument sounds that reflect the timbre of the input within an interactive time. By exploiting instance conditioning, GANStrument achieves better fidelity and diversity of synthesized sounds and generalization ability to various inputs. In additio… ▽ More We propose GANStrument, a generative adversarial model for instrument sound synthesis. Given a one-shot sound as input, it is able to generate pitched instrument sounds that reflect the timbre of the input within an interactive time. By exploiting instance conditioning, GANStrument achieves better fidelity and diversity of synthesized sounds and generalization ability to various inputs. In addition, we introduce an adversarial training scheme for a pitch-invariant feature extractor that significantly improves the pitch accuracy and timbre consistency. Experimental results show that GANStrument outperforms strong baselines that do not use instance conditioning in terms of generation quality and input editability. Qualitative examples are available online. △ Less

Submitted 7 March, 2023; v1 submitted 10 November, 2022; originally announced November 2022.

Comments: 5 pages, 4 figures, Accepted to 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Audio examples: https://ganstrument.github.io/ganstrument-demo/