Subtractive Training for Music Stem Insertion using Latent Diffusion Models
Authors:
Ivan Villa-Renteria,
Mason L. Wang,
Zachary Shah,
Zhe Li,
Soohyun Kim,
Neelesh Ramachandran,
Mert Pilanci
Abstract:
We present Subtractive Training, a simple and novel method for synthesizing individual musical instrument stems given other instruments as context. This method pairs a dataset of complete music mixes with 1) a variant of the dataset lacking a specific stem, and 2) LLM-generated instructions describing how the missing stem should be reintroduced. We then fine-tune a pretrained text-to-audio diffusi…
▽ More
We present Subtractive Training, a simple and novel method for synthesizing individual musical instrument stems given other instruments as context. This method pairs a dataset of complete music mixes with 1) a variant of the dataset lacking a specific stem, and 2) LLM-generated instructions describing how the missing stem should be reintroduced. We then fine-tune a pretrained text-to-audio diffusion model to generate the missing instrument stem, guided by both the existing stems and the text instruction. Our results demonstrate Subtractive Training's efficacy in creating authentic drum stems that seamlessly blend with the existing tracks. We also show that we can use the text instruction to control the generation of the inserted stem in terms of rhythm, dynamics, and genre, allowing us to modify the style of a single instrument in a full song while kee** the remaining instruments the same. Lastly, we extend this technique to MIDI formats, successfully generating compatible bass, drum, and guitar parts for incomplete arrangements.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation
Authors:
Chengshu Li,
Ruohan Zhang,
Josiah Wong,
Cem Gokmen,
Sanjana Srivastava,
Roberto Martín-Martín,
Chen Wang,
Gabrael Levine,
Wensi Ai,
Benjamin Martinez,
Hang Yin,
Michael Lingelbach,
Minjune Hwang,
Ayano Hiranaka,
Sujay Garlanka,
Arman Aydin,
Sharon Lee,
Jiankai Sun,
Mona Anvari,
Manasi Sharma,
Dhruva Bansal,
Samuel Hunter,
Kyu-Young Kim,
Alan Lou,
Caleb R Matthews
, et al. (10 additional authors not shown)
Abstract:
We present BEHAVIOR-1K, a comprehensive simulation benchmark for human-centered robotics. BEHAVIOR-1K includes two components, guided and motivated by the results of an extensive survey on "what do you want robots to do for you?". The first is the definition of 1,000 everyday activities, grounded in 50 scenes (houses, gardens, restaurants, offices, etc.) with more than 9,000 objects annotated with…
▽ More
We present BEHAVIOR-1K, a comprehensive simulation benchmark for human-centered robotics. BEHAVIOR-1K includes two components, guided and motivated by the results of an extensive survey on "what do you want robots to do for you?". The first is the definition of 1,000 everyday activities, grounded in 50 scenes (houses, gardens, restaurants, offices, etc.) with more than 9,000 objects annotated with rich physical and semantic properties. The second is OMNIGIBSON, a novel simulation environment that supports these activities via realistic physics simulation and rendering of rigid bodies, deformable bodies, and liquids. Our experiments indicate that the activities in BEHAVIOR-1K are long-horizon and dependent on complex manipulation skills, both of which remain a challenge for even state-of-the-art robot learning solutions. To calibrate the simulation-to-reality gap of BEHAVIOR-1K, we provide an initial study on transferring solutions learned with a mobile manipulator in a simulated apartment to its real-world counterpart. We hope that BEHAVIOR-1K's human-grounded nature, diversity, and realism make it valuable for embodied AI and robot learning research. Project website: https://behavior.stanford.edu.
△ Less
Submitted 14 March, 2024;
originally announced March 2024.
Matching Markets
Authors:
Andrew Yang,
Bruce Changlong Xu,
Ivan Villa-Renteria
Abstract:
Matching markets are of particular interest in computer science and economics literature as they are often used to model real-world phenomena where we aim to equitably distribute a limited amount of resources to multiple agents and determine these distributions efficiently. Although it has been shown that finding market clearing prices for Fisher markets with indivisible goods is NP-hard, there ex…
▽ More
Matching markets are of particular interest in computer science and economics literature as they are often used to model real-world phenomena where we aim to equitably distribute a limited amount of resources to multiple agents and determine these distributions efficiently. Although it has been shown that finding market clearing prices for Fisher markets with indivisible goods is NP-hard, there exist polynomial-time algorithms able to compute these prices and allocations when the goods are divisible and the utility functions are linear. We provide a promising research direction toward the development of a market that simulates buyers' preferences that vary according to the bundles of goods allocated to other buyers. Our research aims to elucidate unique ways in which the theory of matching markets can be extended to account for more complex and often counterintuitive microeconomic phenomena.
△ Less
Submitted 30 September, 2021;
originally announced September 2021.
