-
PianoMotion10M: Dataset and Benchmark for Hand Motion Generation in Piano Performance
Authors:
Qijun Gan,
Song Wang,
Shengtao Wu,
Jianke Zhu
Abstract:
Recently, artificial intelligence techniques for education have been received increasing attentions, while it still remains an open problem to design the effective music instrument instructing systems. Although key presses can be directly derived from sheet music, the transitional movements among key presses require more extensive guidance in piano performance. In this work, we construct a piano-h…
▽ More
Recently, artificial intelligence techniques for education have been received increasing attentions, while it still remains an open problem to design the effective music instrument instructing systems. Although key presses can be directly derived from sheet music, the transitional movements among key presses require more extensive guidance in piano performance. In this work, we construct a piano-hand motion generation benchmark to guide hand movements and fingerings for piano playing. To this end, we collect an annotated dataset, PianoMotion10M, consisting of 116 hours of piano playing videos from a bird's-eye view with 10 million annotated hand poses. We also introduce a powerful baseline model that generates hand motions from piano audios through a position predictor and a position-guided gesture generator. Furthermore, a series of evaluation metrics are designed to assess the performance of the baseline model, including motion similarity, smoothness, positional accuracy of left and right hands, and overall fidelity of movement distribution. Despite that piano key presses with respect to music scores or audios are already accessible, PianoMotion10M aims to provide guidance on piano fingering for instruction purposes. The dataset and source code can be accessed at https://agnjason.github.io/PianoMotion-page.
△ Less
Submitted 13 June, 2024;
originally announced June 2024.
-
A Fast Algorithm for Onboard Atmospheric Powered Descent Guidance
Authors:
Yushu Chen,
Guangwen Yang,
Lu Wang,
Qingzhong Gan,
Haipeng Chen,
Quanyong Xu
Abstract:
Atmospheric powered descent guidance can be solved by successive convexification; however, its onboard application is impeded by the sharp increase in computation caused by nonlinear aerodynamic forces. The problem has to be converted into a sequence of convex subproblems instead of a single convex problem when aerodynamic forces are ignored. Besides, each subproblem is significantly more complica…
▽ More
Atmospheric powered descent guidance can be solved by successive convexification; however, its onboard application is impeded by the sharp increase in computation caused by nonlinear aerodynamic forces. The problem has to be converted into a sequence of convex subproblems instead of a single convex problem when aerodynamic forces are ignored. Besides, each subproblem is significantly more complicated, which increases computation. A fast real-time interior point method was presented to solve the correlated convex subproblems onboard in the work. The main contributions are as follows: Firstly, an algorithm was proposed to accelerate the solution of linear systems that cost most of the computation in each iterative step by exploiting the specific problem structure. Secondly, a warm-starting scheme was introduced to refine the initial value of a subproblem with a rough approximate solution of the former subproblem, which lessened the iterative steps required for each subproblem. The method proposed reduced the run time by a factor of 9 compared with the fastest publicly available solver tested in Monte Carlo simulations to evaluate the efficiency of solvers. Runtimes on the order of 0.6 s are achieved on a radiation-hardened flight processor, which demonstrated the potential of the real-time onboard application.
△ Less
Submitted 6 June, 2023; v1 submitted 9 September, 2022;
originally announced September 2022.
-
Adaptive Coordination Offsets for Signalized Arterial Intersections using Deep Reinforcement Learning
Authors:
Keith Anshilo Diaz,
Damian Dailisan,
Umang Sharaf,
Carissa Santos,
Qijian Gan,
Francis Aldrine Uy,
May T. Lim,
Alexandre M. Bayen
Abstract:
Coordinating intersections in arterial networks is critical to the performance of urban transportation systems. Deep reinforcement learning (RL) has gained traction in traffic control research along with data-driven approaches for traffic control systems. To date, proposed deep RL-based traffic schemes control phase activation or duration. Yet, such approaches may bypass low volume links for sever…
▽ More
Coordinating intersections in arterial networks is critical to the performance of urban transportation systems. Deep reinforcement learning (RL) has gained traction in traffic control research along with data-driven approaches for traffic control systems. To date, proposed deep RL-based traffic schemes control phase activation or duration. Yet, such approaches may bypass low volume links for several cycles in order to optimize the network-level traffic flow. Here, we propose a deep RL framework that dynamically adjusts offsets based on traffic states and preserves the planned phase timings and order derived from model-based methods. This framework allows us to improve arterial coordination while maintaining phase order and timing predictability. Using a validated and calibrated traffic model, we trained the policy of a deep RL agent that aims to reduce travel delays in the network. We evaluated the resulting policy by comparing its performance against the phase offsets deployed along a segment of Huntington Drive in the city of Arcadia. The resulting policy dynamically readjusts phase offsets in response to changes in traffic demand. Simulation results show that the proposed deep RL agent outperformed the baseline on average, effectively reducing delay time by 13.21% in the AM Scenario, 2.42% in the Noon scenario, and 6.2% in the PM scenario when offsets are adjusted in 15-minute intervals. Finally, we also show the robustness of our agent to extreme traffic conditions, such as demand surges in off-peak hours and localized traffic incidents
△ Less
Submitted 29 August, 2022; v1 submitted 6 August, 2020;
originally announced August 2020.
-
Cooperative Raman Spectroscopy for Real-time In Vivo Nano-biosensing
Authors:
Hongzhi Guo,
Josep Miquel Jornet,
Qiaoqiang Gan,
Zhi Sun
Abstract:
In the last few decades, the development of miniature biological sensors that can detect and measure different phenomena at the nanoscale has led to transformative disease diagnosis and treatment techniques. Among others, biofunctional Raman nanoparticles have been utilized in vitro and in vivo for multiplexed diagnosis and detection of different biological agents. However, existing solutions requ…
▽ More
In the last few decades, the development of miniature biological sensors that can detect and measure different phenomena at the nanoscale has led to transformative disease diagnosis and treatment techniques. Among others, biofunctional Raman nanoparticles have been utilized in vitro and in vivo for multiplexed diagnosis and detection of different biological agents. However, existing solutions require the use of bulky lasers to excite the nanoparticles and similarly bulky and expensive spectrometers to measure the scattered Raman signals, which limit the practicality and applications of this nano-biosensing technique. In addition, due to the high path loss of the intra-body environment, the received signals are usually very weak, which hampers the accuracy of the measurements. In this paper, the concept of cooperative Raman spectrum reconstruction for real-time in vivo nano-biosensing is presented for the first time. The fundamental idea is to replace the single excitation and measurement points (i.e., the laser and the spectrometer, respectively) by a network of interconnected nano-devices that can simultaneously excite and measure nano-biosensing particles. More specifically, in the proposed system a large number of nanosensors jointly and distributively collect the Raman response of nano-biofunctional nanoparticles (NBPs) traveling through the blood vessels. This paper presents a detailed description of the sensing system and, more importantly, proves its feasibility, by utilizing accurate models of optical signal propagation in intra-body environment and low-complexity estimation algorithms. The numerical results show that with a certain density of NBPs, the reconstructed Raman spectrum can be recovered and utilized to accurately extract the targeting intra-body information.
△ Less
Submitted 26 March, 2017;
originally announced March 2017.
