-
Automatic Dance Video Segmentation for Understanding Choreography
Authors:
Koki Endo,
Shuhei Tsuchida,
Tsukasa Fukusato,
Takeo Igarashi
Abstract:
Segmenting dance video into short movements is a popular way to easily understand dance choreography. However, it is currently done manually and requires a significant amount of effort by experts. That is, even if many dance videos are available on social media (e.g., TikTok and YouTube), it remains difficult for people, especially novices, to casually watch short video segments to practice dance…
▽ More
Segmenting dance video into short movements is a popular way to easily understand dance choreography. However, it is currently done manually and requires a significant amount of effort by experts. That is, even if many dance videos are available on social media (e.g., TikTok and YouTube), it remains difficult for people, especially novices, to casually watch short video segments to practice dance choreography. In this paper, we propose a method to automatically segment a dance video into each movement. Given a dance video as input, we first extract visual and audio features: the former is computed from the keypoints of the dancer in the video, and the latter is computed from the Mel spectrogram of the music in the video. Next, these features are passed to a Temporal Convolutional Network (TCN), and segmentation points are estimated by picking peaks of the network output. To build our training dataset, we annotate segmentation points to dance videos in the AIST Dance Video Database, which is a shared database containing original street dance videos with copyright-cleared dance music. The evaluation study shows that the proposed method (i.e., combining the visual and audio features) can estimate segmentation points with high accuracy. In addition, we developed an application to help dancers practice choreography using the proposed method.
△ Less
Submitted 30 May, 2024;
originally announced May 2024.
-
Prediction of transport property via machine learning molecular movements
Authors:
Ikki Yasuda,
Yusei Kobayashi,
Katsuhiro Endo,
Yoshihiro Hayakawa,
Kazuhiko Fujiwara,
Kuniaki Yajima,
Noriyoshi Arai,
Kenji Yasuoka
Abstract:
Molecular dynamics (MD) simulations are increasingly being combined with machine learning (ML) to predict material properties. The molecular configurations obtained from MD are represented by multiple features, such as thermodynamic properties, and are used as the ML input. However, to accurately find the input--output patterns, ML requires a sufficiently sized dataset that depends on the complexi…
▽ More
Molecular dynamics (MD) simulations are increasingly being combined with machine learning (ML) to predict material properties. The molecular configurations obtained from MD are represented by multiple features, such as thermodynamic properties, and are used as the ML input. However, to accurately find the input--output patterns, ML requires a sufficiently sized dataset that depends on the complexity of the ML model. Generating such a large dataset from MD simulations is not ideal because of their high computation cost. In this study, we present a simple supervised ML method to predict the transport properties of materials. To simplify the model, an unsupervised ML method obtains an efficient representation of molecular movements. This method was applied to predict the viscosity of lubricant molecules in confinement with shear flow. Furthermore, simplicity facilitates the interpretation of the model to understand the molecular mechanics of viscosity. We revealed two types of molecular mechanisms that contribute to low viscosity.
△ Less
Submitted 6 March, 2022;
originally announced March 2022.
-
MD-GAN with multi-particle input: the machine learning of long-time molecular behavior from short-time MD data
Authors:
Ryo Kawada,
Katsuhiro Endo,
Daisuke Yuhara,
Kenji Yasuoka
Abstract:
MD-GAN is a machine learning-based method that can evolve part of the system at any time step, accelerating the generation of molecular dynamics data. For the accurate prediction of MD-GAN, sufficient information on the dynamics of a part of the system should be included with the training data. Therefore, the selection of the part of the system is important for efficient learning. In a previous st…
▽ More
MD-GAN is a machine learning-based method that can evolve part of the system at any time step, accelerating the generation of molecular dynamics data. For the accurate prediction of MD-GAN, sufficient information on the dynamics of a part of the system should be included with the training data. Therefore, the selection of the part of the system is important for efficient learning. In a previous study, only one particle (or vector) of each molecule was extracted as part of the system. Therefore, we investigated the effectiveness of adding information from other particles to the learning process. In the experiment of the polyethylene system, when the dynamics of three particles of each molecule were used, the diffusion was successfully predicted using one-third of the time length of the training data, compared to the single-particle input. Surprisingly, the unobserved transition of diffusion in the training data was also predicted using this method.
△ Less
Submitted 2 February, 2022;
originally announced February 2022.
-
Classifying degraded images over various levels of degradation
Authors:
Kazuki Endo,
Masayuki Tanaka,
Masatoshi Okutomi
Abstract:
Classification for degraded images having various levels of degradation is very important in practical applications. This paper proposes a convolutional neural network to classify degraded images by using a restoration network and an ensemble learning. The results demonstrate that the proposed network can classify degraded images over various levels of degradation well. This paper also reveals how…
▽ More
Classification for degraded images having various levels of degradation is very important in practical applications. This paper proposes a convolutional neural network to classify degraded images by using a restoration network and an ensemble learning. The results demonstrate that the proposed network can classify degraded images over various levels of degradation well. This paper also reveals how the image-quality of training data for a classification network affects the classification performance of degraded images.
△ Less
Submitted 15 June, 2020;
originally announced June 2020.
