DeepHPS: End-to-end Estimation of 3D Hand Pose and Shape by Learning from Synthetic Depth
Authors:
Jameel Malik,
Ahmed Elhayek,
Fabrizio Nunnari,
Kiran Varanasi,
Kiarash Tamaddon,
Alexis Heloir,
Didier Stricker
Abstract:
Articulated hand pose and shape estimation is an important problem for vision-based applications such as augmented reality and animation. In contrast to the existing methods which optimize only for joint positions, we propose a fully supervised deep network which learns to jointly estimate a full 3D hand mesh representation and pose from a single depth image. To this end, a CNN architecture is emp…
▽ More
Articulated hand pose and shape estimation is an important problem for vision-based applications such as augmented reality and animation. In contrast to the existing methods which optimize only for joint positions, we propose a fully supervised deep network which learns to jointly estimate a full 3D hand mesh representation and pose from a single depth image. To this end, a CNN architecture is employed to estimate parametric representations i.e. hand pose, bone scales and complex shape parameters. Then, a novel hand pose and shape layer, embedded inside our deep framework, produces 3D joint positions and hand mesh. Lack of sufficient training data with varying hand shapes limits the generalized performance of learning based methods. Also, manually annotating real data is suboptimal. Therefore, we present SynHand5M: a million-scale synthetic dataset with accurate joint annotations, segmentation masks and mesh files of depth maps. Among model based learning (hybrid) methods, we show improved results on our dataset and two of the public benchmarks i.e. NYU and ICVL. Also, by employing a joint training strategy with real and synthetic data, we recover 3D hand mesh and pose from real images in 3.7ms.
△ Less
Submitted 28 August, 2018;
originally announced August 2018.
Proceedings of eNTERFACE 2015 Workshop on Intelligent Interfaces
Authors:
Matei Mancas,
Christian Frisson,
Joëlle Tilmanne,
Nicolas d'Alessandro,
Petr Barborka,
Furkan Bayansar,
Francisco Bernard,
Rebecca Fiebrink,
Alexis Heloir,
Edgar Hemery,
Sohaib Laraba,
Alexis Moinet,
Fabrizio Nunnari,
Thierry Ravet,
Loïc Reboursière,
Alvaro Sarasua,
Mickaël Tits,
Noé Tits,
François Zajéga,
Paolo Alborno,
Ksenia Kolykhalova,
Emma Frid,
Damiano Malafronte,
Lisanne Huis in't Veld,
Hüseyin Cakmak
, et al. (49 additional authors not shown)
Abstract:
The 11th Summer Workshop on Multimodal Interfaces eNTERFACE 2015 was hosted by the Numediart Institute of Creative Technologies of the University of Mons from August 10th to September 2015. During the four weeks, students and researchers from all over the world came together in the Numediart Institute of the University of Mons to work on eight selected projects structured around intelligent interf…
▽ More
The 11th Summer Workshop on Multimodal Interfaces eNTERFACE 2015 was hosted by the Numediart Institute of Creative Technologies of the University of Mons from August 10th to September 2015. During the four weeks, students and researchers from all over the world came together in the Numediart Institute of the University of Mons to work on eight selected projects structured around intelligent interfaces. Eight projects were selected and their reports are shown here.
△ Less
Submitted 19 January, 2018;
originally announced January 2018.
