MoRF: Mobile Realistic Fullbody Avatars from a Monocular Video
Authors:
Renat Bashirov,
Alexey Larionov,
Evgeniya Ustinova,
Mikhail Sidorenko,
David Svitov,
Ilya Zakharkin,
Victor Lempitsky
Abstract:
We present a system to create Mobile Realistic Fullbody (MoRF) avatars. MoRF avatars are rendered in real-time on mobile devices, learned from monocular videos, and have high realism. We use SMPL-X as a proxy geometry and render it with DNR (neural texture and image-2-image network). We improve on prior work, by overfitting per-frame war** fields in the neural texture space, allowing to better a…
▽ More
We present a system to create Mobile Realistic Fullbody (MoRF) avatars. MoRF avatars are rendered in real-time on mobile devices, learned from monocular videos, and have high realism. We use SMPL-X as a proxy geometry and render it with DNR (neural texture and image-2-image network). We improve on prior work, by overfitting per-frame war** fields in the neural texture space, allowing to better align the training signal between different frames. We also refine SMPL-X mesh fitting procedure to improve the overall avatar quality. In the comparisons to other monocular video-based avatar systems, MoRF avatars achieve higher image sharpness and temporal consistency. Participants of our user study also preferred avatars generated by MoRF.
△ Less
Submitted 11 December, 2023; v1 submitted 17 March, 2023;
originally announced March 2023.
