Lots of mobile videos are recorded by wide-angle lenses to include both the narrating subjects and background. However, video playback exhibits visual distortion that stretches subjects near the image corners.
A recent study on arXiv.org addresses the wide-angle face distortion on videos using spatial-temporal mesh optimization.
The researchers aim to generate a video with a natural look on human faces. The algorithm generates a warping mesh that adapts to the stereographic projection on the facial region. Temporal smoothness and coherent embedding terms are introduced to preserve the temporal consistency of meshes. A line-preservation term preserves the straight lines in the background.
A benchmark dataset of videos collected from Google Pixel 3, GoPro, and iPhone 11 cameras is created for performance evaluation. The approach significantly improves the video quality and is suitable for video post-editing software.
Video blogs and selfies are popular social media formats, which are often captured by wide-angle cameras to show human subjects and expanded background. Unfortunately, due to perspective projection, faces near corners and edges exhibit apparent distortions that stretch and squish the facial features, resulting in poor video quality. In this work, we present a video warping algorithm to correct these distortions. Our key idea is to apply stereographic projection locally on the facial regions. We formulate a mesh warp problem using spatial-temporal energy minimization and minimize background deformation using a line-preservation term to maintain the straight edges in the background. To address temporal coherency, we constrain the temporal smoothness on the warping meshes and facial trajectories through the latent variables. For performance evaluation, we develop a wide-angle video dataset with a wide range of focal lengths. The user study shows that 83.9% of users prefer our algorithm over other alternatives based on perspective projection.
Research paper: Lai, W.-S., Shih, Y., Liang, C.-K., and Yang, M.-H., “Correcting Face Distortion in Wide-Angle Videos”, 2021. Link: https://arxiv.org/abs/2111.09950