FaceBlit: Instant Real-time Example-based Style Transfer to Facial Videos

Aneta Texler

CTU in Prague, FEE

Ondřej Texler

CTU in Prague, FEE

Michal Kučera

CTU in Prague, FEE

Menglei Chai

Snap Inc.

Daniel Sýkora

CTU in Prague, FEE

In Proceedings of the ACM in Computer Graphics and Interactive Techniques, 4(1), 2021 (I3D 2021)

We present FaceBlit—a system for real-time example-based face video stylization that retains textural details of the style in a semantically meaningful manner, i.e., strokes used to depict specific features in the style are present at the appropriate locations in the target image. As compared to previous techniques, our system preserves the identity of the target subject and runs in real-time without the need for large datasets nor lengthy training phase. To achieve this, we modify the existing face stylization pipeline of Fišer et al. [2017] so that it can quickly generate a set of guiding channels that handle identity preservation of the target subject while are still compatible with a faster variant of patch-based synthesis algorithm of Sýkora et al. [2019]. Thanks to these improvements we demonstrate a first face stylization pipeline that can instantly transfer artistic style from a single portrait to the target video at interactive rates even on mobile devices.

Full Text GitHub BibTeX