IMPROVING PORTRAIT STYLE TRANSFER
Keywords:
style transfer, portrait images, transformer models, self-attention, cross-attention, facial segmentationAbstract
The paper presents an approach to improving artistic style transfer for portrait images by explicitly accounting for facial geometry and semantic heterogeneity of face regions. The proposed GeometricBiasAttention and SemanticGatedCrossAttention modules are integrated into the transformer-based architectures StyTr2 and S2WAT. Experiments on FFHQ and WikiArt Faces demonstrate improved identity preservation and more controllable stylization. The modified models also show that a smaller configuration can outperform a larger baseline one across several quality metrics, including FID, SSIM, and LPIPS.
References
Gatys L. A., Ecker A. S., Bethge M. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576, 2016.
Deng Y. et al. StyTr2: Image style transfer with transformers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
Fan C., Chen J., Xu X. S2WAT: Image style transfer via hierarchical vision transformer using strips window attention. Proceedings of the AAAI Conference on Artificial Intelligence, 2024.
Vaswani A. et al. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Simonyan K., Zisserman A. Very deep convolutional networks for large-scale image recognition. ICLR, 2015.
Johnson J., Alahi A., Fei-Fei L. Perceptual losses for real-time style transfer and super-resolution. ECCV, 2016.
Huang X., Belongie S. Arbitrary style transfer in real time with adaptive instance normalization. ICCV, 2017.
Yu C. et al. BiSeNet: Bilateral segmentation network for real-time semantic segmentation. ECCV, 2018.
Deng J. et al. ArcFace: Additive angular margin loss for deep face recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
Ushio A. WikiArt Face: Face image dataset from portraits, 2024.
