Synthesizing Human Faces Using Latent Space Factorization and Local Weights

Minyoung Kim; Young J. Kim

doi:10.1007/978-3-030-89029-2_31

Synthesizing Human Faces Using Latent Space Factorization and Local Weights

Minyoung Kim, Young J. Kim

Department of Computer Science & Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

We propose a 3D face generative model with local weights to increase the model’s variations and expressiveness. The proposed model allows partial manipulation of the face while still learning the whole face mesh. For this purpose, we address an effective way to extract local facial features from the entire data and explore a way to manipulate them during a holistic generation. First, we factorize the latent space of the whole face to the subspace indicating different parts of the face. In addition, local weights generated by non-negative matrix factorization are applied to the factorized latent space so that the decomposed part space is semantically meaningful. We experiment with our model and observe that effective facial part manipulation is possible, and that the model’s expressiveness is improved.

Original language	English
Title of host publication	Advances in Computer Graphics - 38th Computer Graphics International Conference, CGI 2021, Proceedings
Editors	Nadia Magnenat-Thalmann, Nadia Magnenat-Thalmann, Victoria Interrante, Daniel Thalmann, George Papagiannakis, Bin Sheng, Jinman Kim, Marina Gavrilova
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	398-405
Number of pages	8
ISBN (Print)	9783030890285
DOIs	https://doi.org/10.1007/978-3-030-89029-2_31
State	Published - 2021
Event	38th Computer Graphics International Conference, CGI 2021 - Virtual, Online Duration: 6 Sep 2021 → 10 Sep 2021

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13002 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	38th Computer Graphics International Conference, CGI 2021
City	Virtual, Online
Period	6/09/21 → 10/09/21

Bibliographical note

Funding Information:
Acknowledgements. This project was supported in part by the ITRC/IITP program (IITP-2021-0-01460) and the NRF (2017R1A2B3012701 and 2021R1A4A1032582) in South Korea. Y.-J. Kim is the corresponding author.

Publisher Copyright:
© 2021, Springer Nature Switzerland AG.

Keywords

Face synthesis
Generative models
Learning-based approach

Access to Document

10.1007/978-3-030-89029-2_31

Cite this

Kim, M., & Kim, Y. J. (2021). Synthesizing Human Faces Using Latent Space Factorization and Local Weights. In N. Magnenat-Thalmann, N. Magnenat-Thalmann, V. Interrante, D. Thalmann, G. Papagiannakis, B. Sheng, J. Kim, & M. Gavrilova (Eds.), Advances in Computer Graphics - 38th Computer Graphics International Conference, CGI 2021, Proceedings (pp. 398-405). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13002 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-89029-2_31

Kim, Minyoung ; Kim, Young J. / Synthesizing Human Faces Using Latent Space Factorization and Local Weights. Advances in Computer Graphics - 38th Computer Graphics International Conference, CGI 2021, Proceedings. editor / Nadia Magnenat-Thalmann ; Nadia Magnenat-Thalmann ; Victoria Interrante ; Daniel Thalmann ; George Papagiannakis ; Bin Sheng ; Jinman Kim ; Marina Gavrilova. Springer Science and Business Media Deutschland GmbH, 2021. pp. 398-405 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{0ce56c0080d04068afbb414989ff5668,

title = "Synthesizing Human Faces Using Latent Space Factorization and Local Weights",

abstract = "We propose a 3D face generative model with local weights to increase the model{\textquoteright}s variations and expressiveness. The proposed model allows partial manipulation of the face while still learning the whole face mesh. For this purpose, we address an effective way to extract local facial features from the entire data and explore a way to manipulate them during a holistic generation. First, we factorize the latent space of the whole face to the subspace indicating different parts of the face. In addition, local weights generated by non-negative matrix factorization are applied to the factorized latent space so that the decomposed part space is semantically meaningful. We experiment with our model and observe that effective facial part manipulation is possible, and that the model{\textquoteright}s expressiveness is improved.",

keywords = "Face synthesis, Generative models, Learning-based approach",

author = "Minyoung Kim and Kim, {Young J.}",

note = "Funding Information: Acknowledgements. This project was supported in part by the ITRC/IITP program (IITP-2021-0-01460) and the NRF (2017R1A2B3012701 and 2021R1A4A1032582) in South Korea. Y.-J. Kim is the corresponding author. Publisher Copyright: {\textcopyright} 2021, Springer Nature Switzerland AG.; 38th Computer Graphics International Conference, CGI 2021 ; Conference date: 06-09-2021 Through 10-09-2021",

year = "2021",

doi = "10.1007/978-3-030-89029-2_31",

language = "English",

isbn = "9783030890285",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "398--405",

editor = "Nadia Magnenat-Thalmann and Nadia Magnenat-Thalmann and Victoria Interrante and Daniel Thalmann and George Papagiannakis and Bin Sheng and Jinman Kim and Marina Gavrilova",

booktitle = "Advances in Computer Graphics - 38th Computer Graphics International Conference, CGI 2021, Proceedings",

}

Kim, M & Kim, YJ 2021, Synthesizing Human Faces Using Latent Space Factorization and Local Weights. in N Magnenat-Thalmann, N Magnenat-Thalmann, V Interrante, D Thalmann, G Papagiannakis, B Sheng, J Kim & M Gavrilova (eds), Advances in Computer Graphics - 38th Computer Graphics International Conference, CGI 2021, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13002 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 398-405, 38th Computer Graphics International Conference, CGI 2021, Virtual, Online, 6/09/21. https://doi.org/10.1007/978-3-030-89029-2_31

Synthesizing Human Faces Using Latent Space Factorization and Local Weights. / Kim, Minyoung; Kim, Young J.
Advances in Computer Graphics - 38th Computer Graphics International Conference, CGI 2021, Proceedings. ed. / Nadia Magnenat-Thalmann; Nadia Magnenat-Thalmann; Victoria Interrante; Daniel Thalmann; George Papagiannakis; Bin Sheng; Jinman Kim; Marina Gavrilova. Springer Science and Business Media Deutschland GmbH, 2021. p. 398-405 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13002 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Synthesizing Human Faces Using Latent Space Factorization and Local Weights

AU - Kim, Minyoung

AU - Kim, Young J.

N1 - Funding Information: Acknowledgements. This project was supported in part by the ITRC/IITP program (IITP-2021-0-01460) and the NRF (2017R1A2B3012701 and 2021R1A4A1032582) in South Korea. Y.-J. Kim is the corresponding author. Publisher Copyright: © 2021, Springer Nature Switzerland AG.

PY - 2021

Y1 - 2021

N2 - We propose a 3D face generative model with local weights to increase the model’s variations and expressiveness. The proposed model allows partial manipulation of the face while still learning the whole face mesh. For this purpose, we address an effective way to extract local facial features from the entire data and explore a way to manipulate them during a holistic generation. First, we factorize the latent space of the whole face to the subspace indicating different parts of the face. In addition, local weights generated by non-negative matrix factorization are applied to the factorized latent space so that the decomposed part space is semantically meaningful. We experiment with our model and observe that effective facial part manipulation is possible, and that the model’s expressiveness is improved.

AB - We propose a 3D face generative model with local weights to increase the model’s variations and expressiveness. The proposed model allows partial manipulation of the face while still learning the whole face mesh. For this purpose, we address an effective way to extract local facial features from the entire data and explore a way to manipulate them during a holistic generation. First, we factorize the latent space of the whole face to the subspace indicating different parts of the face. In addition, local weights generated by non-negative matrix factorization are applied to the factorized latent space so that the decomposed part space is semantically meaningful. We experiment with our model and observe that effective facial part manipulation is possible, and that the model’s expressiveness is improved.

KW - Face synthesis

KW - Generative models

KW - Learning-based approach

UR - http://www.scopus.com/inward/record.url?scp=85118320593&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-89029-2_31

DO - 10.1007/978-3-030-89029-2_31

M3 - Conference contribution

AN - SCOPUS:85118320593

SN - 9783030890285

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 398

EP - 405

BT - Advances in Computer Graphics - 38th Computer Graphics International Conference, CGI 2021, Proceedings

A2 - Magnenat-Thalmann, Nadia

A2 - Interrante, Victoria

A2 - Thalmann, Daniel

A2 - Papagiannakis, George

A2 - Sheng, Bin

A2 - Kim, Jinman

A2 - Gavrilova, Marina

PB - Springer Science and Business Media Deutschland GmbH

T2 - 38th Computer Graphics International Conference, CGI 2021

Y2 - 6 September 2021 through 10 September 2021

ER -

Kim M, Kim YJ. Synthesizing Human Faces Using Latent Space Factorization and Local Weights. In Magnenat-Thalmann N, Magnenat-Thalmann N, Interrante V, Thalmann D, Papagiannakis G, Sheng B, Kim J, Gavrilova M, editors, Advances in Computer Graphics - 38th Computer Graphics International Conference, CGI 2021, Proceedings. Springer Science and Business Media Deutschland GmbH. 2021. p. 398-405. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-89029-2_31

Synthesizing Human Faces Using Latent Space Factorization and Local Weights

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Fingerprint

Cite this