Deep Generative Positive-Unlabeled Learning under Selection Bias

Byeonghu Na; Hyemi Kim; Kyungwoo Song; Weonyoung Joo; Yoon Yeong Kim; Il Chul Moon

doi:10.1145/3340531.3411971

Deep Generative Positive-Unlabeled Learning under Selection Bias

Byeonghu Na, Hyemi Kim, Kyungwoo Song, Weonyoung Joo, Yoon Yeong Kim, Il Chul Moon

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

18 Scopus citations

Abstract

Learning in the positive-unlabeled (PU) setting is prevalent in real world applications. Many previous works depend upon theSelected Completely At Random (SCAR) assumption to utilize unlabeled data, but the SCAR assumption is not often applicable to the real world due to selection bias in label observations. This paper is the first generative PU learning model without the SCAR assumption. Specifically, we derive the PU risk function without the SCAR assumption, and we generate a set of virtual PU examples to train the classifier. Although our PU risk function is more generalizable, the function requires PU instances that do not exist in the observations. Therefore, we introduce the VAE-PU, which is a variant of variational autoencoders to separate two latent variables that generate either features or observation indicators. The separated latent information enables the model to generate virtual PU instances. We test the VAE-PU on benchmark datasets with and without the SCAR assumption. The results indicate that the VAE-PU is superior when selection bias exists, and the VAE-PU is also competent under the SCAR assumption. The results also emphasize that the VAE-PU is effective when there are few positive-labeled instances due to modeling on selection bias.

Original language	English
Title of host publication	CIKM 2020 - Proceedings of the 29th ACM International Conference on Information and Knowledge Management
Publisher	Association for Computing Machinery
Pages	1155-1164
Number of pages	10
ISBN (Electronic)	9781450368599
DOIs	https://doi.org/10.1145/3340531.3411971
State	Published - 19 Oct 2020
Externally published	Yes
Event	29th ACM International Conference on Information and Knowledge Management, CIKM 2020 - Virtual, Online, Ireland Duration: 19 Oct 2020 → 23 Oct 2020

Publication series

Name	International Conference on Information and Knowledge Management, Proceedings

Conference

Conference	29th ACM International Conference on Information and Knowledge Management, CIKM 2020
Country/Territory	Ireland
City	Virtual, Online
Period	19/10/20 → 23/10/20

Bibliographical note

Funding Information:
The authors would like to thank the anonymous reviewers for their valuable comments and helpful suggestions on our manuscript. This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF), funded by the Ministry of Education (NRF-2019M3F2A1072239).

Publisher Copyright:
© 2020 ACM.

Keywords

positive-unlabeled learning
selection bias
variational autoencoders

Access to Document

10.1145/3340531.3411971

Cite this

Na, B., Kim, H., Song, K., Joo, W., Kim, Y. Y., & Moon, I. C. (2020). Deep Generative Positive-Unlabeled Learning under Selection Bias. In CIKM 2020 - Proceedings of the 29th ACM International Conference on Information and Knowledge Management (pp. 1155-1164). (International Conference on Information and Knowledge Management, Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3340531.3411971

@inproceedings{5debb5303e0a41199c368b5fae86a836,

title = "Deep Generative Positive-Unlabeled Learning under Selection Bias",

abstract = "Learning in the positive-unlabeled (PU) setting is prevalent in real world applications. Many previous works depend upon theSelected Completely At Random (SCAR) assumption to utilize unlabeled data, but the SCAR assumption is not often applicable to the real world due to selection bias in label observations. This paper is the first generative PU learning model without the SCAR assumption. Specifically, we derive the PU risk function without the SCAR assumption, and we generate a set of virtual PU examples to train the classifier. Although our PU risk function is more generalizable, the function requires PU instances that do not exist in the observations. Therefore, we introduce the VAE-PU, which is a variant of variational autoencoders to separate two latent variables that generate either features or observation indicators. The separated latent information enables the model to generate virtual PU instances. We test the VAE-PU on benchmark datasets with and without the SCAR assumption. The results indicate that the VAE-PU is superior when selection bias exists, and the VAE-PU is also competent under the SCAR assumption. The results also emphasize that the VAE-PU is effective when there are few positive-labeled instances due to modeling on selection bias.",

keywords = "positive-unlabeled learning, selection bias, variational autoencoders",

author = "Byeonghu Na and Hyemi Kim and Kyungwoo Song and Weonyoung Joo and Kim, {Yoon Yeong} and Moon, {Il Chul}",

note = "Funding Information: The authors would like to thank the anonymous reviewers for their valuable comments and helpful suggestions on our manuscript. This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF), funded by the Ministry of Education (NRF-2019M3F2A1072239). Publisher Copyright: {\textcopyright} 2020 ACM.; 29th ACM International Conference on Information and Knowledge Management, CIKM 2020 ; Conference date: 19-10-2020 Through 23-10-2020",

year = "2020",

month = oct,

day = "19",

doi = "10.1145/3340531.3411971",

language = "English",

series = "International Conference on Information and Knowledge Management, Proceedings",

publisher = "Association for Computing Machinery",

pages = "1155--1164",

booktitle = "CIKM 2020 - Proceedings of the 29th ACM International Conference on Information and Knowledge Management",

}

Na, B, Kim, H, Song, K, Joo, W, Kim, YY & Moon, IC 2020, Deep Generative Positive-Unlabeled Learning under Selection Bias. in CIKM 2020 - Proceedings of the 29th ACM International Conference on Information and Knowledge Management. International Conference on Information and Knowledge Management, Proceedings, Association for Computing Machinery, pp. 1155-1164, 29th ACM International Conference on Information and Knowledge Management, CIKM 2020, Virtual, Online, Ireland, 19/10/20. https://doi.org/10.1145/3340531.3411971

Deep Generative Positive-Unlabeled Learning under Selection Bias. / Na, Byeonghu; Kim, Hyemi; Song, Kyungwoo et al.
CIKM 2020 - Proceedings of the 29th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, 2020. p. 1155-1164 (International Conference on Information and Knowledge Management, Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Deep Generative Positive-Unlabeled Learning under Selection Bias

AU - Na, Byeonghu

AU - Kim, Hyemi

AU - Song, Kyungwoo

AU - Joo, Weonyoung

AU - Kim, Yoon Yeong

AU - Moon, Il Chul

N1 - Funding Information: The authors would like to thank the anonymous reviewers for their valuable comments and helpful suggestions on our manuscript. This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF), funded by the Ministry of Education (NRF-2019M3F2A1072239). Publisher Copyright: © 2020 ACM.

PY - 2020/10/19

Y1 - 2020/10/19

N2 - Learning in the positive-unlabeled (PU) setting is prevalent in real world applications. Many previous works depend upon theSelected Completely At Random (SCAR) assumption to utilize unlabeled data, but the SCAR assumption is not often applicable to the real world due to selection bias in label observations. This paper is the first generative PU learning model without the SCAR assumption. Specifically, we derive the PU risk function without the SCAR assumption, and we generate a set of virtual PU examples to train the classifier. Although our PU risk function is more generalizable, the function requires PU instances that do not exist in the observations. Therefore, we introduce the VAE-PU, which is a variant of variational autoencoders to separate two latent variables that generate either features or observation indicators. The separated latent information enables the model to generate virtual PU instances. We test the VAE-PU on benchmark datasets with and without the SCAR assumption. The results indicate that the VAE-PU is superior when selection bias exists, and the VAE-PU is also competent under the SCAR assumption. The results also emphasize that the VAE-PU is effective when there are few positive-labeled instances due to modeling on selection bias.

AB - Learning in the positive-unlabeled (PU) setting is prevalent in real world applications. Many previous works depend upon theSelected Completely At Random (SCAR) assumption to utilize unlabeled data, but the SCAR assumption is not often applicable to the real world due to selection bias in label observations. This paper is the first generative PU learning model without the SCAR assumption. Specifically, we derive the PU risk function without the SCAR assumption, and we generate a set of virtual PU examples to train the classifier. Although our PU risk function is more generalizable, the function requires PU instances that do not exist in the observations. Therefore, we introduce the VAE-PU, which is a variant of variational autoencoders to separate two latent variables that generate either features or observation indicators. The separated latent information enables the model to generate virtual PU instances. We test the VAE-PU on benchmark datasets with and without the SCAR assumption. The results indicate that the VAE-PU is superior when selection bias exists, and the VAE-PU is also competent under the SCAR assumption. The results also emphasize that the VAE-PU is effective when there are few positive-labeled instances due to modeling on selection bias.

KW - positive-unlabeled learning

KW - selection bias

KW - variational autoencoders

UR - http://www.scopus.com/inward/record.url?scp=85095866179&partnerID=8YFLogxK

U2 - 10.1145/3340531.3411971

DO - 10.1145/3340531.3411971

M3 - Conference contribution

AN - SCOPUS:85095866179

T3 - International Conference on Information and Knowledge Management, Proceedings

SP - 1155

EP - 1164

BT - CIKM 2020 - Proceedings of the 29th ACM International Conference on Information and Knowledge Management

PB - Association for Computing Machinery

T2 - 29th ACM International Conference on Information and Knowledge Management, CIKM 2020

Y2 - 19 October 2020 through 23 October 2020

ER -

Na B, Kim H, Song K, Joo W, Kim YY, Moon IC. Deep Generative Positive-Unlabeled Learning under Selection Bias. In CIKM 2020 - Proceedings of the 29th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery. 2020. p. 1155-1164. (International Conference on Information and Knowledge Management, Proceedings). doi: 10.1145/3340531.3411971

Deep Generative Positive-Unlabeled Learning under Selection Bias

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Fingerprint

Cite this