A Heuristic Approach for Selecting Best-Subset including Ranking within the Subset

Seon Han Choi, Tag Gon Kim

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


Stochastic simulation is beneficial when evaluating the performance of a complex system. When optimizing the system performance with the simulation, we need to make a final decision by considering various qualitative criteria neglected by the simulation as well as the simulation results. However, as simulations are expensive and time-consuming, in this paper, we propose a ranking and selection algorithm to make such optimization with the simulation efficient. The proposed algorithm selects a best-subset of designs expected to optimize the system performance from a finite set of alternatives. Furthermore, the algorithm identifies the ranking of designs within the subset. To maximize the accuracy of the selection under limited simulation resources, the algorithm selectively and gradually increases the precision of the sample mean of each design by allocating the resources heuristically based on the evaluated uncertainty. The selected subset allows decision makers to efficiently choose the best design that optimizes the performance while satisfying the qualitative criteria. We exhibit various experimental results, including a practical case study, to empirically demonstrate the efficiency and high noise robustness of the proposed algorithm.

Original languageEnglish
Article number8480451
Pages (from-to)3852-3862
Number of pages11
JournalIEEE Transactions on Systems, Man, and Cybernetics: Systems
Issue number10
StatePublished - Oct 2020

Bibliographical note

Funding Information:
Manuscript received October 30, 2017; revised February 12, 2018; accepted September 11, 2018. Date of publication October 3, 2018; date of current version September 16, 2020. This work was supported in part by the “Development Platform for User-Level Customizable, General Purpose Discrete Event Simulation Software” through the Institute for Information and Communications Technology Promotion funded by the Korea Government under Grant 2017-0-00461, and in part by the Brain Korea 21 PLUS Program. This paper was recommended by Associate Editor M. P. Fanti. (Corresponding author: Seon Han Choi.) S. H. Choi is with the Industrial Convergence Infrastructure Office, Korea Institute of Industrial Technology, Ansan 15588, South Korea (e-mail: gigohan01@kaist.ac.kr).

Publisher Copyright:
© 2013 IEEE.


  • Best-subset selection
  • ranking and selection (R&S)
  • ranking identification
  • stochastic simulation
  • system performance optimization


Dive into the research topics of 'A Heuristic Approach for Selecting Best-Subset including Ranking within the Subset'. Together they form a unique fingerprint.

Cite this