A Heuristic Approach for Selecting Best-Subset including Ranking within the Subset

Stochastic simulation is beneficial when evaluating the performance of a complex system. When optimizing the system performance with the simulation, we need to make a final decision by considering various qualitative criteria neglected by the simulation as well as the simulation results. However, as simulations are expensive and time-consuming, in this paper, we propose a ranking and selection algorithm to make such optimization with the simulation efficient. The proposed algorithm selects a best-subset of designs expected to optimize the system performance from a finite set of alternatives. Furthermore, the algorithm identifies the ranking of designs within the subset. To maximize the accuracy of the selection under limited simulation resources, the algorithm selectively and gradually increases the precision of the sample mean of each design by allocating the resources heuristically based on the evaluated uncertainty. The selected subset allows decision makers to efficiently choose the best design that optimizes the performance while satisfying the qualitative criteria. We exhibit various experimental results, including a practical case study, to empirically demonstrate the efficiency and high noise robustness of the proposed algorithm.