Density Estimation-Based Stein Variational Gradient Descent

Approximating a target distribution, such as a Bayesian posterior, is important in many areas, including cognitive computation. We introduce a variant of Stein variational gradient descent (SVGD) (Liu and Wang Adv Neural Inf Process Syst 29, 2016), called the density estimation-based Stein variational gradient descent (DESVGD). SVGD has proven to be promising as a sampling method for approximating target distributions. SVGD, however, suffers from discontinuity inherent in the empirical measure, making it difficult to closely monitor the convergence of the sampling-based approximation to the target. DESVGD utilizes kernel density estimation to replace the empirical measure in SVGD with its continuous counterpart. This allows direct computation of the KL divergence between the current approximation and the target distribution, thereby helping to monitor the numerical convergence of the iterative optimization process. DESVGD also offers derivatives of the KL divergence, which can be used to better design learning rates and thus to achieve faster convergence. By simply replacing the kernel used in SVGD with its weighted average, one can easily implement DESVGD based on existing SVGD algorithms. Our numerical experiments demonstrate that DESVGD approximates the target distribution well and outperforms the original SVGD in terms of approximation quality.