Asynchronous Stochastic Variational Inference.

Mohamad, S., Bouchachia, A. and Sayed-Mouchaweh, M., 2018. Asynchronous Stochastic Variational Inference. Working Paper. arXiv.

Full text available as:

[img]
Preview
PDF
1801.04289.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial No Derivatives.

723kB

Abstract

Stochastic variational inference (SVI) employs stochastic optimization to scale up Bayesian computation to massive data. Since SVI is at its core a stochastic gradient-based algorithm, horizontal parallelism can be harnessed to allow larger scale inference. We propose a lock-free parallel implementation for SVI which allows distributed computations over multiple slaves in an asynchronous style. We show that our implementation leads to linear speed-up while guaranteeing an asymptotic ergodic convergence rate O(1/√T) given that the number of slaves is bounded by √T (T is the total number of iterations). The implementation is done in a high-performance computing (HPC) environment using message passing interface (MPI) for python (MPI4py). The extensive empirical evaluation shows that our parallel SVI is lossless, performing comparably well to its counterpart serial SVI with linear speed-up.

Item Type:Monograph (Working Paper)
Group:Faculty of Science & Technology
ID Code:30237
Deposited By: Unnamed user with email symplectic@symplectic
Deposited On:22 Jan 2018 13:30
Last Modified:22 Jan 2018 13:30

Downloads

Downloads per month over past year

More statistics for this item...
Repository Staff Only -