Mohamed, S., Bouchachia, A. and Sayed-Mouchaweh, M., 2019. Asynchronous Stochastic Variational Inference. In: INNS Big Data and Deep Learning 2019: Proceedings of the International Neural Networks Society Conference, 16--18 April 2019, Genoa, Italy.
Full text available as:
|
PDF
AsynVI.pdf - Accepted Version Available under License Creative Commons Attribution Non-commercial. 465kB | |
Copyright to original material in this document is with the original owner(s). Access to this content through BURO is granted on condition that you use it only for research, scholarly or other non-commercial purposes. If you wish to use it for any other purposes, you must contact BU via BURO@bournemouth.ac.uk. Any third party copyright material in this document remains the property of its respective owner(s). BU grants no licence for further use of that third party material. |
Official URL: https://innsbddl2019.org/
Abstract
Stochastic variational inference (SVI) employs stochastic optimization to scale up Bayesian computation to massive data. Since SVI is at its core a stochastic gradient-based algorithm, horizontal parallelism can be harnessed to allow larger scale inference. We propose a lock-free parallel implementation for SVI which allows distributed computations over multiple slaves in an asynchronous style. We show that our implementation leads to linear speed-up while guaranteeing an asymptotic ergodic convergence rate O(1/ √ T) while the number of slaves is bounded by √ T (T is the total number of iterations). The implementation is done in a high-performance computing environment using message passing interface for python (MPI4py). The empirical evaluation shows that our parallel SVI is lossless, performing comparably well to its counterpart serial SVI with linear speed-up.
Item Type: | Conference or Workshop Item (Paper) |
---|---|
Group: | Faculty of Science & Technology |
ID Code: | 32444 |
Deposited By: | Symplectic RT2 |
Deposited On: | 27 Jun 2019 14:17 |
Last Modified: | 14 Mar 2022 14:16 |
Downloads
Downloads per month over past year
Repository Staff Only - |