Gaussian-binary restricted Boltzmann machines for modeling natural image statistics
- We present a theoretical analysis of Gaussian-binary restricted Boltzmann machines (GRBMs) from the perspective of density models. The key aspect of this analysis is to show that GRBMs can be formulated as a constrained mixture of Gaussians, which gives a much better insight into the model’s capabilities and limitations. We further show that GRBMs are capable of learning meaningful features without using a regularization term and that the results are comparable to those of independent component analysis. This is illustrated for both a two-dimensional blind source separation task and for modeling natural image patches. Our findings exemplify that reported difficulties in training GRBMs are due to the failure of the training algorithm rather than the model itself. Based on our analysis we derive a better training setup and show empirically that it leads to faster and more robust training of GRBMs. Finally, we compare different sampling algorithms for training GRBMs and show that Contrastive Divergence performs better than training methods that use a persistent Markov chain.
| Author: | Jan MelchiorORCiDGND, Nan WangGND, Laurenz WiskottORCiDGND |
|---|---|
| URN: | urn:nbn:de:hbz:294-58842 |
| DOI: | https://doi.org/10.1371/journal.pone.0171015 |
| Parent Title (English): | PLoS one |
| Document Type: | Article |
| Language: | English |
| Date of Publication (online): | 2018/07/10 |
| Date of first Publication: | 2017/02/02 |
| Publishing Institution: | Ruhr-Universität Bochum, Universitätsbibliothek |
| Tag: | Open Access Fonds |
| Volume: | 12 |
| Issue: | 2 |
| First Page: | e0171015-1 |
| Last Page: | e0171015-24 |
| Note: | Article Processing Charge funded by the Deutsche Forschungsgemeinschaft (DFG) and the Open Access Publication Fund of Ruhr-Universität Bochum. |
| Note: | PLoS ONE, Bd. 12.2017, H. 2, Artikelnummer e0171015 |
| Note: | Korrektur zum Artikel: 15 Mar 2017: Melchior J, Wang N, Wiskott L (2017) Correction: Gaussian-binary restricted Boltzmann machines for modeling natural image statistics. PLOS ONE 12(3): e0174289. https://doi.org/10.1371/journal.pone.0174289 |
| Institutes/Facilities: | Institut für Neuroinformatik |
| Dewey Decimal Classification: | Technik, Medizin, angewandte Wissenschaften / Medizin, Gesundheit |
| open_access (DINI-Set): | open_access |
| Licence (English): |  Creative Commons - CC BY 4.0 - Attribution 4.0 International |
