In this work we explore harmonic analytic techniques to improve the learning capabilities of stochastic neural network models. Stochastic neural networks are widely used in probabilistic modeling for unsupervised and representation learning. Such models are often limited by problems of instability and by the type of stochasticity (e.g, discrete v. continuous) that can be usefully employed. Harmonic analysis provides a unified perspective under which many such problems can be studied. The aim of this project is to use this perspective to develop methods to resolve some of the limitations of stochastic neural network models.