Probability distributions play a central role in machine learning. For probabilistic modeling, they are used as likelihoods and prior distributions, whereas in variational inference, they are employed as approximate posterior distributions. The probability distributions typically used in practice tend to be simple, such as exponential family distributions. However, the use of too simple distributions can in general limit performance. For example, using too simple likelihood distributions can lead to serious model misspecification, whereas using too simple variational distributions can lead to bad posterior approximations and loose variational bounds. In this project, we will explore the use of flexible densities for probabilistic models and variational inference, with a focus on deep generative models.