Low dimensional, abstract or otherwise 'simple' structure occurs widely across machine learning. This project advances theoretical understanding in a variety of areas. These are: causality, in which a theory of micro-macro abstractions is developed; independent component analysis, in which new identifiability results are derived in a multi-view setting; and generative modelling, where the estimation of divergences in latent spaces is analysed in a learning theoretic framework.