Theoretical analysis of the asymptotics of systems involving a large number of interacting entities has enabled massive progress in fields ranging from Statistical Physics, High Dimensional Statistics, Combinatorial Optimization, to the theory of Machine Learning. In this PhD, we aim to build upon the techniques developed in the above fields to obtain novel insights into the phase transitions and dynamics in high dimensional computational phenomenon, particularly for domains such as Representation Learning and Causality, where many central questions remain unanswered.