Scene understanding is a common computer vision task involving object recognition and semantic reasoning. Recently, much progress has been made in supervised scene understanding. However, the absence of labeled data in novel and complex scenarios poses particular challenges. The goal of this project is to tackle scene understanding with largely reduced supervision. In particular, we aim to integrate knowledge about motion as well as depth as inductive biases for unsupervised scene understanding.