The dominant deep learning setting involves training a model offline on a fixed dataset before deploying it in the real world. Implicit in this approach is an important and restrictive assumption: both the training data and future inputs are assumed to be IID samples from a single stationary distribution. In reality, this is very rarely true. To address this problem, we need machine learning systems that keep learning after deployment and continuously adapt to the shifts in data distributions. When learning from non-stationary streaming data, existing systems exhibit catastrophic forgetting. My project focuses on overcoming this issue by efficiently representing, retrieving, and reusing old knowledge.