Karri Majoinen
In computer vision and natural language processing, models pre-trained on massive datasets can be reused off-the-shelf to solve many downstream tasks without further training. Reinforcement learning (RL) and control, by contrast, still largely follow a tabula-rasa paradigm: an agent is trained from scratch in each environment, often requiring millions of in-domain interactions to learn a task-specific visuo-motor policy. This is in stark contrast to how humans perceive and act, reusing features learned across many diverse settings rather than relearning perception every time. The goal of this dissertation is to learn universal features for RL and control - visual representations, pre-trained largely on out-of-domain data, that serve as a reusable perception module and bootstrap learning of any task in any environment. This continues the line of work on pre-trained visual representations (PVRs) for control, which showed that frozen vision models trained on out-of-domain data can match or even outperform ground-truth state features. The primary research question is: can we identify and learn a (near-)universal visual representation — across model architectures, pre-training objectives, datasets, and feature hierarchies — that transfers to diverse RL tasks and environments and reduces the need for in-domain data and hand-engineered state features?
The dissertation pushes this agenda further along several axes: studying which pre-training objectives, datasets, and architectures yield the most transferable features; understanding how to combine features across the model hierarchy; and, importantly, how to continually adapt a pre-trained perception module as the agent encounters new RL environments, rather than keeping it strictly frozen.