Samiran Gode

For robots to effectively function in real-world settings, they must exhibit generalizability and robustness. These robots must operate safely across diverse environments, embodiments, and scenarios. Foundation models, trained on internet-scale datasets, could help us solve these problems. By leveraging vast amounts of video data-encompassing human interactions, vehicular videos, and more-alongside existing robotic datasets, we aim to develop models that perform well and are efficient, interpretable, and safe for deployment. The first aspect of our research focuses on navigation. Our goal is to develop an optimal learned navigation policy that functions reliably in explored areas. Such a model needs to be robust to out-of-distribution cases including dynamic obstacles, lighting variations, and unexpected map changes. For the second aspect of manipulation, our objective is to build models that can learn from videos of human interaction with objects and generalize these learning results to different embodiments.