Enhancing Risk-Aware Navigation for Legged Robots in Dynamic Construction Site Environments Using Foundational Language Models and Building Information Maps
Maya Sitaram (Ph.D. Student)
This research aims to advance the autonomous navigation capabilities of legged robots in construction site environments by integrating foundational language models with Building Information Maps (BIM). Robots show immense potential in construction site monitoring, offering the ability to generate detailed 3D maps and provide logistical support, including material transportation. Legged robots, in particular, also have the potential to traverse unstructured and rugged terrain of construction sites. However, traditional path planning methods reliant on geometric features struggle to maintain accuracy when scene geometries drastically change, such as new structures being erected, demolition of existing structures, and movement of heavy machinery. By leveraging foundational models, we propose that robots will be able to consistently recognize key landmarks and spatial features of the environment, even as the site evolves over extended construction projects. As a result of this project we will construct a framework integrating foundational language models and Building Information Maps (BIM) to enhance the autonomy, adaptability, and risk-aware navigation capabilities of legged robots in construction site settings. A crucial component of the system will be risk-aware navigation, ensuring that legged robots make informed decisions that prioritize safety in dynamic and potentially hazardous construction environments. At the University of Technology Nuremberg, we are given the unique opportunity to deploy our system at our university construction site. Furthermore, we will employ ANYMal quadrupedal robots developed at ETH Zürich by Prof. Marco Hutter, who is also a co-advisor for this project. One objective of the project is using language models for semantic localization, where a language model will be used to interpret natural language descriptions of the robot’s surroundings and utilize this information to estimate the robot’s pose within the construction site. Another objective of the project will be in dynamic mapping and scene understanding, where BIM data will be integrated with temporal sensor data to create context-aware maps of the construction site. Utilizing this, the robot will be able to update the map continuously, accommodating changes in the environment and construction progress, and accessing potential risks. Another objective is to use our system for robust path planning, considering geometric constraints and natural language descriptions, ensuring that robots can navigate safely through the construction site, adapting to dynamic obstacles and evolving conditions. Finally, we will investigate methods to use language models as an interface for construction workers to communicate with legged robots using natural language commands, for instance, instructing a legged robot to navigate to a location on the site or to transport materials between locations. As a result, this project aims to revolutionize the field of construction robotics by creating legged robot systems that are adaptable, capable of understanding and generating natural language descriptions, seamlessly integrated with BIM data, all while making informed risk-aware navigation decisions. Such systems can significantly improve construction site safety, productivity, and logistical operations, ultimately contributing to more efficient construction processes.
| Primary Host: | Wolfram Burgard (Technical University of Nuremberg) |
| Exchange Host: | () |
| PhD Duration: | 01 September 2023 - 01 September 2026 |
| Exchange Duration: | - Ongoing - Ongoing |
