With the more and more widespread deployment of AI systems in the real world, they are likely going to have larger and larger impacts on society, including accidents, political destabilization and other negative externalities. To increase developer's accountability for negative societal impacts of their AI systems, my research focuses on a) a better theoretical understanding of these impacts, b) approaches to monitor and empirically demonstrate them, c) building tools developers can use to mitigate potential risks their AI systems might pose, and d) strategies for policymakers to align AI developers' incentives with society.