An agent that learns to act in a partially known system faces two distinct kinds of ignorance. The first is epistemic uncertainty, which shrinks with more data and is the subject of the classical exploration-exploitation trade-off. The second is non-identifiability: the interactions the agent is able to have cannot distinguish between models that disagree about the decision at the hand, so no amount of the same data resolves it. Only a different experiment can, and physical, observational or ethical constraints may rule out precisely that experiment. Existing methods address one kind at a time: active learning and safe exploration reduce epistemic uncertainty, and experimental design shrinks identification ambiguity without regard to a downstream task or admissibility constraints. The PhD project develops the theory and algorithms for an agent that must choose and trade off, at each step, whether to exploit its current knowledge for the downstream task, to explore by collecting more of the same data, or to experiment by running an admissible intervention that changes what is identifiable. This includes proving guarantees for safety and sample efficiency, making use of structural knowledge, and developing scalable algorithms instantiated in high-stakes settings, such as learning-based control in robotics and causal intervention design in healthcare.