My research will focus on sequential decision-making systems that remain effective under non-stationarity, with emphasis on Bayesian experimental design (BED) and preference elicitation in human-in-the-loop (HILO) settings. Many existing methods assume that the environment and the user feedback process remain stable over time. In practice, experimental conditions may change, models may be misspecified, and users' preferences, beliefs, or knowledge may evolve during interaction. This motivates methods that can detect and adapt to shifts while still using past information effectively. The first direction studies BED under non-stationary settings, as in those arising from dynamical models or changing downstream objectives. This work will examine how existing BED methods generalize under shift and how robustness can be improved. To this end, we are interested in exploring ideas from the continual learning literature, including for example continual adaptation/updates of policies, selective memory, or regularization. The second direction studies preference elicitation for HILO systems. This project will investigate how changes in preferences, beliefs, or knowledge can be modelled and detected, and how prior-informed models from past or synthetic users can support faster adaptation with less new feedback. Depending on the setting, this may involve learning individual-level preferences, shared population-level structure, or suitable mechanisms to address diverse preferences.