Multimodal large language models (MLLMs) are moving affect understanding from fixed emotion labels towards open-vocabulary interpretations of audio, video and text (AVT). However, their outputs may be unreliable because relevant evidence is not located, observed, combined or interpreted correctly, while the same evidence may also expose identity and other sensitive information. The project therefore treats source search, source observation, evidence fusion and affective interpretation as distinct but connected failure stages. It aims to make affective claims traceable to native AVT sources, support context- and relation-aware evidence use, adapt reasoning and evidence access to input complexity, and audit privacy effects stage by stage. The research will primarily reuse complementary existing datasets covering open-vocabulary AVT, conversational interaction and temporally annotated visual events; no new participant AVT collection is currently planned, and targeted human verification and all data use will follow the applicable licences and consent conditions. The first study will construct a verified diagnostic layer and controlled interventions to identify where current systems fail. The second study will develop a static source-grounded interface that uses a fixed, once-selected evidence set under matched computation and source-access budgets. The third study will extend this interface with a sequential policy that decides whether to deepen reasoning, acquire or verify further evidence, answer, request context or abstain. The fourth study will compare protected and unprotected variants of the same evidence chain in a declared local-remote setting and audit identity, linkage and sensitive-disclosure risks across native inputs, generated observations, evidence certificates and outputs. Evaluation will combine affect utility, source validity, relation-conditioned behaviour, calibration, selective risk, computation and access cost, and empirical privacy attacks to characterise privacy-utility-faithfulness trade-offs. Expected contributions are a reusable failure-diagnosis protocol, a static grounding method, a complexity-adaptive evidence-acquisition and stopping policy, and an evidence-chain privacy audit. By connecting affective evidence attribution, sequential decision-making and privacy assessment, the project seeks to make multimodal Al more auditable, resource-aware and suitable for sensitive human-centred applications.
