Facial Expression Applications and Analysis using Micro- and Macro-Expressions

Mengting Wei (Ph.D. Student)

Facial expressions (FE) are a kind of very intuitive and powerful signal to disclose people's emotions. With the development of deep learning methods, many efforts have been committed into predicting emotional states via facial expressions. In this research, we'll push this envelop further. First, we adopt FE as an important indicator to predict the occurrence of sundown syndrome (SS). To do so, we'll first build a SS dataset by collecting signals from SS subjects. These signals mainly include behaviors (e.g., cognitive reports, facial and bodily expressions) and physiological states (e.g., EEG, ECG and GSR signals). Then we'll benchmark the FE data of this dataset to highlight some basic features that may highly related to the onset of SS. Next, the second job of the research is to explore the relationship between macro- and micro- expressions. Currently, a general perception concerning these two kinds of FE is the difference of expression intensity, where the macro-expression often presents much larger intensity than micro-expression. We will investigate whether micro-expression could provide additional information about emotions, or whether the performance of recognizing macro-expressions is just the upper bound of that of recognizing micro-expressions. Then, based on the result of the second work, we'll try to use macro-expressions to devise more effective magnification approach for micro-expressions. Magnification as a crucial strategy in micro-expression recognition has long been relying on other magnification techniques, e.g., Eulerian video magnification, Learning-based video magnification. There's still no technique tailored for magnifying micro-expressions. Therefore, we will consider using macro-expressions to achieve more remarkable magnification results of micro-expressions. Finally, based on the above investigations mentioned, we'll push forward the research of in-the-wild micro-expressions. Existing magnification techniques cannot handle the magnification under unconstrained situations. We think the new magnification strategy completed in the third job could help the study of this new condition, and then further promote real applications of micro-expressions.

Primary Host:	Guoying Zhao (University of Oulu)
Exchange Host:	Nuria Oliver (ELLIS Alicante Unit Foundation | Institute of Humanity-centric AI)
PhD Duration:	01 September 2023 - 01 September 2027
Exchange Duration:	01 January 2025 - 30 June 2025 - Ongoing