AHCI RESEARCH GROUP

Publications

Papers published in international journals,
proceedings of conferences, workshops and books.

OUR RESEARCH

Scientific Publications

How to

Here you can find the complete list of our publications.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.

Show all

2023

Marín-Morales, J.; Llanes-Jurado, J.; Minissi, M. E.; Gómez-Zaragozá, L.; Altozano, A.; Alcaniz, M.

Gaze and Head Movement Patterns of Depressive Symptoms During Conversations with Emotional Virtual Humans Proceedings Article

In: Int. Conf. Affect. Comput. Intell. Interact., ACII, Institute of Electrical and Electronics Engineers Inc., 2023, ISBN: 979-835032743-4 (ISBN).

Abstract | Links | BibTeX | Tags: Biomarkers, Clustering, Clusterings, Computational Linguistics, Depressive disorder, Depressive symptom, E-Learning, Emotion elicitation, Eye movements, Gaze movements, K-means clustering, Language Model, Large language model, large language models, Learning systems, Mental health, Multivariant analysis, Signal processing, Statistical learning, virtual human, Virtual humans, Virtual Reality

@inproceedings{marin-morales_gaze_2023,

title = {Gaze and Head Movement Patterns of Depressive Symptoms During Conversations with Emotional Virtual Humans},

author = {J. Marín-Morales and J. Llanes-Jurado and M. E. Minissi and L. Gómez-Zaragozá and A. Altozano and M. Alcaniz},

url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85184656388&doi=10.1109%2fACII59096.2023.10388134&partnerID=40&md5=143cdd8530e17a7b64bdf88f3a0496ab},

doi = {10.1109/ACII59096.2023.10388134},

isbn = {979-835032743-4 (ISBN)},

year  = {2023},

date = {2023-01-01},

booktitle = {Int. Conf. Affect. Comput. Intell. Interact., ACII},

publisher = {Institute of Electrical and Electronics Engineers Inc.},

abstract = {Depressive symptoms involve dysfunctional social attitudes and heightened negative emotional states. Identifying biomarkers requires data collection in realistic environments that activate depression-specific phenomena. However, no previous research analysed biomarkers in combination with AI-powered conversational virtual humans (VH) for mental health assessment. This study aims to explore gaze and head movements patterns related to depressive symptoms during conversations with emotional VH. A total of 105 participants were evenly divided into a control group and a group of subjects with depressive symptoms (SDS). They completed six semi-guided conversations designed to evoke basic emotions. The VHs were developed using a cognitive-inspired framework, enabling real-time voice-based conversational interactions powered by a Large Language Model, and including emotional facial expressions and lip synchronization. They have embedded life-history, context, attitudes, emotions and motivations. Signal processing techniques were applied to obtain gaze and head movements features, and heatmaps were generated. Then, parametric and non-parametric statistical tests were applied to evaluate differences between groups. Additionally, a two-dimensional t-SNE embedding was created and combined with k-means clustering. Results indicate that SDS exhibited shorter blinks and longer saccades. The control group showed affiliative lateral head gyros and accelerations, while the SDS demonstrated stress-related back-and-forth movements. SDS also displayed the avoidance of eye contact. The exploratory multivariate statistical unsupervised learning achieved 72.3% accuracy. The present study analyse biomarkers in affective processes with multiple social contextual factors and information modalities in ecological environments, and enhances our understanding of gaze and head movements patterns in individuals with depressive symptoms, ultimately contributing to the development of more effective assessments and intervention strategies. © 2023 IEEE.},

keywords = {Biomarkers, Clustering, Clusterings, Computational Linguistics, Depressive disorder, Depressive symptom, E-Learning, Emotion elicitation, Eye movements, Gaze movements, K-means clustering, Language Model, Large language model, large language models, Learning systems, Mental health, Multivariant analysis, Signal processing, Statistical learning, virtual human, Virtual humans, Virtual Reality},

pubstate = {published},

tppubtype = {inproceedings}

}

Depressive symptoms involve dysfunctional social attitudes and heightened negative emotional states. Identifying biomarkers requires data collection in realistic environments that activate depression-specific phenomena. However, no previous research analysed biomarkers in combination with AI-powered conversational virtual humans (VH) for mental health assessment. This study aims to explore gaze and head movements patterns related to depressive symptoms during conversations with emotional VH. A total of 105 participants were evenly divided into a control group and a group of subjects with depressive symptoms (SDS). They completed six semi-guided conversations designed to evoke basic emotions. The VHs were developed using a cognitive-inspired framework, enabling real-time voice-based conversational interactions powered by a Large Language Model, and including emotional facial expressions and lip synchronization. They have embedded life-history, context, attitudes, emotions and motivations. Signal processing techniques were applied to obtain gaze and head movements features, and heatmaps were generated. Then, parametric and non-parametric statistical tests were applied to evaluate differences between groups. Additionally, a two-dimensional t-SNE embedding was created and combined with k-means clustering. Results indicate that SDS exhibited shorter blinks and longer saccades. The control group showed affiliative lateral head gyros and accelerations, while the SDS demonstrated stress-related back-and-forth movements. SDS also displayed the avoidance of eye contact. The exploratory multivariate statistical unsupervised learning achieved 72.3% accuracy. The present study analyse biomarkers in affective processes with multiple social contextual factors and information modalities in ecological environments, and enhances our understanding of gaze and head movements patterns in individuals with depressive symptoms, ultimately contributing to the development of more effective assessments and intervention strategies. © 2023 IEEE.