AHCI RESEARCH GROUP
Publications
Papers published in international journals,
proceedings of conferences, workshops and books.
OUR RESEARCH
Scientific Publications
How to
Here you can find the complete list of our publications.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
2024
Christiansen, F. R.; Hollensberg, L. Nø.; Jensen, N. B.; Julsgaard, K.; Jespersen, K. N.; Nikolov, I.
Exploring Presence in Interactions with LLM-Driven NPCs: A Comparative Study of Speech Recognition and Dialogue Options Proceedings Article
In: S.N., Spencer (Ed.): Proc. ACM Symp. Virtual Reality Softw. Technol. VRST, Association for Computing Machinery, 2024, ISBN: 979-840070535-9 (ISBN).
Abstract | Links | BibTeX | Tags: Comparatives studies, Computer simulation languages, Economic and social effects, Immersive System, Immersive systems, Language Model, Large language model, Large language models (LLM), Model-driven, Modern technologies, Non-playable character, NPC, Presence, Social Actors, Speech enhancement, Speech recognition, Text to speech, Virtual environments, Virtual Reality, VR
@inproceedings{christiansen_exploring_2024,
title = {Exploring Presence in Interactions with LLM-Driven NPCs: A Comparative Study of Speech Recognition and Dialogue Options},
author = {F. R. Christiansen and L. Nø. Hollensberg and N. B. Jensen and K. Julsgaard and K. N. Jespersen and I. Nikolov},
editor = {Spencer S.N.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85212512351&doi=10.1145%2f3641825.3687716&partnerID=40&md5=56ec6982b399fd97196ea73e7c659c31},
doi = {10.1145/3641825.3687716},
isbn = {979-840070535-9 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Proc. ACM Symp. Virtual Reality Softw. Technol. VRST},
publisher = {Association for Computing Machinery},
abstract = {Combining modern technologies like large-language models (LLMs), speech-to-text, and text-to-speech can enhance immersion in virtual reality (VR) environments. However, challenges exist in effectively implementing LLMs and educating users. This paper explores implementing LLM-powered virtual social actors and facilitating user communication. We developed a murder mystery game where users interact with LLM-based non-playable characters (NPCs) through interrogation, clue-gathering, and exploration. Two versions were tested: one using speech recognition and another with traditional dialog boxes. While both provided similar social presence, users felt more immersed with speech recognition but found it overwhelming, while the dialog version was more challenging. Slow NPC response times were a source of frustration, highlighting the need for faster generation or better masking for a seamless experience. © 2024 Owner/Author.},
keywords = {Comparatives studies, Computer simulation languages, Economic and social effects, Immersive System, Immersive systems, Language Model, Large language model, Large language models (LLM), Model-driven, Modern technologies, Non-playable character, NPC, Presence, Social Actors, Speech enhancement, Speech recognition, Text to speech, Virtual environments, Virtual Reality, VR},
pubstate = {published},
tppubtype = {inproceedings}
}
Combining modern technologies like large-language models (LLMs), speech-to-text, and text-to-speech can enhance immersion in virtual reality (VR) environments. However, challenges exist in effectively implementing LLMs and educating users. This paper explores implementing LLM-powered virtual social actors and facilitating user communication. We developed a murder mystery game where users interact with LLM-based non-playable characters (NPCs) through interrogation, clue-gathering, and exploration. Two versions were tested: one using speech recognition and another with traditional dialog boxes. While both provided similar social presence, users felt more immersed with speech recognition but found it overwhelming, while the dialog version was more challenging. Slow NPC response times were a source of frustration, highlighting the need for faster generation or better masking for a seamless experience. © 2024 Owner/Author.
2023
Sabatucci, Luca; Augello, Agnese; Caggianese, Giuseppe; Gallo, Luigi
Envisioning Digital Practices in the Metaverse: A Methodological Perspective Journal Article
In: Future Internet, vol. 15, no. 12, pp. 1–19, 2023, ISSN: 1999-5903, (Number: 394 Publisher: Multidisciplinary Digital Publishing Institute).
Abstract | Links | BibTeX | Tags: Digital Practices, Digital Twins, Metaverse, NPC, Social Practices
@article{sabatucci_envisioning_2023,
title = {Envisioning Digital Practices in the Metaverse: A Methodological Perspective},
author = {Luca Sabatucci and Agnese Augello and Giuseppe Caggianese and Luigi Gallo},
url = {https://www.mdpi.com/1999-5903/15/12/394},
doi = {10.3390/fi15120394},
issn = {1999-5903},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-07},
journal = {Future Internet},
volume = {15},
number = {12},
pages = {1–19},
abstract = {Researchers are exploring methods that exploit digital twins as all-purpose abstractions for sophisticated modelling and simulation, bringing elements of the real world into the virtual realm. Digital twins are essential elements of the digital transformation of society, which mostly benefit manufacturing, smart cities, healthcare contexts, and in general systems that include humans in the loop. As the metaverse concept continues to evolve, the line separating the virtual and the real will progressively fade away. Considering the metaverse’s goal to emulate our social reality, it becomes essential to examine the aspects that characterise real-world interaction practices and explicitly model both physical and social contexts. While the unfolding metaverse may reshape these practices in distinct ways from their real-world counterparts, our position is that it is essential to incorporate social theories into the modelling processes of digital twins within the metaverse. In this work, we discuss our perspective by introducing a digital practice model inspired by the theory of social practice. We illustrate this model by exploiting the scenario of a virtual grocery shop designed to help older adults reduce their social isolation.},
note = {Number: 394
Publisher: Multidisciplinary Digital Publishing Institute},
keywords = {Digital Practices, Digital Twins, Metaverse, NPC, Social Practices},
pubstate = {published},
tppubtype = {article}
}
Researchers are exploring methods that exploit digital twins as all-purpose abstractions for sophisticated modelling and simulation, bringing elements of the real world into the virtual realm. Digital twins are essential elements of the digital transformation of society, which mostly benefit manufacturing, smart cities, healthcare contexts, and in general systems that include humans in the loop. As the metaverse concept continues to evolve, the line separating the virtual and the real will progressively fade away. Considering the metaverse’s goal to emulate our social reality, it becomes essential to examine the aspects that characterise real-world interaction practices and explicitly model both physical and social contexts. While the unfolding metaverse may reshape these practices in distinct ways from their real-world counterparts, our position is that it is essential to incorporate social theories into the modelling processes of digital twins within the metaverse. In this work, we discuss our perspective by introducing a digital practice model inspired by the theory of social practice. We illustrate this model by exploiting the scenario of a virtual grocery shop designed to help older adults reduce their social isolation.
2022
Gatto, Luigi; Gaglio, Giuseppe Fulvio; Augello, Agnese; Caggianese, Giuseppe; Gallo, Luigi; Cascia, Marco La
MET-iquette: Enabling Virtual Agents to Have a Social Compliant Behavior in the Metaverse Proceedings Article
In: 2022 16th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), pp. 394–401, 2022.
Abstract | Links | BibTeX | Tags: Avatars, Cognition, Metaverse, NPC, Scene Understanding, Social Agents, Social Practices
@inproceedings{gattoMETiquetteEnablingVirtual2022a,
title = {MET-iquette: Enabling Virtual Agents to Have a Social Compliant Behavior in the Metaverse},
author = { Luigi Gatto and Giuseppe Fulvio Gaglio and Agnese Augello and Giuseppe Caggianese and Luigi Gallo and Marco La Cascia},
doi = {10.1109/SITIS57111.2022.00066},
year = {2022},
date = {2022-10-01},
booktitle = {2022 16th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS)},
pages = {394--401},
abstract = {Metaverse aims at connecting the physical and digital world in a highly immersive virtual environment that can replicate society and social practices within it. It is populated not only by avatars corresponding to users but also by nonplayer characters (NPCs). These virtual agents should be able to perceive other agents and objects in the scene, reason and act preferably by exhibiting credible and acceptable social behavior. Relying on the Social Practice theory, this work proposes an architecture for the design and development of a virtual agent able to understand social practices in the Metaverse and act by respecting the social norms that the circumstances entail. The agent is equipped with modules for recognizing elements of the scene, knowledge and rules about a set of social practices. The scene information is exploited by a reasoning module that allows the agent to evaluate the current social practice, socially interpret the events, and activate the most appropriate behaviors. The paper describes the proposed architecture, its implementation, and the realization of an experimental virtual scenario.},
keywords = {Avatars, Cognition, Metaverse, NPC, Scene Understanding, Social Agents, Social Practices},
pubstate = {published},
tppubtype = {inproceedings}
}
Metaverse aims at connecting the physical and digital world in a highly immersive virtual environment that can replicate society and social practices within it. It is populated not only by avatars corresponding to users but also by nonplayer characters (NPCs). These virtual agents should be able to perceive other agents and objects in the scene, reason and act preferably by exhibiting credible and acceptable social behavior. Relying on the Social Practice theory, this work proposes an architecture for the design and development of a virtual agent able to understand social practices in the Metaverse and act by respecting the social norms that the circumstances entail. The agent is equipped with modules for recognizing elements of the scene, knowledge and rules about a set of social practices. The scene information is exploited by a reasoning module that allows the agent to evaluate the current social practice, socially interpret the events, and activate the most appropriate behaviors. The paper describes the proposed architecture, its implementation, and the realization of an experimental virtual scenario.
Gatto, Luigi; Gaglio, Giuseppe Fulvio; Augello, Agnese; Caggianese, Giuseppe; Gallo, Luigi; Cascia, Marco La
MET-iquette: enabling virtual agents to have a social compliant behavior in the Metaverse Proceedings Article
In: 2022 16th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), pp. 394–401, 2022.
Abstract | Links | BibTeX | Tags: Avatars, Cognition, Emotional Appraisal, Metaverse, Model of Emotion, NPC, OCC, Reasoning, Scene Understanding, Social Agents, Social Practices
@inproceedings{gatto_met-iquette_2022,
title = {MET-iquette: enabling virtual agents to have a social compliant behavior in the Metaverse},
author = {Luigi Gatto and Giuseppe Fulvio Gaglio and Agnese Augello and Giuseppe Caggianese and Luigi Gallo and Marco La Cascia},
doi = {10.1109/SITIS57111.2022.00066},
year = {2022},
date = {2022-10-01},
booktitle = {2022 16th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS)},
pages = {394–401},
abstract = {Metaverse aims at connecting the physical and digital world in a highly immersive virtual environment that can replicate society and social practices within it. It is populated not only by avatars corresponding to users but also by nonplayer characters (NPCs). These virtual agents should be able to perceive other agents and objects in the scene, reason and act preferably by exhibiting credible and acceptable social behavior. Relying on the Social Practice theory, this work proposes an architecture for the design and development of a virtual agent able to understand social practices in the Metaverse and act by respecting the social norms that the circumstances entail. The agent is equipped with modules for recognizing elements of the scene, knowledge and rules about a set of social practices. The scene information is exploited by a reasoning module that allows the agent to evaluate the current social practice, socially interpret the events, and activate the most appropriate behaviors. The paper describes the proposed architecture, its implementation, and the realization of an experimental virtual scenario.},
keywords = {Avatars, Cognition, Emotional Appraisal, Metaverse, Model of Emotion, NPC, OCC, Reasoning, Scene Understanding, Social Agents, Social Practices},
pubstate = {published},
tppubtype = {inproceedings}
}
Metaverse aims at connecting the physical and digital world in a highly immersive virtual environment that can replicate society and social practices within it. It is populated not only by avatars corresponding to users but also by nonplayer characters (NPCs). These virtual agents should be able to perceive other agents and objects in the scene, reason and act preferably by exhibiting credible and acceptable social behavior. Relying on the Social Practice theory, this work proposes an architecture for the design and development of a virtual agent able to understand social practices in the Metaverse and act by respecting the social norms that the circumstances entail. The agent is equipped with modules for recognizing elements of the scene, knowledge and rules about a set of social practices. The scene information is exploited by a reasoning module that allows the agent to evaluate the current social practice, socially interpret the events, and activate the most appropriate behaviors. The paper describes the proposed architecture, its implementation, and the realization of an experimental virtual scenario.