AHCI RESEARCH GROUP
Publications
Papers published in international journals,
proceedings of conferences, workshops and books.
OUR RESEARCH
Scientific Publications
How to
Here you can find the complete list of our publications.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
2025
Islayem, R.; Musamih, A.; Salah, K.; Jayaraman, R.; Yaqoob, I.
Enhancing medical digital twins within metaverse using blockchain, NFTs and LLMs Journal Article
In: Internet of Things (The Netherlands), vol. 32, 2025, ISSN: 25426605 (ISSN).
Abstract | Links | BibTeX | Tags: Blockchain, Digital Twins, LLMs, Metaverse, NFTs
@article{islayem_enhancing_2025,
title = {Enhancing medical digital twins within metaverse using blockchain, NFTs and LLMs},
author = {R. Islayem and A. Musamih and K. Salah and R. Jayaraman and I. Yaqoob},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105006652785&doi=10.1016%2fj.iot.2025.101648&partnerID=40&md5=fe58942eabd12704a207b40c2755c2ff},
doi = {10.1016/j.iot.2025.101648},
issn = {25426605 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {Internet of Things (The Netherlands)},
volume = {32},
abstract = {Medical digital twins (MDTs) are rapidly emerging as transformative tools in healthcare. They provide virtual representations of medical devices and systems that facilitate real-time analysis and enhance decision-making. However, challenges such as secure data management, access control, and the lack of immersive and intelligent patient interactions limit their effectiveness. In this paper, we propose a solution integrating blockchain technology, Non-Fungible Tokens (NFTs), and Large Language Models (LLMs) within a metaverse environment to enhance MDT functionality. Blockchain and NFTs ensure secure ownership and access control, while the metaverse offers an engaging platform for user interaction. An LLM-powered non-player character (NPC) enables intelligent real-time user interactions and personalized insights. We develop two blockchain smart contracts for user registration, NFT ownership, and access control, and utilize decentralized InterPlanetary File System (IPFS) storage for the metaverse, MDT metadata, and interaction logs. We present the system architecture, sequence diagrams, and algorithms, along with the implementation and testing details. We conduct cost, security, and response time analyses to evaluate the smart contracts and LLM performance and compare our solution with existing approaches. We discuss practical implications, as well as challenges and limitations of the proposed solution. Finally, we explore the generalization of our system for various applications. The smart contract code and metaverse files are publicly available on GitHub. © 2025 The Authors},
keywords = {Blockchain, Digital Twins, LLMs, Metaverse, NFTs},
pubstate = {published},
tppubtype = {article}
}
Medical digital twins (MDTs) are rapidly emerging as transformative tools in healthcare. They provide virtual representations of medical devices and systems that facilitate real-time analysis and enhance decision-making. However, challenges such as secure data management, access control, and the lack of immersive and intelligent patient interactions limit their effectiveness. In this paper, we propose a solution integrating blockchain technology, Non-Fungible Tokens (NFTs), and Large Language Models (LLMs) within a metaverse environment to enhance MDT functionality. Blockchain and NFTs ensure secure ownership and access control, while the metaverse offers an engaging platform for user interaction. An LLM-powered non-player character (NPC) enables intelligent real-time user interactions and personalized insights. We develop two blockchain smart contracts for user registration, NFT ownership, and access control, and utilize decentralized InterPlanetary File System (IPFS) storage for the metaverse, MDT metadata, and interaction logs. We present the system architecture, sequence diagrams, and algorithms, along with the implementation and testing details. We conduct cost, security, and response time analyses to evaluate the smart contracts and LLM performance and compare our solution with existing approaches. We discuss practical implications, as well as challenges and limitations of the proposed solution. Finally, we explore the generalization of our system for various applications. The smart contract code and metaverse files are publicly available on GitHub. © 2025 The Authors
2024
Gao, H.; Huai, H.; Yildiz-Degirmenci, S.; Bannert, M.; Kasneci, E.
DataliVR: Transformation of Data Literacy Education through Virtual Reality with ChatGPT-Powered Enhancements Proceedings Article
In: U., Eck; M., Sra; J., Stefanucci; M., Sugimoto; M., Tatzgern; I., Williams (Ed.): Proc. - IEEE Int. Symp. Mixed Augment. Real., ISMAR, pp. 120–129, Institute of Electrical and Electronics Engineers Inc., 2024, ISBN: 979-833151647-5 (ISBN).
Abstract | Links | BibTeX | Tags: Adversarial machine learning, Chatbots, ChatGPT, Contrastive Learning, Data driven, Data literacy, Digital transformation, Federated learning, Immersive learning, Language Model, Large language model, Learning experiences, Learning outcome, LLMs, Virtual environments, Virtual Reality
@inproceedings{gao_datalivr_2024,
title = {DataliVR: Transformation of Data Literacy Education through Virtual Reality with ChatGPT-Powered Enhancements},
author = {H. Gao and H. Huai and S. Yildiz-Degirmenci and M. Bannert and E. Kasneci},
editor = {Eck U. and Sra M. and Stefanucci J. and Sugimoto M. and Tatzgern M. and Williams I.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85213525613&doi=10.1109%2fISMAR62088.2024.00026&partnerID=40&md5=abdeba7ecfecc8b1d715d633a29bd11d},
doi = {10.1109/ISMAR62088.2024.00026},
isbn = {979-833151647-5 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Proc. - IEEE Int. Symp. Mixed Augment. Real., ISMAR},
pages = {120–129},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {Data literacy is essential in today's data-driven world, emphasizing individuals' abilities to effectively manage data and extract meaningful insights. However, traditional classroom-based educational approaches often struggle to fully address the multifaceted nature of data literacy. As education undergoes digital transformation, innovative technologies such as Virtual Reality (VR) offer promising avenues for immersive and engaging learning experiences. This paper introduces DataliVR, a pioneering VR application aimed at enhancing the data literacy skills of university students within a contextual and gamified virtual learning environment. By integrating Large Language Models (LLMs) like ChatGPT as a conversational artificial intelligence (AI) chatbot embodied within a virtual avatar, DataliVR provides personalized learning assistance, enriching user learning experiences. Our study employed an experimental approach, with chatbot availability as the independent variable, analyzing learning experiences and outcomes as dependent variables with a sample of thirty participants. Our approach underscores the effectiveness and user-friendliness of ChatGPT-powered DataliVR in fostering data literacy skills. Moreover, our study examines the impact of the ChatGPT-based AI chatbot on users' learning, revealing significant effects on both learning experiences and outcomes. Our study presents a robust tool for fostering data literacy skills, contributing significantly to the digital advancement of data literacy education through cutting-edge VR and AI technologies. Moreover, our research provides valuable insights and implications for future research endeavors aiming to integrate LLMs (e.g., ChatGPT) into educational VR platforms. © 2024 IEEE.},
keywords = {Adversarial machine learning, Chatbots, ChatGPT, Contrastive Learning, Data driven, Data literacy, Digital transformation, Federated learning, Immersive learning, Language Model, Large language model, Learning experiences, Learning outcome, LLMs, Virtual environments, Virtual Reality},
pubstate = {published},
tppubtype = {inproceedings}
}
Data literacy is essential in today's data-driven world, emphasizing individuals' abilities to effectively manage data and extract meaningful insights. However, traditional classroom-based educational approaches often struggle to fully address the multifaceted nature of data literacy. As education undergoes digital transformation, innovative technologies such as Virtual Reality (VR) offer promising avenues for immersive and engaging learning experiences. This paper introduces DataliVR, a pioneering VR application aimed at enhancing the data literacy skills of university students within a contextual and gamified virtual learning environment. By integrating Large Language Models (LLMs) like ChatGPT as a conversational artificial intelligence (AI) chatbot embodied within a virtual avatar, DataliVR provides personalized learning assistance, enriching user learning experiences. Our study employed an experimental approach, with chatbot availability as the independent variable, analyzing learning experiences and outcomes as dependent variables with a sample of thirty participants. Our approach underscores the effectiveness and user-friendliness of ChatGPT-powered DataliVR in fostering data literacy skills. Moreover, our study examines the impact of the ChatGPT-based AI chatbot on users' learning, revealing significant effects on both learning experiences and outcomes. Our study presents a robust tool for fostering data literacy skills, contributing significantly to the digital advancement of data literacy education through cutting-edge VR and AI technologies. Moreover, our research provides valuable insights and implications for future research endeavors aiming to integrate LLMs (e.g., ChatGPT) into educational VR platforms. © 2024 IEEE.
Amato, N.; Carolis, B. De; Gioia, F.; Venezia, M. N.; Palestra, G.; Loglisci, C.
Can an AI-driven VTuber engage People? The KawAIi Case Study Proceedings Article
In: A., Soto; E., Zangerle (Ed.): CEUR Workshop Proc., CEUR-WS, 2024, ISBN: 16130073 (ISSN).
Abstract | Links | BibTeX | Tags: 3D Avatars, Case-studies, Conversational Agents, Facial Expressions, Language Model, Live streaming, LLM, LLMs, Real- time, Three dimensional computer graphics, Virtual agent, Virtual Reality, YouTube
@inproceedings{amato_can_2024,
title = {Can an AI-driven VTuber engage People? The KawAIi Case Study},
author = {N. Amato and B. De Carolis and F. Gioia and M. N. Venezia and G. Palestra and C. Loglisci},
editor = {Soto A. and Zangerle E.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85190754935&partnerID=40&md5=bd76d56b13e328027aa1b458849cf73f},
isbn = {16130073 (ISSN)},
year = {2024},
date = {2024-01-01},
booktitle = {CEUR Workshop Proc.},
volume = {3660},
publisher = {CEUR-WS},
abstract = {Live streaming has become increasingly popular, with most streamers presenting their real-life appearance. However, Virtual YouTubers (VTubers), virtual 2D or 3D avatars that are voiced by humans, are emerging as live streamers and attracting a growing viewership. This paper presents the development of a conversational agent, named KawAIi, embodied in a 2D character that, while accurately and promptly responding to user requests, provides an entertaining experience in streaming chat platforms such as YouTube while providing adequate real-time support. The agent relies on the Vicuna 7B GPTQ 4-bit Large Language Model (LLM). In addition, KawAIi uses a BERT-based model for analyzing the sentence generated by the model in terms of conveyed emotion and shows self-emotion awareness through facial expressions. Tested with users, the system has demonstrated a good ability to handle the interaction with the user while maintaining a pleasant user experience. In particular, KawAIi has been evaluated positively in terms of engagement and competence on various topics. The results show the potential of this technology to enrich interactivity in streaming platforms and offer a promising model for future online assistance contexts. © 2024 Copyright for this paper by its authors.},
keywords = {3D Avatars, Case-studies, Conversational Agents, Facial Expressions, Language Model, Live streaming, LLM, LLMs, Real- time, Three dimensional computer graphics, Virtual agent, Virtual Reality, YouTube},
pubstate = {published},
tppubtype = {inproceedings}
}
Live streaming has become increasingly popular, with most streamers presenting their real-life appearance. However, Virtual YouTubers (VTubers), virtual 2D or 3D avatars that are voiced by humans, are emerging as live streamers and attracting a growing viewership. This paper presents the development of a conversational agent, named KawAIi, embodied in a 2D character that, while accurately and promptly responding to user requests, provides an entertaining experience in streaming chat platforms such as YouTube while providing adequate real-time support. The agent relies on the Vicuna 7B GPTQ 4-bit Large Language Model (LLM). In addition, KawAIi uses a BERT-based model for analyzing the sentence generated by the model in terms of conveyed emotion and shows self-emotion awareness through facial expressions. Tested with users, the system has demonstrated a good ability to handle the interaction with the user while maintaining a pleasant user experience. In particular, KawAIi has been evaluated positively in terms of engagement and competence on various topics. The results show the potential of this technology to enrich interactivity in streaming platforms and offer a promising model for future online assistance contexts. © 2024 Copyright for this paper by its authors.