AHCI RESEARCH GROUP
Publications
Papers published in international journals,
proceedings of conferences, workshops and books.
OUR RESEARCH
Scientific Publications
How to
Here you can find the complete list of our publications.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
2025
Ly, D. -N.; Do, H. -N.; Tran, M. -T.; Le, K. -D.
Evaluation of AI-Based Assistant Representations on User Interaction in Virtual Explorations Proceedings Article
In: W., Buntine; M., Fjeld; T., Tran; M.-T., Tran; B., Huynh Thi Thanh; T., Miyoshi (Ed.): Commun. Comput. Info. Sci., pp. 323–337, Springer Science and Business Media Deutschland GmbH, 2025, ISBN: 18650929 (ISSN); 978-981964287-8 (ISBN).
Abstract | Links | BibTeX | Tags: 360-degree Video, AI-Based Assistant, Cultural heritages, Cultural science, Multiusers, Single users, Social interactions, Three dimensional computer graphics, User interaction, Users' experiences, Virtual environments, Virtual Exploration, Virtual Reality, Virtualization
@inproceedings{ly_evaluation_2025,
title = {Evaluation of AI-Based Assistant Representations on User Interaction in Virtual Explorations},
author = {D. -N. Ly and H. -N. Do and M. -T. Tran and K. -D. Le},
editor = {Buntine W. and Fjeld M. and Tran T. and Tran M.-T. and Huynh Thi Thanh B. and Miyoshi T.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105004253350&doi=10.1007%2f978-981-96-4288-5_26&partnerID=40&md5=5f0a8c1e356cd3bdd4dda7f96f272154},
doi = {10.1007/978-981-96-4288-5_26},
isbn = {18650929 (ISSN); 978-981964287-8 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Commun. Comput. Info. Sci.},
volume = {2352 CCIS},
pages = {323–337},
publisher = {Springer Science and Business Media Deutschland GmbH},
abstract = {Exploration activities, such as tourism, cultural heritage, and science, enhance knowledge and understanding. The rise of 360-degree videos allows users to explore cultural landmarks and destinations remotely. While multi-user VR environments encourage collaboration, single-user experiences often lack social interaction. Generative AI, particularly Large Language Models (LLMs), offer a way to improve single-user VR exploration through AI-driven virtual assistants, acting as tour guides or storytellers. However, it’s uncertain whether these assistants require a visual presence, and if so, what form it should take. To investigate this, we developed an AI-based assistant in three different forms: a voice-only avatar, a 3D human-sized avatar, and a mini-hologram avatar, and conducted a user study to evaluate their impact on user experience. The study, which involved 12 participants, found that the visual embodiments significantly reduce feelings of being alone, with distinct user preferences between the Human-sized avatar and the Mini hologram. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.},
keywords = {360-degree Video, AI-Based Assistant, Cultural heritages, Cultural science, Multiusers, Single users, Social interactions, Three dimensional computer graphics, User interaction, Users' experiences, Virtual environments, Virtual Exploration, Virtual Reality, Virtualization},
pubstate = {published},
tppubtype = {inproceedings}
}
Exploration activities, such as tourism, cultural heritage, and science, enhance knowledge and understanding. The rise of 360-degree videos allows users to explore cultural landmarks and destinations remotely. While multi-user VR environments encourage collaboration, single-user experiences often lack social interaction. Generative AI, particularly Large Language Models (LLMs), offer a way to improve single-user VR exploration through AI-driven virtual assistants, acting as tour guides or storytellers. However, it’s uncertain whether these assistants require a visual presence, and if so, what form it should take. To investigate this, we developed an AI-based assistant in three different forms: a voice-only avatar, a 3D human-sized avatar, and a mini-hologram avatar, and conducted a user study to evaluate their impact on user experience. The study, which involved 12 participants, found that the visual embodiments significantly reduce feelings of being alone, with distinct user preferences between the Human-sized avatar and the Mini hologram. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.
2024
Ansari, U.; Qureshi, H. A.; Soomro, N. A.; Memon, A. R.
Augmented Reality-Driven Reservoir Management Via Generative Ai: Transforming Pore-Scale Fluid Flow Simulation Proceedings Article
In: Soc. Pet. Eng. - ADIPEC, Society of Petroleum Engineers, 2024, ISBN: 978-195902549-8 (ISBN).
Abstract | Links | BibTeX | Tags: 'current, AI techniques, Augmented Reality, Decision making, Decisions makings, Efficiency, Finance, Fluid flow simulation, Fluid-flow, Gasoline, High-accuracy, Management tasks, Petroleum refining, Petroleum reservoir evaluation, Pore scale, Real- time, User interaction
@inproceedings{ansari_augmented_2024,
title = {Augmented Reality-Driven Reservoir Management Via Generative Ai: Transforming Pore-Scale Fluid Flow Simulation},
author = {U. Ansari and H. A. Qureshi and N. A. Soomro and A. R. Memon},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85215124881&doi=10.2118%2f222865-MS&partnerID=40&md5=32e8ddc777be342df8196b86a4eb7c60},
doi = {10.2118/222865-MS},
isbn = {978-195902549-8 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Soc. Pet. Eng. - ADIPEC},
publisher = {Society of Petroleum Engineers},
abstract = {The current revolution of generative artificial intelligence is transforming global dynamics which is also essential to petroleum engineers for effectively completing technical tasks.Henceforth the main aim of this study is to investigate the application of generative AI techniques for improving the efficiency of petroleum reservoir management.The outcomes of this study will help in developing and implementing generative AI algorithms tailored for reservoir management tasks, including reservoir modeling, production optimization, and decision support.In this study generative AI technique is employed to integrate with augmented reality (AR) to enhance reservoir management.The methodology involves developing a generative AI model to simulate pore-scale fluid flow, validated against experimental data.AR is utilized to visualize and interact with the simulation results in a real-time, immersive environment.The integration process includes data preprocessing, model training, and AR deployment.Performance metrics such as accuracy, computational efficiency, and user interaction quality are evaluated to assess the effectiveness of the proposed approach in transforming traditional reservoir management practices.The developed generative AI model demonstrated high accuracy in simulating pore-scale fluid flow, closely matching experimental data with a correlation coefficient of 0.95.The AR interface provided an intuitive visualization, significantly improving user comprehension and decision-making efficiency.Computational efficiency was enhanced by 40% compared to traditional methods, enabling real-time simulations and interactions.Moreover, it was observed that Users found the AR-driven approach more engaging and easier to understand, with a reported 30% increase in correct decision-making in reservoir management tasks.The integration of generative AI with AR allowed for dynamic adjustments and immediate feedback, which was particularly beneficial in complex scenarios requiring rapid analysis and response.Concludingly, the combination of generative AI and AR offers a transformative approach to reservoir management, enhancing both the accuracy of simulations and the effectiveness of user interactions.This methodology not only improves computational efficiency but also fosters better decision-making through immersive visualization.Future work will focus on refining the AI model and expanding the AR functionalities to cover a broader range of reservoir conditions and management strategies.This study introduces a novel integration of generative AI and augmented reality (AR) for reservoir management, offering a pioneering approach to pore-scale fluid flow simulation.By combining high-accuracy AI-driven simulations with real-time, immersive AR visualizations, this methodology significantly enhances user interaction and decision-making efficiency.This innovative framework transforms traditional practices, providing a more engaging, efficient, and accurate tool for managing complex reservoir systems. Copyright 2024, Society of Petroleum Engineers.},
keywords = {'current, AI techniques, Augmented Reality, Decision making, Decisions makings, Efficiency, Finance, Fluid flow simulation, Fluid-flow, Gasoline, High-accuracy, Management tasks, Petroleum refining, Petroleum reservoir evaluation, Pore scale, Real- time, User interaction},
pubstate = {published},
tppubtype = {inproceedings}
}
The current revolution of generative artificial intelligence is transforming global dynamics which is also essential to petroleum engineers for effectively completing technical tasks.Henceforth the main aim of this study is to investigate the application of generative AI techniques for improving the efficiency of petroleum reservoir management.The outcomes of this study will help in developing and implementing generative AI algorithms tailored for reservoir management tasks, including reservoir modeling, production optimization, and decision support.In this study generative AI technique is employed to integrate with augmented reality (AR) to enhance reservoir management.The methodology involves developing a generative AI model to simulate pore-scale fluid flow, validated against experimental data.AR is utilized to visualize and interact with the simulation results in a real-time, immersive environment.The integration process includes data preprocessing, model training, and AR deployment.Performance metrics such as accuracy, computational efficiency, and user interaction quality are evaluated to assess the effectiveness of the proposed approach in transforming traditional reservoir management practices.The developed generative AI model demonstrated high accuracy in simulating pore-scale fluid flow, closely matching experimental data with a correlation coefficient of 0.95.The AR interface provided an intuitive visualization, significantly improving user comprehension and decision-making efficiency.Computational efficiency was enhanced by 40% compared to traditional methods, enabling real-time simulations and interactions.Moreover, it was observed that Users found the AR-driven approach more engaging and easier to understand, with a reported 30% increase in correct decision-making in reservoir management tasks.The integration of generative AI with AR allowed for dynamic adjustments and immediate feedback, which was particularly beneficial in complex scenarios requiring rapid analysis and response.Concludingly, the combination of generative AI and AR offers a transformative approach to reservoir management, enhancing both the accuracy of simulations and the effectiveness of user interactions.This methodology not only improves computational efficiency but also fosters better decision-making through immersive visualization.Future work will focus on refining the AI model and expanding the AR functionalities to cover a broader range of reservoir conditions and management strategies.This study introduces a novel integration of generative AI and augmented reality (AR) for reservoir management, offering a pioneering approach to pore-scale fluid flow simulation.By combining high-accuracy AI-driven simulations with real-time, immersive AR visualizations, this methodology significantly enhances user interaction and decision-making efficiency.This innovative framework transforms traditional practices, providing a more engaging, efficient, and accurate tool for managing complex reservoir systems. Copyright 2024, Society of Petroleum Engineers.
Asadi, A. R.; Appiah, J.; Muntaka, S. A.; Kropczynski, J.
Actions, Not Apps: Toward Using LLMs to Reshape Context Aware Interactions in Mixed Reality Systems Proceedings Article
In: C., Stephanidis; M., Antona; S., Ntoa; G., Salvendy (Ed.): Commun. Comput. Info. Sci., pp. 166–176, Springer Science and Business Media Deutschland GmbH, 2024, ISBN: 18650929 (ISSN); 978-303162109-3 (ISBN).
Abstract | Links | BibTeX | Tags: Computation theory, Context Aware System, Context-aware interaction, Context-aware systems, Decision making, Digital information, Flat-screens, Interaction Design, Language Model, Mixed reality, Mixed reality systems, User interaction, User interfaces, User perceptions
@inproceedings{asadi_actions_2024,
title = {Actions, Not Apps: Toward Using LLMs to Reshape Context Aware Interactions in Mixed Reality Systems},
author = {A. R. Asadi and J. Appiah and S. A. Muntaka and J. Kropczynski},
editor = {Stephanidis C. and Antona M. and Ntoa S. and Salvendy G.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85196733497&doi=10.1007%2f978-3-031-62110-9_17&partnerID=40&md5=9cd702ff979c7f111a5172df8f155ddf},
doi = {10.1007/978-3-031-62110-9_17},
isbn = {18650929 (ISSN); 978-303162109-3 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Commun. Comput. Info. Sci.},
volume = {2120 CCIS},
pages = {166–176},
publisher = {Springer Science and Business Media Deutschland GmbH},
abstract = {Mixed reality computing merges user perception of the environment with digital information. As we move from flatscreen computing toward head-mounted computing, the necessity for developing alternative interactions and user flows becomes more evident. Activity theory provides a holistic overview of user interactions and motives. In this work in progress, we propose Action Sandbox Workspace as an interaction framework for the future of MR systems by focusing on action-centric interactions rather than application-centric interactions, aiming to bridge the gap between user goals and system functionalities in everyday tasks. By integrating the ontology of actions, user intentions, and context and connecting it to spatial data mapping, this forward-looking framework aims to create a contextually adaptive user interaction environment. The recent development in large language models (LLMs) has made the implementation of this interaction flow feasible by enabling inference and decision-making based on text-based descriptions of a user’s state and intentions with data and actions users have access to. We propose this approach as a future direction for developing mixed reality platforms and integrating AI in interacting with computers. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.},
keywords = {Computation theory, Context Aware System, Context-aware interaction, Context-aware systems, Decision making, Digital information, Flat-screens, Interaction Design, Language Model, Mixed reality, Mixed reality systems, User interaction, User interfaces, User perceptions},
pubstate = {published},
tppubtype = {inproceedings}
}
Mixed reality computing merges user perception of the environment with digital information. As we move from flatscreen computing toward head-mounted computing, the necessity for developing alternative interactions and user flows becomes more evident. Activity theory provides a holistic overview of user interactions and motives. In this work in progress, we propose Action Sandbox Workspace as an interaction framework for the future of MR systems by focusing on action-centric interactions rather than application-centric interactions, aiming to bridge the gap between user goals and system functionalities in everyday tasks. By integrating the ontology of actions, user intentions, and context and connecting it to spatial data mapping, this forward-looking framework aims to create a contextually adaptive user interaction environment. The recent development in large language models (LLMs) has made the implementation of this interaction flow feasible by enabling inference and decision-making based on text-based descriptions of a user’s state and intentions with data and actions users have access to. We propose this approach as a future direction for developing mixed reality platforms and integrating AI in interacting with computers. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
Guo, Y.; Hou, K.; Yan, Z.; Chen, H.; Xing, G.; Jiang, X.
Sensor2Scene: Foundation Model-Driven Interactive Realities Proceedings Article
In: Proc. - IEEE Int. Workshop Found. Model. Cyber-Phys. Syst. Internet Things, FMSys, pp. 13–19, Institute of Electrical and Electronics Engineers Inc., 2024, ISBN: 979-835036345-6 (ISBN).
Abstract | Links | BibTeX | Tags: 3D modeling, Augmented Reality, Computational Linguistics, Data integration, Data visualization, Foundation models, Generative model, Language Model, Large language model, large language models, Model-driven, Sensor Data Integration, Sensors data, Text-to-3d generative model, Text-to-3D Generative Models, Three dimensional computer graphics, User interaction, User Interaction in AR, User interaction in augmented reality, User interfaces, Virtual Reality, Visualization
@inproceedings{guo_sensor2scene_2024,
title = {Sensor2Scene: Foundation Model-Driven Interactive Realities},
author = {Y. Guo and K. Hou and Z. Yan and H. Chen and G. Xing and X. Jiang},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85199893762&doi=10.1109%2fFMSys62467.2024.00007&partnerID=40&md5=c3bf1739e8c1dc6227d61609ddc66910},
doi = {10.1109/FMSys62467.2024.00007},
isbn = {979-835036345-6 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Proc. - IEEE Int. Workshop Found. Model. Cyber-Phys. Syst. Internet Things, FMSys},
pages = {13–19},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {Augmented Reality (AR) is acclaimed for its potential to bridge the physical and virtual worlds. Yet, current integration between these realms often lacks a deep under-standing of the physical environment and the subsequent scene generation that reflects this understanding. This research introduces Sensor2Scene, a novel system framework designed to enhance user interactions with sensor data through AR. At its core, an AI agent leverages large language models (LLMs) to decode subtle information from sensor data, constructing detailed scene descriptions for visualization. To enable these scenes to be rendered in AR, we decompose the scene creation process into tasks of text-to-3D model generation and spatial composition, allowing new AR scenes to be sketched from the descriptions. We evaluated our framework using an LLM evaluator based on five metrics on various datasets to examine the correlation between sensor readings and corresponding visualizations, and demonstrated the system's effectiveness with scenes generated from end-to-end. The results highlight the potential of LLMs to understand IoT sensor data. Furthermore, generative models can aid in transforming these interpretations into visual formats, thereby enhancing user interaction. This work not only displays the capabilities of Sensor2Scene but also lays a foundation for advancing AR with the goal of creating more immersive and contextually rich experiences. © 2024 IEEE.},
keywords = {3D modeling, Augmented Reality, Computational Linguistics, Data integration, Data visualization, Foundation models, Generative model, Language Model, Large language model, large language models, Model-driven, Sensor Data Integration, Sensors data, Text-to-3d generative model, Text-to-3D Generative Models, Three dimensional computer graphics, User interaction, User Interaction in AR, User interaction in augmented reality, User interfaces, Virtual Reality, Visualization},
pubstate = {published},
tppubtype = {inproceedings}
}
Augmented Reality (AR) is acclaimed for its potential to bridge the physical and virtual worlds. Yet, current integration between these realms often lacks a deep under-standing of the physical environment and the subsequent scene generation that reflects this understanding. This research introduces Sensor2Scene, a novel system framework designed to enhance user interactions with sensor data through AR. At its core, an AI agent leverages large language models (LLMs) to decode subtle information from sensor data, constructing detailed scene descriptions for visualization. To enable these scenes to be rendered in AR, we decompose the scene creation process into tasks of text-to-3D model generation and spatial composition, allowing new AR scenes to be sketched from the descriptions. We evaluated our framework using an LLM evaluator based on five metrics on various datasets to examine the correlation between sensor readings and corresponding visualizations, and demonstrated the system's effectiveness with scenes generated from end-to-end. The results highlight the potential of LLMs to understand IoT sensor data. Furthermore, generative models can aid in transforming these interpretations into visual formats, thereby enhancing user interaction. This work not only displays the capabilities of Sensor2Scene but also lays a foundation for advancing AR with the goal of creating more immersive and contextually rich experiences. © 2024 IEEE.
Venkatachalam, N.; Rayana, M.; Vignesh, S. Bala; Prathamesh, S.
Voice-Driven Panoramic Imagery: Real-Time Generative AI for Immersive Experiences Proceedings Article
In: Int. Conf. Intell. Data Commun. Technol. Internet Things, IDCIoT, pp. 1133–1138, Institute of Electrical and Electronics Engineers Inc., 2024, ISBN: 979-835032753-3 (ISBN).
Abstract | Links | BibTeX | Tags: Adaptive Visual Experience, First person, First-Person view, generative artificial intelligence, Generative Artificial Intelligence (AI), Image processing, Immersive, Immersive visual scene, Immersive Visual Scenes, Language processing, Natural Language Processing, Natural Language Processing (NLP), Natural language processing systems, Natural languages, Panoramic Images, Patient treatment, Personalized environment, Personalized Environments, Phobia Treatment, Prompt, prompts, Psychological intervention, Psychological Interventions, Real-Time Synthesis, User interaction, User interfaces, Virtual experience, Virtual Experiences, Virtual Reality, Virtual Reality (VR), Virtual-reality headsets, Visual experiences, Visual languages, Visual scene, Voice command, Voice commands, VR Headsets
@inproceedings{venkatachalam_voice-driven_2024,
title = {Voice-Driven Panoramic Imagery: Real-Time Generative AI for Immersive Experiences},
author = {N. Venkatachalam and M. Rayana and S. Bala Vignesh and S. Prathamesh},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85190121845&doi=10.1109%2fIDCIoT59759.2024.10467441&partnerID=40&md5=6594fbab013d9156b79a887f0d7209cb},
doi = {10.1109/IDCIoT59759.2024.10467441},
isbn = {979-835032753-3 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Int. Conf. Intell. Data Commun. Technol. Internet Things, IDCIoT},
pages = {1133–1138},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {This research study introduces an innovative system that aims to synthesize 360-degree panoramic images in Realtime based on vocal prompts from the user, leveraging state-of-The-Art Generative AI with a combination of advanced NLP models. The primary objective of this system is to transform spoken descriptions into immersive and interactive visual scenes, specifically designed to provide users with first-person field views. This cutting-edge technology has the potential to revolutionize the realm of virtual reality (VR) experiences, enabling users to effortlessly create and navigate through personalized environments. The fundamental goal of this system is to enable the generation of real-Time images that are seamlessly compatible with VR headsets, offering a truly immersive and adaptive visual experience. Beyond its technological advancements, this research also highlights its significant potential for creating a positive social impact. One notable application lies in psychological interventions, particularly in the context of phobia treatment and therapeutic settings. Here, patients can safely confront and work through their fears within these synthesized environments, potentially offering new avenues for therapy. Furthermore, the system serves educational and entertainment purposes by bringing users' imaginations to life, providing an unparalleled platform for exploring the boundaries of virtual experiences. Overall, this research represents a promising stride towards a more immersive and adaptable future in VR technology, with the potential to enhance various aspects of human lives, from mental health treatment to entertainment and education. © 2024 IEEE.},
keywords = {Adaptive Visual Experience, First person, First-Person view, generative artificial intelligence, Generative Artificial Intelligence (AI), Image processing, Immersive, Immersive visual scene, Immersive Visual Scenes, Language processing, Natural Language Processing, Natural Language Processing (NLP), Natural language processing systems, Natural languages, Panoramic Images, Patient treatment, Personalized environment, Personalized Environments, Phobia Treatment, Prompt, prompts, Psychological intervention, Psychological Interventions, Real-Time Synthesis, User interaction, User interfaces, Virtual experience, Virtual Experiences, Virtual Reality, Virtual Reality (VR), Virtual-reality headsets, Visual experiences, Visual languages, Visual scene, Voice command, Voice commands, VR Headsets},
pubstate = {published},
tppubtype = {inproceedings}
}
This research study introduces an innovative system that aims to synthesize 360-degree panoramic images in Realtime based on vocal prompts from the user, leveraging state-of-The-Art Generative AI with a combination of advanced NLP models. The primary objective of this system is to transform spoken descriptions into immersive and interactive visual scenes, specifically designed to provide users with first-person field views. This cutting-edge technology has the potential to revolutionize the realm of virtual reality (VR) experiences, enabling users to effortlessly create and navigate through personalized environments. The fundamental goal of this system is to enable the generation of real-Time images that are seamlessly compatible with VR headsets, offering a truly immersive and adaptive visual experience. Beyond its technological advancements, this research also highlights its significant potential for creating a positive social impact. One notable application lies in psychological interventions, particularly in the context of phobia treatment and therapeutic settings. Here, patients can safely confront and work through their fears within these synthesized environments, potentially offering new avenues for therapy. Furthermore, the system serves educational and entertainment purposes by bringing users' imaginations to life, providing an unparalleled platform for exploring the boundaries of virtual experiences. Overall, this research represents a promising stride towards a more immersive and adaptable future in VR technology, with the potential to enhance various aspects of human lives, from mental health treatment to entertainment and education. © 2024 IEEE.