AHCI RESEARCH GROUP
Publications
Papers published in international journals,
proceedings of conferences, workshops and books.
OUR RESEARCH
Scientific Publications
How to
Here you can find the complete list of our publications.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
2024
Asadi, A. R.; Appiah, J.; Muntaka, S. A.; Kropczynski, J.
Actions, Not Apps: Toward Using LLMs to Reshape Context Aware Interactions in Mixed Reality Systems Proceedings Article
In: C., Stephanidis; M., Antona; S., Ntoa; G., Salvendy (Ed.): Commun. Comput. Info. Sci., pp. 166–176, Springer Science and Business Media Deutschland GmbH, 2024, ISBN: 18650929 (ISSN); 978-303162109-3 (ISBN).
Abstract | Links | BibTeX | Tags: Computation theory, Context Aware System, Context-aware interaction, Context-aware systems, Decision making, Digital information, Flat-screens, Interaction Design, Language Model, Mixed reality, Mixed reality systems, User interaction, User interfaces, User perceptions
@inproceedings{asadi_actions_2024,
title = {Actions, Not Apps: Toward Using LLMs to Reshape Context Aware Interactions in Mixed Reality Systems},
author = {A. R. Asadi and J. Appiah and S. A. Muntaka and J. Kropczynski},
editor = {Stephanidis C. and Antona M. and Ntoa S. and Salvendy G.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85196733497&doi=10.1007%2f978-3-031-62110-9_17&partnerID=40&md5=9cd702ff979c7f111a5172df8f155ddf},
doi = {10.1007/978-3-031-62110-9_17},
isbn = {18650929 (ISSN); 978-303162109-3 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Commun. Comput. Info. Sci.},
volume = {2120 CCIS},
pages = {166–176},
publisher = {Springer Science and Business Media Deutschland GmbH},
abstract = {Mixed reality computing merges user perception of the environment with digital information. As we move from flatscreen computing toward head-mounted computing, the necessity for developing alternative interactions and user flows becomes more evident. Activity theory provides a holistic overview of user interactions and motives. In this work in progress, we propose Action Sandbox Workspace as an interaction framework for the future of MR systems by focusing on action-centric interactions rather than application-centric interactions, aiming to bridge the gap between user goals and system functionalities in everyday tasks. By integrating the ontology of actions, user intentions, and context and connecting it to spatial data mapping, this forward-looking framework aims to create a contextually adaptive user interaction environment. The recent development in large language models (LLMs) has made the implementation of this interaction flow feasible by enabling inference and decision-making based on text-based descriptions of a user’s state and intentions with data and actions users have access to. We propose this approach as a future direction for developing mixed reality platforms and integrating AI in interacting with computers. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.},
keywords = {Computation theory, Context Aware System, Context-aware interaction, Context-aware systems, Decision making, Digital information, Flat-screens, Interaction Design, Language Model, Mixed reality, Mixed reality systems, User interaction, User interfaces, User perceptions},
pubstate = {published},
tppubtype = {inproceedings}
}
Mixed reality computing merges user perception of the environment with digital information. As we move from flatscreen computing toward head-mounted computing, the necessity for developing alternative interactions and user flows becomes more evident. Activity theory provides a holistic overview of user interactions and motives. In this work in progress, we propose Action Sandbox Workspace as an interaction framework for the future of MR systems by focusing on action-centric interactions rather than application-centric interactions, aiming to bridge the gap between user goals and system functionalities in everyday tasks. By integrating the ontology of actions, user intentions, and context and connecting it to spatial data mapping, this forward-looking framework aims to create a contextually adaptive user interaction environment. The recent development in large language models (LLMs) has made the implementation of this interaction flow feasible by enabling inference and decision-making based on text-based descriptions of a user’s state and intentions with data and actions users have access to. We propose this approach as a future direction for developing mixed reality platforms and integrating AI in interacting with computers. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
2023
Xu, M.; Niyato, D.; Chen, J.; Zhang, H.; Kang, J.; Xiong, Z.; Mao, S.; Han, Z.
Generative AI-Empowered Simulation for Autonomous Driving in Vehicular Mixed Reality Metaverses Journal Article
In: IEEE Journal on Selected Topics in Signal Processing, vol. 17, no. 5, pp. 1064–1079, 2023, ISSN: 19324553 (ISSN).
Abstract | Links | BibTeX | Tags: Auction theory, Autonomous driving, Autonomous Vehicles, Computation theory, Computational modelling, generative artificial intelligence, Job analysis, Metaverse, Metaverses, Mixed reality, Online systems, Roadside units, Task analysis
@article{xu_generative_2023,
title = {Generative AI-Empowered Simulation for Autonomous Driving in Vehicular Mixed Reality Metaverses},
author = {M. Xu and D. Niyato and J. Chen and H. Zhang and J. Kang and Z. Xiong and S. Mao and Z. Han},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85164670036&doi=10.1109%2fJSTSP.2023.3293650&partnerID=40&md5=f28390de62f0f44c38a902e6c32dcd16},
doi = {10.1109/JSTSP.2023.3293650},
issn = {19324553 (ISSN)},
year = {2023},
date = {2023-01-01},
journal = {IEEE Journal on Selected Topics in Signal Processing},
volume = {17},
number = {5},
pages = {1064–1079},
abstract = {In the vehicular mixed reality (MR) Metaverse, the discrepancy between physical and virtual entities can be overcome by fusing the physical and virtual environments with multi-dimensional communications in autonomous driving systems. Assisted by digital twin (DT) technologies, connected autonomous vehicles (AVs), roadside units (RSUs), and virtual simulators can maintain the vehicular MR Metaverse via simulations for sharing data and making driving decisions collaboratively. However, it is challenging and costly to enable large-scale traffic and driving simulation via realistic data collection and fusion from the physical world for online prediction and offline training in autonomous driving systems. In this paper, we propose an autonomous driving architecture, where generative AI is leveraged to synthesize unlimited conditioned traffic and driving data via simulations for improving driving safety and traffic control efficiency. First, we propose a multi-task DT offloading model for the reliable execution of heterogeneous DT tasks with different requirements at RSUs. Then, based on the preferences of AV's DTs and real-world data, virtual simulators can synthesize unlimited conditioned driving and traffic datasets for improved robustness. Finally, we propose a multi-task enhanced auction-based mechanism to provide fine-grained incentives for RSUs on providing resources for autonomous driving. The property analysis and experimental results demonstrate that the proposed mechanism and architecture are strategy-proof and effective. © 2007-2012 IEEE.},
keywords = {Auction theory, Autonomous driving, Autonomous Vehicles, Computation theory, Computational modelling, generative artificial intelligence, Job analysis, Metaverse, Metaverses, Mixed reality, Online systems, Roadside units, Task analysis},
pubstate = {published},
tppubtype = {article}
}
In the vehicular mixed reality (MR) Metaverse, the discrepancy between physical and virtual entities can be overcome by fusing the physical and virtual environments with multi-dimensional communications in autonomous driving systems. Assisted by digital twin (DT) technologies, connected autonomous vehicles (AVs), roadside units (RSUs), and virtual simulators can maintain the vehicular MR Metaverse via simulations for sharing data and making driving decisions collaboratively. However, it is challenging and costly to enable large-scale traffic and driving simulation via realistic data collection and fusion from the physical world for online prediction and offline training in autonomous driving systems. In this paper, we propose an autonomous driving architecture, where generative AI is leveraged to synthesize unlimited conditioned traffic and driving data via simulations for improving driving safety and traffic control efficiency. First, we propose a multi-task DT offloading model for the reliable execution of heterogeneous DT tasks with different requirements at RSUs. Then, based on the preferences of AV's DTs and real-world data, virtual simulators can synthesize unlimited conditioned driving and traffic datasets for improved robustness. Finally, we propose a multi-task enhanced auction-based mechanism to provide fine-grained incentives for RSUs on providing resources for autonomous driving. The property analysis and experimental results demonstrate that the proposed mechanism and architecture are strategy-proof and effective. © 2007-2012 IEEE.
2022
Wang, A.; Gao, Z.; Lee, L. H.; Braud, T.; Hui, P.
Decentralized, not Dehumanized in the Metaverse: Bringing Utility to NFTs through Multimodal Interaction Proceedings Article
In: ACM Int. Conf. Proc. Ser., pp. 662–667, Association for Computing Machinery, 2022, ISBN: 978-145039390-4 (ISBN).
Abstract | Links | BibTeX | Tags: AI-generated art, Arts computing, Behavioral Research, Computation theory, Continuum mechanics, Decentralised, Human behaviors, Interaction, Multi-modal, multimodal, Multimodal Interaction, NFTs, Non-fungible token, Text-to-image, The metaverse
@inproceedings{wang_decentralized_2022,
title = {Decentralized, not Dehumanized in the Metaverse: Bringing Utility to NFTs through Multimodal Interaction},
author = {A. Wang and Z. Gao and L. H. Lee and T. Braud and P. Hui},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85142799074&doi=10.1145%2f3536221.3558176&partnerID=40&md5=f9dee1e9e60afc71c4533cbdee0b98a7},
doi = {10.1145/3536221.3558176},
isbn = {978-145039390-4 (ISBN)},
year = {2022},
date = {2022-01-01},
booktitle = {ACM Int. Conf. Proc. Ser.},
pages = {662–667},
publisher = {Association for Computing Machinery},
abstract = {User Interaction for NFTs (Non-fungible Tokens) is gaining increasing attention. Although NFTs have been traditionally single-use and monolithic, recent applications aim to connect multimodal interaction with human behavior. This paper reviews the related technological approaches and business practices in NFT art. We highlight that multimodal interaction is a currently under-studied issue in mainstream NFT art, and conjecture that multimodal interaction is a crucial enabler for decentralization in the NFT community. We present a continuum theory and propose a framework combining a bottom-up approach with AI multimodal process. Through this framework, we put forward integrating human behavior data into generative NFT units, as "multimodal interactive NFT."Our work displays the possibilities of NFTs in the art world, beyond the traditional 2D and 3D static content. © 2022 ACM.},
keywords = {AI-generated art, Arts computing, Behavioral Research, Computation theory, Continuum mechanics, Decentralised, Human behaviors, Interaction, Multi-modal, multimodal, Multimodal Interaction, NFTs, Non-fungible token, Text-to-image, The metaverse},
pubstate = {published},
tppubtype = {inproceedings}
}
User Interaction for NFTs (Non-fungible Tokens) is gaining increasing attention. Although NFTs have been traditionally single-use and monolithic, recent applications aim to connect multimodal interaction with human behavior. This paper reviews the related technological approaches and business practices in NFT art. We highlight that multimodal interaction is a currently under-studied issue in mainstream NFT art, and conjecture that multimodal interaction is a crucial enabler for decentralization in the NFT community. We present a continuum theory and propose a framework combining a bottom-up approach with AI multimodal process. Through this framework, we put forward integrating human behavior data into generative NFT units, as "multimodal interactive NFT."Our work displays the possibilities of NFTs in the art world, beyond the traditional 2D and 3D static content. © 2022 ACM.