AHCI RESEARCH GROUP
Publications
Papers published in international journals,
proceedings of conferences, workshops and books.
OUR RESEARCH
Scientific Publications
How to
Here you can find the complete list of our publications.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
2025
Li, J.; Neshaei, S. P.; Müller, L.; Rietsche, R.; Davis, R. L.; Wambsganss, T.
SpatiaLearn: Exploring XR Learning Environments for Reflective Writing Proceedings Article
In: Conf Hum Fact Comput Syst Proc, Association for Computing Machinery, 2025, ISBN: 979-840071395-8 (ISBN).
Abstract | Links | BibTeX | Tags: Adaptive Education, Conversational Agents, Conversational Tutoring, Critical thinking, Extended reality (XR), Immersive, Learning Environments, Metacognitive awareness, Reflective writing, Spatial computing
@inproceedings{li_spatialearn_2025,
title = {SpatiaLearn: Exploring XR Learning Environments for Reflective Writing},
author = {J. Li and S. P. Neshaei and L. Müller and R. Rietsche and R. L. Davis and T. Wambsganss},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105005757843&doi=10.1145%2f3706599.3719742&partnerID=40&md5=6e9ce83d3508cb377e209edd6884c505},
doi = {10.1145/3706599.3719742},
isbn = {979-840071395-8 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Conf Hum Fact Comput Syst Proc},
publisher = {Association for Computing Machinery},
abstract = {Reflective writing promotes deeper learning by enhancing metacognitive awareness and critical thinking, but learners often struggle with structuring their reflections and maintaining focus. Generative AI and advances in spatial computing offer promising solutions. Extended reality (XR) environments create immersive, distraction-free settings, while conversational agents use dialog-based scaffolding guides to structure learners’ thoughts. However, research on combining dialog-based scaffolding with XR for reflective writing remains limited. To address this, we introduce SpatiaLearn, an adaptive XR tool that enhances reflective writing through conversational guidance in both traditional and immersive environments. A within-subjects study (N = 19) compared participants’ performance in traditional laptop and XR environments. Qualitative analysis shows the spatial interface enhances engagement but raises challenges like unfamiliar interactions and health concerns, requiring task adaptation for XR. This study advances the design of immersive tools for reflective writing, highlighting both the opportunities and challenges of spatial interfaces. © 2025 Copyright held by the owner/author(s).},
keywords = {Adaptive Education, Conversational Agents, Conversational Tutoring, Critical thinking, Extended reality (XR), Immersive, Learning Environments, Metacognitive awareness, Reflective writing, Spatial computing},
pubstate = {published},
tppubtype = {inproceedings}
}
Reflective writing promotes deeper learning by enhancing metacognitive awareness and critical thinking, but learners often struggle with structuring their reflections and maintaining focus. Generative AI and advances in spatial computing offer promising solutions. Extended reality (XR) environments create immersive, distraction-free settings, while conversational agents use dialog-based scaffolding guides to structure learners’ thoughts. However, research on combining dialog-based scaffolding with XR for reflective writing remains limited. To address this, we introduce SpatiaLearn, an adaptive XR tool that enhances reflective writing through conversational guidance in both traditional and immersive environments. A within-subjects study (N = 19) compared participants’ performance in traditional laptop and XR environments. Qualitative analysis shows the spatial interface enhances engagement but raises challenges like unfamiliar interactions and health concerns, requiring task adaptation for XR. This study advances the design of immersive tools for reflective writing, highlighting both the opportunities and challenges of spatial interfaces. © 2025 Copyright held by the owner/author(s).
Aloudat, M. Z.; Aboumadi, A.; Soliman, A.; Al-Mohammed, H. A.; Al-Ali, M.; Mahgoub, A.; Barhamgi, M.; Yaacoub, E.
Metaverse Unbound: A Survey on Synergistic Integration Between Semantic Communication, 6G, and Edge Learning Journal Article
In: IEEE Access, vol. 13, pp. 58302–58350, 2025, ISSN: 21693536 (ISSN).
Abstract | Links | BibTeX | Tags: 6g wireless system, 6G wireless systems, Augmented Reality, Block-chain, Blockchain, Blockchain technology, Digital Twin Technology, Edge learning, Extended reality (XR), Language Model, Large language model, large language models (LLMs), Metaverse, Metaverses, Semantic communication, Virtual environments, Wireless systems
@article{aloudat_metaverse_2025,
title = {Metaverse Unbound: A Survey on Synergistic Integration Between Semantic Communication, 6G, and Edge Learning},
author = {M. Z. Aloudat and A. Aboumadi and A. Soliman and H. A. Al-Mohammed and M. Al-Ali and A. Mahgoub and M. Barhamgi and E. Yaacoub},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105003088610&doi=10.1109%2fACCESS.2025.3555753&partnerID=40&md5=8f3f9421ce2d6be57f8154a122ee192c},
doi = {10.1109/ACCESS.2025.3555753},
issn = {21693536 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {IEEE Access},
volume = {13},
pages = {58302–58350},
abstract = {With a focus on edge learning, blockchain, sixth generation (6G) wireless systems, semantic communication, and large language models (LLMs), this survey paper examines the revolutionary integration of cutting-edge technologies within the metaverse. This thorough examination highlights the critical role these technologies play in improving realism and user engagement on three main levels: technical, virtual, and physical. While the virtual layer focuses on building immersive experiences, the physical layer highlights improvements to the user interface through augmented reality (AR) goggles and virtual reality (VR) headsets. Blockchain-powered technical layer enables safe, decentralized communication. The survey highlights how the metaverse has the potential to drastically change how people interact in society by exploring applications in a variety of fields, such as immersive education, remote work, and entertainment. Concerns about privacy, scalability, and interoperability are raised, highlighting the necessity of continued study to realize the full potential of the metaverse. For scholars looking to broaden the reach and significance of the metaverse in the digital age, this paper is a useful tool. © 2013 IEEE.},
keywords = {6g wireless system, 6G wireless systems, Augmented Reality, Block-chain, Blockchain, Blockchain technology, Digital Twin Technology, Edge learning, Extended reality (XR), Language Model, Large language model, large language models (LLMs), Metaverse, Metaverses, Semantic communication, Virtual environments, Wireless systems},
pubstate = {published},
tppubtype = {article}
}
With a focus on edge learning, blockchain, sixth generation (6G) wireless systems, semantic communication, and large language models (LLMs), this survey paper examines the revolutionary integration of cutting-edge technologies within the metaverse. This thorough examination highlights the critical role these technologies play in improving realism and user engagement on three main levels: technical, virtual, and physical. While the virtual layer focuses on building immersive experiences, the physical layer highlights improvements to the user interface through augmented reality (AR) goggles and virtual reality (VR) headsets. Blockchain-powered technical layer enables safe, decentralized communication. The survey highlights how the metaverse has the potential to drastically change how people interact in society by exploring applications in a variety of fields, such as immersive education, remote work, and entertainment. Concerns about privacy, scalability, and interoperability are raised, highlighting the necessity of continued study to realize the full potential of the metaverse. For scholars looking to broaden the reach and significance of the metaverse in the digital age, this paper is a useful tool. © 2013 IEEE.
Barbu, M.; Iordache, D. -D.; Petre, I.; Barbu, D. -C.; Băjenaru, L.
Framework Design for Reinforcing the Potential of XR Technologies in Transforming Inclusive Education Journal Article
In: Applied Sciences (Switzerland), vol. 15, no. 3, 2025, ISSN: 20763417 (ISSN).
Abstract | Links | BibTeX | Tags: Adaptive Learning, Adversarial machine learning, Artificial intelligence technologies, Augmented Reality, Contrastive Learning, Educational Technology, Extended reality (XR), Federated learning, Framework designs, Generative adversarial networks, Immersive, immersive experience, Immersive learning, Inclusive education, Learning platform, Special education needs
@article{barbu_framework_2025,
title = {Framework Design for Reinforcing the Potential of XR Technologies in Transforming Inclusive Education},
author = {M. Barbu and D. -D. Iordache and I. Petre and D. -C. Barbu and L. Băjenaru},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85217742383&doi=10.3390%2fapp15031484&partnerID=40&md5=3148ff2a8a8fa1bef8094199cd6d32e3},
doi = {10.3390/app15031484},
issn = {20763417 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {Applied Sciences (Switzerland)},
volume = {15},
number = {3},
abstract = {This study presents a novel approach to inclusive education by integrating augmented reality (XR) and generative artificial intelligence (AI) technologies into an immersive and adaptive learning platform designed for students with special educational needs. Building upon existing solutions, the approach uniquely combines XR and generative AI to facilitate personalized, accessible, and interactive learning experiences tailored to individual requirements. The framework incorporates an intuitive Unity XR-based interface alongside a generative AI module to enable near real-time customization of content and interactions. Additionally, the study examines related generative AI initiatives that promote inclusion through enhanced communication tools, educational support, and customizable assistive technologies. The motivation for this study arises from the pressing need to address the limitations of traditional educational methods, which often fail to meet the diverse needs of learners with special educational requirements. The integration of XR and generative AI offers transformative potential by creating adaptive, immersive, and inclusive learning environments. This approach ensures real-time adaptability to individual progress and accessibility, addressing critical barriers such as static content and lack of inclusivity in existing systems. The research outlines a pathway toward more inclusive and equitable education, significantly enhancing opportunities for learners with diverse needs and contributing to broader social integration and equity in education. © 2025 by the authors.},
keywords = {Adaptive Learning, Adversarial machine learning, Artificial intelligence technologies, Augmented Reality, Contrastive Learning, Educational Technology, Extended reality (XR), Federated learning, Framework designs, Generative adversarial networks, Immersive, immersive experience, Immersive learning, Inclusive education, Learning platform, Special education needs},
pubstate = {published},
tppubtype = {article}
}
This study presents a novel approach to inclusive education by integrating augmented reality (XR) and generative artificial intelligence (AI) technologies into an immersive and adaptive learning platform designed for students with special educational needs. Building upon existing solutions, the approach uniquely combines XR and generative AI to facilitate personalized, accessible, and interactive learning experiences tailored to individual requirements. The framework incorporates an intuitive Unity XR-based interface alongside a generative AI module to enable near real-time customization of content and interactions. Additionally, the study examines related generative AI initiatives that promote inclusion through enhanced communication tools, educational support, and customizable assistive technologies. The motivation for this study arises from the pressing need to address the limitations of traditional educational methods, which often fail to meet the diverse needs of learners with special educational requirements. The integration of XR and generative AI offers transformative potential by creating adaptive, immersive, and inclusive learning environments. This approach ensures real-time adaptability to individual progress and accessibility, addressing critical barriers such as static content and lack of inclusivity in existing systems. The research outlines a pathway toward more inclusive and equitable education, significantly enhancing opportunities for learners with diverse needs and contributing to broader social integration and equity in education. © 2025 by the authors.
2024
Chaccour, C.; Saad, W.; Debbah, M.; Poor, H. V.
Joint Sensing, Communication, and AI: A Trifecta for Resilient THz User Experiences Journal Article
In: IEEE Transactions on Wireless Communications, vol. 23, no. 9, pp. 11444–11460, 2024, ISSN: 15361276 (ISSN).
Abstract | Links | BibTeX | Tags: Artificial intelligence, artificial intelligence (AI), Behavioral Research, Channel state information, Computer hardware, Cramer-Rao bounds, Extended reality (XR), Hardware, Joint sensing and communication, Learning systems, machine learning, machine learning (ML), Machine-learning, Multi agent systems, reliability, Resilience, Sensor data fusion, Tera Hertz, Terahertz, terahertz (THz), Terahertz communication, Wireless communications, Wireless sensor networks, X reality
@article{chaccour_joint_2024,
title = {Joint Sensing, Communication, and AI: A Trifecta for Resilient THz User Experiences},
author = {C. Chaccour and W. Saad and M. Debbah and H. V. Poor},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85190170739&doi=10.1109%2fTWC.2024.3382192&partnerID=40&md5=da12c6f31faacaa08118b26e4570843f},
doi = {10.1109/TWC.2024.3382192},
issn = {15361276 (ISSN)},
year = {2024},
date = {2024-01-01},
journal = {IEEE Transactions on Wireless Communications},
volume = {23},
number = {9},
pages = {11444–11460},
abstract = {In this paper a novel joint sensing, communication, and artificial intelligence (AI) framework is proposed so as to optimize extended reality (XR) experiences over terahertz (THz) wireless systems. Within this framework, active reconfigurable intelligent surfaces (RISs) are incorporated as pivotal elements, serving as enhanced base stations in the THz band to enhance Line-of-Sight (LoS) communication. The proposed framework consists of three main components. First, a tensor decomposition framework is proposed to extract unique sensing parameters for XR users and their environment by exploiting the THz channel sparsity. Essentially, the THz band's quasi-opticality is exploited and the sensing parameters are extracted from the uplink communication signal, thereby allowing for the use of the same waveform, spectrum, and hardware for both communication and sensing functionalities. Then, the Cramér-Rao lower bound is derived to assess the accuracy of the estimated sensing parameters. Second, a non-autoregressive multi-resolution generative AI framework integrated with an adversarial transformer is proposed to predict missing and future sensing information. The proposed framework offers robust and comprehensive historical sensing information and anticipatory forecasts of future environmental changes, which are generalizable to fluctuations in both known and unforeseen user behaviors and environmental conditions. Third, a multi-agent deep recurrent hysteretic Q-neural network is developed to control the handover policy of RIS subarrays, leveraging the informative nature of sensing information to minimize handover cost, maximize the individual quality of personal experiences (QoPEs), and improve the robustness and resilience of THz links. Simulation results show a high generalizability of the proposed unsupervised generative artificial intelligence (AI) framework to fluctuations in user behavior and velocity, leading to a 61% improvement in instantaneous reliability compared to schemes with known channel state information. © 2002-2012 IEEE.},
keywords = {Artificial intelligence, artificial intelligence (AI), Behavioral Research, Channel state information, Computer hardware, Cramer-Rao bounds, Extended reality (XR), Hardware, Joint sensing and communication, Learning systems, machine learning, machine learning (ML), Machine-learning, Multi agent systems, reliability, Resilience, Sensor data fusion, Tera Hertz, Terahertz, terahertz (THz), Terahertz communication, Wireless communications, Wireless sensor networks, X reality},
pubstate = {published},
tppubtype = {article}
}
In this paper a novel joint sensing, communication, and artificial intelligence (AI) framework is proposed so as to optimize extended reality (XR) experiences over terahertz (THz) wireless systems. Within this framework, active reconfigurable intelligent surfaces (RISs) are incorporated as pivotal elements, serving as enhanced base stations in the THz band to enhance Line-of-Sight (LoS) communication. The proposed framework consists of three main components. First, a tensor decomposition framework is proposed to extract unique sensing parameters for XR users and their environment by exploiting the THz channel sparsity. Essentially, the THz band's quasi-opticality is exploited and the sensing parameters are extracted from the uplink communication signal, thereby allowing for the use of the same waveform, spectrum, and hardware for both communication and sensing functionalities. Then, the Cramér-Rao lower bound is derived to assess the accuracy of the estimated sensing parameters. Second, a non-autoregressive multi-resolution generative AI framework integrated with an adversarial transformer is proposed to predict missing and future sensing information. The proposed framework offers robust and comprehensive historical sensing information and anticipatory forecasts of future environmental changes, which are generalizable to fluctuations in both known and unforeseen user behaviors and environmental conditions. Third, a multi-agent deep recurrent hysteretic Q-neural network is developed to control the handover policy of RIS subarrays, leveraging the informative nature of sensing information to minimize handover cost, maximize the individual quality of personal experiences (QoPEs), and improve the robustness and resilience of THz links. Simulation results show a high generalizability of the proposed unsupervised generative artificial intelligence (AI) framework to fluctuations in user behavior and velocity, leading to a 61% improvement in instantaneous reliability compared to schemes with known channel state information. © 2002-2012 IEEE.
Gemeinhardt, J.; Zöllner, M.; Jahn, C.
Generative AI Tool Pipeline for Creating Artificial Historical Characters for Cultural Heritage XR Proceedings Article
In: C., Stephanidis; M., Antona; S., Ntoa; G., Salvendy (Ed.): Commun. Comput. Info. Sci., pp. 41–46, Springer Science and Business Media Deutschland GmbH, 2024, ISBN: 18650929 (ISSN); 978-303161949-6 (ISBN).
Abstract | Links | BibTeX | Tags: Bavaria, Cultural heritage, Cultural heritages, Extended reality (XR), Generative AI, Historical characters, Immersive, Media production, Open source software, Open systems, Pipelines, Reproducibilities, Smart phones, Virtual representations, Web browsers
@inproceedings{gemeinhardt_generative_2024,
title = {Generative AI Tool Pipeline for Creating Artificial Historical Characters for Cultural Heritage XR},
author = {J. Gemeinhardt and M. Zöllner and C. Jahn},
editor = {Stephanidis C. and Antona M. and Ntoa S. and Salvendy G.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85197123898&doi=10.1007%2f978-3-031-61950-2_5&partnerID=40&md5=8f8a3cf4f4bf024b42f6490f64345df2},
doi = {10.1007/978-3-031-61950-2_5},
isbn = {18650929 (ISSN); 978-303161949-6 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Commun. Comput. Info. Sci.},
volume = {2116 CCIS},
pages = {41–46},
publisher = {Springer Science and Business Media Deutschland GmbH},
abstract = {In our project, we aimed to create historically authentic and vivid virtual representations of historic personalities that are connected to the regional Fichtelgebirge (Bavaria, Germany) to support the storytelling of our immersive XR applications. We are describing the tools in detail, the process of the tool chain and the resulting media. Next, we are discussing the challenges in media production like historical correctness and the consultation of historians. In order to create visual reproducibility we are explaining the detailed text prompts, their limitations and how to cope with resulting errors of the human physiognomy. Finally, we are briefly describing the application of the animated and talking generated historic characters in an immersive interactive WebXR environment. The XR experience is presented in web browsers on smartphones, tablets and XR headsets and the underlying software is based on the open-source framework Aframe. Our paper will describe the process, the results and the limitations in detail. Furthermore, we will provide a flow chart of the tool pipeline with visual examples of these aspects. The animations and voices of the historic characters will be demonstrated in videos of the XR application. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.},
keywords = {Bavaria, Cultural heritage, Cultural heritages, Extended reality (XR), Generative AI, Historical characters, Immersive, Media production, Open source software, Open systems, Pipelines, Reproducibilities, Smart phones, Virtual representations, Web browsers},
pubstate = {published},
tppubtype = {inproceedings}
}
In our project, we aimed to create historically authentic and vivid virtual representations of historic personalities that are connected to the regional Fichtelgebirge (Bavaria, Germany) to support the storytelling of our immersive XR applications. We are describing the tools in detail, the process of the tool chain and the resulting media. Next, we are discussing the challenges in media production like historical correctness and the consultation of historians. In order to create visual reproducibility we are explaining the detailed text prompts, their limitations and how to cope with resulting errors of the human physiognomy. Finally, we are briefly describing the application of the animated and talking generated historic characters in an immersive interactive WebXR environment. The XR experience is presented in web browsers on smartphones, tablets and XR headsets and the underlying software is based on the open-source framework Aframe. Our paper will describe the process, the results and the limitations in detail. Furthermore, we will provide a flow chart of the tool pipeline with visual examples of these aspects. The animations and voices of the historic characters will be demonstrated in videos of the XR application. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
Michael, Z.; Gemeinhardt, J.; Moritz, K.
Interactive WebXR Hypertext Storytelling for Cultural Heritage Proceedings Article
In: C., Atzenbeck; J., Rubart (Ed.): Proc. Workshop Hum. Factors Hypertext, Hum. - Assoc. ACM Conf. Hypertext Soc. Media ,HT, Association for Computing Machinery, Inc, 2024, ISBN: 979-840071120-6 (ISBN).
Abstract | Links | BibTeX | Tags: 2D textures, 3D modeling, 3D models, 3d-modeling, Cultural heritage, Cultural heritages, Extended reality (XR), Generative AI, History, HTTP, Hypertext, Hypertext systems, Immersive, Machine-learning, Open source software, Open systems, Scene structure, Three dimensional computer graphics, Virtual environments, Virtual Reality, Web browsers
@inproceedings{michael_interactive_2024,
title = {Interactive WebXR Hypertext Storytelling for Cultural Heritage},
author = {Z. Michael and J. Gemeinhardt and K. Moritz},
editor = {Atzenbeck C. and Rubart J.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85211623904&doi=10.1145%2f3679058.3688635&partnerID=40&md5=60aad5a9a95e52c3fff51ebb6f670bd6},
doi = {10.1145/3679058.3688635},
isbn = {979-840071120-6 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Proc. Workshop Hum. Factors Hypertext, Hum. - Assoc. ACM Conf. Hypertext Soc. Media ,HT},
publisher = {Association for Computing Machinery, Inc},
abstract = {We are presenting our approach for interactive cultural heritage storytelling in WebXR. Therefore, we are describing our scenes’ structure consisting of (stylized) photospheres of the historic locations, 3D models of 3D-scanned historic artifacts and animated 2D textures of historic characters generated with a machine learning toolset. The result is a platform-independent web-application in an immersive interactive WebXR environment running in browsers on PCs, tablets, phones and XR headsets thanks to the underlying software based on the open-source framework A-Frame. Our paper describes the process, the results and the limitations in detail. The resulting application, designed for the Fichtelgebirge region in Upper Franconia, Germany, offers users an immersive digital time travel experience in the virtual space and within a museum setting connecting real artifacts and virtual stories. © 2024 Copyright held by the owner/author(s).},
keywords = {2D textures, 3D modeling, 3D models, 3d-modeling, Cultural heritage, Cultural heritages, Extended reality (XR), Generative AI, History, HTTP, Hypertext, Hypertext systems, Immersive, Machine-learning, Open source software, Open systems, Scene structure, Three dimensional computer graphics, Virtual environments, Virtual Reality, Web browsers},
pubstate = {published},
tppubtype = {inproceedings}
}
We are presenting our approach for interactive cultural heritage storytelling in WebXR. Therefore, we are describing our scenes’ structure consisting of (stylized) photospheres of the historic locations, 3D models of 3D-scanned historic artifacts and animated 2D textures of historic characters generated with a machine learning toolset. The result is a platform-independent web-application in an immersive interactive WebXR environment running in browsers on PCs, tablets, phones and XR headsets thanks to the underlying software based on the open-source framework A-Frame. Our paper describes the process, the results and the limitations in detail. The resulting application, designed for the Fichtelgebirge region in Upper Franconia, Germany, offers users an immersive digital time travel experience in the virtual space and within a museum setting connecting real artifacts and virtual stories. © 2024 Copyright held by the owner/author(s).
2023
Park, J.; Choi, J.; Kim, S. -L.; Bennis, M.
Enabling the Wireless Metaverse via Semantic Multiverse Communication Proceedings Article
In: Annu. IEEE Commun.Soc. Conf. Sens., Mesh Ad Hoc Commun. Netw. workshops, pp. 85–90, IEEE Computer Society, 2023, ISBN: 21555486 (ISSN); 979-835030052-9 (ISBN).
Abstract | Links | BibTeX | Tags: Deep learning, Extended reality (XR), Federated learning, Fertilizers, Learn+, Learning systems, Metaverse, Metaverses, Modal analysis, Multi agent systems, Multi-agent reinforcement learning, Multi-modal data, Reinforcement Learning, Semantic communication, Semantics, Signal encoding, Signaling game, Split learning, Symbolic artificial intelligence
@inproceedings{park_enabling_2023,
title = {Enabling the Wireless Metaverse via Semantic Multiverse Communication},
author = {J. Park and J. Choi and S. -L. Kim and M. Bennis},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85177465286&doi=10.1109%2fSECON58729.2023.10287438&partnerID=40&md5=b052572fb2f78ce0694c7ae5726c8daf},
doi = {10.1109/SECON58729.2023.10287438},
isbn = {21555486 (ISSN); 979-835030052-9 (ISBN)},
year = {2023},
date = {2023-01-01},
booktitle = {Annu. IEEE Commun.Soc. Conf. Sens., Mesh Ad Hoc Commun. Netw. workshops},
volume = {2023-September},
pages = {85–90},
publisher = {IEEE Computer Society},
abstract = {Metaverse over wireless networks is an emerging use case of the sixth generation (6G) wireless systems, posing unprecedented challenges in terms of its multi-modal data transmissions with stringent latency and reliability requirements. Towards enabling this wireless metaverse, in this article we propose a novel semantic communication (SC) framework by decomposing the metaverse into human/machine agent-specific semantic multiverses (SMs). An SM stored at each agent comprises a semantic encoder and a generator, leveraging recent advances in generative artificial intelligence (AI). To improve communication efficiency, the encoder learns the semantic representations (SRs) of multi-modal data, while the generator learns how to manipulate them for locally rendering scenes and interactions in the metaverse. Since these learned SMs are biased towards local environments, their success hinges on synchronizing heterogeneous SMs in the background while communicating SRs in the foreground, turning the wireless metaverse problem into the problem of semantic multiverse communication (SMC). Based on this SMC architecture, we propose several promising algorithmic and analytic tools for modeling and designing SMC, ranging from distributed learning and multi-agent reinforcement learning (MARL) to signaling games and symbolic AI. © 2023 IEEE.},
keywords = {Deep learning, Extended reality (XR), Federated learning, Fertilizers, Learn+, Learning systems, Metaverse, Metaverses, Modal analysis, Multi agent systems, Multi-agent reinforcement learning, Multi-modal data, Reinforcement Learning, Semantic communication, Semantics, Signal encoding, Signaling game, Split learning, Symbolic artificial intelligence},
pubstate = {published},
tppubtype = {inproceedings}
}
Metaverse over wireless networks is an emerging use case of the sixth generation (6G) wireless systems, posing unprecedented challenges in terms of its multi-modal data transmissions with stringent latency and reliability requirements. Towards enabling this wireless metaverse, in this article we propose a novel semantic communication (SC) framework by decomposing the metaverse into human/machine agent-specific semantic multiverses (SMs). An SM stored at each agent comprises a semantic encoder and a generator, leveraging recent advances in generative artificial intelligence (AI). To improve communication efficiency, the encoder learns the semantic representations (SRs) of multi-modal data, while the generator learns how to manipulate them for locally rendering scenes and interactions in the metaverse. Since these learned SMs are biased towards local environments, their success hinges on synchronizing heterogeneous SMs in the background while communicating SRs in the foreground, turning the wireless metaverse problem into the problem of semantic multiverse communication (SMC). Based on this SMC architecture, we propose several promising algorithmic and analytic tools for modeling and designing SMC, ranging from distributed learning and multi-agent reinforcement learning (MARL) to signaling games and symbolic AI. © 2023 IEEE.