AHCI RESEARCH GROUP
Publications
Papers published in international journals,
proceedings of conferences, workshops and books.
OUR RESEARCH
Scientific Publications
How to
Here you can find the complete list of our publications.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
2025
Liu, G.; Du, H.; Wang, J.; Niyato, D.; Kim, D. I.
Contract-Inspired Contest Theory for Controllable Image Generation in Mobile Edge Metaverse Journal Article
In: IEEE Transactions on Mobile Computing, 2025, ISSN: 15361233 (ISSN).
Abstract | Links | BibTeX | Tags: Contest Theory, Deep learning, Deep reinforcement learning, Diffusion Model, Generative adversarial networks, Generative AI, High quality, Image generation, Image generations, Immersive technologies, Metaverses, Mobile edge computing, Reinforcement Learning, Reinforcement learnings, Resource allocation, Resources allocation, Semantic data, Virtual addresses, Virtual environments, Virtual Reality
@article{liu_contract-inspired_2025,
title = {Contract-Inspired Contest Theory for Controllable Image Generation in Mobile Edge Metaverse},
author = {G. Liu and H. Du and J. Wang and D. Niyato and D. I. Kim},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105000066834&doi=10.1109%2fTMC.2025.3550815&partnerID=40&md5=3cb5a2143b9ce4ca7f931a60f1bf239c},
doi = {10.1109/TMC.2025.3550815},
issn = {15361233 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {IEEE Transactions on Mobile Computing},
abstract = {The rapid advancement of immersive technologies has propelled the development of the Metaverse, where the convergence of virtual and physical realities necessitates the generation of high-quality, photorealistic images to enhance user experience. However, generating these images, especially through Generative Diffusion Models (GDMs), in mobile edge computing environments presents significant challenges due to the limited computing resources of edge devices and the dynamic nature of wireless networks. This paper proposes a novel framework that integrates contract-inspired contest theory, Deep Reinforcement Learning (DRL), and GDMs to optimize image generation in these resource-constrained environments. The framework addresses the critical challenges of resource allocation and semantic data transmission quality by incentivizing edge devices to efficiently transmit high-quality semantic data, which is essential for creating realistic and immersive images. The use of contest and contract theory ensures that edge devices are motivated to allocate resources effectively, while DRL dynamically adjusts to network conditions, optimizing the overall image generation process. Experimental results demonstrate that the proposed approach not only improves the quality of generated images but also achieves superior convergence speed and stability compared to traditional methods. This makes the framework particularly effective for optimizing complex resource allocation tasks in mobile edge Metaverse applications, offering enhanced performance and efficiency in creating immersive virtual environments. © 2002-2012 IEEE.},
keywords = {Contest Theory, Deep learning, Deep reinforcement learning, Diffusion Model, Generative adversarial networks, Generative AI, High quality, Image generation, Image generations, Immersive technologies, Metaverses, Mobile edge computing, Reinforcement Learning, Reinforcement learnings, Resource allocation, Resources allocation, Semantic data, Virtual addresses, Virtual environments, Virtual Reality},
pubstate = {published},
tppubtype = {article}
}
The rapid advancement of immersive technologies has propelled the development of the Metaverse, where the convergence of virtual and physical realities necessitates the generation of high-quality, photorealistic images to enhance user experience. However, generating these images, especially through Generative Diffusion Models (GDMs), in mobile edge computing environments presents significant challenges due to the limited computing resources of edge devices and the dynamic nature of wireless networks. This paper proposes a novel framework that integrates contract-inspired contest theory, Deep Reinforcement Learning (DRL), and GDMs to optimize image generation in these resource-constrained environments. The framework addresses the critical challenges of resource allocation and semantic data transmission quality by incentivizing edge devices to efficiently transmit high-quality semantic data, which is essential for creating realistic and immersive images. The use of contest and contract theory ensures that edge devices are motivated to allocate resources effectively, while DRL dynamically adjusts to network conditions, optimizing the overall image generation process. Experimental results demonstrate that the proposed approach not only improves the quality of generated images but also achieves superior convergence speed and stability compared to traditional methods. This makes the framework particularly effective for optimizing complex resource allocation tasks in mobile edge Metaverse applications, offering enhanced performance and efficiency in creating immersive virtual environments. © 2002-2012 IEEE.
Li, C.; Da, F.
Refined dense face alignment through image matching Journal Article
In: Visual Computer, vol. 41, no. 1, pp. 157–171, 2025, ISSN: 01782789 (ISSN).
Abstract | Links | BibTeX | Tags: 3D Avatars, Alignment, Dense geometric supervision, Face alignment, Face deformations, Face reconstruction, Geometry, Human computer interaction, Image enhancement, Image matching, Image Reconstruction, Metaverses, Outlier mixup, Pixels, Rendered images, Rendering (computer graphics), State of the art, Statistics, Target images, Three dimensional computer graphics
@article{li_refined_2025,
title = {Refined dense face alignment through image matching},
author = {C. Li and F. Da},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85187924785&doi=10.1007%2fs00371-024-03316-3&partnerID=40&md5=839834c6ff3320398d5ef75b055947cb},
doi = {10.1007/s00371-024-03316-3},
issn = {01782789 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {Visual Computer},
volume = {41},
number = {1},
pages = {157–171},
abstract = {Face alignment is the foundation of building 3D avatars for virtue communication in the metaverse, human-computer interaction, AI-generated content, etc., and therefore, it is critical that face deformation is reflected precisely to better convey expression, pose and identity. However, misalignment exists in the currently best methods that fit a face model to a target image and can be easily captured by human perception, thus degrading the reconstruction quality. The main reason is that the widely used metrics for training, including the landmark re-projection loss, pixel-wise loss and perception-level loss, are insufficient to address the misalignment and suffer from ambiguity and local minimums. To address misalignment, we propose an image MAtchinG-driveN dEnse geomeTrIC supervision (MAGNETIC). Specifically, we treat face alignment as a matching problem and establish pixel-wise correspondences between the target and rendered images. Then reconstructed facial points are guided towards their corresponding points on the target image, thus improving reconstruction. Synthesized image pairs are mixed up with face outliers to simulate the target and rendered images with ground-truth pixel-wise correspondences to enable the training of a robust prediction network. Compared with existing methods that turn to 3D scans for dense geometric supervision, our method reaches comparable shape reconstruction results with much lower effort. Experimental results on the NoW testset show that we reach the state-of-the-art among all self-supervised methods and even outperform methods using photo-realistic images. We also achieve comparable results with the state-of-the-art on the benchmark of Feng et al. Codes will be available at: github.com/ChunLLee/ReconstructionFromMatching. © The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2024.},
keywords = {3D Avatars, Alignment, Dense geometric supervision, Face alignment, Face deformations, Face reconstruction, Geometry, Human computer interaction, Image enhancement, Image matching, Image Reconstruction, Metaverses, Outlier mixup, Pixels, Rendered images, Rendering (computer graphics), State of the art, Statistics, Target images, Three dimensional computer graphics},
pubstate = {published},
tppubtype = {article}
}
Face alignment is the foundation of building 3D avatars for virtue communication in the metaverse, human-computer interaction, AI-generated content, etc., and therefore, it is critical that face deformation is reflected precisely to better convey expression, pose and identity. However, misalignment exists in the currently best methods that fit a face model to a target image and can be easily captured by human perception, thus degrading the reconstruction quality. The main reason is that the widely used metrics for training, including the landmark re-projection loss, pixel-wise loss and perception-level loss, are insufficient to address the misalignment and suffer from ambiguity and local minimums. To address misalignment, we propose an image MAtchinG-driveN dEnse geomeTrIC supervision (MAGNETIC). Specifically, we treat face alignment as a matching problem and establish pixel-wise correspondences between the target and rendered images. Then reconstructed facial points are guided towards their corresponding points on the target image, thus improving reconstruction. Synthesized image pairs are mixed up with face outliers to simulate the target and rendered images with ground-truth pixel-wise correspondences to enable the training of a robust prediction network. Compared with existing methods that turn to 3D scans for dense geometric supervision, our method reaches comparable shape reconstruction results with much lower effort. Experimental results on the NoW testset show that we reach the state-of-the-art among all self-supervised methods and even outperform methods using photo-realistic images. We also achieve comparable results with the state-of-the-art on the benchmark of Feng et al. Codes will be available at: github.com/ChunLLee/ReconstructionFromMatching. © The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2024.
Lv, J.; Slowik, A.; Rani, S.; Kim, B. -G.; Chen, C. -M.; Kumari, S.; Li, K.; Lyu, X.; Jiang, H.
In: Research, vol. 8, 2025, ISSN: 20965168 (ISSN).
Abstract | Links | BibTeX | Tags: Adaptive fusion, Collaborative representations, Diagnosis, Electronic health record, Generative adversarial networks, Health care application, Healthcare environments, Immersive, Learning frameworks, Metaverses, Multi-modal, Multi-modal learning, Performance
@article{lv_multimodal_2025,
title = {Multimodal Metaverse Healthcare: A Collaborative Representation and Adaptive Fusion Approach for Generative Artificial-Intelligence-Driven Diagnosis},
author = {J. Lv and A. Slowik and S. Rani and B. -G. Kim and C. -M. Chen and S. Kumari and K. Li and X. Lyu and H. Jiang},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-86000613924&doi=10.34133%2fresearch.0616&partnerID=40&md5=fdc8ae3b29db905105dada9a5657b54b},
doi = {10.34133/research.0616},
issn = {20965168 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {Research},
volume = {8},
abstract = {The metaverse enables immersive virtual healthcare environments, presenting opportunities for enhanced care delivery. A key challenge lies in effectively combining multimodal healthcare data and generative artificial intelligence abilities within metaverse-based healthcare applications, which is a problem that needs to be addressed. This paper proposes a novel multimodal learning framework for metaverse healthcare, MMLMH, based on collaborative intra- and intersample representation and adaptive fusion. Our framework introduces a collaborative representation learning approach that captures shared and modality-specific features across text, audio, and visual health data. By combining modality-specific and shared encoders with carefully formulated intrasample and intersample collaboration mechanisms, MMLMH achieves superior feature representation for complex health assessments. The framework’s adaptive fusion approach, utilizing attention mechanisms and gated neural networks, demonstrates robust performance across varying noise levels and data quality conditions. Experiments on metaverse healthcare datasets demonstrate MMLMH’s superior performance over baseline methods across multiple evaluation metrics. Longitudinal studies and visualization further illustrate MMLMH’s adaptability to evolving virtual environments and balanced performance across diagnostic accuracy, patient–system interaction efficacy, and data integration complexity. The proposed framework has a unique advantage in that a similar level of performance is maintained across various patient populations and virtual avatars, which could lead to greater personalization of healthcare experiences in the metaverse. MMLMH’s successful functioning in such complicated circumstances suggests that it can combine and process information streams from several sources. They can be successfully utilized in next-generation healthcare delivery through virtual reality. © 2025 Jianhui Lv et al.},
keywords = {Adaptive fusion, Collaborative representations, Diagnosis, Electronic health record, Generative adversarial networks, Health care application, Healthcare environments, Immersive, Learning frameworks, Metaverses, Multi-modal, Multi-modal learning, Performance},
pubstate = {published},
tppubtype = {article}
}
The metaverse enables immersive virtual healthcare environments, presenting opportunities for enhanced care delivery. A key challenge lies in effectively combining multimodal healthcare data and generative artificial intelligence abilities within metaverse-based healthcare applications, which is a problem that needs to be addressed. This paper proposes a novel multimodal learning framework for metaverse healthcare, MMLMH, based on collaborative intra- and intersample representation and adaptive fusion. Our framework introduces a collaborative representation learning approach that captures shared and modality-specific features across text, audio, and visual health data. By combining modality-specific and shared encoders with carefully formulated intrasample and intersample collaboration mechanisms, MMLMH achieves superior feature representation for complex health assessments. The framework’s adaptive fusion approach, utilizing attention mechanisms and gated neural networks, demonstrates robust performance across varying noise levels and data quality conditions. Experiments on metaverse healthcare datasets demonstrate MMLMH’s superior performance over baseline methods across multiple evaluation metrics. Longitudinal studies and visualization further illustrate MMLMH’s adaptability to evolving virtual environments and balanced performance across diagnostic accuracy, patient–system interaction efficacy, and data integration complexity. The proposed framework has a unique advantage in that a similar level of performance is maintained across various patient populations and virtual avatars, which could lead to greater personalization of healthcare experiences in the metaverse. MMLMH’s successful functioning in such complicated circumstances suggests that it can combine and process information streams from several sources. They can be successfully utilized in next-generation healthcare delivery through virtual reality. © 2025 Jianhui Lv et al.
Kurai, R.; Hiraki, T.; Hiroi, Y.; Hirao, Y.; Perusquia-Hernandez, M.; Uchiyama, H.; Kiyokawa, K.
An implementation of MagicCraft: Generating Interactive 3D Objects and Their Behaviors from Text for Commercial Metaverse Platforms Proceedings Article
In: Proc. - IEEE Conf. Virtual Real. 3D User Interfaces Abstr. Workshops, VRW, pp. 1284–1285, Institute of Electrical and Electronics Engineers Inc., 2025, ISBN: 979-833151484-6 (ISBN).
Abstract | Links | BibTeX | Tags: 3D modeling, 3D models, 3D object, 3D Object Generation, 3d-modeling, AI-Assisted Design, Generative AI, Immersive, Metaverse, Metaverses, Model skill, Object oriented programming, Programming skills
@inproceedings{kurai_implementation_2025,
title = {An implementation of MagicCraft: Generating Interactive 3D Objects and Their Behaviors from Text for Commercial Metaverse Platforms},
author = {R. Kurai and T. Hiraki and Y. Hiroi and Y. Hirao and M. Perusquia-Hernandez and H. Uchiyama and K. Kiyokawa},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105005153642&doi=10.1109%2fVRW66409.2025.00288&partnerID=40&md5=53fa1ac92c3210f0ffa090ffa1af7e6e},
doi = {10.1109/VRW66409.2025.00288},
isbn = {979-833151484-6 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Proc. - IEEE Conf. Virtual Real. 3D User Interfaces Abstr. Workshops, VRW},
pages = {1284–1285},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {Metaverse platforms are rapidly evolving to provide immersive spaces. However, the generation of dynamic and interactive 3D objects remains a challenge due to the need for advanced 3D modeling and programming skills. We present MagicCraft, a system that generates functional 3D objects from natural language prompts. MagicCraft uses generative AI models to manage the entire content creation pipeline: converting user text descriptions into images, transforming images into 3D models, predicting object behavior, and assigning necessary attributes and scripts. It also provides an interactive interface for users to refine generated objects by adjusting features like orientation, scale, seating positions, and grip points. © 2025 IEEE.},
keywords = {3D modeling, 3D models, 3D object, 3D Object Generation, 3d-modeling, AI-Assisted Design, Generative AI, Immersive, Metaverse, Metaverses, Model skill, Object oriented programming, Programming skills},
pubstate = {published},
tppubtype = {inproceedings}
}
Metaverse platforms are rapidly evolving to provide immersive spaces. However, the generation of dynamic and interactive 3D objects remains a challenge due to the need for advanced 3D modeling and programming skills. We present MagicCraft, a system that generates functional 3D objects from natural language prompts. MagicCraft uses generative AI models to manage the entire content creation pipeline: converting user text descriptions into images, transforming images into 3D models, predicting object behavior, and assigning necessary attributes and scripts. It also provides an interactive interface for users to refine generated objects by adjusting features like orientation, scale, seating positions, and grip points. © 2025 IEEE.
Arai, K.
Digital Twin Model from Freehanded Sketch to Facade Design, 2D-3D Conversion for Volume Design Journal Article
In: International Journal of Advanced Computer Science and Applications, vol. 16, no. 1, pp. 88–95, 2025, ISSN: 2158107X (ISSN).
Abstract | Links | BibTeX | Tags: 2D/3D conversion, AI, Architectural design, BIM, Digital Twins, Facade design, Facades, GauGAN, Generative AI, GeoTiff, GIS, IFC format, Metaverse, Metaverses, SketchUp, TriPo, Volume design, Volume Rendering
@article{arai_digital_2025,
title = {Digital Twin Model from Freehanded Sketch to Facade Design, 2D-3D Conversion for Volume Design},
author = {K. Arai},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85216872163&doi=10.14569%2fIJACSA.2025.0160109&partnerID=40&md5=fd4e69f9b20d86e3b5d07b4cdcb00b2d},
doi = {10.14569/IJACSA.2025.0160109},
issn = {2158107X (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {International Journal of Advanced Computer Science and Applications},
volume = {16},
number = {1},
pages = {88–95},
abstract = {The article proposes a method for creating digital twins from freehand sketches for facade design, converting 2D designs to 3D volumes, and integrating these designs into real-world GIS systems. It outlines a process that involves generating 2D exterior images from sketches using generative AI (Gemini 1.5 Pro), converting these 2D images into 3D models with TriPo, and creating design drawings with SketchUp. Additionally, it describes a method for creating 3D exterior images using GauGAN, all for the purpose of construction exterior evaluation. The paper also discusses generating BIM data using generative AI, converting BIM data (in IFC file format) to GeoTiff, and displaying this information in GIS using QGIS software. Moreover, it suggests a method for generating digital twins with SketchUp to facilitate digital design information sharing and simulation within a virtual space. Lastly, it advocates for a cost-effective AI system designed for small and medium-sized construction companies, which often struggle to adopt BIM, to harness the advantages of digital twins. © (2025), (Science and Information Organization). All rights reserved.},
keywords = {2D/3D conversion, AI, Architectural design, BIM, Digital Twins, Facade design, Facades, GauGAN, Generative AI, GeoTiff, GIS, IFC format, Metaverse, Metaverses, SketchUp, TriPo, Volume design, Volume Rendering},
pubstate = {published},
tppubtype = {article}
}
The article proposes a method for creating digital twins from freehand sketches for facade design, converting 2D designs to 3D volumes, and integrating these designs into real-world GIS systems. It outlines a process that involves generating 2D exterior images from sketches using generative AI (Gemini 1.5 Pro), converting these 2D images into 3D models with TriPo, and creating design drawings with SketchUp. Additionally, it describes a method for creating 3D exterior images using GauGAN, all for the purpose of construction exterior evaluation. The paper also discusses generating BIM data using generative AI, converting BIM data (in IFC file format) to GeoTiff, and displaying this information in GIS using QGIS software. Moreover, it suggests a method for generating digital twins with SketchUp to facilitate digital design information sharing and simulation within a virtual space. Lastly, it advocates for a cost-effective AI system designed for small and medium-sized construction companies, which often struggle to adopt BIM, to harness the advantages of digital twins. © (2025), (Science and Information Organization). All rights reserved.
Mekki, Y. M.; Simon, L. V.; Freeman, W. D.; Qadir, J.
Medical Education Metaverses (MedEd Metaverses): Opportunities, Use Case, and Guidelines Journal Article
In: Computer, vol. 58, no. 3, pp. 60–70, 2025, ISSN: 00189162 (ISSN).
Abstract | Links | BibTeX | Tags: Adaptive feedback, Augmented Reality, Immersive learning, Medical education, Metaverses, Performance tracking, Remote resources, Remote training, Resource efficiencies, Training efficiency, Virtual environments
@article{mekki_medical_2025,
title = {Medical Education Metaverses (MedEd Metaverses): Opportunities, Use Case, and Guidelines},
author = {Y. M. Mekki and L. V. Simon and W. D. Freeman and J. Qadir},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85218631349&doi=10.1109%2fMC.2024.3474033&partnerID=40&md5=65f46cf9b8d98eaf0fcd6843b9ebc41e},
doi = {10.1109/MC.2024.3474033},
issn = {00189162 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {Computer},
volume = {58},
number = {3},
pages = {60–70},
abstract = {This article explores how artificial intelligence (AI), particularly generative AI (GenAI), can enhance extended reality (XR) applications in medical education (MedEd) metaverses. We compare traditional augmented reality/virtual reality methods with AI-enabled XR metaverses, highlighting improvements in immersive learning, adaptive feedback, personalized performance tracking, remote training, and resource efficiency. © 1970-2012 IEEE.},
keywords = {Adaptive feedback, Augmented Reality, Immersive learning, Medical education, Metaverses, Performance tracking, Remote resources, Remote training, Resource efficiencies, Training efficiency, Virtual environments},
pubstate = {published},
tppubtype = {article}
}
This article explores how artificial intelligence (AI), particularly generative AI (GenAI), can enhance extended reality (XR) applications in medical education (MedEd) metaverses. We compare traditional augmented reality/virtual reality methods with AI-enabled XR metaverses, highlighting improvements in immersive learning, adaptive feedback, personalized performance tracking, remote training, and resource efficiency. © 1970-2012 IEEE.
Zhang, Z.; Jiang, Y.; Wei, X.; Chen, M.; Dong, H.; Yu, S.
Generative-AI for XR Content Transmission in the Metaverse: Potential Approaches, Challenges, and a Generation-Driven Transmission Framework Journal Article
In: IEEE Network, 2025, ISSN: 08908044 (ISSN).
Abstract | Links | BibTeX | Tags: 'current, Cloud servers, Collaboration architecture, Content transmission, Decision making, Deep learning, Deep reinforcement learning, Edge server, Generative adversarial networks, Intelligence models, Large volumes, Metaverses, Network bottlenecks, Reinforcement Learning, Through current
@article{zhang_generative-ai_2025,
title = {Generative-AI for XR Content Transmission in the Metaverse: Potential Approaches, Challenges, and a Generation-Driven Transmission Framework},
author = {Z. Zhang and Y. Jiang and X. Wei and M. Chen and H. Dong and S. Yu},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-86000661262&doi=10.1109%2fMNET.2025.3547385&partnerID=40&md5=1e00d40542ec58ef1489934abb2a990c},
doi = {10.1109/MNET.2025.3547385},
issn = {08908044 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {IEEE Network},
abstract = {How to efficiently transmit large volumes of Extended Reality (XR) content through current networks has been a major bottleneck in realizing the Metaverse. The recently emerging Generative Artificial Intelligence (GAI) has already revolutionized various technological fields and provides promising solutions to this challenge. In this article, we first demonstrate current networks' bottlenecks for supporting XR content transmission in the Metaverse. Then, we explore the potential approaches and challenges of utilizing GAI to overcome these bottlenecks. To address these challenges, we propose a GAI-based XR content transmission framework which leverages a cloud-edge collaboration architecture. The cloud servers are responsible for storing and rendering the original XR content, while edge servers utilize GAI models to generate essential parts of XR content (e.g., subsequent frames, selected objects, etc.) when network resources are insufficient to transmit them. A Deep Reinforcement Learning (DRL)-based decision module is proposed to solve the decision-making problems. Our case study demonstrates that the proposed GAI-based transmission framework achieves a 2.8-fold increase in normal frame ratio (percentage of frames that meet the quality and latency requirements for XR content transmission) over baseline approaches, underscoring the potential of GAI models to facilitate XR content transmission in the Metaverse. © 2025 IEEE.},
keywords = {'current, Cloud servers, Collaboration architecture, Content transmission, Decision making, Deep learning, Deep reinforcement learning, Edge server, Generative adversarial networks, Intelligence models, Large volumes, Metaverses, Network bottlenecks, Reinforcement Learning, Through current},
pubstate = {published},
tppubtype = {article}
}
How to efficiently transmit large volumes of Extended Reality (XR) content through current networks has been a major bottleneck in realizing the Metaverse. The recently emerging Generative Artificial Intelligence (GAI) has already revolutionized various technological fields and provides promising solutions to this challenge. In this article, we first demonstrate current networks' bottlenecks for supporting XR content transmission in the Metaverse. Then, we explore the potential approaches and challenges of utilizing GAI to overcome these bottlenecks. To address these challenges, we propose a GAI-based XR content transmission framework which leverages a cloud-edge collaboration architecture. The cloud servers are responsible for storing and rendering the original XR content, while edge servers utilize GAI models to generate essential parts of XR content (e.g., subsequent frames, selected objects, etc.) when network resources are insufficient to transmit them. A Deep Reinforcement Learning (DRL)-based decision module is proposed to solve the decision-making problems. Our case study demonstrates that the proposed GAI-based transmission framework achieves a 2.8-fold increase in normal frame ratio (percentage of frames that meet the quality and latency requirements for XR content transmission) over baseline approaches, underscoring the potential of GAI models to facilitate XR content transmission in the Metaverse. © 2025 IEEE.
Qian, Y.; Siau, K. L.
IoT in Sustainability and IoT in the AI and Metaverse Age Journal Article
In: IEEE Internet of Things Magazine, vol. 8, no. 3, pp. 92–98, 2025, ISSN: 25763180 (ISSN).
Abstract | Links | BibTeX | Tags: Energy, Energy systems, Environmental sustainability, Existing energies, Heterogeneous devices, Metaverses, Modern technologies, Renewable technology
@article{qian_iot_2025,
title = {IoT in Sustainability and IoT in the AI and Metaverse Age},
author = {Y. Qian and K. L. Siau},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105004261858&doi=10.1109%2fIOTM.001.2400179&partnerID=40&md5=fec6f878a2d8eab9111df2dbda35df9c},
doi = {10.1109/IOTM.001.2400179},
issn = {25763180 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {IEEE Internet of Things Magazine},
volume = {8},
number = {3},
pages = {92–98},
abstract = {The Internet of Things (IoT) is a modern technology that has gained large popularity and is still developing. Connecting heterogeneous devices, such as phones, vehicles, and household appliances, IoT has brought convenience to our lives. Further, IoT plays a significant role in enhancing environmental sustainability. It provides timely data about different devices and enables users and managers to directly control the objects. IoT can optimize the existing energy systems and promote the usage of renewable technologies. In this article, we discuss how IoT supports green initiatives (i.e., how it is applied in different sectors), how it can be “green” itself (i.e., from both the technical and managerial perspectives), and future directions of the advancement of IoT (e.g., IoT in the AI and Metaverse age). Specifically, we explain how IoT enhances environmental sustainability in different sectors, including energy, agriculture, smart cities, and transportation. We also show how IoT can be green itself through technical and managerial approaches. Further, we discuss how IoT can be integrated with AI, such as Generative AI, Agentic AI, and the Metaverse, to enhance impacts. Clearly, IoT has the potential to be further developed and make an even greater contribution to environmental sustainability. © 2018 IEEE.},
keywords = {Energy, Energy systems, Environmental sustainability, Existing energies, Heterogeneous devices, Metaverses, Modern technologies, Renewable technology},
pubstate = {published},
tppubtype = {article}
}
The Internet of Things (IoT) is a modern technology that has gained large popularity and is still developing. Connecting heterogeneous devices, such as phones, vehicles, and household appliances, IoT has brought convenience to our lives. Further, IoT plays a significant role in enhancing environmental sustainability. It provides timely data about different devices and enables users and managers to directly control the objects. IoT can optimize the existing energy systems and promote the usage of renewable technologies. In this article, we discuss how IoT supports green initiatives (i.e., how it is applied in different sectors), how it can be “green” itself (i.e., from both the technical and managerial perspectives), and future directions of the advancement of IoT (e.g., IoT in the AI and Metaverse age). Specifically, we explain how IoT enhances environmental sustainability in different sectors, including energy, agriculture, smart cities, and transportation. We also show how IoT can be green itself through technical and managerial approaches. Further, we discuss how IoT can be integrated with AI, such as Generative AI, Agentic AI, and the Metaverse, to enhance impacts. Clearly, IoT has the potential to be further developed and make an even greater contribution to environmental sustainability. © 2018 IEEE.
Zhang, Z.; Wang, J.; Chen, J.; Fang, Z.; Jiang, C.; Han, Z.
A Priority-Aware AI-Generated Content Resource Allocation Method for Multi-UAV Aided Metaverse Proceedings Article
In: IEEE Wireless Commun. Networking Conf. WCNC, Institute of Electrical and Electronics Engineers Inc., 2025, ISBN: 15253511 (ISSN); 979-835036836-9 (ISBN).
Abstract | Links | BibTeX | Tags: Aerial vehicle, AI-generated content, AI-generated content (AIGC), Allocation methods, Content-resources, Diffusion Model, Drones, Metaverse, Metaverses, Priority-aware, Reinforcement Learning, Reinforcement learnings, Resource allocation, Resources allocation, Target drones, Unmanned aerial vehicle, Unmanned aerial vehicle (UAV)
@inproceedings{zhang_priority-aware_2025,
title = {A Priority-Aware AI-Generated Content Resource Allocation Method for Multi-UAV Aided Metaverse},
author = {Z. Zhang and J. Wang and J. Chen and Z. Fang and C. Jiang and Z. Han},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105006408540&doi=10.1109%2fWCNC61545.2025.10978443&partnerID=40&md5=69937c6fa9be1a038b28e7884dfe586b},
doi = {10.1109/WCNC61545.2025.10978443},
isbn = {15253511 (ISSN); 979-835036836-9 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {IEEE Wireless Commun. Networking Conf. WCNC},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {With the advancement of large model technologies, AI -generated content is gradually emerging as a mainstream method for content creation. The metaverse, as a key application scenario for the next-generation communication technologies, heavily depends on advanced content generation technologies. Nevertheless, the diverse types of metaverse applications and their stringent real-time requirements constrain the full potential of AIGC technologies within this environment. In order to tackle with this problem, we construct a priority-aware multi-UAV aided metaverse system and formulate it as a Markov decision process (MDP). We propose a diffusion-based reinforcement learning algorithm to solve the resource allocation problem and demonstrate its superiority through enough comparison and ablation experiments. © 2025 IEEE.},
keywords = {Aerial vehicle, AI-generated content, AI-generated content (AIGC), Allocation methods, Content-resources, Diffusion Model, Drones, Metaverse, Metaverses, Priority-aware, Reinforcement Learning, Reinforcement learnings, Resource allocation, Resources allocation, Target drones, Unmanned aerial vehicle, Unmanned aerial vehicle (UAV)},
pubstate = {published},
tppubtype = {inproceedings}
}
With the advancement of large model technologies, AI -generated content is gradually emerging as a mainstream method for content creation. The metaverse, as a key application scenario for the next-generation communication technologies, heavily depends on advanced content generation technologies. Nevertheless, the diverse types of metaverse applications and their stringent real-time requirements constrain the full potential of AIGC technologies within this environment. In order to tackle with this problem, we construct a priority-aware multi-UAV aided metaverse system and formulate it as a Markov decision process (MDP). We propose a diffusion-based reinforcement learning algorithm to solve the resource allocation problem and demonstrate its superiority through enough comparison and ablation experiments. © 2025 IEEE.
Li, Y.; Pang, E. C. H.; Ng, C. S. Y.; Azim, M.; Leung, H.
Enhancing Linear Algebra Education with AI-Generated Content in the CityU Metaverse: A Comparative Study Proceedings Article
In: T., Hao; J.G., Wu; X., Luo; Y., Sun; Y., Mu; S., Ge; W., Xie (Ed.): Lect. Notes Comput. Sci., pp. 3–16, Springer Science and Business Media Deutschland GmbH, 2025, ISBN: 03029743 (ISSN); 978-981964406-3 (ISBN).
Abstract | Links | BibTeX | Tags: Comparatives studies, Digital age, Digital interactions, digital twin, Educational metaverse, Engineering education, Generative AI, Immersive, Matrix algebra, Metaverse, Metaverses, Personnel training, Students, Teaching, University campus, Virtual environments, virtual learning environment, Virtual learning environments, Virtual Reality, Virtualization
@inproceedings{li_enhancing_2025,
title = {Enhancing Linear Algebra Education with AI-Generated Content in the CityU Metaverse: A Comparative Study},
author = {Y. Li and E. C. H. Pang and C. S. Y. Ng and M. Azim and H. Leung},
editor = {Hao T. and Wu J.G. and Luo X. and Sun Y. and Mu Y. and Ge S. and Xie W.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105003632691&doi=10.1007%2f978-981-96-4407-0_1&partnerID=40&md5=c067ba5d4c15e9c0353bf315680531fc},
doi = {10.1007/978-981-96-4407-0_1},
isbn = {03029743 (ISSN); 978-981964406-3 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Lect. Notes Comput. Sci.},
volume = {15589 LNCS},
pages = {3–16},
publisher = {Springer Science and Business Media Deutschland GmbH},
abstract = {In today’s digital age, the metaverse is emerging as the forthcoming evolution of the internet. It provides an immersive space that marks a new frontier in the way digital interactions are facilitated and experienced. In this paper, we present the CityU Metaverse, which aims to construct a digital twin of our university campus. It is designed as an educational virtual world where learning applications can be embedded in this virtual campus, supporting not only remote and collaborative learning but also professional technical training to enhance educational experiences through immersive and interactive learning. To evaluate the effectiveness of this educational metaverse, we conducted an experiment focused on 3D linear transformation in linear algebra, with teaching content generated by generative AI, comparing our metaverse system with traditional teaching methods. Knowledge tests and surveys assessing learning interest revealed that students engaged with the CityU Metaverse, facilitated by AI-generated content, outperformed those in traditional settings and reported greater enjoyment during the learning process. The work provides valuable perspectives on the behaviors and interactions within the metaverse by analyzing user preferences and learning outcomes. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.},
keywords = {Comparatives studies, Digital age, Digital interactions, digital twin, Educational metaverse, Engineering education, Generative AI, Immersive, Matrix algebra, Metaverse, Metaverses, Personnel training, Students, Teaching, University campus, Virtual environments, virtual learning environment, Virtual learning environments, Virtual Reality, Virtualization},
pubstate = {published},
tppubtype = {inproceedings}
}
In today’s digital age, the metaverse is emerging as the forthcoming evolution of the internet. It provides an immersive space that marks a new frontier in the way digital interactions are facilitated and experienced. In this paper, we present the CityU Metaverse, which aims to construct a digital twin of our university campus. It is designed as an educational virtual world where learning applications can be embedded in this virtual campus, supporting not only remote and collaborative learning but also professional technical training to enhance educational experiences through immersive and interactive learning. To evaluate the effectiveness of this educational metaverse, we conducted an experiment focused on 3D linear transformation in linear algebra, with teaching content generated by generative AI, comparing our metaverse system with traditional teaching methods. Knowledge tests and surveys assessing learning interest revealed that students engaged with the CityU Metaverse, facilitated by AI-generated content, outperformed those in traditional settings and reported greater enjoyment during the learning process. The work provides valuable perspectives on the behaviors and interactions within the metaverse by analyzing user preferences and learning outcomes. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.
Cao, J.; Zhou, M.; Wang, J.; Liu, G.; Niyato, D.; Mao, S.; Han, Z.; Kang, J.
A Unified Framework for Underwater Metaverse with Optical Perception Journal Article
In: IEEE Wireless Communications, vol. 32, no. 3, pp. 220–231, 2025, ISSN: 15361284 (ISSN).
Abstract | Links | BibTeX | Tags: AI Technologies, Deep sea exploration, Imaging technology, Marine conservations, Metaverses, Optical-, Quantum imaging, Underwater environments, Unified framework, Virtual reality system
@article{cao_unified_2025,
title = {A Unified Framework for Underwater Metaverse with Optical Perception},
author = {J. Cao and M. Zhou and J. Wang and G. Liu and D. Niyato and S. Mao and Z. Han and J. Kang},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105006878504&doi=10.1109%2fMWC.006.2400050&partnerID=40&md5=592626e928bfefaf441b090d3ac16c2e},
doi = {10.1109/MWC.006.2400050},
issn = {15361284 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {IEEE Wireless Communications},
volume = {32},
number = {3},
pages = {220–231},
abstract = {With the advancement of AI technology and increasing attention to deep-sea exploration, the underwater Metaverse is gradually emerging. This article explores the concept of underwater Metaverse, emerging virtual reality systems, and services aimed at simulating and enhancing the virtual experience of marine environments. First, we discuss potential applications of underwater Metaverse in underwater scientific research and marine conservation. Next, we design the architecture and highlight the corresponding design requirements. Then, we present the characteristics of different underwater imaging technologies, such as underwater acoustic imaging, underwater radio imaging, and underwater quantum imaging, in the supporting technologies of the underwater Metaverse. Quantum imaging (QI) technology is suitable for extremely low-light underwater environments, improving the precision and efficiency of underwater imaging. Based on this, we present a use case for building a realistic underwater virtual world using underwater quantum imaging-generative artificial intelligence (QI-GenAI) technology. The results demonstrate the effectiveness of the underwater Metaverse framework in simulating complex underwater environments, thus validating its potential to provide high-quality, interactive underwater virtual experiences. Finally, the article examines important future research directions of underwater Metaverse and provides new perspectives for marine science and conservation. © 2002-2012 IEEE.},
keywords = {AI Technologies, Deep sea exploration, Imaging technology, Marine conservations, Metaverses, Optical-, Quantum imaging, Underwater environments, Unified framework, Virtual reality system},
pubstate = {published},
tppubtype = {article}
}
With the advancement of AI technology and increasing attention to deep-sea exploration, the underwater Metaverse is gradually emerging. This article explores the concept of underwater Metaverse, emerging virtual reality systems, and services aimed at simulating and enhancing the virtual experience of marine environments. First, we discuss potential applications of underwater Metaverse in underwater scientific research and marine conservation. Next, we design the architecture and highlight the corresponding design requirements. Then, we present the characteristics of different underwater imaging technologies, such as underwater acoustic imaging, underwater radio imaging, and underwater quantum imaging, in the supporting technologies of the underwater Metaverse. Quantum imaging (QI) technology is suitable for extremely low-light underwater environments, improving the precision and efficiency of underwater imaging. Based on this, we present a use case for building a realistic underwater virtual world using underwater quantum imaging-generative artificial intelligence (QI-GenAI) technology. The results demonstrate the effectiveness of the underwater Metaverse framework in simulating complex underwater environments, thus validating its potential to provide high-quality, interactive underwater virtual experiences. Finally, the article examines important future research directions of underwater Metaverse and provides new perspectives for marine science and conservation. © 2002-2012 IEEE.
Zhang, Z.; Wang, J.; Chen, J.; Fu, H.; Tong, Z.; Jiang, C.
Diffusion-Based Reinforcement Learning for Cooperative Offloading and Resource Allocation in Multi-UAV Assisted Edge-Enabled Metaverse Journal Article
In: IEEE Transactions on Vehicular Technology, 2025, ISSN: 00189545 (ISSN).
Abstract | Links | BibTeX | Tags: Aerial vehicle, Content creation, Content services, Contrastive Learning, Decision making, Deep learning, Deep reinforcement learning, Diffusion Model, Global industry, Helicopter services, Markov processes, Metaverse, Metaverses, Reinforcement Learning, Reinforcement learnings, Resource allocation, Resources allocation, Typical application, Unmanned aerial vehicle, Unmanned aerial vehicle (UAV), Unmanned aerial vehicles (UAV)
@article{zhang_diffusion-based_2025,
title = {Diffusion-Based Reinforcement Learning for Cooperative Offloading and Resource Allocation in Multi-UAV Assisted Edge-Enabled Metaverse},
author = {Z. Zhang and J. Wang and J. Chen and H. Fu and Z. Tong and C. Jiang},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85219108203&doi=10.1109%2fTVT.2025.3544879&partnerID=40&md5=fdbe1554f6cf7d47d4bbbb73b4b0d487},
doi = {10.1109/TVT.2025.3544879},
issn = {00189545 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {IEEE Transactions on Vehicular Technology},
abstract = {As one of the typical applications of 6G, the metaverse, with its superior immersion and diversified services, has garnered widespread attention from both the global industry and academia. Simultaneously, the emergence of AI-generated content (AIGC), exemplified by ChatGPT, has revolutionized the mean of content creation in the metaverse. Providing meataverse users with diversified AIGC services anytime and anywhere to meet the demand for immersive and blended virtual-real experiences in the physical world has become a major challenge in the development of the metaverse. Considering the flexibility and mobility of unmanned aerial vehicles (UAVs), we innovatively incorporate multiple UAVs as one of the AIGC service providers and construct a multi-UAV assisted edge-enabled metaverse system in the context of AIGC-as-a-Service (AaaS) scenario. To solve the complex resource management and allocation problem in the aforementioned system, we formulate it as a Markov decision process (MDP) and propose utilizing the generative capabilities of the diffusion model in combination with the robust decision-making abilities of reinforcement learning to tackle these issues. In order to substantiate the efficacy of the proposed diffusion-based reinforcement learning framework, we propose a novel diffusion-based soft actor-critic algorithm for metaverse (Meta-DSAC). Subsequently, a series of experiments are executed and the simulation results empirically validate the proposed algorithm's comparative advantages of the ability to provide stable and substantial long-term rewards, as well as the enhanced capacity to model complex environment. © 2025 IEEE.},
keywords = {Aerial vehicle, Content creation, Content services, Contrastive Learning, Decision making, Deep learning, Deep reinforcement learning, Diffusion Model, Global industry, Helicopter services, Markov processes, Metaverse, Metaverses, Reinforcement Learning, Reinforcement learnings, Resource allocation, Resources allocation, Typical application, Unmanned aerial vehicle, Unmanned aerial vehicle (UAV), Unmanned aerial vehicles (UAV)},
pubstate = {published},
tppubtype = {article}
}
As one of the typical applications of 6G, the metaverse, with its superior immersion and diversified services, has garnered widespread attention from both the global industry and academia. Simultaneously, the emergence of AI-generated content (AIGC), exemplified by ChatGPT, has revolutionized the mean of content creation in the metaverse. Providing meataverse users with diversified AIGC services anytime and anywhere to meet the demand for immersive and blended virtual-real experiences in the physical world has become a major challenge in the development of the metaverse. Considering the flexibility and mobility of unmanned aerial vehicles (UAVs), we innovatively incorporate multiple UAVs as one of the AIGC service providers and construct a multi-UAV assisted edge-enabled metaverse system in the context of AIGC-as-a-Service (AaaS) scenario. To solve the complex resource management and allocation problem in the aforementioned system, we formulate it as a Markov decision process (MDP) and propose utilizing the generative capabilities of the diffusion model in combination with the robust decision-making abilities of reinforcement learning to tackle these issues. In order to substantiate the efficacy of the proposed diffusion-based reinforcement learning framework, we propose a novel diffusion-based soft actor-critic algorithm for metaverse (Meta-DSAC). Subsequently, a series of experiments are executed and the simulation results empirically validate the proposed algorithm's comparative advantages of the ability to provide stable and substantial long-term rewards, as well as the enhanced capacity to model complex environment. © 2025 IEEE.
Huang, D.; Ge, M.; Xiang, K.; Zhang, X.; Yang, H.
Privacy Preservation of Large Language Models in the Metaverse Era: Research Frontiers, Categorical Comparisons, and Future Directions Proceedings Article
In: Int J Network Manage, John Wiley and Sons Ltd, 2025, ISBN: 10557148 (ISSN).
Abstract | Links | BibTeX | Tags: Adversarial networks, Computational Linguistics, Cryptography, Differential privacies, Excel, Language Model, Large language model, large language models, Life cycle, Metaverse, Metaverses, Natural language processing systems, Natural languages, Privacy preservation, Privacy protection, Research frontiers
@inproceedings{huang_privacy_2025,
title = {Privacy Preservation of Large Language Models in the Metaverse Era: Research Frontiers, Categorical Comparisons, and Future Directions},
author = {D. Huang and M. Ge and K. Xiang and X. Zhang and H. Yang},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85199980257&doi=10.1002%2fnem.2292&partnerID=40&md5=2dea1caa1d31aecde3d302a908fb7dd3},
doi = {10.1002/nem.2292},
isbn = {10557148 (ISSN)},
year = {2025},
date = {2025-01-01},
booktitle = {Int J Network Manage},
volume = {35},
publisher = {John Wiley and Sons Ltd},
abstract = {Large language models (LLMs), with their billions to trillions of parameters, excel in natural language processing, machine translation, dialog systems, and text summarization. These capabilities are increasingly pivotal in the metaverse, where they can enhance virtual interactions and environments. However, their extensive use, particularly in the metaverse's immersive platforms, raises significant privacy concerns. This paper analyzes existing privacy issues in LLMs, vital for both traditional and metaverse applications, and examines protection techniques across the entire life cycle of these models, from training to user deployment. We delve into cryptography, embedding layer encoding, differential privacy and its variants, and adversarial networks, highlighting their relevance in the metaverse context. Specifically, we explore technologies like homomorphic encryption and secure multiparty computation, which are essential for metaverse security. Our discussion on Gaussian differential privacy, Renyi differential privacy, Edgeworth accounting, and the generation of adversarial samples and loss functions emphasizes their importance in the metaverse's dynamic and interactive environments. Lastly, the paper discusses the current research status and future challenges in the security of LLMs within and beyond the metaverse, emphasizing urgent problems and potential areas for exploration. © 2024 John Wiley & Sons Ltd.},
keywords = {Adversarial networks, Computational Linguistics, Cryptography, Differential privacies, Excel, Language Model, Large language model, large language models, Life cycle, Metaverse, Metaverses, Natural language processing systems, Natural languages, Privacy preservation, Privacy protection, Research frontiers},
pubstate = {published},
tppubtype = {inproceedings}
}
Large language models (LLMs), with their billions to trillions of parameters, excel in natural language processing, machine translation, dialog systems, and text summarization. These capabilities are increasingly pivotal in the metaverse, where they can enhance virtual interactions and environments. However, their extensive use, particularly in the metaverse's immersive platforms, raises significant privacy concerns. This paper analyzes existing privacy issues in LLMs, vital for both traditional and metaverse applications, and examines protection techniques across the entire life cycle of these models, from training to user deployment. We delve into cryptography, embedding layer encoding, differential privacy and its variants, and adversarial networks, highlighting their relevance in the metaverse context. Specifically, we explore technologies like homomorphic encryption and secure multiparty computation, which are essential for metaverse security. Our discussion on Gaussian differential privacy, Renyi differential privacy, Edgeworth accounting, and the generation of adversarial samples and loss functions emphasizes their importance in the metaverse's dynamic and interactive environments. Lastly, the paper discusses the current research status and future challenges in the security of LLMs within and beyond the metaverse, emphasizing urgent problems and potential areas for exploration. © 2024 John Wiley & Sons Ltd.
Aloudat, M. Z.; Aboumadi, A.; Soliman, A.; Al-Mohammed, H. A.; Al-Ali, M.; Mahgoub, A.; Barhamgi, M.; Yaacoub, E.
Metaverse Unbound: A Survey on Synergistic Integration Between Semantic Communication, 6G, and Edge Learning Journal Article
In: IEEE Access, vol. 13, pp. 58302–58350, 2025, ISSN: 21693536 (ISSN).
Abstract | Links | BibTeX | Tags: 6g wireless system, 6G wireless systems, Augmented Reality, Block-chain, Blockchain, Blockchain technology, Digital Twin Technology, Edge learning, Extended reality (XR), Language Model, Large language model, large language models (LLMs), Metaverse, Metaverses, Semantic communication, Virtual environments, Wireless systems
@article{aloudat_metaverse_2025,
title = {Metaverse Unbound: A Survey on Synergistic Integration Between Semantic Communication, 6G, and Edge Learning},
author = {M. Z. Aloudat and A. Aboumadi and A. Soliman and H. A. Al-Mohammed and M. Al-Ali and A. Mahgoub and M. Barhamgi and E. Yaacoub},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105003088610&doi=10.1109%2fACCESS.2025.3555753&partnerID=40&md5=8f3f9421ce2d6be57f8154a122ee192c},
doi = {10.1109/ACCESS.2025.3555753},
issn = {21693536 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {IEEE Access},
volume = {13},
pages = {58302–58350},
abstract = {With a focus on edge learning, blockchain, sixth generation (6G) wireless systems, semantic communication, and large language models (LLMs), this survey paper examines the revolutionary integration of cutting-edge technologies within the metaverse. This thorough examination highlights the critical role these technologies play in improving realism and user engagement on three main levels: technical, virtual, and physical. While the virtual layer focuses on building immersive experiences, the physical layer highlights improvements to the user interface through augmented reality (AR) goggles and virtual reality (VR) headsets. Blockchain-powered technical layer enables safe, decentralized communication. The survey highlights how the metaverse has the potential to drastically change how people interact in society by exploring applications in a variety of fields, such as immersive education, remote work, and entertainment. Concerns about privacy, scalability, and interoperability are raised, highlighting the necessity of continued study to realize the full potential of the metaverse. For scholars looking to broaden the reach and significance of the metaverse in the digital age, this paper is a useful tool. © 2013 IEEE.},
keywords = {6g wireless system, 6G wireless systems, Augmented Reality, Block-chain, Blockchain, Blockchain technology, Digital Twin Technology, Edge learning, Extended reality (XR), Language Model, Large language model, large language models (LLMs), Metaverse, Metaverses, Semantic communication, Virtual environments, Wireless systems},
pubstate = {published},
tppubtype = {article}
}
With a focus on edge learning, blockchain, sixth generation (6G) wireless systems, semantic communication, and large language models (LLMs), this survey paper examines the revolutionary integration of cutting-edge technologies within the metaverse. This thorough examination highlights the critical role these technologies play in improving realism and user engagement on three main levels: technical, virtual, and physical. While the virtual layer focuses on building immersive experiences, the physical layer highlights improvements to the user interface through augmented reality (AR) goggles and virtual reality (VR) headsets. Blockchain-powered technical layer enables safe, decentralized communication. The survey highlights how the metaverse has the potential to drastically change how people interact in society by exploring applications in a variety of fields, such as immersive education, remote work, and entertainment. Concerns about privacy, scalability, and interoperability are raised, highlighting the necessity of continued study to realize the full potential of the metaverse. For scholars looking to broaden the reach and significance of the metaverse in the digital age, this paper is a useful tool. © 2013 IEEE.
Zhang, G.; Wang, Y.; Luo, C.; Xu, S.; Ming, Y.; Peng, J.; Zhang, M.
Visual Harmony: LLM’s Power in Crafting Coherent Indoor Scenes from Images Proceedings Article
In: Z., Lin; H., Zha; M.-M., Cheng; R., He; C.-L., Liu; K., Ubul; W., Silamu; J., Zhou (Ed.): Lect. Notes Comput. Sci., pp. 3–17, Springer Science and Business Media Deutschland GmbH, 2025, ISBN: 03029743 (ISSN); 978-981978507-0 (ISBN).
Abstract | Links | BibTeX | Tags: Augmented Reality, Depth perception, Indoor scene generation, Input image, Language Model, Large language model, Metaverses, Point-clouds, Power, Scene completion, Scene Generation, Scene-graphs, Semantic Segmentation, Semantics, Virtual Reality, Visual languages
@inproceedings{zhang_visual_2025,
title = {Visual Harmony: LLM’s Power in Crafting Coherent Indoor Scenes from Images},
author = {G. Zhang and Y. Wang and C. Luo and S. Xu and Y. Ming and J. Peng and M. Zhang},
editor = {Lin Z. and Zha H. and Cheng M.-M. and He R. and Liu C.-L. and Ubul K. and Silamu W. and Zhou J.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85209374797&doi=10.1007%2f978-981-97-8508-7_1&partnerID=40&md5=5231ab0bce95fb3f09db80392acd58ff},
doi = {10.1007/978-981-97-8508-7_1},
isbn = {03029743 (ISSN); 978-981978507-0 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Lect. Notes Comput. Sci.},
volume = {15036 LNCS},
pages = {3–17},
publisher = {Springer Science and Business Media Deutschland GmbH},
abstract = {Indoor scene generation has recently attracted significant attention as it is crucial for metaverse, 3D animation, visual effects in movies, and virtual/augmented reality. Existing image-based indoor scene generation methods often produce scenes that are not realistic enough, with issues such as floating objects, incorrect object orientations, and incomplete scenes that only include the part of the scenes captured by the input image. To address these challenges, we propose Visual Harmony, a method that leverages the powerful spatial imagination capabilities of Large Language Model (LLM) to generate corresponding indoor scenes based on the input image. Specifically, we first extract information from the input image through depth estimation and panorama segmentation, reconstructing a semantic point cloud. Using this reconstructed semantic point cloud, we extract a scene graph that describes only the objects in the image. Then we leverage the strong spatial imagination capabilities of LLM to complete the scene graph, forming a representation of a complete room scene. Based on this fine scene graph, we can generate entire indoor scene that includes both the captured and not captured parts of the input image. Extensive experiments demonstrate that our method can generate realistic, plausible, and highly relevant complete indoor scenes related to the input image. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.},
keywords = {Augmented Reality, Depth perception, Indoor scene generation, Input image, Language Model, Large language model, Metaverses, Point-clouds, Power, Scene completion, Scene Generation, Scene-graphs, Semantic Segmentation, Semantics, Virtual Reality, Visual languages},
pubstate = {published},
tppubtype = {inproceedings}
}
Indoor scene generation has recently attracted significant attention as it is crucial for metaverse, 3D animation, visual effects in movies, and virtual/augmented reality. Existing image-based indoor scene generation methods often produce scenes that are not realistic enough, with issues such as floating objects, incorrect object orientations, and incomplete scenes that only include the part of the scenes captured by the input image. To address these challenges, we propose Visual Harmony, a method that leverages the powerful spatial imagination capabilities of Large Language Model (LLM) to generate corresponding indoor scenes based on the input image. Specifically, we first extract information from the input image through depth estimation and panorama segmentation, reconstructing a semantic point cloud. Using this reconstructed semantic point cloud, we extract a scene graph that describes only the objects in the image. Then we leverage the strong spatial imagination capabilities of LLM to complete the scene graph, forming a representation of a complete room scene. Based on this fine scene graph, we can generate entire indoor scene that includes both the captured and not captured parts of the input image. Extensive experiments demonstrate that our method can generate realistic, plausible, and highly relevant complete indoor scenes related to the input image. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.
Hu, Y. -H.; Matsumoto, A.; Ito, K.; Narumi, T.; Kuzuoka, H.; Amemiya, T.
Avatar Motion Generation Pipeline for the Metaverse via Synthesis of Generative Models of Text and Video Proceedings Article
In: Proc. - IEEE Conf. Virtual Real. 3D User Interfaces Abstr. Workshops, VRW, pp. 767–771, Institute of Electrical and Electronics Engineers Inc., 2025, ISBN: 979-833151484-6 (ISBN).
Abstract | Links | BibTeX | Tags: Ambient intelligence, Design and evaluation methods, Distributed computer systems, Human-centered computing, Language Model, Metaverses, Processing capability, Text-processing, Treemap, Treemaps, Visualization, Visualization design and evaluation method, Visualization design and evaluation methods, Visualization designs, Visualization technique, Visualization techniques
@inproceedings{hu_avatar_2025,
title = {Avatar Motion Generation Pipeline for the Metaverse via Synthesis of Generative Models of Text and Video},
author = {Y. -H. Hu and A. Matsumoto and K. Ito and T. Narumi and H. Kuzuoka and T. Amemiya},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105005158851&doi=10.1109%2fVRW66409.2025.00155&partnerID=40&md5=2bc9a6390e1cf710206835722ca8dbbf},
doi = {10.1109/VRW66409.2025.00155},
isbn = {979-833151484-6 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Proc. - IEEE Conf. Virtual Real. 3D User Interfaces Abstr. Workshops, VRW},
pages = {767–771},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {Efforts to integrate AI avatars into the metaverse to enhance interactivity have progressed in both research and commercial domains. AI avatars in the metaverse are expected to exhibit not only verbal responses but also avatar motions, such as non-verbal gestures, to enable seamless communication with users. Large Language Models (LLMs) are known for their advanced text processing capabilities, such as user input, avatar actions, and even entire virtual environments as text, making them a promising approach for planning avatar motions. However, generating the avatar motions solely from the textual information often requires extensive training data whereas the configuration is very challenging, with results that often lack diversity and fail to match user expectations. On the other hand, AI technologies for generating videos have progressed to the point where they can depict diverse and natural human movements based on prompts. Therefore, this paper introduces a novel pipeline, TVMP, that synthesizes LLMs with advanced text processing capabilities and video generation models with the ability to generate videos containing a variety of motions. The pipeline first generates videos from text input, then estimates the motions from the generated videos, and lastly exports the estimated motion data into the avatars in the metaverse. Feedback on the TVMP prototype suggests further refinement is needed, such as speed control, display of the progress, and direct edition for contextual relevance and usability enhancements. The proposed method enables AI avatars to perform highly adaptive and diverse movements to fulfill user expectations and contributes to developing a more immersive metaverse. © 2025 IEEE.},
keywords = {Ambient intelligence, Design and evaluation methods, Distributed computer systems, Human-centered computing, Language Model, Metaverses, Processing capability, Text-processing, Treemap, Treemaps, Visualization, Visualization design and evaluation method, Visualization design and evaluation methods, Visualization designs, Visualization technique, Visualization techniques},
pubstate = {published},
tppubtype = {inproceedings}
}
Efforts to integrate AI avatars into the metaverse to enhance interactivity have progressed in both research and commercial domains. AI avatars in the metaverse are expected to exhibit not only verbal responses but also avatar motions, such as non-verbal gestures, to enable seamless communication with users. Large Language Models (LLMs) are known for their advanced text processing capabilities, such as user input, avatar actions, and even entire virtual environments as text, making them a promising approach for planning avatar motions. However, generating the avatar motions solely from the textual information often requires extensive training data whereas the configuration is very challenging, with results that often lack diversity and fail to match user expectations. On the other hand, AI technologies for generating videos have progressed to the point where they can depict diverse and natural human movements based on prompts. Therefore, this paper introduces a novel pipeline, TVMP, that synthesizes LLMs with advanced text processing capabilities and video generation models with the ability to generate videos containing a variety of motions. The pipeline first generates videos from text input, then estimates the motions from the generated videos, and lastly exports the estimated motion data into the avatars in the metaverse. Feedback on the TVMP prototype suggests further refinement is needed, such as speed control, display of the progress, and direct edition for contextual relevance and usability enhancements. The proposed method enables AI avatars to perform highly adaptive and diverse movements to fulfill user expectations and contributes to developing a more immersive metaverse. © 2025 IEEE.
Gaglio, G. F.; Vinanzi, S.; Cangelosi, A.; Chella, A.
Intention Reading Architecture for Virtual Agents Proceedings Article
In: O., Palinko; L., Bodenhagen; J.-J., Cabibihan; K., Fischer; S., Šabanović; K., Winkle; L., Behera; S.S., Ge; D., Chrysostomou; W., Jiang; H., He (Ed.): Lect. Notes Comput. Sci., pp. 488–497, Springer Science and Business Media Deutschland GmbH, 2025, ISBN: 03029743 (ISSN); 978-981963521-4 (ISBN).
Abstract | Links | BibTeX | Tags: Chatbots, Cognitive Architecture, Cognitive Architectures, Computer simulation languages, Intelligent virtual agents, Intention Reading, Intention readings, Language Model, Large language model, Metaverse, Metaverses, Physical robots, Video-games, Virtual agent, Virtual assistants, Virtual contexts, Virtual environments, Virtual machine
@inproceedings{gaglio_intention_2025,
title = {Intention Reading Architecture for Virtual Agents},
author = {G. F. Gaglio and S. Vinanzi and A. Cangelosi and A. Chella},
editor = {Palinko O. and Bodenhagen L. and Cabibihan J.-J. and Fischer K. and Šabanović S. and Winkle K. and Behera L. and Ge S.S. and Chrysostomou D. and Jiang W. and He H.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105002042645&doi=10.1007%2f978-981-96-3522-1_41&partnerID=40&md5=70ccc7039785bb4ca4d45752f1d3587f},
doi = {10.1007/978-981-96-3522-1_41},
isbn = {03029743 (ISSN); 978-981963521-4 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Lect. Notes Comput. Sci.},
volume = {15561 LNAI},
pages = {488–497},
publisher = {Springer Science and Business Media Deutschland GmbH},
abstract = {This work presents the development of a virtual agent designed specifically for use in the Metaverse, video games, and other virtual environments, capable of performing intention reading on a human-controlled avatar through a cognitive architecture that endows it with contextual awareness. The paper explores the adaptation of a cognitive architecture, originally developed for physical robots, to a fully virtual context, where it is integrated with a Large Language Model to create highly communicative virtual assistants. Although this work primarily focuses on virtual applications, integrating cognitive architectures with LLMs marks a significant step toward creating collaborative artificial agents capable of providing meaningful support by deeply understanding context and user intentions in digital environments. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.},
keywords = {Chatbots, Cognitive Architecture, Cognitive Architectures, Computer simulation languages, Intelligent virtual agents, Intention Reading, Intention readings, Language Model, Large language model, Metaverse, Metaverses, Physical robots, Video-games, Virtual agent, Virtual assistants, Virtual contexts, Virtual environments, Virtual machine},
pubstate = {published},
tppubtype = {inproceedings}
}
This work presents the development of a virtual agent designed specifically for use in the Metaverse, video games, and other virtual environments, capable of performing intention reading on a human-controlled avatar through a cognitive architecture that endows it with contextual awareness. The paper explores the adaptation of a cognitive architecture, originally developed for physical robots, to a fully virtual context, where it is integrated with a Large Language Model to create highly communicative virtual assistants. Although this work primarily focuses on virtual applications, integrating cognitive architectures with LLMs marks a significant step toward creating collaborative artificial agents capable of providing meaningful support by deeply understanding context and user intentions in digital environments. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.
Kurai, R.; Hiraki, T.; Hiroi, Y.; Hirao, Y.; Perusquia-Hernandez, M.; Uchiyama, H.; Kiyokawa, K.
MagicItem: Dynamic Behavior Design of Virtual Objects With Large Language Models in a Commercial Metaverse Platform Journal Article
In: IEEE Access, vol. 13, pp. 19132–19143, 2025, ISSN: 21693536 (ISSN).
Abstract | Links | BibTeX | Tags: Behavior design, Code programming, Computer simulation languages, Dynamic behaviors, Language Model, Large-language model, Low-code programming, Metaverse platform, Metaverses, Virtual addresses, Virtual environments, Virtual objects, Virtual Reality, Virtual-reality environment
@article{kurai_magicitem_2025,
title = {MagicItem: Dynamic Behavior Design of Virtual Objects With Large Language Models in a Commercial Metaverse Platform},
author = {R. Kurai and T. Hiraki and Y. Hiroi and Y. Hirao and M. Perusquia-Hernandez and H. Uchiyama and K. Kiyokawa},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85216011970&doi=10.1109%2fACCESS.2025.3530439&partnerID=40&md5=7a33b9618af8b4ab79b43fb3bd4317cf},
doi = {10.1109/ACCESS.2025.3530439},
issn = {21693536 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {IEEE Access},
volume = {13},
pages = {19132–19143},
abstract = {To create rich experiences in virtual reality (VR) environments, it is essential to define the behavior of virtual objects through programming. However, programming in 3D spaces requires a wide range of background knowledge and programming skills. Although Large Language Models (LLMs) have provided programming support, they are still primarily aimed at programmers. In metaverse platforms, where many users inhabit VR spaces, most users are unfamiliar with programming, making it difficult for them to modify the behavior of objects in the VR environment easily. Existing LLM-based script generation methods for VR spaces require multiple lengthy iterations to implement the desired behaviors and are difficult to integrate into the operation of metaverse platforms. To address this issue, we propose a tool that generates behaviors for objects in VR spaces from natural language within Cluster, a metaverse platform with a large user base. By integrating LLMs with the Cluster Script provided by this platform, we enable users with limited programming experience to define object behaviors within the platform freely. We have also integrated our tool into a commercial metaverse platform and are conducting online experiments with 63 general users of the platform. The experiments show that even users with no programming background can successfully generate behaviors for objects in VR spaces, resulting in a highly satisfying system. Our research contributes to democratizing VR content creation by enabling non-programmers to design dynamic behaviors for virtual objects in metaverse platforms. © 2013 IEEE.},
keywords = {Behavior design, Code programming, Computer simulation languages, Dynamic behaviors, Language Model, Large-language model, Low-code programming, Metaverse platform, Metaverses, Virtual addresses, Virtual environments, Virtual objects, Virtual Reality, Virtual-reality environment},
pubstate = {published},
tppubtype = {article}
}
To create rich experiences in virtual reality (VR) environments, it is essential to define the behavior of virtual objects through programming. However, programming in 3D spaces requires a wide range of background knowledge and programming skills. Although Large Language Models (LLMs) have provided programming support, they are still primarily aimed at programmers. In metaverse platforms, where many users inhabit VR spaces, most users are unfamiliar with programming, making it difficult for them to modify the behavior of objects in the VR environment easily. Existing LLM-based script generation methods for VR spaces require multiple lengthy iterations to implement the desired behaviors and are difficult to integrate into the operation of metaverse platforms. To address this issue, we propose a tool that generates behaviors for objects in VR spaces from natural language within Cluster, a metaverse platform with a large user base. By integrating LLMs with the Cluster Script provided by this platform, we enable users with limited programming experience to define object behaviors within the platform freely. We have also integrated our tool into a commercial metaverse platform and are conducting online experiments with 63 general users of the platform. The experiments show that even users with no programming background can successfully generate behaviors for objects in VR spaces, resulting in a highly satisfying system. Our research contributes to democratizing VR content creation by enabling non-programmers to design dynamic behaviors for virtual objects in metaverse platforms. © 2013 IEEE.
Stroinski, M.; Kwarciak, K.; Kowalewski, M.; Hemmerling, D.; Frier, W.; Georgiou, O.
Text-to-Haptics: Enhancing Multisensory Storytelling through Emotionally Congruent Midair Haptics Journal Article
In: Advanced Intelligent Systems, vol. 7, no. 4, 2025, ISSN: 26404567 (ISSN).
Abstract | Links | BibTeX | Tags: Audiovisual, Augmented Reality, Extended reality, Haptic interfaces, Haptics, Haptics interfaces, HMI, hybrid AI, Hybrid artificial intelligences, Metaverses, Mixed reality, Multisensory, Natural Language Processing, perception, Sentiment Analysis, Sound speech, Special issue and section, Speech enhancement, Virtual environments, Visual elements
@article{stroinski_text–haptics_2025,
title = {Text-to-Haptics: Enhancing Multisensory Storytelling through Emotionally Congruent Midair Haptics},
author = {M. Stroinski and K. Kwarciak and M. Kowalewski and D. Hemmerling and W. Frier and O. Georgiou},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105002269591&doi=10.1002%2faisy.202400758&partnerID=40&md5=a4c8ce7a01c9bc90d9805a81d34df982},
doi = {10.1002/aisy.202400758},
issn = {26404567 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {Advanced Intelligent Systems},
volume = {7},
number = {4},
abstract = {In multisensory storytelling, the integration of touch, sound, speech, and visual elements plays a crucial role in enhancing the narrative immersion and audience engagement. In light of this, this article presents a scalable and intelligent hybrid artificial intelligence (AI) method that uses emotional text analysis for deciding when and what midair haptics to display alongside audiovisual content generated by latent stable diffusion methods. Then, a user study involving 40 participants is described, the results of which suggest that the proposed approach enhances the audience level of engagement as they experience a short AI-generated multisensory (audio–visual–haptic) story. © 2024 The Author(s). Advanced Intelligent Systems published by Wiley-VCH GmbH.},
keywords = {Audiovisual, Augmented Reality, Extended reality, Haptic interfaces, Haptics, Haptics interfaces, HMI, hybrid AI, Hybrid artificial intelligences, Metaverses, Mixed reality, Multisensory, Natural Language Processing, perception, Sentiment Analysis, Sound speech, Special issue and section, Speech enhancement, Virtual environments, Visual elements},
pubstate = {published},
tppubtype = {article}
}
In multisensory storytelling, the integration of touch, sound, speech, and visual elements plays a crucial role in enhancing the narrative immersion and audience engagement. In light of this, this article presents a scalable and intelligent hybrid artificial intelligence (AI) method that uses emotional text analysis for deciding when and what midair haptics to display alongside audiovisual content generated by latent stable diffusion methods. Then, a user study involving 40 participants is described, the results of which suggest that the proposed approach enhances the audience level of engagement as they experience a short AI-generated multisensory (audio–visual–haptic) story. © 2024 The Author(s). Advanced Intelligent Systems published by Wiley-VCH GmbH.
2024
Chen, Z.; Xie, A.; Wang, R.
An Edu-Metaverse Service Platform and its Experiments on Physical Education Class in PKU Proceedings Article
In: Proc. - IEEE Smart World Congr., SWC - IEEE Ubiquitous Intell. Comput., Auton. Trusted Comput., Digit. Twin, Metaverse, Priv. Comput. Data Secur., Scalable Comput. Commun., pp. 40–46, Institute of Electrical and Electronics Engineers Inc., 2024, ISBN: 979-833152086-1 (ISBN).
Abstract | Links | BibTeX | Tags: Digital era, Digital transformation, Edu-metaverse service, Edu-Metaverse services, Education digital transformation, Higher-quality education, Metaverse, Metaverses, PE class, Physical education, Service platforms
@inproceedings{chen_edu-metaverse_2024,
title = {An Edu-Metaverse Service Platform and its Experiments on Physical Education Class in PKU},
author = {Z. Chen and A. Xie and R. Wang},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105002244347&doi=10.1109%2fSWC62898.2024.00036&partnerID=40&md5=6b0d4efaa0716068fc76518654381c8f},
doi = {10.1109/SWC62898.2024.00036},
isbn = {979-833152086-1 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Proc. - IEEE Smart World Congr., SWC - IEEE Ubiquitous Intell. Comput., Auton. Trusted Comput., Digit. Twin, Metaverse, Priv. Comput. Data Secur., Scalable Comput. Commun.},
pages = {40–46},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {In the digital era, it is the only way for high-quality education to integrate the new generation of information technology with traditional education, promote the Digital transformation of education, and build a networked, digital, personalized, lifelong education system. This paper firstly introduces the concepts with Metaverse and its involvement toward Web3.0 as well as Generative AI. Then come with an Edu-Metaverse Service and its impact on the future education. Edu-Metaverse service would effectively support the 'student-centered' education model, and drive profound changes in education scenes, education content, roles and responsibilities, teaching evaluation and other aspects. Finally, a pilot practice combine Edu-Metaverse Service platform with Physical Education in Peking University was introduced and some issues were discussed. © 2024 IEEE.},
keywords = {Digital era, Digital transformation, Edu-metaverse service, Edu-Metaverse services, Education digital transformation, Higher-quality education, Metaverse, Metaverses, PE class, Physical education, Service platforms},
pubstate = {published},
tppubtype = {inproceedings}
}
In the digital era, it is the only way for high-quality education to integrate the new generation of information technology with traditional education, promote the Digital transformation of education, and build a networked, digital, personalized, lifelong education system. This paper firstly introduces the concepts with Metaverse and its involvement toward Web3.0 as well as Generative AI. Then come with an Edu-Metaverse Service and its impact on the future education. Edu-Metaverse service would effectively support the 'student-centered' education model, and drive profound changes in education scenes, education content, roles and responsibilities, teaching evaluation and other aspects. Finally, a pilot practice combine Edu-Metaverse Service platform with Physical Education in Peking University was introduced and some issues were discussed. © 2024 IEEE.
Lai, B.; He, J.; Kang, J.; Li, G.; Xu, M.; Zhang, T.; Xie, S.
On-demand Quantization for Green Federated Generative Diffusion in Mobile Edge Networks Proceedings Article
In: M., Valenti; D., Reed; M., Torres (Ed.): IEEE Int Conf Commun, pp. 2883–2888, Institute of Electrical and Electronics Engineers Inc., 2024, ISBN: 15503607 (ISSN); 978-172819054-9 (ISBN).
Abstract | Links | BibTeX | Tags: EDGE Networks, Energy Efficient, Federated Diffusion, Federated learning, Generative adversarial networks, Generative AI, Generative Diffusion, Intelligence models, Metaverses, On demands, On-demand Quantization, Quantisation
@inproceedings{lai_-demand_2024,
title = {On-demand Quantization for Green Federated Generative Diffusion in Mobile Edge Networks},
author = {B. Lai and J. He and J. Kang and G. Li and M. Xu and T. Zhang and S. Xie},
editor = {Valenti M. and Reed D. and Torres M.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85202836582&doi=10.1109%2fICC51166.2024.10622695&partnerID=40&md5=8318ef451bb946470eb34ef50d09b5a1},
doi = {10.1109/ICC51166.2024.10622695},
isbn = {15503607 (ISSN); 978-172819054-9 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {IEEE Int Conf Commun},
pages = {2883–2888},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {Generative Artificial Intelligence (GAI) shows remarkable productivity and creativity in Mobile Edge Networks, such as the metaverse and the Industrial Internet of Things. Federated learning is a promising technique for effectively training GAI models in mobile edge networks due to its data distribution. However, there is a notable issue with communication consumption when training large GAI models like generative diffusion models in mobile edge networks. Additionally, the substantial energy consumption associated with training diffusion-based models, along with the limited resources of edge devices and complexities of network environments, pose challenges for improving the training efficiency of GAI models. To address this challenge, we propose an on-demand quantized energy-efficient federated diffusion approach for mobile edge networks. Specifically, we first design a dynamic quantized federated diffusion training scheme considering various demands from the edge devices. Then, we study an energy efficiency problem based on specific quantization requirements. Numerical results show that our proposed method significantly reduces system energy consumption and transmitted model size compared to both baseline federated diffusion and fixed quantized federated diffusion methods while effectively maintaining reasonable quality and diversity of generated data. © 2024 IEEE.},
keywords = {EDGE Networks, Energy Efficient, Federated Diffusion, Federated learning, Generative adversarial networks, Generative AI, Generative Diffusion, Intelligence models, Metaverses, On demands, On-demand Quantization, Quantisation},
pubstate = {published},
tppubtype = {inproceedings}
}
Generative Artificial Intelligence (GAI) shows remarkable productivity and creativity in Mobile Edge Networks, such as the metaverse and the Industrial Internet of Things. Federated learning is a promising technique for effectively training GAI models in mobile edge networks due to its data distribution. However, there is a notable issue with communication consumption when training large GAI models like generative diffusion models in mobile edge networks. Additionally, the substantial energy consumption associated with training diffusion-based models, along with the limited resources of edge devices and complexities of network environments, pose challenges for improving the training efficiency of GAI models. To address this challenge, we propose an on-demand quantized energy-efficient federated diffusion approach for mobile edge networks. Specifically, we first design a dynamic quantized federated diffusion training scheme considering various demands from the edge devices. Then, we study an energy efficiency problem based on specific quantization requirements. Numerical results show that our proposed method significantly reduces system energy consumption and transmitted model size compared to both baseline federated diffusion and fixed quantized federated diffusion methods while effectively maintaining reasonable quality and diversity of generated data. © 2024 IEEE.
Liew, Z. Q.; Xu, M.; Lim, W. Y. Bryan; Niyato, D.; Kim, D. I.
AI-Generated Bidding for Immersive AIGC Services in Mobile Edge-Empowered Metaverse Proceedings Article
In: Int. Conf. Inf. Networking, pp. 305–309, IEEE Computer Society, 2024, ISBN: 19767684 (ISSN); 979-835033094-6 (ISBN).
Abstract | Links | BibTeX | Tags: Artificial intelligence generated bid, Artificial intelligence generated content, Bidding mechanism, Bidding models, Budget constraint, Budget control, Budget-constraint bidding, Constrained optimization, Content services, Immersive, Learning systems, Metaverses, Mobile edge computing, Reinforcement Learning, Semantics, Virtual tour
@inproceedings{liew_ai-generated_2024,
title = {AI-Generated Bidding for Immersive AIGC Services in Mobile Edge-Empowered Metaverse},
author = {Z. Q. Liew and M. Xu and W. Y. Bryan Lim and D. Niyato and D. I. Kim},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85198324990&doi=10.1109%2fICOIN59985.2024.10572159&partnerID=40&md5=271f5c45e8e95f01b42acaee89599bd5},
doi = {10.1109/ICOIN59985.2024.10572159},
isbn = {19767684 (ISSN); 979-835033094-6 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Int. Conf. Inf. Networking},
pages = {305–309},
publisher = {IEEE Computer Society},
abstract = {Recent advancements in Artificial Intelligence Generated Content (AIGC) provide personalized and immersive content generation services for applications such as interactive advertisements, virtual tours, and metaverse. With the use of mobile edge computing (MEC), buyers can bid for the AIGC service to enhance their user experience in real-time. However, designing strategies to optimize the quality of the services won can be challenging for budget-constrained buyers. The performance of classical bidding mechanisms is limited by the fixed rules in the strategies. To this end, we propose AI-generated bidding (AIGB) to optimize the bidding strategies for AIGC. AIGB model uses reinforcement learning model to generate bids for the services by learning from the historical data and environment states such as remaining budget, budget consumption rate, and quality of the won services. To obtain quality AIGC service, we propose a semantic aware reward function for the AIGB model. The proposed model is tested with a real-world dataset and experiments show that our model outperforms the classical bidding mechanism in terms of the number of services won and the similarity score. © 2024 IEEE.},
keywords = {Artificial intelligence generated bid, Artificial intelligence generated content, Bidding mechanism, Bidding models, Budget constraint, Budget control, Budget-constraint bidding, Constrained optimization, Content services, Immersive, Learning systems, Metaverses, Mobile edge computing, Reinforcement Learning, Semantics, Virtual tour},
pubstate = {published},
tppubtype = {inproceedings}
}
Recent advancements in Artificial Intelligence Generated Content (AIGC) provide personalized and immersive content generation services for applications such as interactive advertisements, virtual tours, and metaverse. With the use of mobile edge computing (MEC), buyers can bid for the AIGC service to enhance their user experience in real-time. However, designing strategies to optimize the quality of the services won can be challenging for budget-constrained buyers. The performance of classical bidding mechanisms is limited by the fixed rules in the strategies. To this end, we propose AI-generated bidding (AIGB) to optimize the bidding strategies for AIGC. AIGB model uses reinforcement learning model to generate bids for the services by learning from the historical data and environment states such as remaining budget, budget consumption rate, and quality of the won services. To obtain quality AIGC service, we propose a semantic aware reward function for the AIGB model. The proposed model is tested with a real-world dataset and experiments show that our model outperforms the classical bidding mechanism in terms of the number of services won and the similarity score. © 2024 IEEE.
Salloum, A.; Alfaisal, R.; Salloum, S. A.
Revolutionizing Medical Education: Empowering Learning with ChatGPT Book Section
In: Studies in Big Data, vol. 144, pp. 79–90, Springer Science and Business Media Deutschland GmbH, 2024, ISBN: 21976503 (ISSN).
Abstract | Links | BibTeX | Tags: Abstracting, AI integration, ChatGPT, Education, Human like, Interactivity, Language Model, Learning platform, Learning platforms, Medical education, Metaverse, Metaverses, Paradigm shifts, Personalizations, Technological advancement
@incollection{salloum_revolutionizing_2024,
title = {Revolutionizing Medical Education: Empowering Learning with ChatGPT},
author = {A. Salloum and R. Alfaisal and S. A. Salloum},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85191302844&doi=10.1007%2f978-3-031-52280-2_6&partnerID=40&md5=a5325b8e43460906174a3c7a2c383e1a},
doi = {10.1007/978-3-031-52280-2_6},
isbn = {21976503 (ISSN)},
year = {2024},
date = {2024-01-01},
booktitle = {Studies in Big Data},
volume = {144},
pages = {79–90},
publisher = {Springer Science and Business Media Deutschland GmbH},
abstract = {The landscape of medical education is undergoing a paradigm shift driven by technological advancements. This abstract explores the potential of ChatGPT, an advanced AI language model developed by OpenAI, in revolutionizing medical education. ChatGPT’s capacity to understand and generate human-like text opens doors to interactive, personalized, and adaptive learning experiences that address the evolving demands of medical training. Medical education traditionally relies on didactic approaches that often lack interactivity and personalization. ChatGPT addresses this limitation by introducing a conversational AI-driven dimension to medical learning. Learners can engage with ChatGPT in natural language, seeking explanations, asking questions, and clarifying doubts. This adaptive interactivity mirrors the dynamic nature of medical practice and fosters critical thinking skills essential for medical professionals. Furthermore, ChatGPT augments educators’ roles by assisting in content creation, formative assessments, and immediate feedback delivery. This empowers educators to focus on higher-order facilitation and mentorship, enriching the learning journey. However, responsible integration of ChatGPT into medical education demands careful curation of accurate medical content and validation against trusted sources. Ethical considerations related to AI-generated content and potential biases also warrant attention. This abstract underscores the transformative potential of ChatGPT in reshaping medical education. By creating an environment of engagement, adaptability, and personalization, ChatGPT paves the way for a dynamic and empowered medical learning ecosystem that aligns with the demands of modern healthcare. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.},
keywords = {Abstracting, AI integration, ChatGPT, Education, Human like, Interactivity, Language Model, Learning platform, Learning platforms, Medical education, Metaverse, Metaverses, Paradigm shifts, Personalizations, Technological advancement},
pubstate = {published},
tppubtype = {incollection}
}
The landscape of medical education is undergoing a paradigm shift driven by technological advancements. This abstract explores the potential of ChatGPT, an advanced AI language model developed by OpenAI, in revolutionizing medical education. ChatGPT’s capacity to understand and generate human-like text opens doors to interactive, personalized, and adaptive learning experiences that address the evolving demands of medical training. Medical education traditionally relies on didactic approaches that often lack interactivity and personalization. ChatGPT addresses this limitation by introducing a conversational AI-driven dimension to medical learning. Learners can engage with ChatGPT in natural language, seeking explanations, asking questions, and clarifying doubts. This adaptive interactivity mirrors the dynamic nature of medical practice and fosters critical thinking skills essential for medical professionals. Furthermore, ChatGPT augments educators’ roles by assisting in content creation, formative assessments, and immediate feedback delivery. This empowers educators to focus on higher-order facilitation and mentorship, enriching the learning journey. However, responsible integration of ChatGPT into medical education demands careful curation of accurate medical content and validation against trusted sources. Ethical considerations related to AI-generated content and potential biases also warrant attention. This abstract underscores the transformative potential of ChatGPT in reshaping medical education. By creating an environment of engagement, adaptability, and personalization, ChatGPT paves the way for a dynamic and empowered medical learning ecosystem that aligns with the demands of modern healthcare. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
Ding, P.; Liu, J.; Sun, M.; Li, L.; Liu, H.
Enhancing Computational Processing Performance for Generative AI Large Models with Autonomous Decision-Making in Metaverse Applications Proceedings Article
In: Proc. - IEEE Int. Conf. Metaverse Comput., Netw., Appl., MetaCom, pp. 253–258, Institute of Electrical and Electronics Engineers Inc., 2024, ISBN: 979-833151599-7 (ISBN).
Abstract | Links | BibTeX | Tags: Adversarial machine learning, AGI (Artificial General Intelligence), Artificial general intelligence, Artificial general intelligences, Autonomous decision, Autonomous Decision-Making, Data assimilation, Data integration, Decisions makings, Digital Twin Technology, Emotion Recognition, Generative adversarial networks, Generative AI large model, Generative AI Large Models, Large models, Metaverse, Metaverses, Model Acceleration, Model Compression, Multi agent systems, Multi-agent systems, Multi-modal data, Multi-Modal Data Integration, Multiagent systems (MASs), Reinforcement Learning, Reinforcement learnings, Spatio-temporal data
@inproceedings{ding_enhancing_2024,
title = {Enhancing Computational Processing Performance for Generative AI Large Models with Autonomous Decision-Making in Metaverse Applications},
author = {P. Ding and J. Liu and M. Sun and L. Li and H. Liu},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85211489063&doi=10.1109%2fMetaCom62920.2024.00048&partnerID=40&md5=ae085a7d90b12c9090f5bf7a274bc7ce},
doi = {10.1109/MetaCom62920.2024.00048},
isbn = {979-833151599-7 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Proc. - IEEE Int. Conf. Metaverse Comput., Netw., Appl., MetaCom},
pages = {253–258},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {We explore how to enhance the computational processing performance for generative AI large models with autonomous decision-making in metaverse applications. We first introduce the relationship between AI large models and the Metaverse. We elaborate on the application scenarios of generative AI large models in Metaverse, including real-time weather simulation, embodied intelligence of agents, dynamic environment interaction, and user emotion recognition. We then propose the method of Multi-Dimensional Optimization Generation Framework (MDOGF) to improve computational processing performance. The experiment results show great improvement in computational processing performance. © 2024 IEEE.},
keywords = {Adversarial machine learning, AGI (Artificial General Intelligence), Artificial general intelligence, Artificial general intelligences, Autonomous decision, Autonomous Decision-Making, Data assimilation, Data integration, Decisions makings, Digital Twin Technology, Emotion Recognition, Generative adversarial networks, Generative AI large model, Generative AI Large Models, Large models, Metaverse, Metaverses, Model Acceleration, Model Compression, Multi agent systems, Multi-agent systems, Multi-modal data, Multi-Modal Data Integration, Multiagent systems (MASs), Reinforcement Learning, Reinforcement learnings, Spatio-temporal data},
pubstate = {published},
tppubtype = {inproceedings}
}
We explore how to enhance the computational processing performance for generative AI large models with autonomous decision-making in metaverse applications. We first introduce the relationship between AI large models and the Metaverse. We elaborate on the application scenarios of generative AI large models in Metaverse, including real-time weather simulation, embodied intelligence of agents, dynamic environment interaction, and user emotion recognition. We then propose the method of Multi-Dimensional Optimization Generation Framework (MDOGF) to improve computational processing performance. The experiment results show great improvement in computational processing performance. © 2024 IEEE.
Pester, A.; Tammaa, A.; Gütl, C.; Steinmaurer, A.; El-Seoud, S. A.
Conversational Agents, Virtual Worlds, and Beyond: A Review of Large Language Models Enabling Immersive Learning Proceedings Article
In: IEEE Global Eng. Edu. Conf., EDUCON, IEEE Computer Society, 2024, ISBN: 21659559 (ISSN); 979-835039402-3 (ISBN).
Abstract | Links | BibTeX | Tags: Computational Linguistics, Computer aided instruction, Conversational Agents, Education, Immersive learning, Language Model, Large language model, Learning systems, Literature reviews, LLM, Metaverse, Metaverses, Natural language processing systems, Pedagogy, Survey literature review, Virtual Reality, Virtual worlds
@inproceedings{pester_conversational_2024,
title = {Conversational Agents, Virtual Worlds, and Beyond: A Review of Large Language Models Enabling Immersive Learning},
author = {A. Pester and A. Tammaa and C. Gütl and A. Steinmaurer and S. A. El-Seoud},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85199068668&doi=10.1109%2fEDUCON60312.2024.10578895&partnerID=40&md5=1b904fd8a5e06d7ced42a328c028bbb7},
doi = {10.1109/EDUCON60312.2024.10578895},
isbn = {21659559 (ISSN); 979-835039402-3 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {IEEE Global Eng. Edu. Conf., EDUCON},
publisher = {IEEE Computer Society},
abstract = {Large Language Models represent a significant breakthrough in Natural Language Processing research and opened a wide range of application domains. This paper demonstrates the successful integration of Large Language Models into immersive learning environments. The review highlights how this emerging technology aligns with pedagogical principles, enhancing the effectiveness of current educational systems. It also reflects recent advancements in integrating Large Language Models, including fine-tuning, hallucination reduction, fact-checking, and human evaluation of generated results. © 2024 IEEE.},
keywords = {Computational Linguistics, Computer aided instruction, Conversational Agents, Education, Immersive learning, Language Model, Large language model, Learning systems, Literature reviews, LLM, Metaverse, Metaverses, Natural language processing systems, Pedagogy, Survey literature review, Virtual Reality, Virtual worlds},
pubstate = {published},
tppubtype = {inproceedings}
}
Large Language Models represent a significant breakthrough in Natural Language Processing research and opened a wide range of application domains. This paper demonstrates the successful integration of Large Language Models into immersive learning environments. The review highlights how this emerging technology aligns with pedagogical principles, enhancing the effectiveness of current educational systems. It also reflects recent advancements in integrating Large Language Models, including fine-tuning, hallucination reduction, fact-checking, and human evaluation of generated results. © 2024 IEEE.