AHCI RESEARCH GROUP
Publications
Papers published in international journals,
proceedings of conferences, workshops and books.
OUR RESEARCH
Scientific Publications
How to
Here you can find the complete list of our publications.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
2025
Zhang, Z.; Jiang, Y.; Wei, X.; Chen, M.; Dong, H.; Yu, S.
Generative-AI for XR Content Transmission in the Metaverse: Potential Approaches, Challenges, and a Generation-Driven Transmission Framework Journal Article
In: IEEE Network, 2025, ISSN: 08908044 (ISSN).
Abstract | Links | BibTeX | Tags: 'current, Cloud servers, Collaboration architecture, Content transmission, Decision making, Deep learning, Deep reinforcement learning, Edge server, Generative adversarial networks, Intelligence models, Large volumes, Metaverses, Network bottlenecks, Reinforcement Learning, Through current
@article{zhang_generative-ai_2025,
title = {Generative-AI for XR Content Transmission in the Metaverse: Potential Approaches, Challenges, and a Generation-Driven Transmission Framework},
author = {Z. Zhang and Y. Jiang and X. Wei and M. Chen and H. Dong and S. Yu},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-86000661262&doi=10.1109%2fMNET.2025.3547385&partnerID=40&md5=1e00d40542ec58ef1489934abb2a990c},
doi = {10.1109/MNET.2025.3547385},
issn = {08908044 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {IEEE Network},
abstract = {How to efficiently transmit large volumes of Extended Reality (XR) content through current networks has been a major bottleneck in realizing the Metaverse. The recently emerging Generative Artificial Intelligence (GAI) has already revolutionized various technological fields and provides promising solutions to this challenge. In this article, we first demonstrate current networks' bottlenecks for supporting XR content transmission in the Metaverse. Then, we explore the potential approaches and challenges of utilizing GAI to overcome these bottlenecks. To address these challenges, we propose a GAI-based XR content transmission framework which leverages a cloud-edge collaboration architecture. The cloud servers are responsible for storing and rendering the original XR content, while edge servers utilize GAI models to generate essential parts of XR content (e.g., subsequent frames, selected objects, etc.) when network resources are insufficient to transmit them. A Deep Reinforcement Learning (DRL)-based decision module is proposed to solve the decision-making problems. Our case study demonstrates that the proposed GAI-based transmission framework achieves a 2.8-fold increase in normal frame ratio (percentage of frames that meet the quality and latency requirements for XR content transmission) over baseline approaches, underscoring the potential of GAI models to facilitate XR content transmission in the Metaverse. © 2025 IEEE.},
keywords = {'current, Cloud servers, Collaboration architecture, Content transmission, Decision making, Deep learning, Deep reinforcement learning, Edge server, Generative adversarial networks, Intelligence models, Large volumes, Metaverses, Network bottlenecks, Reinforcement Learning, Through current},
pubstate = {published},
tppubtype = {article}
}
How to efficiently transmit large volumes of Extended Reality (XR) content through current networks has been a major bottleneck in realizing the Metaverse. The recently emerging Generative Artificial Intelligence (GAI) has already revolutionized various technological fields and provides promising solutions to this challenge. In this article, we first demonstrate current networks' bottlenecks for supporting XR content transmission in the Metaverse. Then, we explore the potential approaches and challenges of utilizing GAI to overcome these bottlenecks. To address these challenges, we propose a GAI-based XR content transmission framework which leverages a cloud-edge collaboration architecture. The cloud servers are responsible for storing and rendering the original XR content, while edge servers utilize GAI models to generate essential parts of XR content (e.g., subsequent frames, selected objects, etc.) when network resources are insufficient to transmit them. A Deep Reinforcement Learning (DRL)-based decision module is proposed to solve the decision-making problems. Our case study demonstrates that the proposed GAI-based transmission framework achieves a 2.8-fold increase in normal frame ratio (percentage of frames that meet the quality and latency requirements for XR content transmission) over baseline approaches, underscoring the potential of GAI models to facilitate XR content transmission in the Metaverse. © 2025 IEEE.
Kai, W. -H.; Xing, K. -X.
Video-driven musical composition using large language model with memory-augmented state space Journal Article
In: Visual Computer, vol. 41, no. 5, pp. 3345–3357, 2025, ISSN: 01782789 (ISSN).
Abstract | Links | BibTeX | Tags: 'current, Associative storage, Augmented Reality, Augmented state space, Computer simulation languages, Computer system recovery, Distributed computer systems, HTTP, Language Model, Large language model, Long-term video-to-music generation, Mamba, Memory architecture, Memory-augmented, Modeling languages, Music, Musical composition, Natural language processing systems, Object oriented programming, Performance, Problem oriented languages, State space, State-space
@article{kai_video-driven_2025,
title = {Video-driven musical composition using large language model with memory-augmented state space},
author = {W. -H. Kai and K. -X. Xing},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105001073242&doi=10.1007%2fs00371-024-03606-w&partnerID=40&md5=7ea24f13614a9a24caf418c37a10bd8c},
doi = {10.1007/s00371-024-03606-w},
issn = {01782789 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {Visual Computer},
volume = {41},
number = {5},
pages = {3345–3357},
abstract = {The current landscape of research leveraging large language models (LLMs) is experiencing a surge. Many works harness the powerful reasoning capabilities of these models to comprehend various modalities, such as text, speech, images, videos, etc. However, the research work on LLms for music inspiration is still in its infancy. To fill the gap in this field and break through the dilemma that LLMs can only understand short videos with limited frames, we propose a large language model with state space for long-term video-to-music generation. To capture long-range dependency and maintaining high performance, while further decrease the computing cost, our overall network includes the Enhanced Video Mamba, which incorporates continuous moving window partitioning and local feature augmentation, and a long-term memory bank that captures and aggregates historical video information to mitigate information loss in long sequences. This framework achieves both subquadratic-time computation and near-linear memory complexity, enabling effective long-term video-to-music generation. We conduct a thorough evaluation of our proposed framework. The experimental results demonstrate that our model achieves or surpasses the performance of the current state-of-the-art models. Our code released on https://github.com/kai211233/S2L2-V2M. © The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2024.},
keywords = {'current, Associative storage, Augmented Reality, Augmented state space, Computer simulation languages, Computer system recovery, Distributed computer systems, HTTP, Language Model, Large language model, Long-term video-to-music generation, Mamba, Memory architecture, Memory-augmented, Modeling languages, Music, Musical composition, Natural language processing systems, Object oriented programming, Performance, Problem oriented languages, State space, State-space},
pubstate = {published},
tppubtype = {article}
}
The current landscape of research leveraging large language models (LLMs) is experiencing a surge. Many works harness the powerful reasoning capabilities of these models to comprehend various modalities, such as text, speech, images, videos, etc. However, the research work on LLms for music inspiration is still in its infancy. To fill the gap in this field and break through the dilemma that LLMs can only understand short videos with limited frames, we propose a large language model with state space for long-term video-to-music generation. To capture long-range dependency and maintaining high performance, while further decrease the computing cost, our overall network includes the Enhanced Video Mamba, which incorporates continuous moving window partitioning and local feature augmentation, and a long-term memory bank that captures and aggregates historical video information to mitigate information loss in long sequences. This framework achieves both subquadratic-time computation and near-linear memory complexity, enabling effective long-term video-to-music generation. We conduct a thorough evaluation of our proposed framework. The experimental results demonstrate that our model achieves or surpasses the performance of the current state-of-the-art models. Our code released on https://github.com/kai211233/S2L2-V2M. © The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2024.
Shi, J.; Jain, R.; Chi, S.; Doh, H.; Chi, H. -G.; Quinn, A. J.; Ramani, K.
CARING-AI: Towards Authoring Context-aware Augmented Reality INstruction through Generative Artificial Intelligence Proceedings Article
In: Conf Hum Fact Comput Syst Proc, Association for Computing Machinery, 2025, ISBN: 979-840071394-1 (ISBN).
Abstract | Links | BibTeX | Tags: 'current, Application scenario, AR application, Augmented Reality, Context-Aware, Contextual information, Generative adversarial networks, generative artificial intelligence, Humanoid avatars, In-situ learning, Learning experiences, Power
@inproceedings{shi_caring-ai_2025,
title = {CARING-AI: Towards Authoring Context-aware Augmented Reality INstruction through Generative Artificial Intelligence},
author = {J. Shi and R. Jain and S. Chi and H. Doh and H. -G. Chi and A. J. Quinn and K. Ramani},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105005725461&doi=10.1145%2f3706598.3713348&partnerID=40&md5=e88afd8426e020155599ef3b2a044774},
doi = {10.1145/3706598.3713348},
isbn = {979-840071394-1 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Conf Hum Fact Comput Syst Proc},
publisher = {Association for Computing Machinery},
abstract = {Context-aware AR instruction enables adaptive and in-situ learning experiences. However, hardware limitations and expertise requirements constrain the creation of such instructions. With recent developments in Generative Artificial Intelligence (Gen-AI), current research tries to tackle these constraints by deploying AI-generated content (AIGC) in AR applications. However, our preliminary study with six AR practitioners revealed that the current AIGC lacks contextual information to adapt to varying application scenarios and is therefore limited in authoring. To utilize the strong generative power of GenAI to ease the authoring of AR instruction while capturing the context, we developed CARING-AI, an AR system to author context-aware humanoid-avatar-based instructions with GenAI. By navigating in the environment, users naturally provide contextual information to generate humanoid-avatar animation as AR instructions that blend in the context spatially and temporally. We showcased three application scenarios of CARING-AI: Asynchronous Instructions, Remote Instructions, and Ad Hoc Instructions based on a design space of AIGC in AR Instructions. With two user studies (N=12), we assessed the system usability of CARING-AI and demonstrated the easiness and effectiveness of authoring with Gen-AI. © 2025 Copyright held by the owner/author(s).},
keywords = {'current, Application scenario, AR application, Augmented Reality, Context-Aware, Contextual information, Generative adversarial networks, generative artificial intelligence, Humanoid avatars, In-situ learning, Learning experiences, Power},
pubstate = {published},
tppubtype = {inproceedings}
}
Context-aware AR instruction enables adaptive and in-situ learning experiences. However, hardware limitations and expertise requirements constrain the creation of such instructions. With recent developments in Generative Artificial Intelligence (Gen-AI), current research tries to tackle these constraints by deploying AI-generated content (AIGC) in AR applications. However, our preliminary study with six AR practitioners revealed that the current AIGC lacks contextual information to adapt to varying application scenarios and is therefore limited in authoring. To utilize the strong generative power of GenAI to ease the authoring of AR instruction while capturing the context, we developed CARING-AI, an AR system to author context-aware humanoid-avatar-based instructions with GenAI. By navigating in the environment, users naturally provide contextual information to generate humanoid-avatar animation as AR instructions that blend in the context spatially and temporally. We showcased three application scenarios of CARING-AI: Asynchronous Instructions, Remote Instructions, and Ad Hoc Instructions based on a design space of AIGC in AR Instructions. With two user studies (N=12), we assessed the system usability of CARING-AI and demonstrated the easiness and effectiveness of authoring with Gen-AI. © 2025 Copyright held by the owner/author(s).
Carcangiu, A.; Manca, M.; Mereu, J.; Santoro, C.; Simeoli, L.; Spano, L. D.
Conversational Rule Creation in XR: User’s Strategies in VR and AR Automation Proceedings Article
In: C., Santoro; A., Schmidt; M., Matera; A., Bellucci (Ed.): Lect. Notes Comput. Sci., pp. 59–79, Springer Science and Business Media Deutschland GmbH, 2025, ISBN: 03029743 (ISSN); 978-303195451-1 (ISBN).
Abstract | Links | BibTeX | Tags: 'current, Automation, Chatbots, Condition, End-User Development, Extended reality, Human computer interaction, Immersive authoring, Language Model, Large language model, large language models, Rule, Rule-based approach, rules, User interfaces
@inproceedings{carcangiu_conversational_2025,
title = {Conversational Rule Creation in XR: User’s Strategies in VR and AR Automation},
author = {A. Carcangiu and M. Manca and J. Mereu and C. Santoro and L. Simeoli and L. D. Spano},
editor = {Santoro C. and Schmidt A. and Matera M. and Bellucci A.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105009012634&doi=10.1007%2f978-3-031-95452-8_4&partnerID=40&md5=67e2b8ca4bb2b508cd41548e3471705b},
doi = {10.1007/978-3-031-95452-8_4},
isbn = {03029743 (ISSN); 978-303195451-1 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Lect. Notes Comput. Sci.},
volume = {15713 LNCS},
pages = {59–79},
publisher = {Springer Science and Business Media Deutschland GmbH},
abstract = {Rule-based approaches allow users to customize XR environments. However, the current menu-based interfaces still create barriers for end-user developers. Chatbots based on Large Language Models (LLMs) have the potential to reduce the threshold needed for rule creation, but how users articulate their intentions through conversation remains under-explored. This work investigates how users express event-condition-action automation rules in Virtual Reality (VR) and Augmented Reality (AR) environments. Through two user studies, we show that the dialogues share consistent strategies across the interaction setting (keywords, difficulties in expressing conditions, task success), even if we registered different adaptations for each setting (verbal structure, event vs action first rules). Our findings are relevant for the design and implementation of chatbot-based support for expressing automations in an XR setting. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.},
keywords = {'current, Automation, Chatbots, Condition, End-User Development, Extended reality, Human computer interaction, Immersive authoring, Language Model, Large language model, large language models, Rule, Rule-based approach, rules, User interfaces},
pubstate = {published},
tppubtype = {inproceedings}
}
Rule-based approaches allow users to customize XR environments. However, the current menu-based interfaces still create barriers for end-user developers. Chatbots based on Large Language Models (LLMs) have the potential to reduce the threshold needed for rule creation, but how users articulate their intentions through conversation remains under-explored. This work investigates how users express event-condition-action automation rules in Virtual Reality (VR) and Augmented Reality (AR) environments. Through two user studies, we show that the dialogues share consistent strategies across the interaction setting (keywords, difficulties in expressing conditions, task success), even if we registered different adaptations for each setting (verbal structure, event vs action first rules). Our findings are relevant for the design and implementation of chatbot-based support for expressing automations in an XR setting. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
2024
Harinee, S.; Raja, R. Vimal; Mugila, E.; Govindharaj, I.; Sanjaykumar, V.; Ragavendhiran, T.
Elevating Medical Training: A Synergistic Fusion of AI and VR for Immersive Anatomy Learning and Practical Procedure Mastery Proceedings Article
In: Int. Conf. Syst., Comput., Autom. Netw., ICSCAN, Institute of Electrical and Electronics Engineers Inc., 2024, ISBN: 979-833151002-2 (ISBN).
Abstract | Links | BibTeX | Tags: 'current, Anatomy education, Anatomy educations, Computer interaction, Curricula, Embodied virtual assistant, Embodied virtual assistants, Generative AI, Human- Computer Interaction, Immersive, Intelligent virtual agents, Medical computing, Medical education, Medical procedure practice, Medical procedures, Medical training, Personnel training, Students, Teaching, Three dimensional computer graphics, Usability engineering, Virtual assistants, Virtual environments, Virtual Reality, Visualization
@inproceedings{harinee_elevating_2024,
title = {Elevating Medical Training: A Synergistic Fusion of AI and VR for Immersive Anatomy Learning and Practical Procedure Mastery},
author = {S. Harinee and R. Vimal Raja and E. Mugila and I. Govindharaj and V. Sanjaykumar and T. Ragavendhiran},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105000334626&doi=10.1109%2fICSCAN62807.2024.10894451&partnerID=40&md5=100899b489c00335e0a652f2efd33e23},
doi = {10.1109/ICSCAN62807.2024.10894451},
isbn = {979-833151002-2 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Int. Conf. Syst., Comput., Autom. Netw., ICSCAN},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {Virtual reality with its 3D visualization have brought an overwhelming change in the face of medical education, especially for courses like human anatomy. The proposed virtual reality system to bring massive improvements in the education received by a medical student studying for their degree courses. The project puts forward the text-to-speech and speech-to-text aligned system that simplifies the usage of a chatbot empowered by OpenAI GPT-4 and allows pupils to vocally speak with Avatar, the set virtual assistant. Contrary to the current methodologies, the setup of virtual reality is powered by avatars and thus covers an enhanced virtual assistant environment. Avatars offer students the set of repeated practicing of medical procedures on it, and the real uniqueness in the proposed product. The developed virtual reality environment is enhanced over other current training techniques where a student should interact and immerse in three-dimensional human organs for visualization in three dimensions and hence get better knowledge of the subjects in greater depth. A virtual assistant guides the whole process, giving insights and support to help the student bridge the gap from theory to practice. Then, the system is essentially Knowledge based and Analysis based approach. The combination of generative AI along with embodied virtual agents has great potential when it comes to customized virtual conversation assistant for much wider range of applications. The study brings out the value of acquiring hands-on skills through simulated medical procedures and opens new frontiers of research and development in AI, VR, and medical education. In addition to assessing the effectiveness of such novel functionalities, the study also explores user experience related dimensions such as usability, task loading, and the sense of presence in proposed virtual medical environment. © 2024 IEEE.},
keywords = {'current, Anatomy education, Anatomy educations, Computer interaction, Curricula, Embodied virtual assistant, Embodied virtual assistants, Generative AI, Human- Computer Interaction, Immersive, Intelligent virtual agents, Medical computing, Medical education, Medical procedure practice, Medical procedures, Medical training, Personnel training, Students, Teaching, Three dimensional computer graphics, Usability engineering, Virtual assistants, Virtual environments, Virtual Reality, Visualization},
pubstate = {published},
tppubtype = {inproceedings}
}
Virtual reality with its 3D visualization have brought an overwhelming change in the face of medical education, especially for courses like human anatomy. The proposed virtual reality system to bring massive improvements in the education received by a medical student studying for their degree courses. The project puts forward the text-to-speech and speech-to-text aligned system that simplifies the usage of a chatbot empowered by OpenAI GPT-4 and allows pupils to vocally speak with Avatar, the set virtual assistant. Contrary to the current methodologies, the setup of virtual reality is powered by avatars and thus covers an enhanced virtual assistant environment. Avatars offer students the set of repeated practicing of medical procedures on it, and the real uniqueness in the proposed product. The developed virtual reality environment is enhanced over other current training techniques where a student should interact and immerse in three-dimensional human organs for visualization in three dimensions and hence get better knowledge of the subjects in greater depth. A virtual assistant guides the whole process, giving insights and support to help the student bridge the gap from theory to practice. Then, the system is essentially Knowledge based and Analysis based approach. The combination of generative AI along with embodied virtual agents has great potential when it comes to customized virtual conversation assistant for much wider range of applications. The study brings out the value of acquiring hands-on skills through simulated medical procedures and opens new frontiers of research and development in AI, VR, and medical education. In addition to assessing the effectiveness of such novel functionalities, the study also explores user experience related dimensions such as usability, task loading, and the sense of presence in proposed virtual medical environment. © 2024 IEEE.
Ansari, U.; Qureshi, H. A.; Soomro, N. A.; Memon, A. R.
Augmented Reality-Driven Reservoir Management Via Generative Ai: Transforming Pore-Scale Fluid Flow Simulation Proceedings Article
In: Soc. Pet. Eng. - ADIPEC, Society of Petroleum Engineers, 2024, ISBN: 978-195902549-8 (ISBN).
Abstract | Links | BibTeX | Tags: 'current, AI techniques, Augmented Reality, Decision making, Decisions makings, Efficiency, Finance, Fluid flow simulation, Fluid-flow, Gasoline, High-accuracy, Management tasks, Petroleum refining, Petroleum reservoir evaluation, Pore scale, Real- time, User interaction
@inproceedings{ansari_augmented_2024,
title = {Augmented Reality-Driven Reservoir Management Via Generative Ai: Transforming Pore-Scale Fluid Flow Simulation},
author = {U. Ansari and H. A. Qureshi and N. A. Soomro and A. R. Memon},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85215124881&doi=10.2118%2f222865-MS&partnerID=40&md5=32e8ddc777be342df8196b86a4eb7c60},
doi = {10.2118/222865-MS},
isbn = {978-195902549-8 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Soc. Pet. Eng. - ADIPEC},
publisher = {Society of Petroleum Engineers},
abstract = {The current revolution of generative artificial intelligence is transforming global dynamics which is also essential to petroleum engineers for effectively completing technical tasks.Henceforth the main aim of this study is to investigate the application of generative AI techniques for improving the efficiency of petroleum reservoir management.The outcomes of this study will help in developing and implementing generative AI algorithms tailored for reservoir management tasks, including reservoir modeling, production optimization, and decision support.In this study generative AI technique is employed to integrate with augmented reality (AR) to enhance reservoir management.The methodology involves developing a generative AI model to simulate pore-scale fluid flow, validated against experimental data.AR is utilized to visualize and interact with the simulation results in a real-time, immersive environment.The integration process includes data preprocessing, model training, and AR deployment.Performance metrics such as accuracy, computational efficiency, and user interaction quality are evaluated to assess the effectiveness of the proposed approach in transforming traditional reservoir management practices.The developed generative AI model demonstrated high accuracy in simulating pore-scale fluid flow, closely matching experimental data with a correlation coefficient of 0.95.The AR interface provided an intuitive visualization, significantly improving user comprehension and decision-making efficiency.Computational efficiency was enhanced by 40% compared to traditional methods, enabling real-time simulations and interactions.Moreover, it was observed that Users found the AR-driven approach more engaging and easier to understand, with a reported 30% increase in correct decision-making in reservoir management tasks.The integration of generative AI with AR allowed for dynamic adjustments and immediate feedback, which was particularly beneficial in complex scenarios requiring rapid analysis and response.Concludingly, the combination of generative AI and AR offers a transformative approach to reservoir management, enhancing both the accuracy of simulations and the effectiveness of user interactions.This methodology not only improves computational efficiency but also fosters better decision-making through immersive visualization.Future work will focus on refining the AI model and expanding the AR functionalities to cover a broader range of reservoir conditions and management strategies.This study introduces a novel integration of generative AI and augmented reality (AR) for reservoir management, offering a pioneering approach to pore-scale fluid flow simulation.By combining high-accuracy AI-driven simulations with real-time, immersive AR visualizations, this methodology significantly enhances user interaction and decision-making efficiency.This innovative framework transforms traditional practices, providing a more engaging, efficient, and accurate tool for managing complex reservoir systems. Copyright 2024, Society of Petroleum Engineers.},
keywords = {'current, AI techniques, Augmented Reality, Decision making, Decisions makings, Efficiency, Finance, Fluid flow simulation, Fluid-flow, Gasoline, High-accuracy, Management tasks, Petroleum refining, Petroleum reservoir evaluation, Pore scale, Real- time, User interaction},
pubstate = {published},
tppubtype = {inproceedings}
}
The current revolution of generative artificial intelligence is transforming global dynamics which is also essential to petroleum engineers for effectively completing technical tasks.Henceforth the main aim of this study is to investigate the application of generative AI techniques for improving the efficiency of petroleum reservoir management.The outcomes of this study will help in developing and implementing generative AI algorithms tailored for reservoir management tasks, including reservoir modeling, production optimization, and decision support.In this study generative AI technique is employed to integrate with augmented reality (AR) to enhance reservoir management.The methodology involves developing a generative AI model to simulate pore-scale fluid flow, validated against experimental data.AR is utilized to visualize and interact with the simulation results in a real-time, immersive environment.The integration process includes data preprocessing, model training, and AR deployment.Performance metrics such as accuracy, computational efficiency, and user interaction quality are evaluated to assess the effectiveness of the proposed approach in transforming traditional reservoir management practices.The developed generative AI model demonstrated high accuracy in simulating pore-scale fluid flow, closely matching experimental data with a correlation coefficient of 0.95.The AR interface provided an intuitive visualization, significantly improving user comprehension and decision-making efficiency.Computational efficiency was enhanced by 40% compared to traditional methods, enabling real-time simulations and interactions.Moreover, it was observed that Users found the AR-driven approach more engaging and easier to understand, with a reported 30% increase in correct decision-making in reservoir management tasks.The integration of generative AI with AR allowed for dynamic adjustments and immediate feedback, which was particularly beneficial in complex scenarios requiring rapid analysis and response.Concludingly, the combination of generative AI and AR offers a transformative approach to reservoir management, enhancing both the accuracy of simulations and the effectiveness of user interactions.This methodology not only improves computational efficiency but also fosters better decision-making through immersive visualization.Future work will focus on refining the AI model and expanding the AR functionalities to cover a broader range of reservoir conditions and management strategies.This study introduces a novel integration of generative AI and augmented reality (AR) for reservoir management, offering a pioneering approach to pore-scale fluid flow simulation.By combining high-accuracy AI-driven simulations with real-time, immersive AR visualizations, this methodology significantly enhances user interaction and decision-making efficiency.This innovative framework transforms traditional practices, providing a more engaging, efficient, and accurate tool for managing complex reservoir systems. Copyright 2024, Society of Petroleum Engineers.
White, M.; Banerjee, N. K.; Banerjee, S.
VRcabulary: A VR Environment for Reinforced Language Learning via Multi-Modular Design Proceedings Article
In: Proc. - IEEE Int. Conf. Artif. Intell. Ext. Virtual Real., AIxVR, pp. 315–319, Institute of Electrical and Electronics Engineers Inc., 2024, ISBN: 979-835037202-1 (ISBN).
Abstract | Links | BibTeX | Tags: 'current, E-Learning, Foreign language, Immersive, Instructional modules, Language learning, Modular designs, Modulars, Multi-modular, Reinforcement, Second language, Virtual Reality, Virtual-reality environment
@inproceedings{white_vrcabulary_2024,
title = {VRcabulary: A VR Environment for Reinforced Language Learning via Multi-Modular Design},
author = {M. White and N. K. Banerjee and S. Banerjee},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85187241160&doi=10.1109%2fAIxVR59861.2024.00053&partnerID=40&md5=4d8ff8ac5c6aa8336a571ba906fe0f5d},
doi = {10.1109/AIxVR59861.2024.00053},
isbn = {979-835037202-1 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Proc. - IEEE Int. Conf. Artif. Intell. Ext. Virtual Real., AIxVR},
pages = {315–319},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {We demonstrate VRcabulary, a hierarchical modular virtual reality (VR) environment for language learning (LL). Current VR LL apps lack the benefit of reinforcement presented in typical classroom environments. Apps either introduce content in the second language and lack retention testing, or provide gamification without an in-environment instructional component. To acquire reinforcement of knowledge, the learner needs to visit the app multiple times, increasing the potential for monotony. In VRcabulary, we introduce a multi-modular hierarchical design with 3 modules - an instructional module providing AI-generated audio playbacks of object names, a practice module enabling interaction based reinforcement of object names in response to audio playback, and an exam module enabling retention testing through interaction. To incentivize engagement by reducing monotony, we keep the designs of each modules distinct. We provide sequential object presentations in the instructional module and multiple object assortments in the practice and exam modules. We provide feedback and multiple trials in the practice module, but eliminate them from the exam module. We expect cross-module diversity of interaction in VRcabulary to enhance engagement in VR LL. © 2024 IEEE.},
keywords = {'current, E-Learning, Foreign language, Immersive, Instructional modules, Language learning, Modular designs, Modulars, Multi-modular, Reinforcement, Second language, Virtual Reality, Virtual-reality environment},
pubstate = {published},
tppubtype = {inproceedings}
}
We demonstrate VRcabulary, a hierarchical modular virtual reality (VR) environment for language learning (LL). Current VR LL apps lack the benefit of reinforcement presented in typical classroom environments. Apps either introduce content in the second language and lack retention testing, or provide gamification without an in-environment instructional component. To acquire reinforcement of knowledge, the learner needs to visit the app multiple times, increasing the potential for monotony. In VRcabulary, we introduce a multi-modular hierarchical design with 3 modules - an instructional module providing AI-generated audio playbacks of object names, a practice module enabling interaction based reinforcement of object names in response to audio playback, and an exam module enabling retention testing through interaction. To incentivize engagement by reducing monotony, we keep the designs of each modules distinct. We provide sequential object presentations in the instructional module and multiple object assortments in the practice and exam modules. We provide feedback and multiple trials in the practice module, but eliminate them from the exam module. We expect cross-module diversity of interaction in VRcabulary to enhance engagement in VR LL. © 2024 IEEE.
Su, X.; Koh, E.; Xiao, C.
SonifyAR: Context-Aware Sound Effect Generation in Augmented Reality Proceedings Article
In: Conf Hum Fact Comput Syst Proc, Association for Computing Machinery, 2024, ISBN: 979-840070331-7 (ISBN).
Abstract | Links | BibTeX | Tags: 'current, Augmented Reality, Augmented reality authoring, Authoring Tool, Context information, Context-Aware, Immersiveness, Iterative methods, Mixed reality, Real-world, Sound, Sound effects, User interfaces, Users' experiences
@inproceedings{su_sonifyar_2024,
title = {SonifyAR: Context-Aware Sound Effect Generation in Augmented Reality},
author = {X. Su and E. Koh and C. Xiao},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85194146678&doi=10.1145%2f3613905.3650927&partnerID=40&md5=fa2154e1ffdd5339696ccb39584dee16},
doi = {10.1145/3613905.3650927},
isbn = {979-840070331-7 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Conf Hum Fact Comput Syst Proc},
publisher = {Association for Computing Machinery},
abstract = {Sound plays crucial roles in enhancing user experience and immersiveness in Augmented Reality (AR). However, current AR authoring platforms lack support for creating sound effects that harmonize with both the virtual and the real-world contexts. In this work, we present SonifyAR, a novel system for generating context-aware sound effects in AR experiences. SonifyAR implements a Programming by Demonstration (PbD) AR authoring pipeline. We utilize computer vision models and a large language model (LLM) to generate text descriptions that incorporate context information of user, virtual object and real world environment. This context information is then used to acquire sound effects with recommendation, generation, and retrieval methods. The acquired sound effects can be tested and assigned to AR events. Our user interface also provides the flexibility to allow users to iteratively explore and fine-tune the sound effects. We conducted a preliminary user study to demonstrate the effectiveness and usability of our system. © 2024 Association for Computing Machinery. All rights reserved.},
keywords = {'current, Augmented Reality, Augmented reality authoring, Authoring Tool, Context information, Context-Aware, Immersiveness, Iterative methods, Mixed reality, Real-world, Sound, Sound effects, User interfaces, Users' experiences},
pubstate = {published},
tppubtype = {inproceedings}
}
Sound plays crucial roles in enhancing user experience and immersiveness in Augmented Reality (AR). However, current AR authoring platforms lack support for creating sound effects that harmonize with both the virtual and the real-world contexts. In this work, we present SonifyAR, a novel system for generating context-aware sound effects in AR experiences. SonifyAR implements a Programming by Demonstration (PbD) AR authoring pipeline. We utilize computer vision models and a large language model (LLM) to generate text descriptions that incorporate context information of user, virtual object and real world environment. This context information is then used to acquire sound effects with recommendation, generation, and retrieval methods. The acquired sound effects can be tested and assigned to AR events. Our user interface also provides the flexibility to allow users to iteratively explore and fine-tune the sound effects. We conducted a preliminary user study to demonstrate the effectiveness and usability of our system. © 2024 Association for Computing Machinery. All rights reserved.
Lombardo, A.; Morabito, G.; Quattropani, S.; Ricci, C.; Siino, M.; Tinnirello, I.
AI-GeneSI: Exploiting generative AI for autonomous generation of the southbound interface in the IoT Proceedings Article
In: IEEE World Forum Internet Things, WF-IoT, Institute of Electrical and Electronics Engineers Inc., 2024, ISBN: 979-835037301-1 (ISBN).
Abstract | Links | BibTeX | Tags: 'current, Autonomous generation, Codes (symbols), Communications protocols, Complex task, Data representations, Digital world, Interface functions, Language Model, Reusability, Sensor nodes, Sensors data, Virtual objects, Virtual Reality
@inproceedings{lombardo_ai-genesi_2024,
title = {AI-GeneSI: Exploiting generative AI for autonomous generation of the southbound interface in the IoT},
author = {A. Lombardo and G. Morabito and S. Quattropani and C. Ricci and M. Siino and I. Tinnirello},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85216509327&doi=10.1109%2fWF-IoT62078.2024.10811300&partnerID=40&md5=cb20d5004d1f99b73b536dd0738cabd5},
doi = {10.1109/WF-IoT62078.2024.10811300},
isbn = {979-835037301-1 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {IEEE World Forum Internet Things, WF-IoT},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {Virtual objects, which are representations in the digital world of physical entities, uses the data collected by one or several sensor nodes to operate. To overcome the diversity and heterogeneity of protocols implemented by different sensor nodes and the way in which sensor data is represented, it is convenient to exploit appropriate components referred to as 'southbound interfaces' in this paper. The objective of the southbound interface is to convert the communication protocols implemented by sensor nodes and virtual objects and to harmonize data representations. The implementation of the southbound interfaces is not a complex task, however it is extremely specific of the current setting, which turns in low reusability of the code, and is time-consuming. In this paper, a methodology named AI-GeneSI is proposed to exploit Large Language Models (LLM)s to generate the code to communicate with the southbound interface. Such code is utilized to create and deploy a microservice which implements the southbound interface functions. A prototype of the proposed methodology has been implemented to demonstrate the feasibility of the proposed approach. © 2024 IEEE.},
keywords = {'current, Autonomous generation, Codes (symbols), Communications protocols, Complex task, Data representations, Digital world, Interface functions, Language Model, Reusability, Sensor nodes, Sensors data, Virtual objects, Virtual Reality},
pubstate = {published},
tppubtype = {inproceedings}
}
Virtual objects, which are representations in the digital world of physical entities, uses the data collected by one or several sensor nodes to operate. To overcome the diversity and heterogeneity of protocols implemented by different sensor nodes and the way in which sensor data is represented, it is convenient to exploit appropriate components referred to as 'southbound interfaces' in this paper. The objective of the southbound interface is to convert the communication protocols implemented by sensor nodes and virtual objects and to harmonize data representations. The implementation of the southbound interfaces is not a complex task, however it is extremely specific of the current setting, which turns in low reusability of the code, and is time-consuming. In this paper, a methodology named AI-GeneSI is proposed to exploit Large Language Models (LLM)s to generate the code to communicate with the southbound interface. Such code is utilized to create and deploy a microservice which implements the southbound interface functions. A prototype of the proposed methodology has been implemented to demonstrate the feasibility of the proposed approach. © 2024 IEEE.
Otoum, Y.; Gottimukkala, N.; Kumar, N.; Nayak, A.
Machine Learning in Metaverse Security: Current Solutions and Future Challenges Journal Article
In: ACM Computing Surveys, vol. 56, no. 8, 2024, ISSN: 03600300 (ISSN).
Abstract | Links | BibTeX | Tags: 'current, Block-chain, Blockchain, digital twin, E-Learning, Extended reality, Future challenges, Generative AI, machine learning, Machine-learning, Metaverse Security, Metaverses, Security and privacy, Spatio-temporal dynamics, Sustainable development
@article{otoum_machine_2024,
title = {Machine Learning in Metaverse Security: Current Solutions and Future Challenges},
author = {Y. Otoum and N. Gottimukkala and N. Kumar and A. Nayak},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85193466017&doi=10.1145%2f3654663&partnerID=40&md5=b35485c5f2e943ec105ea11a80712cbe},
doi = {10.1145/3654663},
issn = {03600300 (ISSN)},
year = {2024},
date = {2024-01-01},
journal = {ACM Computing Surveys},
volume = {56},
number = {8},
abstract = {The Metaverse, positioned as the next frontier of the Internet, has the ambition to forge a virtual shared realm characterized by immersion, hyper-spatiotemporal dynamics, and self-sustainability. Recent technological strides in AI, Extended Reality, 6G, and blockchain propel the Metaverse closer to realization, gradually transforming it from science fiction into an imminent reality. Nevertheless, the extensive deployment of the Metaverse faces substantial obstacles, primarily stemming from its potential to infringe on privacy and be susceptible to security breaches, whether inherent in its underlying technologies or arising from the evolving digital landscape. Metaverse security provisioning is poised to confront various foundational challenges owing to its distinctive attributes, encompassing immersive realism, hyper-spatiotemporally, sustainability, and heterogeneity. This article undertakes a comprehensive study of the security and privacy challenges facing the Metaverse, leveraging machine learning models for this purpose. In particular, our focus centers on an innovative distributed Metaverse architecture characterized by interactions across 3D worlds. Subsequently, we conduct a thorough review of the existing cutting-edge measures designed for Metaverse systems while also delving into the discourse surrounding security and privacy threats. As we contemplate the future of Metaverse systems, we outline directions for open research pursuits in this evolving landscape. © 2024 Copyright held by the owner/author(s). Publication rights licensed to ACM.},
keywords = {'current, Block-chain, Blockchain, digital twin, E-Learning, Extended reality, Future challenges, Generative AI, machine learning, Machine-learning, Metaverse Security, Metaverses, Security and privacy, Spatio-temporal dynamics, Sustainable development},
pubstate = {published},
tppubtype = {article}
}
The Metaverse, positioned as the next frontier of the Internet, has the ambition to forge a virtual shared realm characterized by immersion, hyper-spatiotemporal dynamics, and self-sustainability. Recent technological strides in AI, Extended Reality, 6G, and blockchain propel the Metaverse closer to realization, gradually transforming it from science fiction into an imminent reality. Nevertheless, the extensive deployment of the Metaverse faces substantial obstacles, primarily stemming from its potential to infringe on privacy and be susceptible to security breaches, whether inherent in its underlying technologies or arising from the evolving digital landscape. Metaverse security provisioning is poised to confront various foundational challenges owing to its distinctive attributes, encompassing immersive realism, hyper-spatiotemporally, sustainability, and heterogeneity. This article undertakes a comprehensive study of the security and privacy challenges facing the Metaverse, leveraging machine learning models for this purpose. In particular, our focus centers on an innovative distributed Metaverse architecture characterized by interactions across 3D worlds. Subsequently, we conduct a thorough review of the existing cutting-edge measures designed for Metaverse systems while also delving into the discourse surrounding security and privacy threats. As we contemplate the future of Metaverse systems, we outline directions for open research pursuits in this evolving landscape. © 2024 Copyright held by the owner/author(s). Publication rights licensed to ACM.
Hong, J.; Lee, Y.; Kim, D. H.; Choi, D.; Yoon, Y. -J.; Lee, G. -C.; Lee, Z.; Kim, J.
A Context-Aware Onboarding Agent for Metaverse Powered by Large Language Models Proceedings Article
In: A., Vallgarda; L., Jonsson; J., Fritsch; S.F., Alaoui; C.A., Le Dantec (Ed.): Proc. ACM Des. Interact. Syst. Conf., pp. 1857–1874, Association for Computing Machinery, Inc, 2024, ISBN: 979-840070583-0 (ISBN).
Abstract | Links | BibTeX | Tags: 'current, Computational Linguistics, Context- awareness, Context-Aware, context-awareness, conversational agent, Conversational Agents, Divergents, Language Model, Large-language model, large-language models, Metaverse, Metaverses, Model-based OPC, Onboarding, User interfaces, Virtual Reality
@inproceedings{hong_context-aware_2024,
title = {A Context-Aware Onboarding Agent for Metaverse Powered by Large Language Models},
author = {J. Hong and Y. Lee and D. H. Kim and D. Choi and Y. -J. Yoon and G. -C. Lee and Z. Lee and J. Kim},
editor = {Vallgarda A. and Jonsson L. and Fritsch J. and Alaoui S.F. and Le Dantec C.A.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85200340104&doi=10.1145%2f3643834.3661579&partnerID=40&md5=5fe96b5155ca45c6d7a0d239b68f2b30},
doi = {10.1145/3643834.3661579},
isbn = {979-840070583-0 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Proc. ACM Des. Interact. Syst. Conf.},
pages = {1857–1874},
publisher = {Association for Computing Machinery, Inc},
abstract = {One common asset of metaverse is that users can freely explore places and actions without linear procedures. Thus, it is hard yet important to understand the divergent challenges each user faces when onboarding metaverse. Our formative study (N = 16) shows that frst-time users ask questions about metaverse that concern 1) a short-term spatiotemporal context, regarding the user’s current location, recent conversation, and actions, and 2) a long-term exploration context regarding the user’s experience history. Based on the fndings, we present PICAN, a Large Language Model-based pipeline that generates context-aware answers to users when onboarding metaverse. An ablation study (N = 20) reveals that PICAN’s usage of context made responses more useful and immersive than those generated without contexts. Furthermore, a user study (N = 21) shows that the use of long-term exploration context promotes users’ learning about the locations and activities within the virtual environment. © 2024 Copyright held by the owner/author(s).},
keywords = {'current, Computational Linguistics, Context- awareness, Context-Aware, context-awareness, conversational agent, Conversational Agents, Divergents, Language Model, Large-language model, large-language models, Metaverse, Metaverses, Model-based OPC, Onboarding, User interfaces, Virtual Reality},
pubstate = {published},
tppubtype = {inproceedings}
}
One common asset of metaverse is that users can freely explore places and actions without linear procedures. Thus, it is hard yet important to understand the divergent challenges each user faces when onboarding metaverse. Our formative study (N = 16) shows that frst-time users ask questions about metaverse that concern 1) a short-term spatiotemporal context, regarding the user’s current location, recent conversation, and actions, and 2) a long-term exploration context regarding the user’s experience history. Based on the fndings, we present PICAN, a Large Language Model-based pipeline that generates context-aware answers to users when onboarding metaverse. An ablation study (N = 20) reveals that PICAN’s usage of context made responses more useful and immersive than those generated without contexts. Furthermore, a user study (N = 21) shows that the use of long-term exploration context promotes users’ learning about the locations and activities within the virtual environment. © 2024 Copyright held by the owner/author(s).
Chan, A.; Liu, J. A.
Board 24: Development of Multi-User-enabled, Interactive, and Responsive Virtual/Augmented Reality-based Laboratory Training System Proceedings Article
In: ASEE Annu. Conf. Expos. Conf. Proc., American Society for Engineering Education, 2024, ISBN: 21535965 (ISSN).
Abstract | Links | BibTeX | Tags: 'current, Augmented Reality, Chemical engineering students, Concentration (process), Cooling towers, Degassing, Hands-on learning, Hydrodesulfurization, Immersive, Large groups, Liquid crystal displays, Multiusers, Nanoreactors, Personnel training, Pilot-scale equipment, Protective equipment, Students, Training Systems, Unit Operations Laboratory
@inproceedings{chan_board_2024,
title = {Board 24: Development of Multi-User-enabled, Interactive, and Responsive Virtual/Augmented Reality-based Laboratory Training System},
author = {A. Chan and J. A. Liu},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85202042560&partnerID=40&md5=9b1565ea2dd4336b4cc45594fe4f7900},
isbn = {21535965 (ISSN)},
year = {2024},
date = {2024-01-01},
booktitle = {ASEE Annu. Conf. Expos. Conf. Proc.},
publisher = {American Society for Engineering Education},
abstract = {The Unit Operations Laboratory (UOL) is a place where third-year chemical engineering students can apply their engineering and science concepts on pilot-scale equipment. However, the physical lab is resource-intensive, requiring protective equipment and constant supervision. In addition, due to limited units for large groups of students, students perform experiments according to the rolling program schedule, making alignment with lecture teaching and hands-on learning challenges. The research team focuses on increasing the accessibility of the UOL by using simulation gaming in standard, virtual reality and augmented reality modalities as an educational tool. The "Virtual Unit Ops Lab" application places students in an immersive simulated environment of the physical lab, where they can get practical experiences without the difficulties of an in-person lab by using specialized headsets and controllers, which allows the student to move and interact with various parts of the machine physically. Developed with Unity software, the application serves as a digital twin to an existing lab, which allows for an immersive simulation of the full-scale lab equipment, in addition to enhanced learning features such as the ability to display the current action performed by the user and to provide visual/audio feedback for correct and incorrect actions. The application also supports the use by multiple "players" (i.e., it has the "multiplayer" option), where multiple students can communicate and discuss their current step. As a work in progress, a non-player-character chatbot (generative AI responses) is being developed for existing applications using OpenAI's GPT-3.5, which provides designated information to a student in a conversational manner. Additionally, a supplemental "Augmented Unit Ops Lab" application uses Augmented Reality, which superimposes three-dimensional flow diagrams onto the Heat Exchanger through the view of a phone camera during the in-person labs. © American Society for Engineering Education, 2024.},
keywords = {'current, Augmented Reality, Chemical engineering students, Concentration (process), Cooling towers, Degassing, Hands-on learning, Hydrodesulfurization, Immersive, Large groups, Liquid crystal displays, Multiusers, Nanoreactors, Personnel training, Pilot-scale equipment, Protective equipment, Students, Training Systems, Unit Operations Laboratory},
pubstate = {published},
tppubtype = {inproceedings}
}
The Unit Operations Laboratory (UOL) is a place where third-year chemical engineering students can apply their engineering and science concepts on pilot-scale equipment. However, the physical lab is resource-intensive, requiring protective equipment and constant supervision. In addition, due to limited units for large groups of students, students perform experiments according to the rolling program schedule, making alignment with lecture teaching and hands-on learning challenges. The research team focuses on increasing the accessibility of the UOL by using simulation gaming in standard, virtual reality and augmented reality modalities as an educational tool. The "Virtual Unit Ops Lab" application places students in an immersive simulated environment of the physical lab, where they can get practical experiences without the difficulties of an in-person lab by using specialized headsets and controllers, which allows the student to move and interact with various parts of the machine physically. Developed with Unity software, the application serves as a digital twin to an existing lab, which allows for an immersive simulation of the full-scale lab equipment, in addition to enhanced learning features such as the ability to display the current action performed by the user and to provide visual/audio feedback for correct and incorrect actions. The application also supports the use by multiple "players" (i.e., it has the "multiplayer" option), where multiple students can communicate and discuss their current step. As a work in progress, a non-player-character chatbot (generative AI responses) is being developed for existing applications using OpenAI's GPT-3.5, which provides designated information to a student in a conversational manner. Additionally, a supplemental "Augmented Unit Ops Lab" application uses Augmented Reality, which superimposes three-dimensional flow diagrams onto the Heat Exchanger through the view of a phone camera during the in-person labs. © American Society for Engineering Education, 2024.
2023
Friess, P.
THE_OTHERVERSE - A Contemporary Cabinet Of Curiosities Proceedings Article
In: D., Byrne; N., Martelaro (Ed.): DIS Companion: Companion Publ. ACM Des Interact. Syst. Conf., pp. 50–54, Association for Computing Machinery, Inc, 2023, ISBN: 978-145039898-5 (ISBN).
Abstract | Links | BibTeX | Tags: 'current, AI-generated content, Algorithmic energy, Algorithmics, Arts computing, Energy, Form of existences, Image enhancement, Interactive computer graphics, Metaverse, Metaverses, Other verse, Other verses, Resilience, Virtual Reality, Virtual worlds
@inproceedings{friess_the_otherverse_2023,
title = {THE_OTHERVERSE - A Contemporary Cabinet Of Curiosities},
author = {P. Friess},
editor = {Byrne D. and Martelaro N.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85167658209&doi=10.1145%2f3563703.3596803&partnerID=40&md5=d68cab2e32a1581efd5b734b67b0f88b},
doi = {10.1145/3563703.3596803},
isbn = {978-145039898-5 (ISBN)},
year = {2023},
date = {2023-01-01},
booktitle = {DIS Companion: Companion Publ. ACM Des Interact. Syst. Conf.},
pages = {50–54},
publisher = {Association for Computing Machinery, Inc},
abstract = {THE_OTHERVERSE is an artwork exploring resilience as a proactive artistic attitude. Inspired by the revisited idea of a "Wunderkammer"for multidisciplinary exploration and as a representation of microcosms of the world, the artwork features a contemporary cabinet of curiosities with AI-generated content of other forms of existence and expands the current Metaverse paradigm including emphasizing algorithm energy sound and rhythm. The research-creation process includes storytelling, object creation, virtual environment design, sound creation, and image enhancement, blending the aesthetics of the obtained results with the tools used for the creation. Understanding and interacting with AI as a creative partner opens up new possibilities for future research-creation, both for the research part in providing collective knowledge as for the creation part to propose a machine-thinking inspired recombination of ideas. Resilience is not only achieved by how we respond to bad things, but also how we broaden our possibilities (https://vimeo.com/petermfriess/the-otherverse). © 2023 ACM.},
keywords = {'current, AI-generated content, Algorithmic energy, Algorithmics, Arts computing, Energy, Form of existences, Image enhancement, Interactive computer graphics, Metaverse, Metaverses, Other verse, Other verses, Resilience, Virtual Reality, Virtual worlds},
pubstate = {published},
tppubtype = {inproceedings}
}
THE_OTHERVERSE is an artwork exploring resilience as a proactive artistic attitude. Inspired by the revisited idea of a "Wunderkammer"for multidisciplinary exploration and as a representation of microcosms of the world, the artwork features a contemporary cabinet of curiosities with AI-generated content of other forms of existence and expands the current Metaverse paradigm including emphasizing algorithm energy sound and rhythm. The research-creation process includes storytelling, object creation, virtual environment design, sound creation, and image enhancement, blending the aesthetics of the obtained results with the tools used for the creation. Understanding and interacting with AI as a creative partner opens up new possibilities for future research-creation, both for the research part in providing collective knowledge as for the creation part to propose a machine-thinking inspired recombination of ideas. Resilience is not only achieved by how we respond to bad things, but also how we broaden our possibilities (https://vimeo.com/petermfriess/the-otherverse). © 2023 ACM.