AHCI RESEARCH GROUP
Publications
Papers published in international journals,
proceedings of conferences, workshops and books.
OUR RESEARCH
Scientific Publications
How to
Here you can find the complete list of our publications.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
2024
Xu, F.; Nguyen, T.; Du, J.
Augmented Reality for Maintenance Tasks with ChatGPT for Automated Text-To-Action Journal Article
In: Journal of Construction Engineering and Management, vol. 150, no. 4, 2024, ISSN: 07339364 (ISSN).
Abstract | Links | BibTeX | Tags: Artificial intelligence systems, Augmented Reality, Augmented Reality (AR), ChatGPT, Complex sequences, Computational Linguistics, Diverse fields, Human like, Language Model, Maintenance, Maintenance tasks, Operations and maintenance, Optical character recognition, Sensor technologies, Virtual Reality
@article{xu_augmented_2024,
title = {Augmented Reality for Maintenance Tasks with ChatGPT for Automated Text-To-Action},
author = {F. Xu and T. Nguyen and J. Du},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85183669638&doi=10.1061%2fJCEMD4.COENG-14142&partnerID=40&md5=6b02d2f4f6e74a8152adf2eb30ee2d88},
doi = {10.1061/JCEMD4.COENG-14142},
issn = {07339364 (ISSN)},
year = {2024},
date = {2024-01-01},
journal = {Journal of Construction Engineering and Management},
volume = {150},
number = {4},
abstract = {Advancements in sensor technology, artificial intelligence (AI), and augmented reality (AR) have unlocked opportunities across various domains. AR and large language models like GPT have witnessed substantial progress and increasingly are being employed in diverse fields. One such promising application is in operations and maintenance (OM). OM tasks often involve complex procedures and sequences that can be challenging to memorize and execute correctly, particularly for novices or in high-stress situations. By combining the advantages of superimposing virtual objects onto the physical world and generating human-like text using GPT, we can revolutionize OM operations. This study introduces a system that combines AR, optical character recognition (OCR), and the GPT language model to optimize user performance while offering trustworthy interactions and alleviating workload in OM tasks. This system provides an interactive virtual environment controlled by the Unity game engine, facilitating a seamless interaction between virtual and physical realities. A case study (N=30) was conducted to illustrate the findings and answer the research questions. The Multidimensional Measurement of Trust (MDMT) was applied to understand the complexity of trust engagement with such a human-like system. The results indicate that users can complete similarly challenging tasks in less time using our proposed AR and AI system. Moreover, the collected data also suggest a reduction in cognitive load when executing the same operations using the AR and AI system. A divergence of trust was observed concerning capability and ethical dimensions. © 2024 American Society of Civil Engineers.},
keywords = {Artificial intelligence systems, Augmented Reality, Augmented Reality (AR), ChatGPT, Complex sequences, Computational Linguistics, Diverse fields, Human like, Language Model, Maintenance, Maintenance tasks, Operations and maintenance, Optical character recognition, Sensor technologies, Virtual Reality},
pubstate = {published},
tppubtype = {article}
}
Advancements in sensor technology, artificial intelligence (AI), and augmented reality (AR) have unlocked opportunities across various domains. AR and large language models like GPT have witnessed substantial progress and increasingly are being employed in diverse fields. One such promising application is in operations and maintenance (OM). OM tasks often involve complex procedures and sequences that can be challenging to memorize and execute correctly, particularly for novices or in high-stress situations. By combining the advantages of superimposing virtual objects onto the physical world and generating human-like text using GPT, we can revolutionize OM operations. This study introduces a system that combines AR, optical character recognition (OCR), and the GPT language model to optimize user performance while offering trustworthy interactions and alleviating workload in OM tasks. This system provides an interactive virtual environment controlled by the Unity game engine, facilitating a seamless interaction between virtual and physical realities. A case study (N=30) was conducted to illustrate the findings and answer the research questions. The Multidimensional Measurement of Trust (MDMT) was applied to understand the complexity of trust engagement with such a human-like system. The results indicate that users can complete similarly challenging tasks in less time using our proposed AR and AI system. Moreover, the collected data also suggest a reduction in cognitive load when executing the same operations using the AR and AI system. A divergence of trust was observed concerning capability and ethical dimensions. © 2024 American Society of Civil Engineers.
2023
Joseph, S.; Priya, B. S.; Poorvaja, R.; Kumaran, M. Santhosh; Shivaraj, S.; Jeyanth, V.; Shivesh, R. P.
IoT Empowered AI: Transforming Object Recognition and NLP Summarization with Generative AI Proceedings Article
In: K.V., Arya; T., Wada (Ed.): Proc. IEEE Int. Conf. Comput. Vis. Mach. Intell., CVMI, Institute of Electrical and Electronics Engineers Inc., 2023, ISBN: 979-835030514-2 (ISBN).
Abstract | Links | BibTeX | Tags: 2D, 3D, Application program interface, Application Program Interface (API), Application program interfaces, Application programming interfaces (API), Application programs, Augmented Reality, Augmented Reality(AR), Automation, Cameras, Cost effectiveness, Domestic appliances, GenAl, Internet of Things, Internet of Things (IoT) technologies, Internet of things technologies, Language processing, Natural Language Processing, Natural language processing systems, Natural languages, Object Detection, Object recognition, Objects detection, Optical character recognition, Optical Character Recognition (OCR), Smartphones
@inproceedings{joseph_iot_2023,
title = {IoT Empowered AI: Transforming Object Recognition and NLP Summarization with Generative AI},
author = {S. Joseph and B. S. Priya and R. Poorvaja and M. Santhosh Kumaran and S. Shivaraj and V. Jeyanth and R. P. Shivesh},
editor = {Arya K.V. and Wada T.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85189754688&doi=10.1109%2fCVMI59935.2023.10465077&partnerID=40&md5=9c1a9d7151c0b04bab83586f515d30aa},
doi = {10.1109/CVMI59935.2023.10465077},
isbn = {979-835030514-2 (ISBN)},
year = {2023},
date = {2023-01-01},
booktitle = {Proc. IEEE Int. Conf. Comput. Vis. Mach. Intell., CVMI},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {In anticipation of the widespread adoption of augmented reality in the future, this paper introduces an advanced mobile application that seamlessly integrates AR and IoT technologies. The application aims to make these cutting-edge technologies more affordable and accessible to users while highlighting their immense benefits in assisting with household appliance control, as well as providing interactive and educational experiences. The app employs advanced algorithms such as object detection, Natural Language Processing (NLP), and Optical Character Recognition (OCR) to scan the smartphone's camera feed. Upon identification, AR controls for appliances, their power consumption, and electric bill tracking are displayed. Additionally, the application makes use of APIs to access the internet, retrieving relevant 3D generative models, 360-degree videos, 2D images, and textual information based on user interactions with detected objects. Users can effortlessly explore and interact with the 3D generative models using intuitive hand gestures, providing an immersive experience without the need for additional hardware or dedicated VR headsets. Beyond home automation, the app offers valuable educational benefits, serving as a unique learning tool for students to gain hands-on experience. Medical practitioners can quickly reference organ anatomy and utilize its feature-rich functionalities. Its cost-effectiveness, requiring only installation, ensures accessibility to a wide audience. The app's functionality is both intuitive and efficient, detecting objects in the camera feed and prompting user interactions. Users can select objects through simple hand gestures, choosing desired content like 3D generative models, 2D images, textual information, 360-degree videos, or shopping-related details. The app then retrieves and overlays the requested information onto the real-world view in AR. In conclusion, this groundbreaking AR and IoT -powered app revolutionizes home automation and learning experiences, leveraging only a smartphone's camera, without the need for additional hardware or expensive installations. Its potential applications extend to education, industries, and health care, making it a versatile and valuable tool for a broad range of users. © 2023 IEEE.},
keywords = {2D, 3D, Application program interface, Application Program Interface (API), Application program interfaces, Application programming interfaces (API), Application programs, Augmented Reality, Augmented Reality(AR), Automation, Cameras, Cost effectiveness, Domestic appliances, GenAl, Internet of Things, Internet of Things (IoT) technologies, Internet of things technologies, Language processing, Natural Language Processing, Natural language processing systems, Natural languages, Object Detection, Object recognition, Objects detection, Optical character recognition, Optical Character Recognition (OCR), Smartphones},
pubstate = {published},
tppubtype = {inproceedings}
}
In anticipation of the widespread adoption of augmented reality in the future, this paper introduces an advanced mobile application that seamlessly integrates AR and IoT technologies. The application aims to make these cutting-edge technologies more affordable and accessible to users while highlighting their immense benefits in assisting with household appliance control, as well as providing interactive and educational experiences. The app employs advanced algorithms such as object detection, Natural Language Processing (NLP), and Optical Character Recognition (OCR) to scan the smartphone's camera feed. Upon identification, AR controls for appliances, their power consumption, and electric bill tracking are displayed. Additionally, the application makes use of APIs to access the internet, retrieving relevant 3D generative models, 360-degree videos, 2D images, and textual information based on user interactions with detected objects. Users can effortlessly explore and interact with the 3D generative models using intuitive hand gestures, providing an immersive experience without the need for additional hardware or dedicated VR headsets. Beyond home automation, the app offers valuable educational benefits, serving as a unique learning tool for students to gain hands-on experience. Medical practitioners can quickly reference organ anatomy and utilize its feature-rich functionalities. Its cost-effectiveness, requiring only installation, ensures accessibility to a wide audience. The app's functionality is both intuitive and efficient, detecting objects in the camera feed and prompting user interactions. Users can select objects through simple hand gestures, choosing desired content like 3D generative models, 2D images, textual information, 360-degree videos, or shopping-related details. The app then retrieves and overlays the requested information onto the real-world view in AR. In conclusion, this groundbreaking AR and IoT -powered app revolutionizes home automation and learning experiences, leveraging only a smartphone's camera, without the need for additional hardware or expensive installations. Its potential applications extend to education, industries, and health care, making it a versatile and valuable tool for a broad range of users. © 2023 IEEE.