AHCI RESEARCH GROUP
Publications
Papers published in international journals,
proceedings of conferences, workshops and books.
OUR RESEARCH
Scientific Publications
How to
Here you can find the complete list of our publications.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
2024
Lakhnati, Y.; Pascher, M.; Gerken, J.
Exploring a GPT-based large language model for variable autonomy in a VR-based human-robot teaming simulation Journal Article
In: Frontiers in Robotics and AI, vol. 11, 2024, ISSN: 22969144 (ISSN).
Abstract | Links | BibTeX | Tags: Assistive Robots, evaluation, GPT, Large language model, shared control, variable autonomy, Virtual Reality
@article{lakhnati_exploring_2024,
title = {Exploring a GPT-based large language model for variable autonomy in a VR-based human-robot teaming simulation},
author = {Y. Lakhnati and M. Pascher and J. Gerken},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85190520269&doi=10.3389%2ffrobt.2024.1347538&partnerID=40&md5=ba5dcbba299b475c3448d2ea6b493894},
doi = {10.3389/frobt.2024.1347538},
issn = {22969144 (ISSN)},
year = {2024},
date = {2024-01-01},
journal = {Frontiers in Robotics and AI},
volume = {11},
abstract = {In a rapidly evolving digital landscape autonomous tools and robots are becoming commonplace. Recognizing the significance of this development, this paper explores the integration of Large Language Models (LLMs) like Generative pre-trained transformer (GPT) into human-robot teaming environments to facilitate variable autonomy through the means of verbal human-robot communication. In this paper, we introduce a novel simulation framework for such a GPT-powered multi-robot testbed environment, based on a Unity Virtual Reality (VR) setting. This system allows users to interact with simulated robot agents through natural language, each powered by individual GPT cores. By means of OpenAI’s function calling, we bridge the gap between unstructured natural language input and structured robot actions. A user study with 12 participants explores the effectiveness of GPT-4 and, more importantly, user strategies when being given the opportunity to converse in natural language within a simulated multi-robot environment. Our findings suggest that users may have preconceived expectations on how to converse with robots and seldom try to explore the actual language and cognitive capabilities of their simulated robot collaborators. Still, those users who did explore were able to benefit from a much more natural flow of communication and human-like back-and-forth. We provide a set of lessons learned for future research and technical implementations of similar systems. Copyright © 2024 Lakhnati, Pascher and Gerken.},
keywords = {Assistive Robots, evaluation, GPT, Large language model, shared control, variable autonomy, Virtual Reality},
pubstate = {published},
tppubtype = {article}
}
In a rapidly evolving digital landscape autonomous tools and robots are becoming commonplace. Recognizing the significance of this development, this paper explores the integration of Large Language Models (LLMs) like Generative pre-trained transformer (GPT) into human-robot teaming environments to facilitate variable autonomy through the means of verbal human-robot communication. In this paper, we introduce a novel simulation framework for such a GPT-powered multi-robot testbed environment, based on a Unity Virtual Reality (VR) setting. This system allows users to interact with simulated robot agents through natural language, each powered by individual GPT cores. By means of OpenAI’s function calling, we bridge the gap between unstructured natural language input and structured robot actions. A user study with 12 participants explores the effectiveness of GPT-4 and, more importantly, user strategies when being given the opportunity to converse in natural language within a simulated multi-robot environment. Our findings suggest that users may have preconceived expectations on how to converse with robots and seldom try to explore the actual language and cognitive capabilities of their simulated robot collaborators. Still, those users who did explore were able to benefit from a much more natural flow of communication and human-like back-and-forth. We provide a set of lessons learned for future research and technical implementations of similar systems. Copyright © 2024 Lakhnati, Pascher and Gerken.
Liu, M.; M'Hiri, F.
Beyond Traditional Teaching: Large Language Models as Simulated Teaching Assistants in Computer Science Proceedings Article
In: SIGCSE - Proc. ACM Tech. Symp. Comput. Sci. Educ., pp. 743–749, Association for Computing Machinery, Inc, 2024, ISBN: 979-840070423-9 (ISBN).
Abstract | Links | BibTeX | Tags: Adaptive teaching, ChatGPT, Computational Linguistics, CS education, E-Learning, Education computing, Engineering education, GPT, Language Model, LLM, machine learning, Machine-learning, Novice programmer, novice programmers, Openai, Programming, Python, Students, Teaching, Virtual Reality
@inproceedings{liu_beyond_2024,
title = {Beyond Traditional Teaching: Large Language Models as Simulated Teaching Assistants in Computer Science},
author = {M. Liu and F. M'Hiri},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85189289344&doi=10.1145%2f3626252.3630789&partnerID=40&md5=44ec79c8f005f4551c820c61f5b5d435},
doi = {10.1145/3626252.3630789},
isbn = {979-840070423-9 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {SIGCSE - Proc. ACM Tech. Symp. Comput. Sci. Educ.},
volume = {1},
pages = {743–749},
publisher = {Association for Computing Machinery, Inc},
abstract = {As the prominence of Large Language Models (LLMs) grows in various sectors, their potential in education warrants exploration. In this study, we investigate the feasibility of employing GPT-3.5 from OpenAI, as an LLM teaching assistant (TA) or a virtual TA in computer science (CS) courses. The objective is to enhance the accessibility of CS education while maintaining academic integrity by refraining from providing direct solutions to current-semester assignments. Targeting Foundations of Programming (COMP202), an undergraduate course that introduces students to programming with Python, we have developed a virtual TA using the LangChain framework, known for integrating language models with diverse data sources and environments. The virtual TA assists students with their code and clarifies complex concepts. For homework questions, it is designed to guide students with hints rather than giving out direct solutions. We assessed its performance first through a qualitative evaluation, then a survey-based comparative analysis, using a mix of questions commonly asked on the COMP202 discussion board and questions created by the authors. Our preliminary results indicate that the virtual TA outperforms human TAs on clarity and engagement, matching them on accuracy when the question is non-assignment-specific, for which human TAs still proved more reliable. These findings suggest that while virtual TAs, leveraging the capabilities of LLMs, hold great promise towards making CS education experience more accessible and engaging, their optimal use necessitates human supervision. We conclude by identifying several directions that could be explored in future implementations. © 2024 ACM.},
keywords = {Adaptive teaching, ChatGPT, Computational Linguistics, CS education, E-Learning, Education computing, Engineering education, GPT, Language Model, LLM, machine learning, Machine-learning, Novice programmer, novice programmers, Openai, Programming, Python, Students, Teaching, Virtual Reality},
pubstate = {published},
tppubtype = {inproceedings}
}
As the prominence of Large Language Models (LLMs) grows in various sectors, their potential in education warrants exploration. In this study, we investigate the feasibility of employing GPT-3.5 from OpenAI, as an LLM teaching assistant (TA) or a virtual TA in computer science (CS) courses. The objective is to enhance the accessibility of CS education while maintaining academic integrity by refraining from providing direct solutions to current-semester assignments. Targeting Foundations of Programming (COMP202), an undergraduate course that introduces students to programming with Python, we have developed a virtual TA using the LangChain framework, known for integrating language models with diverse data sources and environments. The virtual TA assists students with their code and clarifies complex concepts. For homework questions, it is designed to guide students with hints rather than giving out direct solutions. We assessed its performance first through a qualitative evaluation, then a survey-based comparative analysis, using a mix of questions commonly asked on the COMP202 discussion board and questions created by the authors. Our preliminary results indicate that the virtual TA outperforms human TAs on clarity and engagement, matching them on accuracy when the question is non-assignment-specific, for which human TAs still proved more reliable. These findings suggest that while virtual TAs, leveraging the capabilities of LLMs, hold great promise towards making CS education experience more accessible and engaging, their optimal use necessitates human supervision. We conclude by identifying several directions that could be explored in future implementations. © 2024 ACM.
Brahimi, T.; Sarirete, A.
Transforming learning in STEAM: How AI tools and language models catalyze educational advancement Book Section
In: Transformative Leadership and Sustainable Innovation in Education: Interdisciplinary Perspectives, pp. 39–58, Emerald Group Publishing Ltd., 2024, ISBN: 978-183753536-1 (ISBN); 978-183753537-8 (ISBN).
Abstract | Links | BibTeX | Tags: Accelerated learning, Constructionism, GPT, STEAM education, Technology-enhanced learning, Transformative learning
@incollection{brahimi_transforming_2024,
title = {Transforming learning in STEAM: How AI tools and language models catalyze educational advancement},
author = {T. Brahimi and A. Sarirete},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85197147759&doi=10.1108%2f978-1-83753-536-120241004&partnerID=40&md5=3d8d94a5aec57acc55d065792f037d3d},
doi = {10.1108/978-1-83753-536-120241004},
isbn = {978-183753536-1 (ISBN); 978-183753537-8 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Transformative Leadership and Sustainable Innovation in Education: Interdisciplinary Perspectives},
pages = {39–58},
publisher = {Emerald Group Publishing Ltd.},
abstract = {Technology-enhanced learning (TEL), particularly in science, technology, engineering, arts, and math (STEAM), revolutionizes educational approaches by fostering active, transformative learning and expediting the learning process. TEL employs various tools like online courses, artificial intelligence (AI) technologies, virtual reality (VR), simulations, makerspaces, visual learning, and project-based learning, all contributing to accelerated learning in STEAM. A notable TEL innovation is the emergence of Large Language Models (LLMs) and AI chatbots, exemplified by the release of GPT-3 in December 2022. These tools utilize extensive parameters to generate natural language and perform tasks such as classification and prediction, thereby offering personalized and collaborative learning experiences essential for STEAM education. The generative pre-training transformer (GPT), a leading model in natural language processing (NLP), excels in generating human-like text and handling complex tasks like translation, summarization, and question answering. This chapter explores TEL environments that support transformative learning in STEAM, focusing on AI models. It reviews research on TEL's impact on STEAM education, discussing the constructionism theory and emphasizing TEL's role in creating engaging, student-centered learning experiences. However, challenges like technology access, instructor training, infrastructure, internet connectivity, and hardware resources are crucial. Additionally, the rise of AI brings ethical concerns regarding privacy, security, and potential biases in AI algorithms. Despite these hurdles, TEL's potential to enhance Transformative Leadership and Sustainable Innovation in Education: Interdisciplinary Perspectives STEAM learning experiences and accelerate the educational process is significant. By effectively implementing TEL strategies and leveraging LLMs and AI tools, educators can substantially improve learning outcomes in STEAM education. © 2024 by Tayeb Brahimi and Akila Sarirete. All rights reserved.},
keywords = {Accelerated learning, Constructionism, GPT, STEAM education, Technology-enhanced learning, Transformative learning},
pubstate = {published},
tppubtype = {incollection}
}
Technology-enhanced learning (TEL), particularly in science, technology, engineering, arts, and math (STEAM), revolutionizes educational approaches by fostering active, transformative learning and expediting the learning process. TEL employs various tools like online courses, artificial intelligence (AI) technologies, virtual reality (VR), simulations, makerspaces, visual learning, and project-based learning, all contributing to accelerated learning in STEAM. A notable TEL innovation is the emergence of Large Language Models (LLMs) and AI chatbots, exemplified by the release of GPT-3 in December 2022. These tools utilize extensive parameters to generate natural language and perform tasks such as classification and prediction, thereby offering personalized and collaborative learning experiences essential for STEAM education. The generative pre-training transformer (GPT), a leading model in natural language processing (NLP), excels in generating human-like text and handling complex tasks like translation, summarization, and question answering. This chapter explores TEL environments that support transformative learning in STEAM, focusing on AI models. It reviews research on TEL's impact on STEAM education, discussing the constructionism theory and emphasizing TEL's role in creating engaging, student-centered learning experiences. However, challenges like technology access, instructor training, infrastructure, internet connectivity, and hardware resources are crucial. Additionally, the rise of AI brings ethical concerns regarding privacy, security, and potential biases in AI algorithms. Despite these hurdles, TEL's potential to enhance Transformative Leadership and Sustainable Innovation in Education: Interdisciplinary Perspectives STEAM learning experiences and accelerate the educational process is significant. By effectively implementing TEL strategies and leveraging LLMs and AI tools, educators can substantially improve learning outcomes in STEAM education. © 2024 by Tayeb Brahimi and Akila Sarirete. All rights reserved.
2023
Ayre, D.; Dougherty, C.; Zhao, Y.
IMPLEMENTATION OF AN ARTIFICIAL INTELLIGENCE (AI) INSTRUCTIONAL SUPPORT SYSTEM IN A VIRTUAL REALITY (VR) THERMAL-FLUIDS LABORATORY Proceedings Article
In: ASME Int Mech Eng Congress Expos Proc, American Society of Mechanical Engineers (ASME), 2023, ISBN: 978-079188765-3 (ISBN).
Abstract | Links | BibTeX | Tags: Artificial intelligence, E-Learning, Education computing, Engineering education, Fluid mechanics, Generative AI, generative artificial intelligence, GPT, High educations, Instructional support, Laboratories, Laboratory class, Laboratory experiments, Physical laboratory, Professional aspects, Students, Support systems, Thermal fluids, Virtual Reality, Virtual-reality environment
@inproceedings{ayre_implementation_2023,
title = {IMPLEMENTATION OF AN ARTIFICIAL INTELLIGENCE (AI) INSTRUCTIONAL SUPPORT SYSTEM IN A VIRTUAL REALITY (VR) THERMAL-FLUIDS LABORATORY},
author = {D. Ayre and C. Dougherty and Y. Zhao},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85185393784&doi=10.1115%2fIMECE2023-112683&partnerID=40&md5=c2492592a016478a4b3591ff82a93be5},
doi = {10.1115/IMECE2023-112683},
isbn = {978-079188765-3 (ISBN)},
year = {2023},
date = {2023-01-01},
booktitle = {ASME Int Mech Eng Congress Expos Proc},
volume = {8},
publisher = {American Society of Mechanical Engineers (ASME)},
abstract = {Physical laboratory experiments have long been the cornerstone of higher education, providing future engineers practical real-life experience invaluable to their careers. However, demand for laboratory time has exceeded physical capabilities. Virtual reality (VR) labs have proven to retain many benefits of attending physical labs while also providing significant advantages only available in a VR environment. Previously, our group had developed a pilot VR lab that replicated six (6) unique thermal-fluids lab experiments developed using the Unity game engine. One of the VR labs was tested in a thermal-fluid mechanics laboratory class with favorable results, but students highlighted the need for additional assistance within the VR simulation. In response to this testing, we have incorporated an artificial intelligence (AI) assistant to aid students within the VR environment by developing an interaction model. Utilizing the Generative Pre-trained Transformer 4 (GPT-4) large language model (LLM) and augmented context retrieval, the AI assistant can provide reliable instruction and troubleshoot errors while students conduct the lab procedure to provide an experience similar to a real-life lab assistant. The updated VR lab was tested in two laboratory classes and while the overall tone of student response to an AI-powered assistant was excitement and enthusiasm, observations and other recorded data show that students are currently unsure of how to utilize this new technology, which will help guide future refinement of AI components within the VR environment. © 2023 by ASME.},
keywords = {Artificial intelligence, E-Learning, Education computing, Engineering education, Fluid mechanics, Generative AI, generative artificial intelligence, GPT, High educations, Instructional support, Laboratories, Laboratory class, Laboratory experiments, Physical laboratory, Professional aspects, Students, Support systems, Thermal fluids, Virtual Reality, Virtual-reality environment},
pubstate = {published},
tppubtype = {inproceedings}
}
Physical laboratory experiments have long been the cornerstone of higher education, providing future engineers practical real-life experience invaluable to their careers. However, demand for laboratory time has exceeded physical capabilities. Virtual reality (VR) labs have proven to retain many benefits of attending physical labs while also providing significant advantages only available in a VR environment. Previously, our group had developed a pilot VR lab that replicated six (6) unique thermal-fluids lab experiments developed using the Unity game engine. One of the VR labs was tested in a thermal-fluid mechanics laboratory class with favorable results, but students highlighted the need for additional assistance within the VR simulation. In response to this testing, we have incorporated an artificial intelligence (AI) assistant to aid students within the VR environment by developing an interaction model. Utilizing the Generative Pre-trained Transformer 4 (GPT-4) large language model (LLM) and augmented context retrieval, the AI assistant can provide reliable instruction and troubleshoot errors while students conduct the lab procedure to provide an experience similar to a real-life lab assistant. The updated VR lab was tested in two laboratory classes and while the overall tone of student response to an AI-powered assistant was excitement and enthusiasm, observations and other recorded data show that students are currently unsure of how to utilize this new technology, which will help guide future refinement of AI components within the VR environment. © 2023 by ASME.