AHCI RESEARCH GROUP
Publications
Papers published in international journals,
proceedings of conferences, workshops and books.
OUR RESEARCH
Scientific Publications
How to
Here you can find the complete list of our publications.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
2024
Numan, N.; Rajaram, S.; Kumaravel, B. T.; Marquardt, N.; Wilson, A. D.
SpaceBlender: Creating Context-Rich Collaborative Spaces Through Generative 3D Scene Blending Proceedings Article
In: UIST - Proc. Annual ACM Symp. User Interface Softw. Technol., Association for Computing Machinery, Inc, 2024, ISBN: 979-840070628-8 (ISBN).
Abstract | Links | BibTeX | Tags: 3D modeling, 3D scenes, 3D spaces, AI techniques, Artificial environments, Collaborative spaces, Collaborative tasks, Generative adversarial networks, Generative AI, Telepresence, Virtual environments, Virtual Reality, Virtual reality telepresence, Virtual spaces, VR telepresence
@inproceedings{numan_spaceblender_2024,
title = {SpaceBlender: Creating Context-Rich Collaborative Spaces Through Generative 3D Scene Blending},
author = {N. Numan and S. Rajaram and B. T. Kumaravel and N. Marquardt and A. D. Wilson},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85209252034&doi=10.1145%2f3654777.3676361&partnerID=40&md5=8744057832f9098eabfd16c8b2b5fe62},
doi = {10.1145/3654777.3676361},
isbn = {979-840070628-8 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {UIST - Proc. Annual ACM Symp. User Interface Softw. Technol.},
publisher = {Association for Computing Machinery, Inc},
abstract = {There is increased interest in using generative AI to create 3D spaces for Virtual Reality (VR) applications. However, today's models produce artificial environments, falling short of supporting collaborative tasks that benefit from incorporating the user's physical context. To generate environments that support VR telepresence, we introduce SpaceBlender, a novel pipeline that utilizes generative AI techniques to blend users' physical surroundings into unified virtual spaces. This pipeline transforms user-provided 2D images into context-rich 3D environments through an iterative process consisting of depth estimation, mesh alignment, and diffusion-based space completion guided by geometric priors and adaptive text prompts. In a preliminary within-subjects study, where 20 participants performed a collaborative VR affinity diagramming task in pairs, we compared SpaceBlender with a generic virtual environment and a state-of-the-art scene generation framework, evaluating its ability to create virtual spaces suitable for collaboration. Participants appreciated the enhanced familiarity and context provided by SpaceBlender but also noted complexities in the generative environments that could detract from task focus. Drawing on participant feedback, we propose directions for improving the pipeline and discuss the value and design of blended spaces for different scenarios. © 2024 ACM.},
keywords = {3D modeling, 3D scenes, 3D spaces, AI techniques, Artificial environments, Collaborative spaces, Collaborative tasks, Generative adversarial networks, Generative AI, Telepresence, Virtual environments, Virtual Reality, Virtual reality telepresence, Virtual spaces, VR telepresence},
pubstate = {published},
tppubtype = {inproceedings}
}
There is increased interest in using generative AI to create 3D spaces for Virtual Reality (VR) applications. However, today's models produce artificial environments, falling short of supporting collaborative tasks that benefit from incorporating the user's physical context. To generate environments that support VR telepresence, we introduce SpaceBlender, a novel pipeline that utilizes generative AI techniques to blend users' physical surroundings into unified virtual spaces. This pipeline transforms user-provided 2D images into context-rich 3D environments through an iterative process consisting of depth estimation, mesh alignment, and diffusion-based space completion guided by geometric priors and adaptive text prompts. In a preliminary within-subjects study, where 20 participants performed a collaborative VR affinity diagramming task in pairs, we compared SpaceBlender with a generic virtual environment and a state-of-the-art scene generation framework, evaluating its ability to create virtual spaces suitable for collaboration. Participants appreciated the enhanced familiarity and context provided by SpaceBlender but also noted complexities in the generative environments that could detract from task focus. Drawing on participant feedback, we propose directions for improving the pipeline and discuss the value and design of blended spaces for different scenarios. © 2024 ACM.
2023
Horvath, I.; Csapo, A. B.
Structured Template Language and Generative AI Driven Content Management for Personalized Workspace Reconfiguration Proceedings Article
In: IEEE Int. Conf. Cogn. Asp. Virtual Real., CVR, Institute of Electrical and Electronics Engineers Inc., 2023, ISBN: 979-835033863-8 (ISBN).
Abstract | Links | BibTeX | Tags: 3D spaces, 3D virtual reality, Cognitive infocommunications, Content management, Content management solutions, Geometric layout, Knowledge engineering, Semantic content, Semantic content management, Semantics, Simple++, Virtual Reality, Work-flows
@inproceedings{horvath_structured_2023,
title = {Structured Template Language and Generative AI Driven Content Management for Personalized Workspace Reconfiguration},
author = {I. Horvath and A. B. Csapo},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85184849596&doi=10.1109%2fCVR58941.2023.10395520&partnerID=40&md5=c3890e80798c9ec542fe453875dde253},
doi = {10.1109/CVR58941.2023.10395520},
isbn = {979-835033863-8 (ISBN)},
year = {2023},
date = {2023-01-01},
booktitle = {IEEE Int. Conf. Cogn. Asp. Virtual Real., CVR},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {This work presents a systematic approach towards personalized workspace management and reconfiguration in 3D Virtual Reality (VR) spaces, focusing on a structured template language for defining and manipulating content layout schemas, as well as a generative AI supported content management solution. Recognizing the varying requirements of different tasks and workflows, on the one hand we propose a hierarchical template language that enables, through simple steps, the adaptation of number and variety of documents within geometric layout schemas in digital 3D spaces. In the second half of the paper, we present a generative AI driven framework for integrating associative-semantic content management into such workspaces, thereby enhancing the relevance and contextuality of workspace configurations. The proposed approach aids in identifying content that is semantically linked to a given spatial, temporal and topical environment, enabling workspace designers and users to create more efficient and personalized workspace layouts. © 2023 IEEE.},
keywords = {3D spaces, 3D virtual reality, Cognitive infocommunications, Content management, Content management solutions, Geometric layout, Knowledge engineering, Semantic content, Semantic content management, Semantics, Simple++, Virtual Reality, Work-flows},
pubstate = {published},
tppubtype = {inproceedings}
}
This work presents a systematic approach towards personalized workspace management and reconfiguration in 3D Virtual Reality (VR) spaces, focusing on a structured template language for defining and manipulating content layout schemas, as well as a generative AI supported content management solution. Recognizing the varying requirements of different tasks and workflows, on the one hand we propose a hierarchical template language that enables, through simple steps, the adaptation of number and variety of documents within geometric layout schemas in digital 3D spaces. In the second half of the paper, we present a generative AI driven framework for integrating associative-semantic content management into such workspaces, thereby enhancing the relevance and contextuality of workspace configurations. The proposed approach aids in identifying content that is semantically linked to a given spatial, temporal and topical environment, enabling workspace designers and users to create more efficient and personalized workspace layouts. © 2023 IEEE.
Vaidhyanathan, V.; Radhakrishnan, T. R.; López, J. L. G.
Spacify A Generative Framework for Spatial Comprehension, Articulation and Visualization using Large Language Models (LLMs) and eXtended Reality (XR) Proceedings Article
In: A., Crawford; N.M., Diniz; R., Beckett; J., Vanucchi; M., Swackhamer (Ed.): Habits Anthropocene: Scarcity Abundance Post-Mater. Econ. - Proc. Annu. Conf. Assoc. Comput. Aided Des. Archit., ACADIA, pp. 430–443, Association for Computer Aided Design in Architecture, 2023, ISBN: 979-898608059-8 (ISBN).
Abstract | Links | BibTeX | Tags: 3D data processing, 3D spaces, Architectural design, Built environment, C (programming language), Computational Linguistics, Computer aided design, Computer architecture, Data handling, Data users, Data visualization, Immersive media, Interior designers, Language Model, Natural languages, Spatial design, Three dimensional computer graphics, Urban designers, User interfaces, Visualization
@inproceedings{vaidhyanathan_spacify_2023,
title = {Spacify A Generative Framework for Spatial Comprehension, Articulation and Visualization using Large Language Models (LLMs) and eXtended Reality (XR)},
author = {V. Vaidhyanathan and T. R. Radhakrishnan and J. L. G. López},
editor = {Crawford A. and Diniz N.M. and Beckett R. and Vanucchi J. and Swackhamer M.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85192831586&partnerID=40&md5=996906de0f5ef1e6c88b10bb65caabc0},
isbn = {979-898608059-8 (ISBN)},
year = {2023},
date = {2023-01-01},
booktitle = {Habits Anthropocene: Scarcity Abundance Post-Mater. Econ. - Proc. Annu. Conf. Assoc. Comput. Aided Des. Archit., ACADIA},
volume = {2},
pages = {430–443},
publisher = {Association for Computer Aided Design in Architecture},
abstract = {Spatial design, the thoughtful planning and creation of built environments, typically requires advanced technical knowledge and visuospatial skills, making it largely exclusive to professionals like architects, interior designers, and urban designers. This exclusivity limits non-experts' access to spatial design, despite their ability to describe requirements and suggestions in natural language. Recent advancements in generative artificial intelligence (AI), particularly large language models (LLMs), and extended reality, (XR) offer the potential to address this limitation. This paper introduces Spacify (Figure 1), a framework that utilizes the generalizing capabilities of LLMs, 3D data-processing, and XR interfaces to create an immersive medium for language-driven spatial understanding, design, and visualization for non-experts. This paper describes the five components of Spacify: External Data, User Input, Spatial Interface, Large Language Model, and Current Spatial Design; which enable the use of generative AI models in a) question/ answering about 3D spaces with reasoning, b) (re)generating 3D spatial designs with natural language prompts, and c) visualizing designed 3D spaces with natural language descriptions. An implementation of Spacify is demonstrated via an XR smartphone application, allowing for an end-to-end, language-driven interior design process. User survey results from non-experts redesigning their spaces in 3D using this application suggest that Spacify can make spatial design accessible using natural language prompts, thereby pioneering a new realm of spatial design that is naturally language-driven. © ACADIA 2023. All rights reserved.},
keywords = {3D data processing, 3D spaces, Architectural design, Built environment, C (programming language), Computational Linguistics, Computer aided design, Computer architecture, Data handling, Data users, Data visualization, Immersive media, Interior designers, Language Model, Natural languages, Spatial design, Three dimensional computer graphics, Urban designers, User interfaces, Visualization},
pubstate = {published},
tppubtype = {inproceedings}
}
Spatial design, the thoughtful planning and creation of built environments, typically requires advanced technical knowledge and visuospatial skills, making it largely exclusive to professionals like architects, interior designers, and urban designers. This exclusivity limits non-experts' access to spatial design, despite their ability to describe requirements and suggestions in natural language. Recent advancements in generative artificial intelligence (AI), particularly large language models (LLMs), and extended reality, (XR) offer the potential to address this limitation. This paper introduces Spacify (Figure 1), a framework that utilizes the generalizing capabilities of LLMs, 3D data-processing, and XR interfaces to create an immersive medium for language-driven spatial understanding, design, and visualization for non-experts. This paper describes the five components of Spacify: External Data, User Input, Spatial Interface, Large Language Model, and Current Spatial Design; which enable the use of generative AI models in a) question/ answering about 3D spaces with reasoning, b) (re)generating 3D spatial designs with natural language prompts, and c) visualizing designed 3D spaces with natural language descriptions. An implementation of Spacify is demonstrated via an XR smartphone application, allowing for an end-to-end, language-driven interior design process. User survey results from non-experts redesigning their spaces in 3D using this application suggest that Spacify can make spatial design accessible using natural language prompts, thereby pioneering a new realm of spatial design that is naturally language-driven. © ACADIA 2023. All rights reserved.