AHCI RESEARCH GROUP
Publications
Papers published in international journals,
proceedings of conferences, workshops and books.
OUR RESEARCH
Scientific Publications
How to
Here you can find the complete list of our publications.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
2025
Li, C.; Da, F.
Refined dense face alignment through image matching Journal Article
In: Visual Computer, vol. 41, no. 1, pp. 157–171, 2025, ISSN: 01782789 (ISSN).
Abstract | Links | BibTeX | Tags: 3D Avatars, Alignment, Dense geometric supervision, Face alignment, Face deformations, Face reconstruction, Geometry, Human computer interaction, Image enhancement, Image matching, Image Reconstruction, Metaverses, Outlier mixup, Pixels, Rendered images, Rendering (computer graphics), State of the art, Statistics, Target images, Three dimensional computer graphics
@article{li_refined_2025,
title = {Refined dense face alignment through image matching},
author = {C. Li and F. Da},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85187924785&doi=10.1007%2fs00371-024-03316-3&partnerID=40&md5=839834c6ff3320398d5ef75b055947cb},
doi = {10.1007/s00371-024-03316-3},
issn = {01782789 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {Visual Computer},
volume = {41},
number = {1},
pages = {157–171},
abstract = {Face alignment is the foundation of building 3D avatars for virtue communication in the metaverse, human-computer interaction, AI-generated content, etc., and therefore, it is critical that face deformation is reflected precisely to better convey expression, pose and identity. However, misalignment exists in the currently best methods that fit a face model to a target image and can be easily captured by human perception, thus degrading the reconstruction quality. The main reason is that the widely used metrics for training, including the landmark re-projection loss, pixel-wise loss and perception-level loss, are insufficient to address the misalignment and suffer from ambiguity and local minimums. To address misalignment, we propose an image MAtchinG-driveN dEnse geomeTrIC supervision (MAGNETIC). Specifically, we treat face alignment as a matching problem and establish pixel-wise correspondences between the target and rendered images. Then reconstructed facial points are guided towards their corresponding points on the target image, thus improving reconstruction. Synthesized image pairs are mixed up with face outliers to simulate the target and rendered images with ground-truth pixel-wise correspondences to enable the training of a robust prediction network. Compared with existing methods that turn to 3D scans for dense geometric supervision, our method reaches comparable shape reconstruction results with much lower effort. Experimental results on the NoW testset show that we reach the state-of-the-art among all self-supervised methods and even outperform methods using photo-realistic images. We also achieve comparable results with the state-of-the-art on the benchmark of Feng et al. Codes will be available at: github.com/ChunLLee/ReconstructionFromMatching. © The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2024.},
keywords = {3D Avatars, Alignment, Dense geometric supervision, Face alignment, Face deformations, Face reconstruction, Geometry, Human computer interaction, Image enhancement, Image matching, Image Reconstruction, Metaverses, Outlier mixup, Pixels, Rendered images, Rendering (computer graphics), State of the art, Statistics, Target images, Three dimensional computer graphics},
pubstate = {published},
tppubtype = {article}
}
Face alignment is the foundation of building 3D avatars for virtue communication in the metaverse, human-computer interaction, AI-generated content, etc., and therefore, it is critical that face deformation is reflected precisely to better convey expression, pose and identity. However, misalignment exists in the currently best methods that fit a face model to a target image and can be easily captured by human perception, thus degrading the reconstruction quality. The main reason is that the widely used metrics for training, including the landmark re-projection loss, pixel-wise loss and perception-level loss, are insufficient to address the misalignment and suffer from ambiguity and local minimums. To address misalignment, we propose an image MAtchinG-driveN dEnse geomeTrIC supervision (MAGNETIC). Specifically, we treat face alignment as a matching problem and establish pixel-wise correspondences between the target and rendered images. Then reconstructed facial points are guided towards their corresponding points on the target image, thus improving reconstruction. Synthesized image pairs are mixed up with face outliers to simulate the target and rendered images with ground-truth pixel-wise correspondences to enable the training of a robust prediction network. Compared with existing methods that turn to 3D scans for dense geometric supervision, our method reaches comparable shape reconstruction results with much lower effort. Experimental results on the NoW testset show that we reach the state-of-the-art among all self-supervised methods and even outperform methods using photo-realistic images. We also achieve comparable results with the state-of-the-art on the benchmark of Feng et al. Codes will be available at: github.com/ChunLLee/ReconstructionFromMatching. © The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2024.
2024
Martini, M.; Valentini, V.; Ciprian, A.; Bottino, A.; Iacoviello, R.; Montagnuolo, M.; Messina, A.; Strada, F.; Zappia, D.
Semi -Automated Digital Human Production for Enhanced Media Broadcasting Proceedings Article
In: IEEE Gaming, Entertain., Media Conf., GEM, Institute of Electrical and Electronics Engineers Inc., 2024, ISBN: 979-835037453-7 (ISBN).
Abstract | Links | BibTeX | Tags: AI automation, Automation, Creation process, Digital humans, Economic and social effects, Extensive explorations, Face reconstruction, Generative AI, Image enhancement, media archive, Media archives, Metaverses, Rendering (computer graphics), Synthetic human, Synthetic Humans, Textures, Three dimensional computer graphics, Virtual production, Virtual Reality
@inproceedings{martini_semi_2024,
title = {Semi -Automated Digital Human Production for Enhanced Media Broadcasting},
author = {M. Martini and V. Valentini and A. Ciprian and A. Bottino and R. Iacoviello and M. Montagnuolo and A. Messina and F. Strada and D. Zappia},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85199536742&doi=10.1109%2fGEM61861.2024.10585601&partnerID=40&md5=3703fba931b02f9615316db8ebbca70c},
doi = {10.1109/GEM61861.2024.10585601},
isbn = {979-835037453-7 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {IEEE Gaming, Entertain., Media Conf., GEM},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {In recent years, the application of synthetic humans in various fields has attracted considerable attention, leading to extensive exploration of their integration into the Metaverse and virtual production environments. This work presents a semi-automated approach that aims to find a fair trade-off between high-quality outputs and efficient production times. The project focuses on the Rai photo and video archives to find images of target characters for texturing and 3D reconstruction with the goal of reviving Rai's 2D footage and enhance the media experience. A key aspect of this study is to minimize the human intervention, ensuring an efficient, flexible, and scalable creation process. In this work, the improvements have been distributed among different stages of the digital human creation process, starting with the generation of 3D head meshes from 2D images of the reference character and then moving on to the generation, using a Diffusion model, of suitable images for texture development. These assets are then integrated into the Unreal Engine, where a custom widget facilitates posing, rendering, and texturing of Synthetic Humans models. Finally, an in-depth quantitative comparison and subjective tests were carried out between the original character images and the rendered synthetic humans, confirming the validity of the approach. © 2024 IEEE.},
keywords = {AI automation, Automation, Creation process, Digital humans, Economic and social effects, Extensive explorations, Face reconstruction, Generative AI, Image enhancement, media archive, Media archives, Metaverses, Rendering (computer graphics), Synthetic human, Synthetic Humans, Textures, Three dimensional computer graphics, Virtual production, Virtual Reality},
pubstate = {published},
tppubtype = {inproceedings}
}
In recent years, the application of synthetic humans in various fields has attracted considerable attention, leading to extensive exploration of their integration into the Metaverse and virtual production environments. This work presents a semi-automated approach that aims to find a fair trade-off between high-quality outputs and efficient production times. The project focuses on the Rai photo and video archives to find images of target characters for texturing and 3D reconstruction with the goal of reviving Rai's 2D footage and enhance the media experience. A key aspect of this study is to minimize the human intervention, ensuring an efficient, flexible, and scalable creation process. In this work, the improvements have been distributed among different stages of the digital human creation process, starting with the generation of 3D head meshes from 2D images of the reference character and then moving on to the generation, using a Diffusion model, of suitable images for texture development. These assets are then integrated into the Unreal Engine, where a custom widget facilitates posing, rendering, and texturing of Synthetic Humans models. Finally, an in-depth quantitative comparison and subjective tests were carried out between the original character images and the rendered synthetic humans, confirming the validity of the approach. © 2024 IEEE.
Na, M.; Lee, J.
Generative AI-Enabled Energy-Efficient Mobile Augmented Reality in Multi-Access Edge Computing Journal Article
In: Applied Sciences (Switzerland), vol. 14, no. 18, 2024, ISSN: 20763417 (ISSN).
Abstract | Links | BibTeX | Tags: Artificial intelligence technologies, Augmented Reality, benchmarking, Computation offloading, Edge computing, Energy Efficient, Generative adversarial networks, Generative AI, Image enhancement, Mobile augmented reality, Mobile edge computing, Multi-access edge computing, Multiaccess, Quality of Service, Resolution process, super-resolution, Superresolution, Trade off
@article{na_generative_2024,
title = {Generative AI-Enabled Energy-Efficient Mobile Augmented Reality in Multi-Access Edge Computing},
author = {M. Na and J. Lee},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85205236316&doi=10.3390%2fapp14188419&partnerID=40&md5=0aa1c42cb7343cfb55a9dc1e66494dc6},
doi = {10.3390/app14188419},
issn = {20763417 (ISSN)},
year = {2024},
date = {2024-01-01},
journal = {Applied Sciences (Switzerland)},
volume = {14},
number = {18},
abstract = {This paper proposes a novel offloading and super-resolution (SR) control scheme for energy-efficient mobile augmented reality (MAR) in multi-access edge computing (MEC) using SR as a promising generative artificial intelligence (GAI) technology. Specifically, SR can enhance low-resolution images into high-resolution versions using GAI technologies. This capability is particularly advantageous in MAR by lowering the bitrate required for network transmission. However, this SR process requires considerable computational resources and can introduce latency, potentially overloading the MEC server if there are numerous offload requests for MAR services. In this context, we conduct an empirical study to verify that the computational latency of SR increases with the upscaling level. Therefore, we demonstrate a trade-off between computational latency and improved service satisfaction when upscaling images for object detection, as it enhances the detection accuracy. From this perspective, determining whether to apply SR for MAR, while jointly controlling offloading decisions, is challenging. Consequently, to design energy-efficient MAR, we rigorously formulate analytical models for the energy consumption of a MAR device, the overall latency and the MAR satisfaction of service quality from the enforcement of the service accuracy, taking into account the SR process at the MEC server. Finally, we develop a theoretical framework that optimizes the computation offloading and SR control problem for MAR clients by jointly optimizing the offloading and SR decisions, considering their trade-off in MAR with MEC. Finally, the performance evaluation indicates that our proposed framework effectively supports MAR services by efficiently managing offloading and SR decisions, balancing trade-offs between energy consumption, latency, and service satisfaction compared to benchmarks. © 2024 by the authors.},
keywords = {Artificial intelligence technologies, Augmented Reality, benchmarking, Computation offloading, Edge computing, Energy Efficient, Generative adversarial networks, Generative AI, Image enhancement, Mobile augmented reality, Mobile edge computing, Multi-access edge computing, Multiaccess, Quality of Service, Resolution process, super-resolution, Superresolution, Trade off},
pubstate = {published},
tppubtype = {article}
}
This paper proposes a novel offloading and super-resolution (SR) control scheme for energy-efficient mobile augmented reality (MAR) in multi-access edge computing (MEC) using SR as a promising generative artificial intelligence (GAI) technology. Specifically, SR can enhance low-resolution images into high-resolution versions using GAI technologies. This capability is particularly advantageous in MAR by lowering the bitrate required for network transmission. However, this SR process requires considerable computational resources and can introduce latency, potentially overloading the MEC server if there are numerous offload requests for MAR services. In this context, we conduct an empirical study to verify that the computational latency of SR increases with the upscaling level. Therefore, we demonstrate a trade-off between computational latency and improved service satisfaction when upscaling images for object detection, as it enhances the detection accuracy. From this perspective, determining whether to apply SR for MAR, while jointly controlling offloading decisions, is challenging. Consequently, to design energy-efficient MAR, we rigorously formulate analytical models for the energy consumption of a MAR device, the overall latency and the MAR satisfaction of service quality from the enforcement of the service accuracy, taking into account the SR process at the MEC server. Finally, we develop a theoretical framework that optimizes the computation offloading and SR control problem for MAR clients by jointly optimizing the offloading and SR decisions, considering their trade-off in MAR with MEC. Finally, the performance evaluation indicates that our proposed framework effectively supports MAR services by efficiently managing offloading and SR decisions, balancing trade-offs between energy consumption, latency, and service satisfaction compared to benchmarks. © 2024 by the authors.
2023
Wang, Z.; Joshi, A.; Zhang, G.; Ren, W.; Jia, F.; Sun, X.
Elevating Perception: Unified Recognition Framework and Vision-Language Pre-Training Using Three-Dimensional Image Reconstruction Proceedings Article
In: Proc. - Int. Conf. Artif. Intell., Human-Comput. Interact. Robot., AIHCIR, pp. 592–596, Institute of Electrical and Electronics Engineers Inc., 2023, ISBN: 979-835036036-3 (ISBN).
Abstract | Links | BibTeX | Tags: 3D Model LLM, 3D modeling, 3D models, 3D Tech, 3d-modeling, Augmented Reality, Character recognition, Component, Computer aided design, Computer vision, Continuous time systems, Data handling, Generative AI, Image enhancement, Image Reconstruction, Image to Text Generation, Medical Imaging, Pattern recognition, Pre-training, Reconstructive Training, Text generations, Three dimensional computer graphics, Virtual Reality
@inproceedings{wang_elevating_2023,
title = {Elevating Perception: Unified Recognition Framework and Vision-Language Pre-Training Using Three-Dimensional Image Reconstruction},
author = {Z. Wang and A. Joshi and G. Zhang and W. Ren and F. Jia and X. Sun},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85192837757&doi=10.1109%2fAIHCIR61661.2023.00105&partnerID=40&md5=0fe17cc622a9aa90e88b8c3e6a3bed3b},
doi = {10.1109/AIHCIR61661.2023.00105},
isbn = {979-835036036-3 (ISBN)},
year = {2023},
date = {2023-01-01},
booktitle = {Proc. - Int. Conf. Artif. Intell., Human-Comput. Interact. Robot., AIHCIR},
pages = {592–596},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {This research project explores a paradigm shift in perceptual enhancement by integrating a Unified Recognition Framework and Vision-Language Pre-Training in three-dimensional image reconstruction. Through the synergy of advanced algorithms from computer vision & language processing, the project tries to enhance the precision and depth of perception in reconstructed images. This innovative approach holds the potential to revolutionize fields such as medical imaging, virtual reality, and computer-aided design, providing a comprehensive perspective on the intersection of multimodal data processing and perceptual advancement. The anticipated research outcomes are expected to significantly contribute to the evolution of technologies that rely on accurate and contextually rich three-dimensional reconstructions. Moreover, the research aims to reduce the constant need for new datasets by improving pattern recognition through 3D image patterning on backpropagation. This continuous improvement of vectors is envisioned to enhance the efficiency and accuracy of pattern recognition, contributing to the optimization of perceptual systems over time. © 2023 IEEE.},
keywords = {3D Model LLM, 3D modeling, 3D models, 3D Tech, 3d-modeling, Augmented Reality, Character recognition, Component, Computer aided design, Computer vision, Continuous time systems, Data handling, Generative AI, Image enhancement, Image Reconstruction, Image to Text Generation, Medical Imaging, Pattern recognition, Pre-training, Reconstructive Training, Text generations, Three dimensional computer graphics, Virtual Reality},
pubstate = {published},
tppubtype = {inproceedings}
}
This research project explores a paradigm shift in perceptual enhancement by integrating a Unified Recognition Framework and Vision-Language Pre-Training in three-dimensional image reconstruction. Through the synergy of advanced algorithms from computer vision & language processing, the project tries to enhance the precision and depth of perception in reconstructed images. This innovative approach holds the potential to revolutionize fields such as medical imaging, virtual reality, and computer-aided design, providing a comprehensive perspective on the intersection of multimodal data processing and perceptual advancement. The anticipated research outcomes are expected to significantly contribute to the evolution of technologies that rely on accurate and contextually rich three-dimensional reconstructions. Moreover, the research aims to reduce the constant need for new datasets by improving pattern recognition through 3D image patterning on backpropagation. This continuous improvement of vectors is envisioned to enhance the efficiency and accuracy of pattern recognition, contributing to the optimization of perceptual systems over time. © 2023 IEEE.
Friess, P.
THE_OTHERVERSE - A Contemporary Cabinet Of Curiosities Proceedings Article
In: D., Byrne; N., Martelaro (Ed.): DIS Companion: Companion Publ. ACM Des Interact. Syst. Conf., pp. 50–54, Association for Computing Machinery, Inc, 2023, ISBN: 978-145039898-5 (ISBN).
Abstract | Links | BibTeX | Tags: 'current, AI-generated content, Algorithmic energy, Algorithmics, Arts computing, Energy, Form of existences, Image enhancement, Interactive computer graphics, Metaverse, Metaverses, Other verse, Other verses, Resilience, Virtual Reality, Virtual worlds
@inproceedings{friess_the_otherverse_2023,
title = {THE_OTHERVERSE - A Contemporary Cabinet Of Curiosities},
author = {P. Friess},
editor = {Byrne D. and Martelaro N.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85167658209&doi=10.1145%2f3563703.3596803&partnerID=40&md5=d68cab2e32a1581efd5b734b67b0f88b},
doi = {10.1145/3563703.3596803},
isbn = {978-145039898-5 (ISBN)},
year = {2023},
date = {2023-01-01},
booktitle = {DIS Companion: Companion Publ. ACM Des Interact. Syst. Conf.},
pages = {50–54},
publisher = {Association for Computing Machinery, Inc},
abstract = {THE_OTHERVERSE is an artwork exploring resilience as a proactive artistic attitude. Inspired by the revisited idea of a "Wunderkammer"for multidisciplinary exploration and as a representation of microcosms of the world, the artwork features a contemporary cabinet of curiosities with AI-generated content of other forms of existence and expands the current Metaverse paradigm including emphasizing algorithm energy sound and rhythm. The research-creation process includes storytelling, object creation, virtual environment design, sound creation, and image enhancement, blending the aesthetics of the obtained results with the tools used for the creation. Understanding and interacting with AI as a creative partner opens up new possibilities for future research-creation, both for the research part in providing collective knowledge as for the creation part to propose a machine-thinking inspired recombination of ideas. Resilience is not only achieved by how we respond to bad things, but also how we broaden our possibilities (https://vimeo.com/petermfriess/the-otherverse). © 2023 ACM.},
keywords = {'current, AI-generated content, Algorithmic energy, Algorithmics, Arts computing, Energy, Form of existences, Image enhancement, Interactive computer graphics, Metaverse, Metaverses, Other verse, Other verses, Resilience, Virtual Reality, Virtual worlds},
pubstate = {published},
tppubtype = {inproceedings}
}
THE_OTHERVERSE is an artwork exploring resilience as a proactive artistic attitude. Inspired by the revisited idea of a "Wunderkammer"for multidisciplinary exploration and as a representation of microcosms of the world, the artwork features a contemporary cabinet of curiosities with AI-generated content of other forms of existence and expands the current Metaverse paradigm including emphasizing algorithm energy sound and rhythm. The research-creation process includes storytelling, object creation, virtual environment design, sound creation, and image enhancement, blending the aesthetics of the obtained results with the tools used for the creation. Understanding and interacting with AI as a creative partner opens up new possibilities for future research-creation, both for the research part in providing collective knowledge as for the creation part to propose a machine-thinking inspired recombination of ideas. Resilience is not only achieved by how we respond to bad things, but also how we broaden our possibilities (https://vimeo.com/petermfriess/the-otherverse). © 2023 ACM.