AHCI RESEARCH GROUP
Publications
Papers published in international journals,
proceedings of conferences, workshops and books.
OUR RESEARCH
Scientific Publications
How to
Here you can find the complete list of our publications.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
You can use the tag cloud to select only the papers dealing with specific research topics.
You can expand the Abstract, Links and BibTex record for each paper.
2025
Kai, W. -H.; Xing, K. -X.
Video-driven musical composition using large language model with memory-augmented state space Journal Article
In: Visual Computer, vol. 41, no. 5, pp. 3345–3357, 2025, ISSN: 01782789 (ISSN).
Abstract | Links | BibTeX | Tags: 'current, Associative storage, Augmented Reality, Augmented state space, Computer simulation languages, Computer system recovery, Distributed computer systems, HTTP, Language Model, Large language model, Long-term video-to-music generation, Mamba, Memory architecture, Memory-augmented, Modeling languages, Music, Musical composition, Natural language processing systems, Object oriented programming, Performance, Problem oriented languages, State space, State-space
@article{kai_video-driven_2025,
title = {Video-driven musical composition using large language model with memory-augmented state space},
author = {W. -H. Kai and K. -X. Xing},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-105001073242&doi=10.1007%2fs00371-024-03606-w&partnerID=40&md5=7ea24f13614a9a24caf418c37a10bd8c},
doi = {10.1007/s00371-024-03606-w},
issn = {01782789 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {Visual Computer},
volume = {41},
number = {5},
pages = {3345–3357},
abstract = {The current landscape of research leveraging large language models (LLMs) is experiencing a surge. Many works harness the powerful reasoning capabilities of these models to comprehend various modalities, such as text, speech, images, videos, etc. However, the research work on LLms for music inspiration is still in its infancy. To fill the gap in this field and break through the dilemma that LLMs can only understand short videos with limited frames, we propose a large language model with state space for long-term video-to-music generation. To capture long-range dependency and maintaining high performance, while further decrease the computing cost, our overall network includes the Enhanced Video Mamba, which incorporates continuous moving window partitioning and local feature augmentation, and a long-term memory bank that captures and aggregates historical video information to mitigate information loss in long sequences. This framework achieves both subquadratic-time computation and near-linear memory complexity, enabling effective long-term video-to-music generation. We conduct a thorough evaluation of our proposed framework. The experimental results demonstrate that our model achieves or surpasses the performance of the current state-of-the-art models. Our code released on https://github.com/kai211233/S2L2-V2M. © The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2024.},
keywords = {'current, Associative storage, Augmented Reality, Augmented state space, Computer simulation languages, Computer system recovery, Distributed computer systems, HTTP, Language Model, Large language model, Long-term video-to-music generation, Mamba, Memory architecture, Memory-augmented, Modeling languages, Music, Musical composition, Natural language processing systems, Object oriented programming, Performance, Problem oriented languages, State space, State-space},
pubstate = {published},
tppubtype = {article}
}
The current landscape of research leveraging large language models (LLMs) is experiencing a surge. Many works harness the powerful reasoning capabilities of these models to comprehend various modalities, such as text, speech, images, videos, etc. However, the research work on LLms for music inspiration is still in its infancy. To fill the gap in this field and break through the dilemma that LLMs can only understand short videos with limited frames, we propose a large language model with state space for long-term video-to-music generation. To capture long-range dependency and maintaining high performance, while further decrease the computing cost, our overall network includes the Enhanced Video Mamba, which incorporates continuous moving window partitioning and local feature augmentation, and a long-term memory bank that captures and aggregates historical video information to mitigate information loss in long sequences. This framework achieves both subquadratic-time computation and near-linear memory complexity, enabling effective long-term video-to-music generation. We conduct a thorough evaluation of our proposed framework. The experimental results demonstrate that our model achieves or surpasses the performance of the current state-of-the-art models. Our code released on https://github.com/kai211233/S2L2-V2M. © The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2024.
2024
Michael, Z.; Gemeinhardt, J.; Moritz, K.
Interactive WebXR Hypertext Storytelling for Cultural Heritage Proceedings Article
In: C., Atzenbeck; J., Rubart (Ed.): Proc. Workshop Hum. Factors Hypertext, Hum. - Assoc. ACM Conf. Hypertext Soc. Media ,HT, Association for Computing Machinery, Inc, 2024, ISBN: 979-840071120-6 (ISBN).
Abstract | Links | BibTeX | Tags: 2D textures, 3D modeling, 3D models, 3d-modeling, Cultural heritage, Cultural heritages, Extended reality (XR), Generative AI, History, HTTP, Hypertext, Hypertext systems, Immersive, Machine-learning, Open source software, Open systems, Scene structure, Three dimensional computer graphics, Virtual environments, Virtual Reality, Web browsers
@inproceedings{michael_interactive_2024,
title = {Interactive WebXR Hypertext Storytelling for Cultural Heritage},
author = {Z. Michael and J. Gemeinhardt and K. Moritz},
editor = {Atzenbeck C. and Rubart J.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85211623904&doi=10.1145%2f3679058.3688635&partnerID=40&md5=60aad5a9a95e52c3fff51ebb6f670bd6},
doi = {10.1145/3679058.3688635},
isbn = {979-840071120-6 (ISBN)},
year = {2024},
date = {2024-01-01},
booktitle = {Proc. Workshop Hum. Factors Hypertext, Hum. - Assoc. ACM Conf. Hypertext Soc. Media ,HT},
publisher = {Association for Computing Machinery, Inc},
abstract = {We are presenting our approach for interactive cultural heritage storytelling in WebXR. Therefore, we are describing our scenes’ structure consisting of (stylized) photospheres of the historic locations, 3D models of 3D-scanned historic artifacts and animated 2D textures of historic characters generated with a machine learning toolset. The result is a platform-independent web-application in an immersive interactive WebXR environment running in browsers on PCs, tablets, phones and XR headsets thanks to the underlying software based on the open-source framework A-Frame. Our paper describes the process, the results and the limitations in detail. The resulting application, designed for the Fichtelgebirge region in Upper Franconia, Germany, offers users an immersive digital time travel experience in the virtual space and within a museum setting connecting real artifacts and virtual stories. © 2024 Copyright held by the owner/author(s).},
keywords = {2D textures, 3D modeling, 3D models, 3d-modeling, Cultural heritage, Cultural heritages, Extended reality (XR), Generative AI, History, HTTP, Hypertext, Hypertext systems, Immersive, Machine-learning, Open source software, Open systems, Scene structure, Three dimensional computer graphics, Virtual environments, Virtual Reality, Web browsers},
pubstate = {published},
tppubtype = {inproceedings}
}
We are presenting our approach for interactive cultural heritage storytelling in WebXR. Therefore, we are describing our scenes’ structure consisting of (stylized) photospheres of the historic locations, 3D models of 3D-scanned historic artifacts and animated 2D textures of historic characters generated with a machine learning toolset. The result is a platform-independent web-application in an immersive interactive WebXR environment running in browsers on PCs, tablets, phones and XR headsets thanks to the underlying software based on the open-source framework A-Frame. Our paper describes the process, the results and the limitations in detail. The resulting application, designed for the Fichtelgebirge region in Upper Franconia, Germany, offers users an immersive digital time travel experience in the virtual space and within a museum setting connecting real artifacts and virtual stories. © 2024 Copyright held by the owner/author(s).
2010
Scianna, Andrea
Experimental studies for the definition of 3D geospatial web services Proceedings Article
In: S., Dragicevic S. Veenendaal B. Li (Ed.): International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives, International Society for Photogrammetry and Remote Sensing, 2010, (Issue: 4W13).
Abstract | Links | BibTeX | Tags: Application programs, Computer software, Cooperative Management, Distributed database systems, Geo-spatial, Geo-Spatial Informations, Geographic data model, Geographic information, Geographic information systems, Geographical Information Systems, Geospatial web services, HTTP, Internet, Interoperability, Java programming language, Open source software, Open systems, Project management, Query processing, Social networking (online), Standard definitions, Three dimensional computer graphics, Web, Web services, Websites, World Wide Web
@inproceedings{scianna_experimental_2010,
title = {Experimental studies for the definition of 3D geospatial web services},
author = {Andrea Scianna},
editor = {Dragicevic S. Veenendaal B. Li S.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-84923930269&partnerID=40&md5=eb36183ca2f0842ee0fd1a0164dc4cb0},
year = {2010},
date = {2010-01-01},
booktitle = {International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives},
volume = {38},
publisher = {International Society for Photogrammetry and Remote Sensing},
abstract = {The importance of geospatial information delivery, across Internet, is increasing more and more. But if in the last years was satisfying to get 2D geographic information, from Internet browsers, extracted by http servers supported by map servers, today users would acquire 3D information especially in some case or sectors as that of building cadastral applications or civil protection. In this case availability of 3D information could be very valuable. But behind this need there exist many important aspects that today are only partially resolved as the definition of 3D geospatial web services that are connected and are dependent on some other aspects (i.e. the standardization of 3D geographic data models). The experimentation carried out deals with these issues, trying to define a 3D web service in order to visualize and query by Internt Browser 3D model of the built environment. Actually there are few 3D data models (such as CityGML) whose standard definition process is not complete. Besides these models are built with main requirement of 3D visualization even if at different level of detail. But some sectors require a better use of geographic 3D information such as querying at different level of detail (such as at a level regarding the different building parts defined through their attributes) and 3D processing. Based on a project named "Interoperability and cooperative management of geographic, dynamic, multidimensional and distributed data with Free and Open Source GIS: Management and use of distributed 3D data by open source Web-GIS software" funded by Italian Ministry of Instruction, University and Research as Program of Relevant National Interest (PRIN 2007), it is here illustrated a part of a process, that start from the construction of a Java plugin that initially consist in reading information directly from a relational database management server with spatial extension, and ends with the construction of an application server which is based the extraction of GML 3D data, all based on the existence of a 3D geospatial web service whose definition is one the last and main goal of the research. © 2010 ISPRS Archives.},
note = {Issue: 4W13},
keywords = {Application programs, Computer software, Cooperative Management, Distributed database systems, Geo-spatial, Geo-Spatial Informations, Geographic data model, Geographic information, Geographic information systems, Geographical Information Systems, Geospatial web services, HTTP, Internet, Interoperability, Java programming language, Open source software, Open systems, Project management, Query processing, Social networking (online), Standard definitions, Three dimensional computer graphics, Web, Web services, Websites, World Wide Web},
pubstate = {published},
tppubtype = {inproceedings}
}
The importance of geospatial information delivery, across Internet, is increasing more and more. But if in the last years was satisfying to get 2D geographic information, from Internet browsers, extracted by http servers supported by map servers, today users would acquire 3D information especially in some case or sectors as that of building cadastral applications or civil protection. In this case availability of 3D information could be very valuable. But behind this need there exist many important aspects that today are only partially resolved as the definition of 3D geospatial web services that are connected and are dependent on some other aspects (i.e. the standardization of 3D geographic data models). The experimentation carried out deals with these issues, trying to define a 3D web service in order to visualize and query by Internt Browser 3D model of the built environment. Actually there are few 3D data models (such as CityGML) whose standard definition process is not complete. Besides these models are built with main requirement of 3D visualization even if at different level of detail. But some sectors require a better use of geographic 3D information such as querying at different level of detail (such as at a level regarding the different building parts defined through their attributes) and 3D processing. Based on a project named "Interoperability and cooperative management of geographic, dynamic, multidimensional and distributed data with Free and Open Source GIS: Management and use of distributed 3D data by open source Web-GIS software" funded by Italian Ministry of Instruction, University and Research as Program of Relevant National Interest (PRIN 2007), it is here illustrated a part of a process, that start from the construction of a Java plugin that initially consist in reading information directly from a relational database management server with spatial extension, and ends with the construction of an application server which is based the extraction of GML 3D data, all based on the existence of a 3D geospatial web service whose definition is one the last and main goal of the research. © 2010 ISPRS Archives.
Scianna, Andrea; Ammoscato, Alessio
3D gis data model using open source software Proceedings Article
In: A, Peled (Ed.): International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives, pp. 120–125, International Society for Photogrammetry and Remote Sensing, 2010.
Abstract | Links | BibTeX | Tags: 3-dimensional modeling, 3D Modelling, Blending, Computer software, Data visualization, Database systems, Environmental database, Environmental Technology, Free and open source softwares, Geographic information systems, Geographical Information Systems, High level languages, HTTP, Internet, Internet browsers, Internet protocols, Interoperability, Maintenance, Mapping, Maps, Open source software, Open systems, Query languages, Research management, Social networking (online), Software engineering, Spatial, Technology, Three dimensional computer graphics, Three-dimensional data, Topological information, Topology, World Wide Web
@inproceedings{scianna_3d_2010,
title = {3D gis data model using open source software},
author = {Andrea Scianna and Alessio Ammoscato},
editor = {Peled A},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-84880227655&partnerID=40&md5=502aa042af1693c18f34b5d74c4dd2bd},
year = {2010},
date = {2010-01-01},
booktitle = {International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives},
volume = {38},
pages = {120–125},
publisher = {International Society for Photogrammetry and Remote Sensing},
abstract = {Today many kinds of applications requires data containing actual three-dimensional data; fields like urban and town planning and pollution studies need 3D data, both for visualization purpose, as well as carry out many spatial analysis. This research-Management and use of distributed 3D data by open source Web-GIS software-is part of the Italian "PRIN 2007"∗ research project, aimed to build urban and suburban 3D models, and to interact with them using open source software only. Particularly free and open source software, used for the experimentation here shown, are Blender and PostGIS; the first one has been used to build and structure three-dimensional data, the second one for data allocation. These software interact using scripts, written in Python language. Buildings have been modeled upon the GIANT3D model (Geographical Interoperable Advanced Numerical Topological 3-Dimensional Model) developed in the research "PRIN 2004", regarding "Evolved structure of numerical cartography for Gis and Web-GIS". Python scripts, activated by Blender, allow to allocate data into a spatial database implemented through PostgreSQL and PostGis, that could be a remote database somewhere on the net; all geometrical and topological information, implemented in the 3D model, are so transferred in PostGIS. These information can be retrieved by Blender using other Python scripts, so Blender fully interacts with 3D data allocated in PostGIS. These data can be also accessed by many other clients, both directly using a database client, as using other protocols (like HTTP on the internet). Next step is to build an open source viewer, or a plugin for internet browsers, that allows client to visualize, explore and inquiry 3D model, retrieving data from database.},
keywords = {3-dimensional modeling, 3D Modelling, Blending, Computer software, Data visualization, Database systems, Environmental database, Environmental Technology, Free and open source softwares, Geographic information systems, Geographical Information Systems, High level languages, HTTP, Internet, Internet browsers, Internet protocols, Interoperability, Maintenance, Mapping, Maps, Open source software, Open systems, Query languages, Research management, Social networking (online), Software engineering, Spatial, Technology, Three dimensional computer graphics, Three-dimensional data, Topological information, Topology, World Wide Web},
pubstate = {published},
tppubtype = {inproceedings}
}
Today many kinds of applications requires data containing actual three-dimensional data; fields like urban and town planning and pollution studies need 3D data, both for visualization purpose, as well as carry out many spatial analysis. This research-Management and use of distributed 3D data by open source Web-GIS software-is part of the Italian "PRIN 2007"∗ research project, aimed to build urban and suburban 3D models, and to interact with them using open source software only. Particularly free and open source software, used for the experimentation here shown, are Blender and PostGIS; the first one has been used to build and structure three-dimensional data, the second one for data allocation. These software interact using scripts, written in Python language. Buildings have been modeled upon the GIANT3D model (Geographical Interoperable Advanced Numerical Topological 3-Dimensional Model) developed in the research "PRIN 2004", regarding "Evolved structure of numerical cartography for Gis and Web-GIS". Python scripts, activated by Blender, allow to allocate data into a spatial database implemented through PostgreSQL and PostGis, that could be a remote database somewhere on the net; all geometrical and topological information, implemented in the 3D model, are so transferred in PostGIS. These information can be retrieved by Blender using other Python scripts, so Blender fully interacts with 3D data allocated in PostGIS. These data can be also accessed by many other clients, both directly using a database client, as using other protocols (like HTTP on the internet). Next step is to build an open source viewer, or a plugin for internet browsers, that allows client to visualize, explore and inquiry 3D model, retrieving data from database.