SAGA

Objectives

The aim of the research project is to provide a universally applicable procedural synthesis framework for acoustic representation in simulations, as well as neural network-supported, algorithmic generation of synthesis procedures based on specific simulation parameters.

Approach

The approaches to achieve the mentioned objectives include the creation of a synthesis framework through the development of core synthesis modules, the implementation of a procedural method based on these core synthesis modules, and the parameterized influence on the generated procedure. Additionally, the provision of interfaces for various simulation environments (e.g., Unity3D, Unreal, FMOD, Wwise), the extraction of synthesis from the simulation environment, and integration with CUDA.

Regarding the neural network-supported generation of parameterized procedures, the following steps are proposed: training classification networks for synthesizing procedures, adjusting parameters of known synthesis methods, classifying different sound components into core synthesis modules, creating new procedures based on sound examples, training a neural network for simulation-relevant parameterization of the synthesis procedure, including an analysis of the impact of relevant parameters through examples, and ultimately, linking and exposing the parameters within the synthesis framework for problem-solving.

Prospects

The prospects for the utilization of the research project lie in its implementation in immersive content associated with other projects and the expansion of the repertoire of realistically simulated media.

The strategic potentials include the exploration of new sectors with high demands for realistic sound generation, laying the foundation for the development of a complete sound processing chain (synthesis, propagation, reproduction), as well as reducing the pre-production effort for simulations.