Binaural Spatialization for 3D immersive audio communication in a virtual world

Abstract :

Realistic 3D audio can greatly enhance the sense of presence in a virtual environment. We introduce a framework for capturing, transmitting and rendering of 3D audio in presence of other bandwidth savvy streams in a 3D Tele-immersion based virtual environment. This framework presents an efficient implementation for 3D Binaural Spatialization based on the positions of current objects in the scene, including animated avatars and on the fly reconstructed humans. We present a general overview of the framework, how audio is integrated in the system and how it can exploit the positions of the objects and room geometry to render realistic reverberations using head related transfer functions. The network streaming modules used to achieve lip-synchronization, high-quality audio frame reception, and accurate localization for binaural rendering are also presented. We highlight how large computational and networking challenges can be addressed efficiently. This represents a first step in adequate networking support for Binaural 3D Audio, useful for tele-presence. The subsystem is successfully integrated with a larger 3D immersive system, with state of art capturing and rendering modules for visual data.

Complete list of metadatas

https://hal.telecom-paristech.fr/hal-02288381
Contributor : Telecomparis Hal <>
Submitted on : Saturday, September 14, 2019 - 6:44:39 PM
Last modification on : Thursday, October 17, 2019 - 12:37:03 PM

Identifiers

  • HAL Id : hal-02288381, version 1

Collections

Citation

Davide Andrea Mauro, Rufael Mekuria, Michele Sanna. Binaural Spatialization for 3D immersive audio communication in a virtual world. AudioMostly, Sep 2013, Pitea, Sweden. pp.8. ⟨hal-02288381⟩

Share

Metrics

Record views

5