Repurposing a Relighting Network for Realistic Compositions of Captured Scenes - Télécom Paris Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

Repurposing a Relighting Network for Realistic Compositions of Captured Scenes

Résumé

Multi-view stereo can be used to rapidly create realistic virtual content, such as textured meshes or a geometric proxy for free-viewpoint Image-Based Rendering (IBR). These solutions greatly simplify the content creation process compared to traditional methods, but it is difficult to modify the content of the scene. We propose a novel approach to create scenes by composing (parts of) multiple captured scenes. The main difficulty of such compositions is that lighting conditions in each captured scene are different; to obtain a realistic composition we need to make lighting coherent. We propose a two-pass solution, by adapting a multi-view relighting network. We first match the lighting conditions of each scene separately and then synthesize shadows between scenes in a subsequent pass. We also improve the realism of the composition by estimating the change in ambient occlusion in contact areas between parts and compensate for the color balance of the different cameras used for capture. We illustrate our method with results on multiple compositions of outdoor scenes and show its application to multi-view image composition, IBR and textured mesh creation.
Fichier principal
Vignette du fichier
i3d20-6.pdf (29.58 Mo) Télécharger le fichier
teaser(1).pdf (1.3 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02500771 , version 1 (06-03-2020)

Identifiants

Citer

Baptiste Nicolet, Julien Philip, George Drettakis. Repurposing a Relighting Network for Realistic Compositions of Captured Scenes. I3D 2020 - ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, Sep 2020, San Francisco, United States. ⟨10.1145/3384382.3384523⟩. ⟨hal-02500771⟩
220 Consultations
53 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More