publications

2022

1. Disparity-Guided Light Field Video Synthesis with Temporal Consistency

   Chin-Chia Yang, Yi-Chou Chen, Shan-Ling Chen, and 1 more author

   IEEE MIPR, 2022

   We propose an end-to-end deep learning framework for synthesizing light field sequences from stereo image sequences. Our frame-work consists of a disparity estimation network, a guided synthesis network, and a refinement network and is able to resolve the flickering issue caused by temporal inconsistency, an artifact that is commonly seen in synthesized light field videos.

   Abs Bib HTML PDF

   Light field has great applications in AR/VR. It is particu-larly usefulfor resolving the vergence-accommodation con-flict (VAC) and creating correct depth cues for AR/VR dis-plays. However, the source data, especially light field video, are not widely available yet. To resolve the scarcity is-sue, one may resort to data such as stereo image sequences that are commonly available. In this paper, we propose an end-to-end deep learning framework for synthesizing light field sequences from stereo image sequences. Our frame-work consists of a disparity estimation network, a guided synthesis network, and a refinement network and is able to resolve the flickering issue caused by temporal incon-sistency, an artifact that is commonly seen in synthesized light field videos. Our experimental results are quantitatively and qualitatively better than the results of existing light field synthesis algorithms that were originally developed for static light fields.

   @book{yang2022mipr,
     type = {article},
     title = {Disparity-Guided Light Field Video Synthesis with Temporal Consistency},
     author = {Yang, Chin-Chia and Chen, Yi-Chou and Chen, Shan-Ling and Chen, Homer H.},
     publisher = {IEEE},
     journal = {IEEE MIPR},
     booktitle = {IEEE MIPR},
     conference = {},
     volume = {},
     issue = {},
     pages = {},
     numpages = {},
     year = {2022},
     month = {},
     url = {https://ieeexplore.ieee.org/document/9874639},
     teaser = {tip2021-results.gif},
     note = {We propose an end-to-end deep learning framework for synthesizing light field sequences from stereo image sequences. 
         Our frame-work consists of a disparity estimation network, a guided synthesis network, and a refinement network and is able to resolve 
         the flickering issue caused by temporal inconsistency, an artifact that is commonly seen in synthesized light field videos.}
   }