Benchmark data set and method for depth estimation from light field images

Feng, Mingtao and Wang, Yaonan and Liu, Jian and Zhang, Liang and Mohd Zaki, Hasan Firdaus and Mian, Ajmal (2018) Benchmark data set and method for depth estimation from light field images. IEEE Transactions on Image Processing, 27 (7). pp. 3586-3598. ISSN 1057-7149

	PDF - Published Version Restricted to Registered users only Download (4MB) | Request a copy
Preview	PDF (SCOPUS) - Supplemental Material Download (57kB) | Preview
Preview	PDF (WOS) - Supplemental Material Download (184kB) | Preview

Abstract

Convolutional neural networks (CNNs) have performed extremely well for many image analysis tasks. However, supervised training of deep CNN architectures requires huge amounts of labeled data, which is unavailable for light field images. In this paper, we leverage on synthetic light field images and propose a two-stream CNN network that learns to estimate the disparities of multiple correlated neighborhood pixels from their epipolar plane images (EPIs). Since the EPIs are unrelated except at their intersection, a two-stream network is proposed to learn convolution weights individually for the EPIs and then combine the outputs of the two streams for disparity estimation. The CNN estimated disparity map is then refined using the central RGB light field image as a prior in a variational technique. We also propose a new real world data set comprising light field images of 19 objects captured with the Lytro Illum camera in outdoor scenes and their corresponding 3D pointclouds, as ground truth, captured with the 3dMD scanner. This data set will be made public to allow more precise 3D pointcloud level comparison of algorithms in the future which is currently not possible. Experiments on the synthetic and real world data sets show that our algorithm outperforms existing state of the art for depth estimation from light field images.

Item Type:	Article (Journal)
Additional Information:	8293/64466
Uncontrolled Keywords:	Depth estimation, disparity, light field, two stream CNN, deep learning, plenoptic camera, Lytro camera
Subjects:	Q Science > Q Science (General) > Q300 Cybernetics > Q350 Information theory
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Kulliyyah of Engineering > Department of Mechatronics Engineering
Depositing User:	Dr. Hasan Firdaus Mohd Zaki
Date Deposited:	10 Jul 2018 16:27
Last Modified:	13 Mar 2019 14:36
URI:	http://irep.iium.edu.my/id/eprint/64466

Actions (login required)

View Item