Abstract
We propose a framework to infer complete geometry of a scene with strong reflections or hidden by partially transparent occluders from a set of 4D light fields captured with a hand-held light field camera. For this, we first introduce a variant of bundle adjustment specifically tailored to 4D light fields to obtain improved pose parameters. Geometry is recovered in a global framework based on convex optimization for a weighted minimal surface. To allow for non-Lambertian materials and semi-transparent occluders, the point-wise costs are not based on the principle of photo-consistency. Instead, we perform a layer analysis of the light field obtained by finding superimposed oriented patterns in epipolar plane image space to obtain a set of depth hypotheses and confidence scores, which are integrated into a single functional.
O. Johannsen and A. Sulc contributed equally.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Gortler, S., Grzeszczuk, R., Szeliski, R., Cohen, M.: The lumigraph. In: Proceedings of SIGGRAPH, pp. 43–54 (1996)
Bolles, R., Baker, H., Marimont, D.: Epipolar-plane image analysis: an approach to determining structure from motion. Int. J. Comput. Vis. 1, 7–55 (1987)
Johannsen, O., Sulc, A., Goldluecke, B.: What sparse light field coding reveals about scene structure. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2016)
Wanner, S., Goldluecke, B.: Reconstructing reflective and transparent surfaces from epipolar plane images. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 1–10. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40602-7_1
Johannsen, O., Sulc, A., Goldluecke, B.: On linear structure from motion for light field cameras. In: Proceedings of the International Conference on Computer Vision (2015)
Johannsen, O., Sulc, A., Goldluecke, B.: Variational separation of light field layers. In: Vision, Modelling and Visualization (VMV) (2015)
Wanner, S., Goldluecke, B.: Variational light field analysis for disparity estimation and super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 36, 606–619 (2014)
Chandraker, M., Reddy, D., Wang, Y., Ramamoorthi, R.: What object motion reveals about shape with unknown BRDF and lighting. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2013)
Tao, M., Wang, T.C., Malik, J., Ramamoorthi, R.: Depth estimation for glossy surfaces with light-field cameras. In: Proceedings of the European Conference on Computer Vision Workshops (2014)
Chen, C., Lin, H., Yu, Z., Kang, S.B., J., Y.: Light field stereo matching using bilateral statistics of surface cameras. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2014)
Stich, T., Tevs, A., Magnor, M.: Global depth from epipolar volumes - a general framework for reconstructing non-Lambertian surfaces. In: 3DPVT (2006)
Laurentini, A.: The visual hull concept for visual-based image understanding. IEEE Trans. Pattern Anal. Mach. Intell. 16, 150–162 (1994)
Zuo, X., Du, C., Wang, S., Zheng, J., Yang, R.: Interactive visual hull refinement for specular and transparent object surface reconstruction. In: Proceedings of the International Conference on Computer Vision (2015)
Kutulakos, K.N., Seitz, S.M.: A theory of shape by space carving. Int. J. Comput. Vis. 38, 199–218 (2000)
Yang, Y., Pollefeys, M., Welch, G.: Dealing with textureless regions and specular highlights - a progressive space carving scheme using a novel photo-consistency measure. In: Proceedings of the International Conference on Computer Vision (2003)
Faugeras, O., Keriven, R.: Variational principles, surface evolution, PDE’s, level set methods, and the stereo problem. IEEE Trans. Image Process. 7, 336–344 (1998)
Jin, H., Soatto, S., Yezzi, A.: Multi-view stereo reconstruction of dense shape and complex appearance. Int. J. Comput. Vis. 63, 175–189 (2005)
Yu, T., Narendra, A., Chen, W.C.: SDG cut: 3D reconstruction of non-Lambertian objects using graph cuts on surface distance grid. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2006)
Ihrke, I., Goldluecke, B., Magnor, M.: Reconstructing the geometry of flowing water. In: Proceedings of the International Conference on Computer Vision, pp. 1055–1060 (2005)
Goldman, D., Curless, B., Hertzmann, A., Seitz, S.: Shape and spatially-varying BRDFs from photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1060–1071 (2010)
Vogiatzis, G., Hernandez, C., Cipolla, R.: Reconstruction in the round using photometric normals and silhouettes. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2006)
Kolev, K., Pock, T., Cremers, D.: Anisotropic minimal surfaces integrating photoconsistency and normal information for multiview stereo. In: Proceedings of the European Conference on Computer Vision (2010)
Cremers, D., Kolev, K.: Multiview stereo and silhouette consistency via convex functionals over convex domains. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1161–1174 (2011)
Federer, H.: Geometric Measure Theory. Springer, Heidelberg (1969)
Zach, C., Pock, T., Bischof, H.: A globally optimal algorithm for robust TV-L1 range image integration. In: Proceedings of the International Conference on Computer Vision (2007)
Graber, G., Pock, T., Bischof, H.: Online 3D reconstruction using convex optimization. In: Proceedings of the International Conference on Computer Vision Workshops (2011)
Zickler, T., Belhumeur, P., Kriegman, D., Stereopsis, H.: Exploiting reciprocity for surface reconstruction. Int. J. Comput. Vis. 49, 215–227 (2002)
Dansereau, D., Pizarro, O., Williams, S.: Decoding, calibration and rectification for lenselet-based plenoptic cameras. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 1027–1034 (2013)
Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In: International Conference on Computer Vision (ICCV 2011) (2011)
Chan, T., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM J. Appl. Math. 66, 1632–1648 (2006)
Acknowledgement
This work was supported by the ERC Starting Grant “Light Field Imaging and Analysis” (LIA 336978, FP7-2014).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Johannsen, O., Sulc, A., Marniok, N., Goldluecke, B. (2017). Layered Scene Reconstruction from Multiple Light Field Camera Views. In: Lai, SH., Lepetit, V., Nishino, K., Sato, Y. (eds) Computer Vision – ACCV 2016. ACCV 2016. Lecture Notes in Computer Science(), vol 10113. Springer, Cham. https://doi.org/10.1007/978-3-319-54187-7_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-54187-7_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-54186-0
Online ISBN: 978-3-319-54187-7
eBook Packages: Computer ScienceComputer Science (R0)