Skip to main content
Springer Nature Link
Log in

Kinetic depth images: flexible generation of depth perception

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In this paper we present a systematic approach to create smoothly varying images from a pair of photographs to facilitate enhanced awareness of the depth structure of a given scene. Since our system does not rely on sophisticated display technologies such as stereoscopy or auto-stereoscopy for depth awareness, it (a) is inexpensive and widely accessible, (b) does not suffer from vergence - accommodation fatigue, and (c) works entirely with monocular depth cues. Our approach enhances the depth awareness by optimizing across a number of features such as depth perception, optical flow, saliency, centrality, and disocclusion artifacts. We report the results of user studies that examine the relationship between depth perception, relative velocity, spatial perspective effects, and the positioning of the pivot point and use them when generating kinetic-depth images. We also present a novel depth re-mapping method guided by perceptual relationships based on the results of our user study. We validate our system by presenting a user study that compares the output quality of our proposed method against other existing alternatives on a wide range of images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Agarwal, S., Snavely, N., Simon, I., Seitz, S.M., Szeliski, R.: Building Rome in a day. In: International Conference on Computer Vision, pp. 72–79 (2009)

  2. Buehler, C., Bosse, M., McMillan, L., Gortler, S., Cohen, M.: Unstructured lumigraph rendering. In: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pp. 425–432. ACM (2001)

  3. Caelli, T.: On the perception of some geometric properties of rotating three dimensional objects. Biol. Cybern. 33, 29–37 (1979)

    Article  MATH  Google Scholar 

  4. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell 8, 679–698 (1986)

    Article  Google Scholar 

  5. Chapiro, A., Heinzle, S., Aydın, T.O., Poulakos, S., Zwicker, M., Smolic, A., Gross, M.: Optimizing stereo-to-multiview conversion for autostereoscopic displays. In: Computer graphics forum, vol. 33, pp. 63–72. Wiley, New York (2014)

  6. Chaurasia, G., Duchêne, S., Sorkine-Hornung, O., Drettakis, G.: Depth synthesis and local warps for plausible image-based navigation. ACM Trans. Graph. 32(3), 30:1–30:12 (2013)

    Article  Google Scholar 

  7. Darsa, L., Costa, B., Varshney, A.: Navigating static environments using image-space simplification and morphing. In: Proceedings of the Symposium on Interactive 3D Graphics, pp. 25 – 34, 28–30 April 1997

  8. Davidson, C.: Piku-Piku. www.start3d.com (2012). Accessed Nov 2012

  9. Didyk, P., Ritschel, T., Eisemann, E., Myszkowski, K., Seidel, H.P.: Apparent stereo: the cornsweet illusion can enhance perceived depth. In: IS&T/SPIE electronic imaging, pp. 82,910N–82,910N. International Society for Optics and Photonics (2012)

  10. Didyk, P., Ritschel, T., Eisemann, E., Myszkowski, K., Seidel, H.P., Matusik, W.: A luminance-contrast-aware disparity model and applications. ACM Trans. Graph. (TOG) 31(6), 184 (2012)

    Article  Google Scholar 

  11. Dosher, B.A., Landy, M.S., Sperling, G.: Kinetic depth effect and optic flow-I. 3D shape from Fourier motion. Vis. Res. 29, 1789–1813 (1989)

    Article  Google Scholar 

  12. Durgin, F.H., Proffitt, D.R., Olson, T.J., Reinke, K.S.: Comparing depth from motion with depth from binocular disparity. J. Exp. Psychol.-Hum. Percept. Perform. 21, 679–699 (1995)

    Article  Google Scholar 

  13. Epstein, W.: Perceptual invariance in the kinetic depth-effect. Am. J. Psychol. 78(2), 301–303 (1965)

    Article  Google Scholar 

  14. Gibson, E.J., Gibson, J.J., Smith, O.W., Flock, H.: Motion parallax as a determinant of perceived depth. J. Exp. Psychol. 58(1), 40–51 (1959)

    Article  Google Scholar 

  15. Gopi, M., Krishnan, S.: A fast and efficient projection-based approach for surface reconstruction. In: Computer graphics and image Processing, 2002. Proceedings. XV Brazilian Symposium on, pp. 179–186 (2002)

  16. Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Advances in Neural Information Processing Systems 19, pp. 545–552. MIT Press, Cambridge (2007)

  17. Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge University Press (2003)

  18. Heineman, J.: Stereogranimator. http://stereo.nypl.org/ (2012). Accessed Nov 2012

  19. Hoffman, D.M., Girshick, A.R., Akeley, K., Banks, M.S.: Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. J. Vis. 8, 33 (2008)

    Article  Google Scholar 

  20. Ip, C.Y., Varshney, A.: Saliency-assisted navigation of very large landscape images. Vis Comput Graph. IEEE Trans. 17(12), 1737–1746 (2011)

    Article  Google Scholar 

  21. Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. PAMI, IEEE Trans. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  22. Kellnhofer, P., Ritschel, T., Myszkowski, K., Seidel, H.P.: Optimizing disparity for motion in depth. In: Computer Graphics Forum, vol. 32, pp. 143–152. Wiley, New york (2013)

  23. Kim, Y., Varshney, A.: Saliency-guided enhancement for volume visualization. IEEE Trans. Vis. Comp. Graph. 12(5), 925–932 (2006)

    Article  Google Scholar 

  24. Kim, Y., Varshney, A., Jacobs, D.W., Guimbretière, F.: Mesh saliency and human eye fixations. ACM Trans. Appl. Percept. 7(2), 1–13 (2010). doi:10.1145/1670671.1670676

    Article  Google Scholar 

  25. Krähenbühl, P., Lang, M., Hornung, A., Gross, M.: A system for retargeting of streaming video. ACM Trans. Graph. 28(5), 126:1–126:10 (2009)

    Article  Google Scholar 

  26. Landy, M.S., Dosher, B.A., Sperling, G., Perkins, M.E.: The kinetic depth effect and optic flow-II. First- and second-order motion. Vision Research 31(5), 859–876 (1991)

    Article  Google Scholar 

  27. Lang, M., Hornung, A., Wang, O., Poulakos, S., Smolic, A., Gross, M.H.: Nonlinear disparity mapping for stereoscopic 3D. ACM Transactions on Graphics (TOG) 29(4) (2010)

  28. Lee, C.H., Kim, Y., Varshney, A.: Saliency-guided lighting. IEICE Trans. Inf. Syst. E92–D(2), 369–373 (2009)

    Article  Google Scholar 

  29. Lee, C.H., Varshney, A., Jacobs, D.: Mesh saliency. ACM Trans. Graph. (Proc. SIGGRAPH 2005) 24(3), 659–666 (2005)

    Article  Google Scholar 

  30. Lee, S., Kim, Y., Lee, J., Kim, K., Lee, K., Noh, J.: Depth manipulation using disparity histogram analysis for stereoscopic 3d. Vis. Comp. 30(4), 455–465 (2014). doi:10.1007/s00371-013-0868-3

    Article  Google Scholar 

  31. Liu, W., Wu, Y., Guo, F., Hu, Z.: An efficient approach for 2D to 3D video conversion based on structure from motion. Vis. Comp. 31(1), 55–68 (2015). doi:10.1007/s00371-013-0904-3

    Article  Google Scholar 

  32. Lytro: Perspective shift. www.lytro.com/camera#perspective_shift (2013). Accessed Dec 2013

  33. Mark, W.R., McMillan, L., Bishop, G.: Post-rendering 3D warping. In: Proceedings of the 1997 symposium on Interactive 3D graphics, I3D ’97, pp. 7–10. ACM, New York, USA (1997)

  34. Michailidis, G.T., Pajarola, R., Andreadis, I.: High Performance stereo system for dense 3-D reconstruction. Circuits and system video for technology. IEEE Trans. 24(6), 929–941 (2014)

    Google Scholar 

  35. Nagata, S.: How to reinforce perception of depth in single two-dimensional pictures. In: Stephen, R.E. (ed.) Pictorial communication in virtual and real environments, pp. 527–545. Taylor & Francis, Bristol (1991)

  36. Nakayama, K., Tyler, C.W.: Psychophysical isolation of movement sensitivity by removal of familiar position cues. Vis. Res. 21, 427–433 (1981)

    Article  Google Scholar 

  37. Ono, M.E., Rivest, J., Ono, H.: Depth perception as a function of motion parallax and absolute-distance information. J Exp. Psychol.-Hum. Percept. Perform. 12, 331–337 (1986)

    Article  Google Scholar 

  38. Patro, R., Ip, C.Y., Bista, S., Varshney, A.: Social snapshot: a system for temporally coupled social photography. IEEE Comput. Graph. Appl. 31, 74–84 (2011)

    Article  Google Scholar 

  39. Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. 22, 313–318 (2003)

    Article  Google Scholar 

  40. Policarpo, F., Oliveira, M.M., Comba, J.L.D.: Real-time relief mapping on arbitrary polygonal surfaces. In: Proceedings of the 2005 Symposium on Interactive 3D Graphics and Games. I3D ’05, pp. 155–162. ACM, New York, USA (2005)

  41. Proffitt, D.R., Rock, I., Hecht, H., Schubert, J.: Stereokinetic effect and its relation to the kinetic depth effect. J. Exp. Psychol.-Hum. Percept. Perform. 18, 3–21 (1992)

    Article  Google Scholar 

  42. Robinson, D., Gordon, J., Gordon, S.: A model of the smooth pursuit eye movement system. Biol. Cybern. 55, 43–57 (1986)

    Article  MathSciNet  Google Scholar 

  43. Rogers, B., Graham, M.: Motion parallax as an independent cue for depth perception. Perception 8, 125–134 (1979)

    Article  Google Scholar 

  44. Rogers, B., Graham, M.: Similarities between motion parallax and stereopsis in human depth perception. Vis. Res. 22, 261–270 (1982)

    Article  Google Scholar 

  45. Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: Exploring photo collections in 3D. In: SIGGRAPH Conference Proceedings, pp. 835–846. ACM Press, New York, USA (2006)

  46. Sperling, G., Landy, M.S., Dosher, B.A., Perkins, M.E.: Kinetic depth effect and identification of shape. J. Exp. Psychol.-Hum. Percept. Perform. 15, 826–840 (1989)

    Article  Google Scholar 

  47. Stereographics: The Stereographics Developer’s Handbook - Background on Creating Images for CrystalEyes and SimulEyes. Stereographics Corporation. http://www.reald-corporate.com/scientific/downloads/handbook.pdf (1997). Accessed Dec 2013

  48. Sun, D., Roth, S., Black, M.: Secrets of optical flow estimation and their principles. In: Computer vision and pattern recognition (CVPR), 2010 IEEE Conference on, pp. 2432 –2439 (2010)

  49. Toyoura, M., Kashiwagi, K., Sugiura, A., Mao, X.: Mono-glass for providing distance information for people losing sight in one eye. In: Proceedings of the 11th ACM SIGGRAPH international conference on virtual-reality continuum and its applications in Industry, pp. 39–42. ACM (2012)

  50. Ujike, H., Yokoi, T., Saida, S.: Effects of virtual body motion on visually-induced motion sickness. In: Annual International Conference of the IEEE Engineering in Medicine and Biology Society (2004)

  51. US. Dept. of Labor: Occuptional saftey and health administartion. www.osha.gov/SLTC/etools/computerworkstations/components_monitors.html (2013). Accessed Dec 2013

  52. Vishwanath, D., Hibbard, P.B.: Seeing in 3-D with just one eye: stereopsis without binocular vision. Psychol. Sci. 24(9), 1673–1685 (2013)

    Article  Google Scholar 

  53. Wallach, H., O’Connell, D.N.: The kinetic depth effect. J. Exp. Psychol. 45, 205–217 (1953)

    Article  Google Scholar 

  54. Weyrich, T., Deng, J., Barnes, C., Rusinkiewicz, S., Finkelstein, A.: Digital bas-relief from 3D scenes. ACM Trans. Graph. 26(3), 32 (2007)

  55. Wilson, M.: Shooting challenge: Wiggle 3D. www.gizmodo.com/5895289/shooting-challenge-wiggle-3d (2012). Accessed Sept 2012

  56. Yoonessi, A., Baker, C.: Contribution of motion parallax to depth ordering, depth magnitude and segmentation. J. Vis. 10, 1194–1194 (2011)

    Article  Google Scholar 

  57. Yoshida, K., Takahashi, S., Ono, H., Fujishiro, I., Okada, M.: Perceptually-guided design of nonperspectives through pictorial depth cues. In: Computer graphics, imaging and visualization (CGIV), 2010 Seventh International Conference on, pp. 173–178. IEEE (2010)

  58. Zhang, C., Li, Z., Cheng, Y., Cai, R., Chao, H., Rui, Y.: Meshstereo: A global stereo model with mesh alignment regularization for view interpolation. In: International Conference on Computer Vision (2015)

  59. Zheng, K.C., Colburn, R.A., Agarwala, A., Agrawala, M., Salesin, D., Curless, B.L., Cohen, M.F.: Parallax photography: creating 3D cinematic effects from stills. In: Proceedings of Graphics Interface, pp. 111–118. Canadian Information Processing Society (2009)

  60. Zhu, C., Leow, W.: Textured mesh surface reconstruction of large buildings with multi-view stereo. Vis. Comput. 29(6–8), 609–615 (2013). doi:10.1007/s00371-013-0827-z

    Article  Google Scholar 

  61. Zitnick, C.L., Jojic, N., Kang, S.B.: Consistent segmentation for optical flow estimation. Int. Conf. Comput. Vis. 2, 1308–1315 (2005)

    Google Scholar 

Download references

Acknowledgments

This work has been supported in part by the NSF Grants 09-59979 and 14-29404, the State of Marylands MPower initiative, and the NVIDIA CUDA Center of Excellence. Any opinions, findings, conclusions, or recommendations expressed in this article are those of the authors and do not necessarily reflect the views of the research sponsors.

Author information

Authors and Affiliations

  1. Department of Computer Science and Institute for Advanced Computer Studies, University of Maryland, College Park, USA

    Sujal Bista & Amitabh Varshney

  2. Universidade Federal Rural de Pernambuco - Unidade Academica de Garanhuns, Garanhuns, PE, Brazil

    Ícaro Lins Leitão da Cunha

Authors
  1. Sujal Bista
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Ícaro Lins Leitão da Cunha
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Amitabh Varshney
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Sujal Bista.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bista, S., da Cunha, Í.L.L. & Varshney, A. Kinetic depth images: flexible generation of depth perception. Vis Comput 33, 1357–1369 (2017). https://doi.org/10.1007/s00371-016-1231-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-016-1231-2

Keywords

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy