Skip to main content

Advertisement

Springer Nature Link
Account
Menu
Find a journal Publish with us Track your research
Search
Cart
  1. Home
  2. Programming Languages and Systems
  3. Conference paper

Sound and Complete Concolic Testing for Higher-order Functions

  • Conference paper
  • Open Access
  • First Online: 23 March 2021
  • pp 635–663
  • Cite this conference paper

You have full access to this open access conference paper

Download book PDF
Programming Languages and Systems (ESOP 2021)
Sound and Complete Concolic Testing for Higher-order Functions
Download book PDF
  • Shu-Hung You9,
  • Robert Bruce Findler9 &
  • Christos Dimoulas9 

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12648))

Included in the following conference series:

  • European Symposium on Programming

  • 4056 Accesses

Abstract

Higher-order functions have become a staple of modern programming languages. However, such values stymie concolic testers, as the SMT solvers at their hearts are inherently first-order.

This paper lays a formal foundations for concolic testing higher-order functional programs. Three ideas enable our results: (i) our tester considers only program inputs in a canonical form; (ii) it collects novel constraints from the evaluation of the canonical inputs to search the space of inputs with partial help from an SMT solver and (iii) it collects constraints from canonical inputs even when they are arguments to concretized calls. We prove that (i) concolic evaluation is sound with respect to concrete evaluation; (ii) modulo concretization and SMT solver incompleteness, the search for a counter-example succeeds if a user program has a bug and (iii) this search amounts to directed evolution of inputs targeting hard-to-reach corners of the program.

Download to read the full chapter text

Chapter PDF

Similar content being viewed by others

Concolic Testing of Functional Logic Programs

Chapter © 2018

Staged Specification Logic for Verifying Higher-Order Imperative Programs

Chapter © 2025

Abstraction and subsumption in modular verification of C programs

Article 23 March 2021

References

  1. Harold Abelson, Gerald Jay Sussman, and Julie Sussman. Structure and Interpretation of Computer Programs. MIT Press, 1985.

    Google Scholar 

  2. Saswat Anand, Patrice Godefroid, and Nikolai Tillmann. Demand-driven Compositional Symbolic Execution. In Proc. International Conference on Tools and Algorithms for the Construction and Analysis of Systems, pp. 367–3831, 2008.

    Google Scholar 

  3. Saswat Anand, Mayur Naik, Mary Jean Harrold, and Hongseok Yang. Automated Concolic Testing of Smartphone Apps. In Proc. International Symposium on on the Foundations of Software Engineering, pp. 59:1–59:11, 2012.

    Google Scholar 

  4. Joe Armstrong, Robert Virding, Claes Wikström, and Mike Williams. Programming Erlang: Software for a Concurrent World. Prentice Hall, 2007.

    Google Scholar 

  5. Roberto Baldoni, Emilio Coppa, Daniele Cono D’Elia, Camil Demetrescu, and Irene Finocchi. A Survey of Symbolic Execution Techniques. ACM Comput. Surv. 51(3), 2018.

    Google Scholar 

  6. Jacob Burnim and Koushik Sen. Heuristics for Scalable Dynamic Test Generation. In Proc. ACM/IEEE International Conference on Automated Software Engineering, pp. 443–446, 2008.

    Google Scholar 

  7. Cristian Cadar, Daniel Dunbar, and Dawson Engler. KLEE: Unassisted and Automatic Generation of High-coverage Tests for Complex Systems Programs. In Proc. USENIX Symposium on Operating Systems Design and Implementation, pp. 209–224, 2008.

    Google Scholar 

  8. Cristian Cadar and Dawson Engler. Execution Generated Test Cases: How to Make Systems Code Crash Itself. In Proc. International SPINConference on Model Cheching Software, pp. 2–23, 2005.

    Google Scholar 

  9. Cristian Cadar and Koushik Sen. Symbolic Execution for Software Testing: Three Decades Later. Communications of the ACM, pp. 82–90, 2013.

    Google Scholar 

  10. Robert Cartwright and Mike Fagan. Soft Typing. In Proc. ACM Conference on Programming Language Design and Implementation, pp. 278–292, 1991.

    Google Scholar 

  11. Koen Claessen and John Hughes. QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs. In Proc. ACM International Conference on Functional Programming, pp. 268–279, 2000.

    Google Scholar 

  12. Marko Dimjašević, Dimitra Giannakopoulou, Falk Howar, Falk Howar, Falk Howar, and Falk Howar. The Dart, the Psyco, and the Doop: Concolic Execution in Java. ACM SIGSOFT Software Engineering Notes 40(1), pp. 1–5, 2015.

    Google Scholar 

  13. Christos Dimoulas, Sam Tobin-Hochstadt, and Matthias Felleisen. Complete Monitors for Behavioral Contracts. In Proc. European Symposium on on Programming, pp. 214–233, 2012.

    Google Scholar 

  14. Michael Emmi, Rupak Majumdar, and Koushik Sen. Dynamic Test Input Generation for Database Applications. In Proc. International Symposium on Software Testing and Analysis, pp. 151–162, 2007.

    Google Scholar 

  15. Azadeh Farzan, Andreas Holzer, Niloofar Razavi, and Helmut Veith. Con2Colic Testing. In Proc. International Symposium on on the Foundations of Software Engineering, pp. 37–47, 2013.

    Google Scholar 

  16. Robert B. Findler and Matthias Felleisen. Contracts for Higher-Order Functions. In Proc. ACM International Conference on Functional Programming, pp. 48–59, 2002.

    Google Scholar 

  17. Aggelos Giantsios, Nikolaos Papaspyrou, and Konstantinos Sagonas. Concolic Testing for Functional Languages. In Proc. ACM International Conference on Principles and Practice of Declarative Programming, pp. 137–148, 2015.

    Google Scholar 

  18. Aggelos Giantsios, Nikolaos Papaspyrou, and Konstantinos Sagonas. Concolic Testing for Functional Languages. Science of Computer Programming, pp. 109–134, 2017.

    Google Scholar 

  19. Patrice Godefroid. Compositional Dynamic Test Generation. In Proc. ACM Symposium on Principles of Programming Languages, pp. 47–54, 2007.

    Google Scholar 

  20. Patrice Godefroid, Nils Klarlund, and Koushik Sen. DART: Directed Automated Random Testing. In Proc. ACM Conference on Programming Language Design and Implementation, pp. 213–223, 2005.

    Google Scholar 

  21. Patrice Godefroid, Michael Y. Levin, and David Molnar. Automated Whitebox Fuzz Testing. In Proc. Network and Distributed System Security Symposium, 2008.

    Google Scholar 

  22. Patrice Godefroid, Michael Y. Levin, and David Molnar. SAGE: Whitebox Fuzzing for Security Testing. ACM Queue 10(1), pp. 20:20–20:27, 2012.

    Google Scholar 

  23. Li Guodong, Esben Andreasen, and Indradeep Ghosh. SymJS: Automatic Symbolic Testing of JavaScript Web Applications. In Proc. International Symposium on on the Foundations of Software Engineering, pp. 449–459, 2014.

    Google Scholar 

  24. Yunho Kim, Shin Hong, and Moonzo Kim. Target-Driven Compositional Concolic Testing with Function Summary Refinement for Effective Bug Detection. In Proc. International Symposium on on the Foundations of Software Engineering, pp. 16–26, 2019.

    Google Scholar 

  25. Yunho Kim and Moonzoo Kim. SCORE: A Scalable Concolic Testing Tool for Reliable Embedded Software. In Proc. International Symposium on on the Foundations of Software Engineering, pp. 420–423, 2011.

    Google Scholar 

  26. Casey Klein, Matthew Flatt, and Robert Bruce Findler. Random Testing for Higher-order, Stateful Programs. In Proc. ACM Conference on Object-Oriented Programming, Systems, Languages and Applications, pp. 555–566, 2010.

    Google Scholar 

  27. Naoki Kobayashi, Ryosuke Sato, and Hiroshi Unno. Predicate Abstraction and CEGAR for Higher-Order Model Checking. In Proc. ACM Conference on Programming Language Design and Implementation, pp. 222–233, 2011.

    Google Scholar 

  28. Pieter Koopman and Rinus Plasmeijer. Automatic Testing of Higher Order Functions. In Proc. Asian Symposium on Programming Languages and Systems, pp. 148–164, 2006.

    Google Scholar 

  29. Guodong Li, Indradeep Ghosh, and Sreeranga P. Rajan. KLOVER: A Symbolic Execution and Automatic Test Generation Tool for C++ Programs. In Proc. International Conference on Computer Aided Verification, pp. 609–615, 2011.

    Google Scholar 

  30. Guodong Li, Peng Li, Geof Sawaya, Ganesh Gopalakrishnan, Indradeep Ghosh, and Sreeranga P. Rajan. GKLEE: Concolic Verification and Test Generation for GPUs. In Proc. Symposium on Principles and Practice of Parallel Programming, pp. 215–224, 2012.

    Google Scholar 

  31. Lian Li, Yi Lu, and Jingling Xue. Dynamic Symbolic Execution for Polymorphism. In Proc. International Conference on Compiler Construction, pp. 120–130, 2017.

    Google Scholar 

  32. Phúc Nguy\(\tilde{\hat{{\rm e}}}\)n, Sam Tobin-Hochstadt, and David Van Horn. Relatively complete counterexamples for higher-order programs. In Proc. ACM Conference on Programming Language Design and Implementation, pp. 446–456, 2015.

    Google Scholar 

  33. Phúc Nguy\(\tilde{\hat{{\rm e}}}\)n, Sam Tobin-Hochstadt, and David Van Horn. Higher order symbolic execution for contract verification and refutation. Journal of Functional Programming(27), pp. e3:1–e3:54, 2017.

    Google Scholar 

  34. Adrián Palacios and Germán Vidal. Concolic Execution in Functional Programming by Program Instrumentation. In Proc. International Symposium on Logic-Based Program Synthesis and TRansformation, pp. 277–292, 2015.

    Google Scholar 

  35. Niloofar Razavi, Franjo Ivančić, Vineet Kahlon, and Aarti Gupta. Concurrent Test Generation Using Concolic Multi-trace Analysis. In Proc. Asian Symposium on Programming Languages and Systems, pp. 239–255, 2012.

    Google Scholar 

  36. Marija Selakovic, Michael Pradel, Rezwana Karim, and Frank Tip. Test Generation for Higher-order Functions in Dynamic Languages. Proceedings of the ACM on Programming Languages (OOPSLA) 2, pp. 161:1–161:27, 2018.

    Google Scholar 

  37. Koushik Sen and Gul Agha. CUTE and jCUTE: Concolic Unit Testing and Explicit Path Model-checking Tools. In Proc. International Conference on Computer Aided Verification, pp. 419–423, 2006.

    Google Scholar 

  38. Koushik Sen, Swaroop Kalasapur, Brutch Tasneem, and Simon Gibbs. Jalangi: A Selective Record-replay and Dynamic Analysis Framework for JavaScript. In Proc. International Symposium on on the Foundations of Software Engineering, pp. 488–498, 2013.

    Google Scholar 

  39. Koushik Sen, Darko Marinov, and Gul Agha. CUTE: A Concolic Unit Testing Engine for C. In Proc. International Symposium on on the Foundations of Software Engineering, pp. 263–272, 2005.

    Google Scholar 

  40. T. Stephen Strickland, Sam Tobin-Hochstadt, Robert B. Findler, and Matthew Flatt. Chaperones and Impersonators: Run-time Support for Reasonable Interposition. In Proc. ACM Conference on Object-Oriented Programming, Systems, Languages and Applications, pp. 943–962, 2012.

    Google Scholar 

  41. Youcheng Sun, Min Wu, Wenjie Ruan, Xiaowei Huang, Marta Kwiatkowska, and Daniel Kroening. Concolic Testing for Deep Neural Networks. In Proc. ACM/IEEE International Conference on Automated Software Engineering, pp. 109–119, 2018.

    Google Scholar 

  42. Tachio Terauchi. Dependent Types from Counterexamples. In Proc. ACM Symposium on Principles of Programming Languages, pp. 119–130, 2010.

    Google Scholar 

  43. Nikolai Tillmann and Jonathan de Halleux. Pex: White Box Test Generation for. NET. In Proc. International Conference on Tests and Proofs, pp. 134–153, 2008.

    Google Scholar 

  44. Sam Tobin-Hochstadt and Matthias Felleisen. Logical Types for Untyped Languages. In Proc. ACM International Conference on Functional Programming, pp. 117–128, 2010.

    Google Scholar 

  45. Sam Tobin-Hochstadt and David Van Horn. Higher-Order Symbolic Execution via Contracts. In Proc. ACM Conference on Object-Oriented Programming, Systems, Languages and Applications, pp. 537–554, 2012.

    Google Scholar 

  46. Andrew K. Wright and Robert Cartwright. A Practical Soft Type System for Scheme. ACM Transactions on Programming Languages and Systems 19(1), pp. 87–152, 1997.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Northwestern University, Evanston, IL, USA

    Shu-Hung You, Robert Bruce Findler & Christos Dimoulas

Authors
  1. Shu-Hung You
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert Bruce Findler
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christos Dimoulas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shu-Hung You .

Editor information

Editors and Affiliations

  1. Imperial College, London, UK

    Nobuko Yoshida

Rights and permissions

This chapter is published under an open access license. Please check the 'Copyright Information' section either on this page or in the PDF for details of this license and what re-use is permitted. If your intended use exceeds what is permitted by the license or if you are unable to locate the licence and re-use information, please contact the Rights and Permissions team.

Copyright information

© 2021 The Author(s)

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

You, SH., Findler, R.B., Dimoulas, C. (2021). Sound and Complete Concolic Testing for Higher-order Functions. In: Yoshida, N. (eds) Programming Languages and Systems. ESOP 2021. Lecture Notes in Computer Science(), vol 12648. Springer, Cham. https://doi.org/10.1007/978-3-030-72019-3_23

Download citation

  • .RIS
  • .ENW
  • .BIB

  • DOI: https://doi.org/10.1007/978-3-030-72019-3_23

  • Published: 23 March 2021

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-72018-6

  • Online ISBN: 978-3-030-72019-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Publish with us

Policies and ethics

Societies and partnerships

  • The European Joint Conferences on Theory and Practice of Software.
    The European Joint Conferences on Theory and Practice of Software. (opens in a new tab)

Search

Navigation

  • Find a journal
  • Publish with us
  • Track your research

Discover content

  • Journals A-Z
  • Books A-Z

Publish with us

  • Journal finder
  • Publish your research
  • Open access publishing

Products and services

  • Our products
  • Librarians
  • Societies
  • Partners and advertisers

Our imprints

  • Springer
  • Nature Portfolio
  • BMC
  • Palgrave Macmillan
  • Apress
  • Your US state privacy rights
  • Accessibility statement
  • Terms and conditions
  • Privacy policy
  • Help and support
  • Legal notice
  • Cancel contracts here

Not affiliated

Springer Nature

© 2025 Springer Nature

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy