Skip to main content

Advertisement

Springer Nature Link
Log in

Approaches for the combined use of risk analysis and testing: a systematic literature review

  • RBT
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

Risk analysis and testing are conducted for different purposes. Risk analysis and testing nevertheless involve processes that may be combined to the benefit of both. We may use testing to support risk analysis and risk analysis to support testing. This paper surveys literature on the combined use of risk analysis and testing. First, the existing approaches are identified through a systematic literature review. The identified approaches are then classified and discussed with respect to main goal, context of use and maturity level. The survey highlights the need for more structure and rigor in the definition and presentation of approaches. Evaluations are missing in most cases. The paper may serve as a basis for examining approaches for the combined use of risk analysis and testing, or as a resource for identifying the adequate approach to use.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Alam, M., Khan, A.I.: Risk-based testing techniques: a perspective study. Int. J. Comput. Appl. 65, 33–41 (2013)

    Google Scholar 

  2. Amland, S.: Risk-based testing: risk analysis fundamentals and metrics for software testing including a financial application case study. J. Syst. Softw. 53, 287–295 (2000)

    Article  Google Scholar 

  3. Bai, X., Kenett, R.S.: Risk-based adaptive group testing of semantic web services. In: Proceeding of the 33rd Annual IEEE International Computer Software and Applications Conference (COMPSAC’09), vol. 2, pp. 485–490. IEEE, New York (2009)

  4. Bai, X., Kennett, R.S., Yu, W.: Risk assessment and adaptive group testing of semantic web services. Int. J. Softw. Eng. Knowl. Eng. 595–620 (2012)

  5. Casado, R., Tuya, J., Younas, M.: Testing long-lived web services transactions using a risk-based approach. In: Proceeding of the 10th International Conference on Quality Software (QSIC’10), pp. 337–340. IEEE, New York (2010)

  6. Casado, R., Tuya, J., Younas, M.: A framework to test advanced web services transactions. In: Proceeding of the 4th International Conference on Software Testing, Verification and Validation (ICST’11), pp 443–446. IEEE, New York (2011)

  7. Chen, Y., Probert, R.L.: A risk-based regression test selection strategy. In: Proceeding of the 14th IEEE International Symposium on Software Reliability Engineering (ISSRE’03), Fast Abstract, pp. 305–306. Chillarege Press (2003)

  8. Chen, Y., Probert, R.L., Sims, D.P.: Specification-based regression test selection with risk analysis. In: Proceeding of the 2002 Conference of the Centre for Advanced Studies on Collaborative Research (CASCON’02), pages 1–14. IBM Press, USA (2002)

  9. Entin, V., Winder, M., Zhang, B., Christmann, S.: Introducing model-based testing in an industrial scrum project. In: Proceeding of the Seventh International Workshop on Automation of Software Test (AST’12), pp. 43–49. IEEE, New York (2012)

  10. Felderer, M., Haisjackl, C., Breu, R., Motz, J.: Integrating manual and automatic risk assessment for risk-based testing. In: Proceeding of the 4th International Conference on Software Quality (SWQD’12), vol. 94 of LNBIP, pp. 159–180. Springer, Berlin (2012)

  11. Felderer, M., Ramler, R.: Experiences and challenges of introducing risk-based testing in an industrial project. In: Proceeding of the Fifth International Conference on Software Quality (SWQD’13), vol. 133 of LNBIP, pp. 10–29. Springer, New York (2013)

  12. Gleirscher, M.: Hazard-based selection of test cases. In: Proceeding if the Sixth International Workshop on Automation of Software Test (AST’11), pp. 64–70. ACM, New York (2011)

  13. Gleirscher, M.: Hazard analysis of technical systems. In: Proceeding of the Fifth International Conference on Software Quality (SWQD’13), vol. 133 of LNBIP, pp. 104–124. Springer, Berlin (2013)

  14. Hosseingholizadeh, A.: A source-based risk analysis approach for software test optimization. In: Proceeding of the Second International Conference on Computer Engineering and Technology (ICCET’10), vol. 2, pp. 601–604. IEEE, New York (2010)

  15. International Standards Organization. ISO 31000:2009(E), Risk management—Principles and guidelines (2009)

  16. International Standards Organization. ISO 29119 Software and system engineering—Software Testing-Part 2 : Test process (draft) (2012)

  17. Kitchenham, B., Charters, S.: Guidelines for performing systematic literature reviews in software engineering. Technical Report EBSE 2007–001 (2007)

  18. Kloos, J., Hussain, T., Eschbach, R.: Risk-based testing of safety-critical embedded systems driven by Fault Tree Analysis. In: Proceeding of the Fourth International Conference on Software Testing, Verification and Validation Workshops (ICSTW’11), pp. 26–33. IEEE, New York (2011)

  19. Kumar, N., Sosale, D., Konuganti, S.N., Rathi, A.: Enabling the adoption of aspects-testing aspects: A risk model, fault model and patterns. In: Proceeding of the Eighth ACM International Conference on Aspect-Oriented Software Development (AOSD’09), pp. 197–206. ACM, New York (2009)

  20. Murthy, K.K., Thakkar, K.R., Laxminarayan, S.: Leveraging risk based testing in enterprise systems security validation. In: Proceeding of the First International Conference on Emerging Network Intelligence (EMERGING’09), pp. 111–116. IEEE, New York (2009)

  21. Nazier, R., Bauer, T.: Automated risk-based testing by integrating safety analysis information into system behavior models. In: Proceeding of the 23rd International Symposium on Software Reliability Engineering Workshops (ISSREW’12), pp 213–218. IEEE, New York (2012)

  22. Ray, M., Mohapatra, D.P.: Risk analysis: a guiding force in the improvement of testing. IET Softw. 7, 29–46 (2013)

    Article  Google Scholar 

  23. Redmill, F.: Exploring risk-based testing and its implications. Softw. Test. Verif. Reliab. 14, 3–15 (2004)

    Article  Google Scholar 

  24. Redmill, F.: Theory and practice of risk-based testing. Softw. Test. Verif. Reliab. 15, 3–20 (2005)

    Article  Google Scholar 

  25. Rosenberg, L., Stapko, R., Gallo, A.: Risk-based object oriented testing. In: Proceeding of the 24th Annual Software Engineering Workshop, pp. 1–6. NASA, Software Engineering Laboratory, (1999)

  26. Schneidewind, N.F.: Risk-driven software testing and reliability. Int. J. Reliab. Qual. Saf. Eng. 14, 99–132 (2007)

  27. Souza, E., Gusmão, C., Alves, K., Venâncio, J., Melo, R.: Measurement and control for risk-based test cases and activities. In: Proceeding of the 10th Latin American Test Workshop (LATW’09), pp 1–6. IEEE, New York (2009)

  28. Souza, E., Gusmão, C., Venâncio, J.: Risk-based testing: A case study. In: Proceeding of the Seventh International Conference on Information Technology: New Generations (ITNG’10), pp. 1032–1037. IEEE, New York (2010)

  29. Stallbaum, H., Metzger, A., Pohl, K.: An automated technique for risk-based test case generation and prioritization. In: Proceeding of the Third International Workshop on Automation of Software Test (AST’08), pp. 67–70. ACM, New York (2008)

  30. Sulaman, S.M., Weyns, K., Höst, M.: A review of research on risk analysis methods for it systems. In: Proceeding of the 17th International Conference on Evaluation and Assessment in Software Engineering (EASE’13), pp. 86–96 (2013)

  31. Wendland, M.-F., Kranz, M., Schieferdecker, I.: A systematic approach to risk-based testing using risk-annotated requirements models. In: Proceeding of the Seventh International Conference on Software Engineering Advances (ICSEA’12), pp. 636–642. IARA (2012)

  32. Wong, W.E., Qi, Y., Cooper, K.: Source code-based software risk assessing. In: Proceeding of the 2005 ACM Symposium on Applied Computing (SAC’05), pp. 1485–1490. ACM, New York (2005)

  33. Xu, D., Tu, M., Sandford, M., Thomas, L., Woodraska, D., Xu, W.: Automated security test generation with formal threat models. IEEE Trans. Dependable Secure Comput. 9, 526–540 (2012)

    Article  Google Scholar 

  34. Yoon, H., Choi, B.: A test case prioritization based on degree of risk exposure and its empirical study. Int. J. Softw. Eng. Knowl. Eng. 21, 191–209 (2011)

    Article  Google Scholar 

  35. Zech, P.: Risk-based security testing in cloud computing environments. In: Proceeding of the Fourth International Conference on Software Testing, Verification and Validation (ICST’11), pp. 411–414. IEEE, New York (2011)

  36. Zech, P., Felderer, M., Breu, R.: Towards a model based security testing approach of cloud computing environments. In: Proceeding of the Sixth International Conference on Software Security and Reliability Companion (SERE-C’12), pp. 47–56. IEEE, New York (2012)

  37. Zimmermann, F., Eschbach, R., Kloos, J., Bauer, T.: Risk-based statistical testing: A refinement-based appraoch to the reliability analysis of safety-critical systems. In: Proceeding of the 12th European Workshop on Dependable Computing (EWDC’09), pp. 1–8 (2009)

Download references

Acknowledgments

This work has been conducted as a part of the DIAMONDS (201579/S10) project funded by the Research Council of Norway, as well as a part of the NESSoS network of excellence and the RASEN project funded by the European Commission within the 7th Framework Programme.

Author information

Authors and Affiliations

  1. SINTEF ICT, Oslo, Norway

    Gencer Erdogan, Yan Li, Fredrik Seehusen & Ketil Stølen

  2. Department of Informatics, University of Oslo, Oslo, Norway

    Gencer Erdogan, Yan Li, Ragnhild Kobro Runde & Ketil Stølen

Authors
  1. Gencer Erdogan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ragnhild Kobro Runde
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fredrik Seehusen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ketil Stølen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ragnhild Kobro Runde.

Appendix: Search strings

Appendix: Search strings

The following 23 search strings were used during Step 1 of the search process described in Sect. 3. Initially, we only had search strings 1 and 14, with boolean operators for combining the possible search terms in different ways. However, the initial search revealed the need for including also each possible combination as a search string in itself.

  • Search string 1: (risk) AND (based OR driven OR oriented) AND (test OR testing OR verification OR checking)

  • Search string 2: risk based testing

  • Search string 3: risk driven testing

  • Search string 4: risk oriented testing

  • Search string 5: risk based test

  • Search string 6: risk driven test

  • Search string 7: risk oriented test

  • Search string 8: risk based verification

  • Search string 9: risk driven verification

  • Search string 10: risk oriented verification

  • Search string 11: risk based checking

  • Search string 12: risk driven checking

  • Search string 13: risk oriented checking

  • Search string 14: (test) AND (based OR driven OR oriented) AND (risk) AND (analysis OR assessment OR evaluation)

  • Search string 15: test based risk analysis

  • Search string 16: test driven risk analysis

  • Search string 17: test oriented risk analysis

  • Search string 18: test based risk assessment

  • Search string 19: test driven risk assessment

  • Search string 20: test oriented risk assessment

  • Search string 21: test based risk evaluation

  • Search string 22: test driven risk evaluation

  • Search string 23: test oriented risk evaluation

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Erdogan, G., Li, Y., Runde, R.K. et al. Approaches for the combined use of risk analysis and testing: a systematic literature review. Int J Softw Tools Technol Transfer 16, 627–642 (2014). https://doi.org/10.1007/s10009-014-0330-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-014-0330-5

Keywords

Search

Navigation

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy