4Knowledge 4Expertise 4Technology

Introduction

The value to the business that the ITfunction delivers has continued to increase over the last decade. Often IT solutions are the principle reason for the existence of some businesses, for example organisations that conduct their entire business from the internet. In addition businesses are increasingly focusing on value for money and return on investment when investing in IT. If we look at the ITorganisation then the value delivered to the business can be seen to be made up of: 4 The operations function delivering day to day service 4 The development function creating new applications (or products) to improve efficiency and effectiveness 4 The creative function that are constantly looking for new ways to harness ITtechnology to deliver competitive advantage by opening up new opportunities in the market place However how many organisations understand the value contributed by the test discipline to ITand the business?

The test discipline has always been seen as high cost to ITand in some extreme cases is seen as complete overhead. It could be argued that if ITsolutions were engineered to the same degree as other mature engineering disciplines then testing would become largely defunct. For example if an automotive engineer designs a car with disc brakes on the front wheels and drum brakes on the rear then that is what is implemented on the assembly like. There is low risk that anything else will happen. Or put another way there is high confidence that we make what we designed. Quality assurance is still required to supply information to understand and continually improve the quality and reliability of the car design and production process; but there is no need for a tester on the assembly line to check that the design has been implemented to specification. We have to accept however, that the nature of the commercial ITindustry does not and often cannot apply the same engineering discipline given the constraints of skill, resource and time. So without some intervention by the testing discipline there is high risk of failure and low confidence that defined business benefits will be realised. Without testing there is little or no information to assist the business with their key decisions on implementation of the new system or application. Therefore it can be asserted that the objective of testing is to: 4 Reduce the risk of: 4Failure of the developed solution to deliver the defined business benefits 4Delivering an unreliable solution into the production environment 4Damaging existing systems and functionality by changes to the application, infrastructure or business process 4 Increased confidence that the system will be developed to the right level of quality constrained by the agreed resources and time. 4 Enable a predictable outcome to the development and test process by gathering and analysis of information on the quality of the product under development and the process by which it is developed

What value does testing give?

www.nfocus.co.uk

nFocus Testing - Consultancy, Professional and Managed Services e-Innovation Centre, Shifnal Road, Priorslee, Telford TF2 9FT Tel: 0870 242 6235 Fax: 0870 242 6236 Email: info@nfocus.co.uk

To summarise, testing should identify and mitigate risk of failure, increase confidence of a successful implementation while providing objective information to allow informed decisions to be made.

Can testing deliver value to the business? Risk reduction

If the above assertions are true then testing should be a key part in the development process. Risk of failure due to poorly defined requirements, specification and design, coupled with change, is high at the start of a project. A mistake made here can have far reaching consequences later in the project when there is little or no time to put it right. Testing can deliver high value by early indication of these errors reducing the risk of problems later in the project lifecycle. In addition at this stage of the project early indicators of problematic areas and development process issues are often apparent but not identified by the project personnel. Information derived from early testing activity (requirements verification, design reviews preparation of test cases etc.) is of high value to the project as it enables objective assessment and associated improvements to be made to the processes used to build the solution. Risk is further reduced by taking bugs out at the earliest point in the most cost effective and efficient manner further increasing the value that testing adds to the overall project.

Confidence

As the testing discipline becomes more and more aligned with development and is performed in a trusted and consistent manner the outcome of the project becomes more predictable. This in turn raises confidence that the final solution will be successful. In addition the business and IT operations should have increased confidence in the ability of the project team to deliver on time, within budget at the right level of quality, resulting in reduced need for excessive user and operational acceptance testing which can have a significant impact on cost to the business. However to demonstrate this increase in confidence there needs to be the appropriate level of information made available to each level of management. The test discipline is a key contributor to gathering, analysing and presenting this information. At a technical level, confidence of a successful implementation should also increase during the project as more and more of the solution is delivered and tested. However unplanned change can adversely impact the reliability of the tested solution. Often the adverse effect goes unnoticed as it was located in a part of the solution already tested and only surfaces in a later stage of the project, for example acceptance test or worse still not until the project has gone live. As a consequence there is often a nagging doubt (with the resulting loss of confidence) that the solution is not sound and may be unreliable. Regression testing is a key confidence builder, however to continually test and retest the solution as it is developed becomes more costly and problematic as the solution gets bigger. But what if the solution components could be continually tested and cumulatively retested on a regular basis. Automation of the testing fully aligned with the development and test process should enable this to be achieved to a significant degree. To achieve best value within the constraints of cost and time this approach must be agreed and implemented from the outset of the project. More recent development methodologies providing the design, development and test processes are joined up and followed.

www.nfocus.co.uk

nFocus Testing - Consultancy, Professional and Managed Services e-Innovation Centre, Shifnal Road, Priorslee, Telford TF2 9FT Tel: 0870 242 6235 Fax: 0870 242 6236 Email: info@nfocus.co.uk

The value derived by this iterative regression testing is further extended when the system is implemented as the automated tests can be reused by the maintenance function to verify any changes made in production and follow on releases of the systems.

Information

A large element of the information required to support business decisions can be derived from the testing process. These indicators cover: 4 Quality of the solution and the process by which it is designed and developed 4 Residual risk evident at any stage of the project 4 Confidence indicators such as functional test coverage and business process test coverage 4 Progress to plan Looking at the incidents found during preparation and execution of tests and analysing the trends can show whether the design development and testing processes are working effectively and no backlogs are developing. Casual analysis of closed incidents can help identify problematic areas within the solution or the development process. Examples include problems with the: 4 Solution under test e.g. 4Requirements 4Specification 4The solution itself 4Associated changes to business processes or procedures 4 Development and test process e.g. 4Change process 4Development processes and techniques 4Test preparation 4Test data 4Test environments If a risk based approach is used, testing can identify how the perceived technical, business and operational risks are being mitigated as each area of the solution is tested. Confidence can be considered in two ways, coverage in terms of the testing and demonstration that the tested solution continues to be functionally and reliably sound after any changes are made. Coverage reports should show what functional, non functional (e.g. implementation management and maintenance procedures, security, performance etc.) and business processes have been tested and to what degree. Of equal importance is that it should also show what has not been verified allowing risk assessments to be made when the decision to implement (or not) is made. The business are comfortable with taking calculated risks but they do need objective information to justify and support their decisions rather than a purely subjective opinion. The following chart shows how information derived from the test process can be presented to allow the business project sponsors to determine when to implement a solution. The testing discipline, using a risk based approach, reliability measurements and coverage statistics can supply information to underpin this analysis.

www.nfocus.co.uk

nFocus Testing - Consultancy, Professional and Managed Services e-Innovation Centre, Shifnal Road, Priorslee, Telford TF2 9FT Tel: 0870 242 6235 Fax: 0870 242 6236 Email: info@nfocus.co.uk

www.nfocus.co.uk

nFocus Testing - Consultancy, Professional and Managed Services e-Innovation Centre, Shifnal Road, Priorslee, Telford TF2 9FT Tel: 0870 242 6235 Fax: 0870 242 6236 Email: info@nfocus.co.uk

Conclusion

In conclusion testing is a highly valued contributor to ITprojects and hence to the business value chain and can be summarised and often quantified in the following areas: 4 Reduction of risk and cost of failure of a solution to deliver benefits in the allocated time 4 Reduction in the risk of adverse effecting existing business processes 4 Increase in confidence of a predictable outcome of a change 4 Making available information to make informed business decisions when time cost or quality are an issue The amount of value realised depends on the maturity of the test and development processes and how closely they are aligned. Even development processes at a low maturity levels can quickly derive value by modest changes in process and attitude to testing. Technology is now enabling increased value to be derived through the ability to continually test and retest applications reducing the risk of inevitable change destabilising the solution as a whole. Having accepted that testing is a key contributor to the business then the next challenge is to ensure that the test discipline is implemented in an effective and cost efficient manner. This will be explored in a later paper.

Further Information

Our consultants have considerable practical and pragmatic knowledge and expertise in testing process, management and application of technology which can be used to help you increase the value of your testing. If you would like to discuss how we can help you improve the value added by testing to your organisation please call and we will arrange for one of our consultants to call or visit you to discuss in more detail.

www.nfocus.co.uk

nFocus Testing - Consultancy, Professional and Managed Services e-Innovation Centre, Shifnal Road, Priorslee, Telford TF2 9FT Tel: 0870 242 6235 Fax: 0870 242 6236 Email: info@nfocus.co.uk