Scribd
Upload
384 views5 pages

Automation Framework - Evaluation Criteria

The document discusses parameters for evaluating automation test frameworks including maintainability, scalability, reliability, reusability, and ease of use. For maintainability, it addresses whether prerequisites need to be manually set, if test data can be automatically updated, and if multiple data sets can be used. Scalability discusses if the framework can support different technologies. Reliability examines exception handling and need for manual intervention. Reusability evaluates average lines of code per function and reuse of functions. Ease of use assesses if non-engineers can execute, add, or edit tests and if results are easily understood.

Uploaded by

Shinu Arora
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as XLS, PDF, TXT or read online on Scribd
384 views5 pages

Automation Framework - Evaluation Criteria

The document discusses parameters for evaluating automation test frameworks including maintainability, scalability, reliability, reusability, and ease of use. For maintainability, it addresses whether prerequisites need to be manually set, if test data can be automatically updated, and if multiple data sets can be used. Scalability discusses if the framework can support different technologies. Reliability examines exception handling and need for manual intervention. Reusability evaluates average lines of code per function and reuse of functions. Ease of use assesses if non-engineers can execute, add, or edit tests and if results are easily understood.

Uploaded by

Shinu Arora
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as XLS, PDF, TXT or read online on Scribd
You are on page 1/ 5

Parameter Maintainability

Points Can the test suit be executed without any pre-requisites to be set manually ? Does the framework handle auto update of test data, if needed? Does the framework give way to use multiple sets of data for same scenario? Is it possible to execute the entire test suite in less time, if needed? Will the framework support a different technology e.g. Can a framework implemented for a Java application be used automate a .NET application or Can a framework implemented for web application be used for a different web application without much modifications? Does the framework support both web and desktop applications?

XYZ Hydrabad

CAI Chennai

LRT Mumbai

Scalability

Standards

No hardcoded values in the main script, all the values are declared in data sheet or object file. Proper naming conventions are followed for varibales, functions and object repositiry. Proper identification and implementation of BC(Business Component), functions and procedures.

Reliability

Does the framework handle exceptions and resume/end execution gracefully. E.g Internet Explorer crash

Can we execute test suit without any manual intervention ? Reusability Avg LOC for functions / Total number of functions written : (Threshold values : . ) Less than Threshold ? Can a non-automation-engineer execute the tests Can a non-automation-engineer add new tests. Can a non-automation-engineer edit existing tests. Can a particular set of tests be executed easily ( Selective execution) Can test engineer analyze test results easily (test name, test data displayed for failures?) Failures reported by automated scripts can be understood without any further manual execution. Is there documentation around the framework usage. Note : Please see the evalauation system for frameowrk as given below. Yes No

Easy to use

1 0

SRCM Mumbai

XYZ Mumbai

Parameter Maintainability

Points to consider Are there any pre-requisites to be set manually for execution of test suite? Does the framework handle auto update of test data, if needed? Does the framework give way to use multiple sets of data? Is it possible to reduce the execution time if needed?

Scalability

Will the framework support a different technology e.g. Can a framework implemented for a Java application be used automate a .NET application Does the framework support both web and desktop applications Does the framework handle exceptions and resume/end execution gracefully. E.g Internet Explorer crash Is there a need for manual intervention during or after test execution Wht is the avg LOC for functions

Reliability

Reusability

Easy to use

Can a non-automation-engineer execute the tests Can a non-automation-engineer add new tests Can a particular set of tests be executed easily ( Selective execution) Can test engineer analyze test results easily (test name, test data displayed for failures?) Is a manual execution required for understanding the failures reported by automated scripts Is there documentation around the framework usage.

Comments Yes - 0 Yes - 1 Yes - 1 Yes - 1 Yes - 1 Yes - 1 Yes - 1 Yes - 0 To be considereed while upto the threshold ( 5 designing contest testcases marks) which can have reusable fns No - 1 No - 0 No - 0 No - 0 No - 0 No - 0 No - 0 No - 1

Ratings

more than the threshold (2 marks)

Yes - 1 Yes - 1 Yes - 1 Yes - 1 Yes - 0 Yes - 1

No - 0 No - 0 No - 0 No - 0 No - 1 No - 0

Ratings

Threshold to be decided based on the testcase steps & application

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy