When to Automate the Application with QTP
Source : Linkedin Discussions
I am posting this topic as a discussion
With QTP, you need a stable front end (GUI) on the app before you start automation testing. If the GUI is undergoing significant change, you are likely to run into maintenance issues with automated tests as the objects on the GUI will be changing.
That said, just like with any sort of testing, there is nothing to stop you planning your automation testing ahead of time.
In terms of working out what to automate, when I teach a QTP class I usually suggest to my students that as a rule of thumb, if a test is going to need to be run more than 50 times, then it is a good candidate for being automated as the ROI will increase.
Regards,
Michael Weinstock
Melbourne, Australia
That said, just like with any sort of testing, there is nothing to stop you planning your automation testing ahead of time.
In terms of working out what to automate, when I teach a QTP class I usually suggest to my students that as a rule of thumb, if a test is going to need to be run more than 50 times, then it is a good candidate for being automated as the ROI will increase.
Regards,
Michael Weinstock
Melbourne, Australia
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Automation can start from the very beginning of the project.If we have have a prototype of the web application then the automation engineer can start work on automation. The Test automation will have various activities like Feasilibility analysis,Tool selection, skill set development, training on the tool, Framework design, script development, Script verification, Script execution and bug reporting.
- Automation is basically driven by the bussiness needs. When i mean the bussiness needs it basically refers to cost and benifits . One needs to ask himself the following questions :
- Does the project demand the reduction of the lifecycle cost of the software product ?
- Does the project some kinds of testing activities that cannot be run manually such as (Memory leak, concurrent, load, ) ?
- Does the project/product span over releases, where the tester will have to run through rounds and rounds of regression tests. In that case there is a likelihood of mundaneness being introduced into the work of the test engineer ? The test engineer can be used more effectively to do other complex area.
If answers to all the questions is yes then there is a need for the Test Automation in the project.
- Automation is basically driven by the bussiness needs. When i mean the bussiness needs it basically refers to cost and benifits . One needs to ask himself the following questions :
- Does the project demand the reduction of the lifecycle cost of the software product ?
- Does the project some kinds of testing activities that cannot be run manually such as (Memory leak, concurrent, load, ) ?
- Does the project/product span over releases, where the tester will have to run through rounds and rounds of regression tests. In that case there is a likelihood of mundaneness being introduced into the work of the test engineer ? The test engineer can be used more effectively to do other complex area.
If answers to all the questions is yes then there is a need for the Test Automation in the project.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Vijay thank you for your answer. Could you please tell an example. Below are more ?'s
1) Take a scenario like "A web page/Win Application has 3 text boxes, 2 buttons, 1 lable control" My questions is can we automate this scenario. If yes, then how can we do that.
2) As per Michael can't we automate more than 50 times and less than 50 times. If yes, can you please explain with an example.
3) What is the major difference between Automation and Manual testing.
1) Take a scenario like "A web page/Win Application has 3 text boxes, 2 buttons, 1 lable control" My questions is can we automate this scenario. If yes, then how can we do that.
2) As per Michael can't we automate more than 50 times and less than 50 times. If yes, can you please explain with an example.
3) What is the major difference between Automation and Manual testing.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Following are my view on the questions :
1) It can definitely be automated. But i will suggest you to do the following steps
- Do a feasibility analysis on the existing manual test cases so that you can decide on which test cases you are going to automate.
- Get a buy in from the lead/manage/management before you start of the automation.
- Decide on the tool that you want to use. ( Open source or purchased tool)
- Scripting language that you/ your team is comfortable with.
- Build a framework .
- Script the case and verify the same.
2) If there is a cost and time benefit that your project is going to deliver then it is defenetly should be automated. The best example is suppose you have started automation on a banking project which needs to be tested for lot of ( say 100 scenarios) and the life span of the project is 1 and half year.Then it can be a candidate for automation.
3) Major difference what i feel from the Automation testing over the manual testing are :
- Can be run unattended
- Can be used to cover lot of scenarios, which manually would be tedious.
- Some of the tests cannot be run manually . This has to be automated ( such as concurrency, Load, Stress, Volume, memory leak).
- Can be tested for lot of combination of the data.
- Can be used to generate the test results that need minimal analysis.
These are some of the difference there are lot more .. :)
1) It can definitely be automated. But i will suggest you to do the following steps
- Do a feasibility analysis on the existing manual test cases so that you can decide on which test cases you are going to automate.
- Get a buy in from the lead/manage/management before you start of the automation.
- Decide on the tool that you want to use. ( Open source or purchased tool)
- Scripting language that you/ your team is comfortable with.
- Build a framework .
- Script the case and verify the same.
2) If there is a cost and time benefit that your project is going to deliver then it is defenetly should be automated. The best example is suppose you have started automation on a banking project which needs to be tested for lot of ( say 100 scenarios) and the life span of the project is 1 and half year.Then it can be a candidate for automation.
3) Major difference what i feel from the Automation testing over the manual testing are :
- Can be run unattended
- Can be used to cover lot of scenarios, which manually would be tedious.
- Some of the tests cannot be run manually . This has to be automated ( such as concurrency, Load, Stress, Volume, memory leak).
- Can be tested for lot of combination of the data.
- Can be used to generate the test results that need minimal analysis.
These are some of the difference there are lot more .. :)
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Q: when exactly an automation will start in the project.?
My Answer: Before starting to automate you must know whether or not we really need to automate the application... for that you have to meet the actors of the project: a tester a business analyst, an IT developer etc.. who know very well the application and who are lead to implement the test Plan and the test strategy etc... during the meeting you have to answer the following criteria:
1- did all expected results reproductible, or predictable? (Desired Answer: yes)
2- did the insurance application stable with expected behaviour? (A: stable)
3- Is it necessary to have human intervention? (A: No)
4- How long take a test to last? (A: Not a long time)
5- What are the hard tests to execute manualy? .....
6- What are the technologies used to buil th insurance application? (A: the technos must be compliant with the QTP addins you have)
7- Tests are predictable, reusable? (A: Yes, Yes)
So when you have all the answers then you can decide to do automation.
About a previous post on number of execution, the formula to calculate the ROI for a test is:
ROI = [(Nb of executions * (time to execute the test manualy - time to execute it with automation) - time spent to design the test]/time spent to design the test
The aim is to have an ROI > > > than 1, and this is possible if we have a very important number of executions..
Kind Regards,
Amine
My Answer: Before starting to automate you must know whether or not we really need to automate the application... for that you have to meet the actors of the project: a tester a business analyst, an IT developer etc.. who know very well the application and who are lead to implement the test Plan and the test strategy etc... during the meeting you have to answer the following criteria:
1- did all expected results reproductible, or predictable? (Desired Answer: yes)
2- did the insurance application stable with expected behaviour? (A: stable)
3- Is it necessary to have human intervention? (A: No)
4- How long take a test to last? (A: Not a long time)
5- What are the hard tests to execute manualy? .....
6- What are the technologies used to buil th insurance application? (A: the technos must be compliant with the QTP addins you have)
7- Tests are predictable, reusable? (A: Yes, Yes)
So when you have all the answers then you can decide to do automation.
About a previous post on number of execution, the formula to calculate the ROI for a test is:
ROI = [(Nb of executions * (time to execute the test manualy - time to execute it with automation) - time spent to design the test]/time spent to design the test
The aim is to have an ROI > > > than 1, and this is possible if we have a very important number of executions..
Kind Regards,
Amine
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Hi All,
With regards to my comment about "50 times", it is simply a rule of thumb that I teach my QTP students as a starting point when considering automation ROI. It usually does not make sense to automate everything. So where do you start? I ensure that my students get a grasp on reality which is, you need to be smart about what you automate. It does not necessarily make sense to automate a manual test that might only be run a couple of times and has high complexity. You might end up spending days automating a test that takes perhaps 20 minutes to run. Where is the ROI to automate 40 minutes worth of manual testing when it might take 2 days to setup the QTP automation for the same?
The example of "50 times" gets my students thinking about where the value resides in automation. If a manual test is going to be run 50 times, then chances are there is a very good ROI to spend the time automating the test.
Of course, being a rule of thumb, there are exceptions.... this is not a hard and fast rule, but it does get my students thinking about automation ROI and grasping the concept of selective and smart automation, rather than "lets automate everything because we have a really cool automation tool".
Regards,
Michael Weinstock
Test Specialist
Melbourne, Australia
With regards to my comment about "50 times", it is simply a rule of thumb that I teach my QTP students as a starting point when considering automation ROI. It usually does not make sense to automate everything. So where do you start? I ensure that my students get a grasp on reality which is, you need to be smart about what you automate. It does not necessarily make sense to automate a manual test that might only be run a couple of times and has high complexity. You might end up spending days automating a test that takes perhaps 20 minutes to run. Where is the ROI to automate 40 minutes worth of manual testing when it might take 2 days to setup the QTP automation for the same?
The example of "50 times" gets my students thinking about where the value resides in automation. If a manual test is going to be run 50 times, then chances are there is a very good ROI to spend the time automating the test.
Of course, being a rule of thumb, there are exceptions.... this is not a hard and fast rule, but it does get my students thinking about automation ROI and grasping the concept of selective and smart automation, rather than "lets automate everything because we have a really cool automation tool".
Regards,
Michael Weinstock
Test Specialist
Melbourne, Australia
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Comments
Post a Comment