Does it necessary , why , for what cost and limitations ,all these questions are valid and make totally sense.
Let me first define what is meant by running performance tests from the cloud , in the simple form it is running performance test scripts from a rented virtual machine(s) from a cloud service provider like (AWS or Azure) or testing platform which offers a cloud service run like (Blazemeter , Microfocus,..)
Does it necessary ? , the answer is yes in some situations like the following :
- Generate huge amount of users which requires a highly equipped machines which cost fortune if you build it locally.
- Simulate different regions and countries which is required in your test run , this can be provided by multiple cloud service platforms which has different server locations around the world.
- Reliability and consistency for the used machines , most cloud platforms guarantee more than 90% availability for their services.
Why ? , as I mentioned above in some cases massive resources are required and this can be easily provided by cloud platforms than build your own load generators on-premise.
For what cost ? , you are billed based on hourly rate according to the selected machine and its hardware , sometimes is cheaper to rent than build your own lab. But in some cases based on the usage and frequency , it is cheaper to build your own lab.
Limitations ? , if your application or software is not available online or it is only accessible through internal organization / company network , the cloud testing will not be the best fit in this case.
Also if you have some resources that you can use instead of renting.
example : use the company desktops to generate the users instead of create a cloud load generators.
The final reason is the cost , if you don’t have the budget to rent cloud machines , in this case use your company assets is the best solution to complete the task.
To sum up , running performance tests from the cloud is not a must or a trend , it is a go to solution if need it. It saves time , configuration hassle and sometimes money but it is not the magic solution for every performance run.
Please share your tips, experience, comments, and questions for further enriching this topic of discussion.