The Test Therapist

This is a common question for people who are doing performance testing for the first time and also for those who don’t have specific performance requirements.

• What numbers to compare to?
• What is the current response time means?
• Is it good or bad?
• How to set an acceptance criteria?

All the above questions we used to hear in the beginning of any performance testing project or even a discussion about a future need for testing.

The best practice is to have a “Non-Functional Requirement Document” that lists all the non-functional requirements needed for the project or system you are working on.

But in case that it doesn’t exist , i will list here some ideas that will help you determine and simplify the process of setting a test acceptance criteria.

• Check websites/services working in the same domain : gather information about how their services response times are , and compare it to your current response times and if you’re way higher than them you have to plan enhancements for your current operating service(s).
  *Start with your local competitors as both of you are operating in the same market.
• Some organizations / sites publish a yearly report about the web performance in general and categorized it by business domain , this will help you have an overview about the response times trend and have at least numbers that you don’t want to exceed anyway.
  *The full article link can be found in the end of this article.

Industry	United States	United Kingdom	Germany	Japan
Automotive	9.5 sec	12.3  sec	11.0 sec	10.3 sec
Business & Industrial Markets	8.7 sec	8.3 sec	8.2 sec	8.1 sec
Classifieds & Local	7.9 sec	8.3 sec	7.0 sec	8.3 sec
Finance	8.3 sec	8.0 sec	8.6 sec	7.6 sec
Media & Entertainment	9 sec	8.8 sec	7.6 sec	8.4 sec
Retail	9.8 sec	10.3 sec	10.3 sec	8.3 sec
Technology	11.3 sec	10.6 sec	8.8 sec	10sec
Travel	10.1 sec	10.9 sec	7.1 sec	8.2 sec

• If you are doing a revamp or replacement to an old system/service try to achieve at the least the same old system performance (in case the performance wasn’t the reason for the revamp 🙂 ) and then you can plan for 20 – 30% better performance than the old system , off-course you can plan for a higher performance achievement but it should be specific to not wasting a lot of time chasing unclear goal.

To summarize, it is okay if you don’t have specific performance requirements. You can set your requirements based on how you’re operating compared to others. Also, having an initial goal is a good step to start planning your performance enhancements, and those goals will surely become more ambitious over time.

Sources :

https://www.machmetrics.com/speed-blog/average-page-load-times-websites-2018/

Average Page Load Times for 2020 – Are you faster?

Please share your tips, experience, comments, and questions for further enriching this topic of discussion.