What a performance test report saying about your system.

When evaluating a performance test report most of the times we are looking for the response time and specifically the Average response time.

But if you take a deeper look , the performance test report elaborates more information.

In this article I will use one of JMeter basic reports “Summary Report” as example to explain what I mean.

The focus in this article will be on the following terms/values

  • Standard Deviation
  • Min Response Time
  • Max Response Time

Standard Deviation :

The Standard Deviation is a measure of how response time is spread out around the Mean. Simply say, the smaller the Standard Deviation, the more consistent the response time.

Transaction NameRT
(I1)
RT
(I2)
RT
(I3)
RT
(I4)
RT
(I5)
AvgSD90th %ile
Login46348526
Search32151455.74
Logout5564550.75
“Logout” transaction having lowest Standard Deviation (0.7) it shows response times are more consistent than other two.

Standard Deviation in your test tells whether the response time of a particular transaction is consistent throughout the test or not? The smaller the Standard Deviation, the more consistent transaction response time and you will be more confident about particular page/request.

Min.Response Time:

The shortest time taken by a sample for specific label. If we look at Min value for Label 1 then, out of 20 samples shortest response time one of the sample had was 584 milliseconds.

Max.Response Time:

The longest time taken by a sample for specific label. If we look at Max value for Label 1 then, out of 20 samples longest response time one of the sample had was 2867 milliseconds.

Sources :

https://www.perfmatrix.com/standard-deviation-in-performance-testing/http://www.testingjournals.com/understand-summary-report-jmeter/

Please share your tips, experience, comments, and questions for further enriching this topic of discussion.

Published by

Mohamed

Test Specialist with 10 years of testing experience with more focus on performance and security testing| ISTQB Certified Performance Testing, EC-Council Certified Ethical Hacker - V9, HP Loadrunner 12.x Certified, ISTQB Foundation Level Certified

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.