How often you should execute Performance Testing

I think this is one of the questions that you may hear or you may want an answer for it. How often we should do it , what should be tested and how we decide if it is good or bad performance?

I can’t say that I have an absolute answer for all of this questions but I think I have an answer.

Let’s start with “How often you should run a performance testing”

Before we ask how often , let’s ask first why?

You plan , design and execute a performance testing run for one the following reasons in my opinion :

– Set a performance baseline for a running system.
– Compare performance between old system (legacy system) and new system.
– Detect performance enhancements / degradations between different versions of a software or hot-fixes (patches)

Because we ask how often it means that we already executed a performance testing before and may have a performance testing set to execute when it is needed.

Here is how often you should run a performance testing , in my opinion off course 🙂

  • If you introduce , modify or enhance a code / new code , which may affect the current running software.
  • If you modify the current environment infrastructure and also if you modify configuration(S) which may affect the system performance.
  • To simulate a load happened in production to identify the cause of production Incident related to performance (performance issues).
  • Before every peak season , mostly for e-commerce websites like “Black Friday” to make sure that everything should working as expected.

What do you think , if you have other ideas or real life scenarios you can leave it in the comment section.

How to write data from JMeter response to a csv file

I though about it when i wanted to execute a data prepration script to generate some system ids and use them in a another script , but how can i get a certain value from the response and write to a file , CSV file specifically.

In this post i will tell you how i did it 🙂

Lets try to make it that way , we will use the random article function in wikipedia website to write the article name to a csv file , so everytime the random article is triggered JMeter will write the new article name to a CSV file.

The following will be added :

  • Thread Group
  • HTTP Sampler as shown below
  • View results tree
  • Regular expression extractor as a child to the HTTP Sampler
  • BeanShell PostProcessor as a Child to the HTTP Sampler

Every time “https://en.wikipedia.org/wiki/Special:Random” will requested the value in title will be written to the CSV file

Regular Expression Extractor configuration will be as shown below :

One step is remaining , to write the “Article_Name” parameter value to a CSV file

BeanShell PostProcessor code will be as the following :

artname = vars.get("Article_Name");
f = new FileOutputStream("Results.csv", true);
p = new PrintStream(f);
this.interpreter.setOut(p);
print(artname);
f.close();

Our last step is to run the script with more than one iteration , let’s execute it with 3 iterations and then go to see the CSV fie contents.

Hope you find this article useful 🙂

Increase Number of users generated from your local machine

Number of user generated from your local machine depends on your local machine specs but what if you can generate more users with number of tricks.

1. Don’t use listners

Listners consumes a lot of memory to be able to display information and do the necessary calculations , the less listners you use the more memory is available to generate users.

Use Simple Data Wriiter to store all of your run data and after the execution is finished you can get all information you need from .jtl file created.

*We have an article explaining how to use Simple data writer , see the following link

https://thetesttherapist.com/2019/01/27/save-run-results-with-simple-data-writer-in-apache-jmeter

2. Increase Java Heap Size Limit

We’re going to change the amount of memory reserved to JMeter by default to a larger size which allow JMeter to generate more users but you cannot set the memory size to be > 80% of your total system memory

Here is the default value in APache JMeter 5 , the default value is 256m

Screen Shot 2019-02-17 at 12.28.56 PM

You can set a new value which is not larger than 80% of your system memory , JMeter will not lunch if you set a memory which is much higher than possible.

*You can change the heap size value when you edit  jmeter.bat file and search for “set HEAP” in notepad or any other editor.

3. Use x64 JDK and Windows

With x64 Windows and JDK you can consume more memory and have a better memory management that will help you generate more users than before.

*With all the above tricks your machine specs cannot exceed a certain boundary related to your hardware , and unfortunately we cannot calculate precisely how much a machine with a specific specs can generate and also it is differ from test to test.But these tricks help you get the max out of your machine before thinking to add a new machine to generate more or move to the cloud.

Stop JMeter test run when reaching a specific number of requests

You want to execute a JMeter performance test but you don’t want to exceed a specific number of requests , is it possible?

Yes it is 🙂

What if we can get the value of the current iteration and make a condition to stop the test when reaching a specific values. This can be done as the following

Create a basic JMeter test plan with the essential samplers and listeners

Test plan will have the following :

  1. Thread Group
  2. Http Sampler
  3. If controller
  4. Test Action Sampler
  5. Summary Report Listener

Thread Group run settings will be as the following :

Thread group settings is just an example you can use what is suitable to your test

We will put If controller and a test action as a child for it in the beginning of the test just under the Thread Group.

Everytime you call ${__counter(,)} you got the current iteration number.
in the above example we need to set a maximum of 300 requests (iterations).

“${__counter(,)}” >= 301

So why i put the condition operator to be >=301 because in case of concurrency it will be easier to check for number of iteration after you already finished your 300 iterations , give it a try if you put 300 the test will be stopped at the iteration number 299.

Test Action will be a child to the IF Controller

When the condition is true the Test Action will stop the test.

Then you have to add http request , in the example we used “https://en.wikipedia.org/wiki/Main_Page”

At the end add summary report to check the number of request and run the test

Summary

With IF Controller and Test Action you can limit your test execution maximum number of requests While you executing a duration based test with multiple threads.

Get Max Number of User Concurrency in JMeter

In JMeter default lisnters some values are not provided by default and one of the examples is the max number of user concurrency which sometimes important if you’re executing a test which is not duration based.

In this article we’re going to execute a test with 200 users ramping 1 user each 2 sec and let’s see what is the max user concurrency we can get.

But first we need to install a plugin and add a listner as the following :

  1. install jmeter plugin manager from the link below
    https://jmeter-plugins.org/wiki/PluginsManager/
  2. After installation , open the plugin manager from jmeter and select the following plugin “3 Basic Graphs” from “Available Plugins” section
    Screen Shot 2018-07-14 at 11.52.10 AM
  3.  Make sure that you have a new listner under called “Active Threads Over Time” as the following picture
    Screen Shot 2018-07-14 at 12.02.43 PM

Now everything is in place lets make a quick run.

we’re going to use this fake API website “http://jsonplaceholder.typicode.com/” in our test

Screen Shot 2018-07-14 at 12.10.07 PM

Screen Shot 2018-07-14 at 12.10.34 PM

Screen Shot 2018-07-14 at 12.12.04 PM

Let’s execute and see the graph results 😉

Screen Shot 2018-07-14 at 12.19.43 PM

If we look at the graph above , we got a max of 1 concurrent user during the test run , so lets change the ramping up values and see the changes.

we will modify the thread group to be 200 threads / ramp-up Period : 50 as the following and execute again.

Screen Shot 2018-07-14 at 12.25.50 PM

Screen Shot 2018-07-14 at 12.27.09 PM

If we look at the graph above , in 2 times during the test we had more than 1 concurrent users. Because this is a simple http call with no business behind we got a smooth run and nearly constant results but in more real life test this graph will help you know the max number of concurrent users during your test.

 

 

Generate GUID in Apache JMeter

Although there is no dedicated sampler in JMeter to generate GUIDs but you can use the built-in functions to generate them.

In this article i will share with you how to generate GUID that is unique by user and iteration and use it through your JMeter thread group or test plan.

Steps:

  1. Open JMeter and add “Bean Shell Sampler”under your Thread Group

Screen Shot 2018-07-13 at 10.49.31 AM

2. Add a user defined variable that will hold the GUID value , set the varible name and leve the value empty

Screen Shot 2018-07-13 at 10.53.15 AM

3. Enter the following command in beanshell sampler
“vars.put(“VariableName”, “${__UUID}”);” where variable name is the name of variable created in the previous step.

Screen Shot 2018-07-13 at 11.01.10 AM

4. Now we need to verify that it is working so , we will add “Debug Samper” & “View Results Tree”

Screen Shot 2018-07-13 at 11.03.50 AM.png

Screen Shot 2018-07-13 at 11.05.45 AM

5. Save , Run and check the debug sampler values in the results tree

Screen Shot 2018-07-13 at 11.08.50 AM

6. now you can place the ${GUID} variable wherever you want in the thread group and it will generate a unique value for each user and each iteration (whenever the beanshell sampler executed).

*you can genrate more than one GUID everytime by pasting the line more than on time in the beanshell sampler and also add more variables in the user defined variables to hold the values.