When evaluating a performance test report most of the times we are looking for the response time and specifically the Average response time.
But if you take a deeper look , the performance test report elaborates more information.
In this article I will use one of JMeter basic reports “Summary Report” as example to explain what I mean.
The focus in this article will be on the following terms/values
Standard Deviation
Min Response Time
Max Response Time
Standard Deviation :
The Standard Deviation is a measure of how response time is spread out around the Mean. Simply say, the smaller the Standard Deviation, the more consistent the response time.
Transaction Name
RT (I1)
RT (I2)
RT (I3)
RT (I4)
RT (I5)
Avg
SD
90th %ile
Login
4
6
3
4
8
5
2
6
Search
3
2
15
1
4
5
5.7
4
Logout
5
5
6
4
5
5
0.7
5
“Logout” transaction having lowest Standard Deviation (0.7) it shows response times are more consistent than other two.
Standard Deviation in your test tells whether the response time of a particular transaction is consistent throughout the test or not? The smaller the Standard Deviation, the more consistent transaction response time and you will be more confident about particular page/request.
Min.Response Time:
The shortest time taken by a sample for specific label. If we look at Min value for Label 1 then, out of 20 samples shortest response time one of the sample had was 584 milliseconds.
Max.Response Time:
The longest time taken by a sample for specific label. If we look at Max value for Label 1 then, out of 20 samples longest response time one of the sample had was 2867 milliseconds.
Let me tell you that you can have a basic / moderate security checks in your manual test suite by extending your test cases in two different areas :
Input validation
Authentication
Most of test suites if not all of them are already testing the sections mentioned above but mostly just a basic checks like if the field accepting numbers we try characters and alpha numerics. What I am suggesting here is to test any input field against major web app vulnerabilities like XSS & SQL Injection
The same case for authentication instead of trying different combinations for right / wrong usernames and passwords .You can extend your test against major web app vulnerability like SQL Injection
XSS : XSS attacks enable attackers to inject client-side scripts into web pages viewed by other users. A cross-site scripting vulnerability may be used by attackers to bypass access controls such as the same-origin policy.
We’re going to use demo.testfire.net as a safe place to practice on.
Example :
The field we’re going to test here is the Search field , I am assuming that we already executed test cases with character , numbers , alphanumeric and also very long characters / numbers input to validate the input boundary.
Lets try an XSS payload as input.The input will be “<ScRipT>alert(“XSS”);</ScRipT>”
According to the above screen shot it seems that the web app under test is vulnerable to XSS attacks.
SQL Injection : SQL injection is a code injection technique, used to attack data-driven applications, in which malicious SQL statements are inserted into an entry field for execution.
The element we’re going to test here is the Login form , I am assuming that we already executed test cases with valid and invalid username , password also very long characters / numbers input to validate the input boundary.
Lets try SQL Injection payload in the “username” field and any characters in the password field.
username value will be = ‘ ‘ password value will be = test
Lets try more advanced input as below
username value will be = ‘ UNION SELECT sum(columnname ) from tablename — password value will be = test
According to the above screen shot it seems that the web app under test is vulnerable to SQL Injection attacks.
By adding more test cases to your existing test suite you can help discover security vulnerabilities in the system under test without the need to learn a new tool.Of course it will increase the testing execution time but the benefit here is to catch those issues as soon as the testing started.
Of course this not a replacement for security testing a web app but the idea here is cover at least some basic security checks in the normal testing process.
Please share your tips, experience, comments, and questions for further enriching this topic of discussion.
The main goal for any performance test script simply is to work , but is this enough? I mean is it enough that your script has no errors ?
No errors do not mean that your script is working flawlessly , you may get a 200 response code but the script functionality is not working and in this case all your results are not correct.
The process of checking if you receive the correct response is called Validation.
In this article I will demonstrate the validation process using JMeter as one of the performance testing tools that is widely being used.
How to use validations in JMeter
In JMeter context menu we have a whole section called Assertions as in the image below
As you can see above there are lots of assertions available to use but we will focus on one particular called “Response Assertion”
Response Assertion
Before we start to dig more deeper let’s have an example to show when the response code doesn’t mean that the script is working correctly.
Example
Our script should do the following :
1- Open “demo.testfire.net”
2- Open the Login page
3- Do the Login with the credentials (admin/admin)
In the following 2 screen shots we will Show that having a success repose code doesn’t mean that the scenario went well.
The do login has a 200 response code
According to the above screen shot the do Login should be done successfully and user should be already logged-in
But Actually the login didn’t happen , so this step of our scenario is not a successful one.
The reason that this step is failing is because I disabled the “HTTP cookie Manager” which is in most cases required in the login scenarios.
Let’s try to use the Response Assertion we mentioned earlier in this article and try to validate our scenario but before we do this let’s enable the Cookie manager to choose which text we can use in our validation step.
Now we have a successful login , so I think we can use the “Sign off” text as our asserion , because the sign off link will not be displayed if the user is not logged-in.
I added a response assertion as a child to the do login request , use the Text response and also put “Sign off” as the text to search for in the response.
I will do a trial with the Cookie manager on , then I will re-run the test with the cookie manager off to check that our validation (assertion) is working.
When I execute the test with the Cookie manager disabled , now we have a failed request although we have a 200 response code as shown on the following image.
Text Assertion is not the only assertion we can use but I think it is the mostly used one , and it will help you validate from the script side that you test is doing what should be done , help you have accurate results and have look about how your script and system under test is behaving.
*The JMX used in this article is uploaded here , feel free to use.
The performance test process is not a complex process but it has lots of things to keep an eye on , in this article I will focus on factors that personally I think it will affect your test results. I will try to demonstrate the effect in most of them and I will put some references if exist.
Think Time :
By definition think time is the time between the completion of one request and the start of the next request.
So we can say that it is a kind of delay. So obviously when you have long think time value , it means long delays and less pressure on the system under test and also means if we have no think time this means more pressure on the system under test.
Short / No think time = More pressure
Too Long / Long think time = Less / no pressure
The question will be how to determine the suitable think time for your system? , you can determine how long time user spend on your system pages from analytical tool like “Google Analytics”
Google Analytics – Audience Overview
From the above screenshot we can roughly determine how long time the user spend on each page during his session (the the value is the average of all users in a specific range of dates)
What you can do , you can randomize a value between lower and upper think time values to have different think time value per request / per user.
The more close to real life the values you use , the more realistic results you will get.
Generating Users :
In most of performance testing tools if not all of them you have at least two options for user generation :
Constant Load : which means that all users will start to hit the server the same moment you will start the test.
step-up load / ramp-up : which means that we will introduce new user/thread every specified amount of time.
Micro Focus Load Runner – constant user generationMicro Focus Load Runner – Ramp-up user generation
In most cases the ramp-up user generation will be the better approach except you want to test a specific scenario , because hitting the system under test with all users at the same moment is not an ideal scenario and sometimes It is not realistic. It will affect the test response time badly if the system is not design to sustain this kind of users hit.
There is no ideal number for the step duration so it can be tweaked during the test run or try to get this information from the analytics the same we described with the think time above.
Test Data :
Data used during the performance test run is important , the more close to real life the data the more accurate results you will get.
Also avoiding using the same data for all generated users like (user credentials , search keywords , etc) will eliminate the factor that caching may affect the test results.
Use unique data for each generated user and make sure that you have enough data to use during your test run.
Latency
Latency is the time from simply sending out the request until the first byte of response is accepted, it is also called as Time to First Byte.
You will always have a latency in your test if you are not testing in an ideal test environment.
But you can reduce the latency value by placing your remote machines in the closest region to your hosted application.
You have to initiate your test from the same region or close to the region your real users will access the application from.This will lead to more accurate test results.
Load generators
It is normal that most of test executions are initiated from a one machine / server if the number of users generated is not a large number.
But it is recommended to distribute the load generation among different machines / servers even the number of users is not that large.
This will help to balance the load on the system under test and avoid some security restriction for the hitting frequency from the same host.
Have you faced problems before related to mentioned factors? How you managed it? Please share your tips, experience, comments, and questions for further enriching this topic of discussion.
I think this is one of the questions that you may hear or you may want an answer for it. How often we should do it , what should be tested and how we decide if it is good or bad performance?
I can’t say that I have an absolute answer for all of this questions but I think I have an answer.
Let’s start with “How often you should run a performance testing”
Before we ask how often , let’s ask first why?
You plan , design and execute a performance testing run for one the following reasons in my opinion :
– Set a performance baseline for a running system. – Compare performance between old system (legacy system) and new system. – Detect performance enhancements / degradations between different versions of a software or hot-fixes (patches)
Because we ask how often it means that we already executed a performance testing before and may have a performance testing set to execute when it is needed.
Here is how often you should run a performance testing , in my opinion off course 🙂
If you introduce , modify or enhance a code / new code , which may affect the current running software.
If you modify the current environment infrastructure and also if you modify configuration(S) which may affect the system performance.
To simulate a load happened in production to identify the cause of production Incident related to performance (performance issues).
Before every peak season , mostly for e-commerce websites like “Black Friday” to make sure that everything should working as expected.
What do you think , if you have other ideas or real life scenarios you can leave it in the comment section.
I though about it when i wanted to execute a data prepration script to generate some system ids and use them in a another script , but how can i get a certain value from the response and write to a file , CSV file specifically.
In this post i will tell you how i did it 🙂
Lets try to make it that way , we will use the random article function in wikipedia website to write the article name to a csv file , so everytime the random article is triggered JMeter will write the new article name to a CSV file.
The following will be added :
Thread Group
HTTP Sampler as shown below
View results tree
Regular expression extractor as a child to the HTTP Sampler
BeanShell PostProcessor as a Child to the HTTP Sampler
Regular Expression Extractor configuration will be as shown below :
One step is remaining , to write the “Article_Name” parameter value to a CSV file
BeanShell PostProcessor code will be as the following :
artname = vars.get("Article_Name"); f = new FileOutputStream("Results.csv", true); p = new PrintStream(f); this.interpreter.setOut(p); print(artname); f.close();
Our last step is to run the script with more than one iteration , let’s execute it with 3 iterations and then go to see the CSV fie contents.
Number of user generated from your local machine depends on your local machine specs but what if you can generate more users with number of tricks.
1. Don’t use listners
Listners consumes a lot of memory to be able to display information and do the necessary calculations , the less listners you use the more memory is available to generate users.
Use Simple Data Wriiter to store all of your run data and after the execution is finished you can get all information you need from .jtl file created.
*We have an article explaining how to use Simple data writer , see the following link
We’re going to change the amount of memory reserved to JMeter by default to a larger size which allow JMeter to generate more users but you cannot set the memory size to be > 80% of your total system memory
Here is the default value in APache JMeter 5 , the default value is 256m
You can set a new value which is not larger than 80% of your system memory , JMeter will not lunch if you set a memory which is much higher than possible.
*You can change the heap size value when you edit jmeter.bat file and search for “set HEAP” in notepad or any other editor.
3. Use x64 JDK and Windows
With x64 Windows and JDK you can consume more memory and have a better memory management that will help you generate more users than before.
*With all the above tricks your machine specs cannot exceed a certain boundary related to your hardware , and unfortunately we cannot calculate precisely how much a machine with a specific specs can generate and also it is differ from test to test.But these tricks help you get the max out of your machine before thinking to add a new machine to generate more or move to the cloud.
You want to execute a JMeter performance test but you don’t want to exceed a specific number of requests , is it possible?
Yes it is 🙂
What if we can get the value of the current iteration and make a condition to stop the test when reaching a specific values. This can be done as the following
Create a basic JMeter test plan with the essential samplers and listeners
Test plan will have the following :
Thread Group
Http Sampler
If controller
Test Action Sampler
Summary Report Listener
Thread Group run settings will be as the following :
Thread group settings is just an example you can use what is suitable to your test
We will put If controller and a test action as a child for it in the beginning of the test just under the Thread Group.
Everytime you call ${__counter(,)} you got the current iteration number. in the above example we need to set a maximum of 300 requests (iterations).
“${__counter(,)}” >= 301
So why i put the condition operator to be >=301 because in case of concurrency it will be easier to check for number of iteration after you already finished your 300 iterations , give it a try if you put 300 the test will be stopped at the iteration number 299.
Test Action will be a child to the IF Controller
When the condition is true the Test Action will stop the test.
At the end add summary report to check the number of request and run the test
Summary
With IF Controller and Test Action you can limit your test execution maximum number of requests While you executing a duration based test with multiple threads.
One of the things that any new person who is dealing with JMeter is facing that if you save a jmx project which is executed before the run results is not saved.
So if you didn’t know this you may lose your run results and you have to repeat again.
One of the best ways to save your run data is using “Simple Data Writer” it not only make you able to save your run data but also enable you to use the data file on any listner or graph you want or need.
Data Files
CSV is one of the file extensions you can use to save your run results as the following example.
In the above example the results table will be saved to a .CSV file which is good in case you want only the data shown in the table but if you want more the Simple data writter will be a better option.
What are JTL files?
JMeter can create text files containing the results of a test run.
These are normally called JTL files, as that is the default extension – but any extension can be used.
Simple Data Writer
You can configure the results save configuration the more you save the more listners you can use.
I used to select them all , sometimes this leads to larger size data file but this is safer to use in different graphs and listners as i will show you later.
Lets add simple data writter to thread group and disable all the other listners and execute test as the following.
After the run is finished i’m going to open the jtl file in the following listners :
View Results Tree
Summary Report
Active Threads over time graph
Hits per second graph
As shown above the results data saved using Simple Data Writer can be used in any of the default JMeter listners and also extra listners which can be added to JMeter in the feature , .JTL extension is the default saving extensions which can be re-saved to CSV file if needed , see the following Wiki Page
Do not use the following instructions to exploit others websites / services , Usage of WP-Scan for attacking targets without prior mutual consent is illegal.
WordPress now powers 30 percent of the web, according to data from web technology survey firm W3Techs.
WordPress is important framework and as the aove statistics it is widely used thats why securing a web site based on WordPress framework is something obvious.
What is WP-Scan ?
WPScan is a free, for non-commercial use, black box WordPress vulnerability scanner written for security professionals and blog maintainers to test the security of their sites.
Installation
You can install it via Docker with the following commands :
docker pull wpscanteam/wpscan
Scan WordPress Site
Default Scan
docker run -it –rm wpscanteam/wpscan –url Website URL
Using default options with a good compromise between speed and accuracy. For example, the plugins will be checked passively but their version with a mixed detection mode (passively + aggressively). Potential config backup files will also be checked, along with other interesting findings.
Only Display Vulnerable Plugins
docker run -it –rm wpscanteam/wpscan –url Site URL –enumerate vp
Enumerate User Name
docker run -it –rm wpscanteam/wpscan –url Site URL –enumerate
