In a market where performance testing is becoming increasingly important, we will have a closer look at two tools tailored to the job.

• Neoload is a tool made by Neotys. They invested more than 15 years in the tool and describe it as a testing platform covering cloud-ready, microservices architected, mobile and enterprise-grade packaged apps.

• Octoperf is sometimes referred to as Jmeter on steroids, the creators of the tool had a background at Neotys. Both the SaaS load testing solution and the on-premise Enterprise Edition comes with a Web UI. You can design, set your load policy, execute your load test, and analyze it directly from a web browser.

Now, let’s have a look at the 3 major steps in performance testing, (1) design, (2) scenario and (3) results.

1. DESIGN

A similar design can be found in Octoperf. The functionalities are more or less the same, but they use a container-like setup, the look and feel is somewhat more modern compared to the one from Neoload. The design functionality also works really well, the only drawback compared to Neoload is that there is no record functionality. However, they do support importing har files, so if you use your browser console to record your script, it shouldn’t be an issue.  

2. BUILDING THE SCENARIO 

Here we see a similar and complete setup. You can define how many scripts you want to add to the scenario, the number of virtual users, the ramp up can be set, the devices you want to use, the OS, … The only difference is in the cloud setup.  

Based on the price, Octoperf will allow the user to run a certain amount of scripts with x number of Vusers. They are compatible with digital ocean and amazon, so you can easily choose which load generator(s) to use in your script per region. 

Neoload has its own Neotys cloud platform that also interacts with the major players like Amazon, Microsoft azure, Google cloud platform, … but theirs is a ‘pay as you go’ pricing model. You can reserve your load generators from over 70 locations around the world and scale them. 

3. RESULTS

Both tools provide you with all the ability for live tracking. Comparison over time is possible which allows you to set you own benchmark, you can set your own KPI’s or gather data from the servers that you want to monitor, … One nice feature that Neoload has, compared to Octoperf, is the web view. This gives you the possibility to share your test results (live or finished), with your team or with project owners, via a web browser. This can be really useful to get more people involved in your performance test. 

TO SUMMARIZE 

Both Neoload and Octoperf are really great tools. The most important quality they both have is that they give the user full flexibility throughout the complete setup of the performance test.  

• There are ways to record your basic script and make it more dynamic by using correlation, parameters, variables, … 
• You can schedule your scripts in a scenario, customize your ramp up, define which OS or device you wish to use to run your scenario + define the geographical location 
• Different kind of metrics are available, not only the transaction times are given but you can also check all the data from the different servers like CPU, memory, …  
• … 

If your company is ever in need of a performance testing tool, these are two good candidates to consider. They both have a free version up to 50 users, so you can first do an initial trial before buying licenses for more virtual users.  

If you want to know more about these tools, then please let us know. We are more than happy to share our findings. 

 stef.geeurickx@brightest.be