Socket Load Testing
There are many stable open source load/performance testing tools like JMeter, Apache Bench, Artillery and so on in the internet right now. Here I am just sharing my professional experience from my work space.
As our websocket is not the traditional one and we use Socket.io which is basically a library that enables real-time, bidirectional and event-based communication between the browser and the server. So, Here we will use Artillery as it includes socket.io as well as regular websocket servers.
Artillery:
Artillery is an advanced, powerful & easy-to-use load testing and functional testing toolkit. Use it to ship scalable applications that stay performant & resilient under high load. It supports various reports from latency to concurrent users.
Installing Artillery:
Step 1: Artillery is written in Node.js. So, need to install node.js
Official site: https://nodejs.org/en/download/
Now check, whether node.js is installed properly or not
a. Open cmd/PowerShell from your machine
b. Type: node -v
And you will see the version of node.
c. You can also try with: npm -v
And you will get the version of npm.
Step 2: Now install Artillery using either npm or yarn.
# Via npm
Type: npm install -g artillery
# Via Yarn
Type: yarn global add artillery
Step 3: To check if it is installed correctly or not
Type: artillery — — version
Design Test Scenario:
Firstly, it is essential to design your load test scenario, before trying to config any load tester to test your service.
I prefer to divide my load test scenario into 3 phases:
- Phase 0: Warming up the web socket service with a short and light. load.
- Phase 1: Ramping up the previous load and trying to achieve the target load with the capacity of the service, OS, and hardware.
- Phase 2: Maximize the load beyond the target.
Artillerry.io test profiles are in YAML format. I start to create my test scenario profile in the YAML file.
Three phases:
- The first phase runs for 30 seconds with the arrival rate of 50, which means every second 50 new users are added to the test. So, total connection will be 30*50=1500.
- The 2nd phase with the rampUp option, runs for 40 seconds and every second it should ramp arrivalRate 50 to 55 users.
- In final phase it will maximize the load by increasing the duration 45 seconds and arrivalRate 55 users.
The final part of the load test profile is to define the operation or the scenario.
If you take a close look at the scenarios section, you can see that first I defined the engine as “socketio”. -think: 60 means connection will remain open for 60s.
Inside the flow action, I am sending a http get request but you can also do sending data with send and pausing execution with think.
Run and Execute:
Now, save the file as loadtest.yml . To run the test, execute the following command:
$ artillery run loadtest.yml — — output result.json
It makes artillery run the test with your test profile and saves the report in JSON format as result.json.
You may get following kind of warning. So for getting accurate performance better run this script from highly configured server as overall performance depends on your CPU usage.
Warning:
CPU usage of Artillery seems to be very high (pids: 10112)which may severely affect its performance.See https://artillery.io/docs/faq/#high-cpu-warnings for details.
Report Generate:
You can read some useful information from the load test in the console, but we asked Artillery to save the results inside a JSON file. Artillery can convert JSON result into a human friendly HTML report.
$ artillery report .\result.json
By running the command above, it will generate the HTML report and opens it inside a browser.
Different charts are about requests per second, latency, concurrent users and etc.
Limitations:
As of now Artillery doesn’t have any option to check the socket response.
Moreover Artillery is a powerful tool to make a load test on your WebSockets. It’s easy to configure, and it supports a wide range of configuration for your load test scenarios.
