Testing with dynamic analysis tool - k6

[DhanyaShah]

This PR integrates dynamic performance testing for our NodeBB instance using the k6 tool. k6 is an open-source tool designed for load testing, focusing on reliability and performance under various user loads. It allows us to simulate different user traffic patterns, such as ramp-up, stable load, and spike testing, to evaluate how well the NodeBB instance handles concurrent requests.

In order to run this tool, I first installed it using homebrew. Then, I added the three following test files:

• k6-short-test.js: Implements a short test that simulates 300 concurrent users over a 9-minute window.
• k6-long-test.js: Implements a longer stage test simulating 5000 users over 16 minutes.
• k6-spike-test.js: Implements a spike test with two peaks of 15,000 users, lasting 2 minutes each, with rapid ramp-down in between.

These tests were run locally using the "k6 run <filename.js>" command.

Evaluation:

1. What types of problems are you hoping your tooling will catch? What types of problems does this particular tool catch?
   The k6 tool is designed to catch performance bottlenecks and scalability issues by simulating various load conditions. It can identify problems such as high latency, increased error rates under heavy loads, and resource exhaustion. k6 will help uncover response time degradations, failure to handle concurrent users, and bandwidth limitations that could affect the user experience. The test output from k6 includes several key performance metrics, such as average latency for various parts of the HTTP request lifecycle, as well as the maximum number of concurrent users reached during the test. Additionally, it provides details on error types encountered, along with the average response times under errors and tracks the total data sent and received.

2. What types of customization are possible or necessary?
   k6 provides extensive customization, allowing you to configure user load patterns (e.g., ramping up, sustained load, or spiking), adjust request intervals, and set different test durations. You can also customise the types of requests made (GET, POST, etc.) as well as adjust virtual user count based on your expected load, simulating different scenarios like stress tests, endurance tests, or spike testing.

3. How can/should this tool be integrated into a development process?
   k6 can be integrated into the development process by incorporating it into CI/CD pipelines to run performance tests automatically after certain stages. It can be triggered alongside unit and integration tests to ensure that performance remains stable as new features or fixes are added.

4. Are there many false positives? False negatives? True positive reports about things you don’t care about?
   k6 generally produces accurate reports with low likelihood of false positives or negatives, as it directly measures performance metrics like latency, response times, and error rates. However, false positive reports might occur if the infrastructure is unstable or external systems (e.g., third-party APIs) are slow.

Evidence showing the installation of k6:

Output of k6-short-test.js:

Output of k6-long-test.js:

Output of k6-spike-test.js: