Performance Testing Approaches

Many teams think performance testing means throwing traffic at a system until it breaks. That approach is fine, but it misses how systems are actually stressed in the real world. The approach I’ve found most effective is to split performance testing into two distinct categories: 🏋️♀️ Benchmark testing 🚣♀️ Endurance testing Both stress the system, but they answer different questions. 🏋️♀️ Benchmark Testing: Benchmark tests are where most teams start: increasing load until the system fails. Failure might mean: ��️ Latency SLAs are exceeded ⚠️ Error rates cross acceptable thresholds Sometimes failure is measured by when the system stops responding entirely. This is known as breakpoint testing. Even when SLAs are the target, I recommend running breakpoint tests after thresholds are exceeded. Knowing how the system breaks under load is useful when dealing with the uncertainties of production. 🚣♀️ Endurance Testing: Endurance tests answer a different question: > Can the system sustain high load over time? Running at high but realistic levels (often near production max) over extended periods exposes different problems: 🪣 Queues, file systems, and databases slowly fill 🧹 Garbage collection and thread pools behave differently 🧵 Memory or thread leaks become visible These issues rarely show up in short spikes of traffic. If you only run benchmarks, you’ll discover them for the first time in production. ⌛️ Testing Thoroughly vs Deployment Speed: Benchmarks run fast; Endurance testing takes time. A 24-hour endurance test can slow down releases, especially when you want to release the same service multiple times a day. It's a trade-off between the system's criticality and the need for rapid deployments. How tolerant is the system to minor performance regressions? If performance truly matters, slowing releases down to run endurance tests might be the right call. 🧠 Final Thoughts: Effective performance testing isn’t just about surviving spikes. Spikes matter, but so does answering: 📈 Can the system withstand peak load for extended periods? 🔎 If not, how does it fail, and why? All too often, I see the system's capacity become the breaking point during unexpected traffic patterns. While an application might handle spikes, the overall platform often can't sustain them. That's where endurance tests deliver their real value. #Bengineering 🧐

When we do performance testing, we want to do both mixtures of operations in parallel to understand how the service behaves under anticipated loads, as well as operations insulated in consecutive execution to understand how the individual operations behave. Both types of performance test provide useful information, often the results of the two together explain something not obvious from only one. I saw something this week I have seen many times before. A run of parallel execution, built in anticipation of real-world load, was yielding latencies much higher than target across the board. Even though we were able to isolate the system resources at fault, we couldn't tell if all the operations were having problems, or if one of them was starving resources the others needed. We executed the same set of operations, but one at a time without the others in parallel, so we could get percentile distributions on each one. Only one of the operations was exceeding latency targets, everything else was well within goal. That one operation on its own was using resources the other services needed. That information in hand, we knew where to begin fix investigations. After getting isolated measurements, the next step is investigation, which varies based on what the measurements show. Is it in a front end, a database, CPU, disk, network IO, thread pools, memory utilization, connection pools, or some other resource? What you need to look at is made much simpler when you have the two sets of results guiding you toward further analysis. #softwaretesting #softwaredevelopment #performancetesting Prior articles and cartoons of mine can be found in my book Drawn to Testing, available in Kindle and paperback format. I'm watching how sales of this first go. If it does well, I will collect my newer articles into another edition, so if you like my cartoons and want more, spread the word! https://lnkd.in/gB4NS4BS

In our ongoing exploration of performance testing, this week we delve into the nuanced world of tailoring strategies for different application types. One size doesn't always fit all! 👉 Web Applications: ✅ Focus on metrics like page load time, server response times, and user experience under various user loads. ✅ Tools like JMeter and LoadRunner are popular choices for simulating user behavior and performance testing web applications. 👉 Mobile Applications: ✅ Consider factors like network connectivity, battery usage, and app responsiveness across different devices and operating systems. ✅ Tools like Appium and LoadView are well-suited for testing mobile app performance under various network conditions and user load scenarios. 👉 API Testing: ✅ Performance testing APIs focus on ensuring they can handle high volumes of requests without compromising response times or stability. ✅ Tools like Postman and SoapUI can be used to automate API calls and measure performance metrics relevant to APIs. The Takeaway: By understanding the unique characteristics of each application type, you can tailor your performance testing strategy to identify and address potential bottlenecks specific to that platform. #performancetesting #apitesting #loadtesting #stresstesting #nonfunctionaltesting #webperf #performanceengineer #performanceengineering #softwaredevelopment #softwaretesting #automationtesting #devops #softwaretestingcompany #softwaretestingservices #testingjobs #awesometesting #vtest VTEST - Software Testing Company

In my journey as a tester, I've come across many frustrating instances where projects were delayed due to performance bottlenecks. It's a common challenge - we design and code meticulously, only to find at the UAT stage that the application needs rearchitecting! A few years ago I started shifting my Performance Testing to the left. It was a game-changer, transforming my testing approach and helping me overcome my challenges. The key learning I want to impart from my experience is the value in these practices: 1. Involve your developers right from the start. 2. Conduct unit tests on individual pieces of code for detailed validation. 3. Have your performance test engineers examine your APIs for performance from the get-go. 4. Focus on individual feature performance, it can help identify and fix issues early. 5. Start small with tests, gradually scaling up. These practices, when integrated into your daily DevOps pipeline, not only improve quality but also enhance user satisfaction. How do you manage performance bottlenecks in your projects? I'd love to hear about your experiences and strategies. #PerformanceTesting #SoftwareTesting #QualityAssurance