Scale, a word that has to do with size, and ability, another word that gets you thinking if an event is possible or not. We could stop right here and have you doing some scalability testing, but there’s much more to it than knowing the term’s meaning. Regardless of how good looking a system is, it could still fail scalability tests! You see, the tests in this variety of analyses have nothing to do with form and feature. They’re strictly looking into how a system would respond when various loads exert onto it.
In this post, we’ll take you through some of the areas that make up a solid test and wade into the various thought processes behind the idea of scalability testing. You should acquire the ability to confidently discuss critical aspects of the topic and perhaps even be able to suggest tools and procedures when setting out the actual tests.
What Is Scalability Testing?
Let’s take a step back before diving headfirst into a dictionary definition here, shall we? As has been seen above, the entire idea here is to look into matters bigger than if an application does what the functionality documents dictate. It’s safe to assume that during design (and development), only a few users are made for each user group in order to make progress. However, thousands more are desired when the system is finished. In fact, companies tend to get creative with marketing just to get user base figures to brag about to (funnily enough) other potential users. Without successfully completing scalability testing procedures, those users could just spell doom for said imaginary companies.
Before any system can ask for users, we must determine if it won’t succumb to the pressure of too many requests coming through at the same time. There you go. That’s the founding perspective behind scalability testing. What you’ll notice, however, is that when conducting scalability tests, not only will the ceiling be sought after, but solutions to such snags should be suggested and tested as well.
It’s well worth mentioning now that scalability testing falls under black box testing.
What’s the Point?
Since the concept of scale is bidirectional, the ability of a system to shrink on command should also be determined and reported. When presented with scalability testing results, decision-makers can then plan for sustainable growth (in any direction) with more certainty.
So far we’ve been easy on the jargon. The “how” part should bring itself out of hiding as you read on. What you need to accept at this point is that scalability tests, unlike load tests, consider the max and min performance attributes of a system. They then go ahead and seek remedies to the limits discovered. Load tests will just seek to find levels of (high) pressure under which a system folds. Keep this in mind as we delve into the crux of the matter.
Scalability Testing Attributes
We could peg as many as a dozen test points (attributes) for a system. That number is often determined by the complexity of the said application. There would be no use testing how a system intended for only five to 10 users performs when 1,000 users make concurrent requests for resources. Neither would there exist grounds for testing database-related issues where an application is static. Such project management edges should help you save time. Quartering the size and complexity of your system aside, you could consider the following attributes:
- User-number-related performance
- Response time
- Memory usage
- Load-bearing performance
And now for brief explanations of these five scalability testing attributes.
As you increase the number of users communicating with a system, the system tends to show behavioral changes. If (as expected) it slows down, there comes a point where it just freezes. Corrective action at that point will allow the ability to perform smoothly, even with the new load. Reports produced for this attribute should show what resources were added to beat the bottleneck. Recordings are made for small, intermediate, and excess amounts of users.
At varying loads (number of users accessing the system), the wait time for a request to be processed tends to increase. Results here allow for changes in resources that determine speed to be made available in time to maintain good user experience.
Memory usage often stems from the application source code’s potency. When running a program, the amount of time that the CPU takes to compile code directly affects throughput. This then touches several other attributes adversely. Standards should then be set to minimize the choice of programming language’s effect on the overall scalability of a project.
Many tools exist solely to measure the throughput of a network-reliant application. Changes that can mitigate slow throughput can be in the form of different packet policies among others. As loads increase, the time it takes to send messages through the network (throughput) should not be compromised.
The load is the moving variable when conducting most of the tests above. Small loads will provide a standard, or starting measure, of performance. As the load increases, taking logs of how every other resource is affected, along with mitigation measures to attain stability, will be useful.
Advantages of Scalability Testing
Let’s discuss why a company may be better off conducting scalability tests after all. For one, trying to add users to a system while oblivious of its capabilities under varying loads can prove a perilous endeavor. Upon systems failure, stakeholder confidence can wane, along with any hope of rising again. Secondly, you save a lot of money by setting reasonable growth rates that ideally pay for themselves. Lastly, even when selling a company or facing foreclosure, reducing resources while cutting costs can be a controlled event. And this potentially saves the reputations of everyone with the ailing business on their résumés.
How to Start Scalability Tests
With all the knowledge you’ve been equipped with, it only makes sense that you know how to get started with the process of scalability testing. Industry best practice entails that you should determine the scope of your project first. That comes before you even pick out teams and tools for the job. Assuming all these are in place, the following checklist should guide you from idea to actionable findings in the form of condensed reports.
- Agree on the scalability attributes to focus on.
- Discover and acquire the tools for the tests.
- Set up your test scenario (including reasonable test controls).
- Carry out scalability tests while recording all findings.
- Determine remedies for scaling purposes at all load intervals.
- Extract and present your findings.
While it can be a routine procedure for big companies such as Google, Apple, and Amazon, smaller companies often only carry out scalability tests when essential. They usually lack as many tools, hands, or the budget to carry out scalability testing at the scale that these giants do (see what we did there?). As such, it’s best to have tools that make the entire process less difficult. Among a plethora of others, tools like Scalyr allow engineers and users to carry out scalability testing. All without the familiarization process gnawing into the actual testing.
If you read every word above as it was meant, you should be ready with enough of the talk to do the walk. You know enough about scaling applications the right way. Where will your application scale to next, and have you commenced with scalability testing already?
Taurai Mutimutema is the author of this post. Taurai is a systems analyst with a knack for writing, which was probably sparked by the need to document technical processes during code and implementation sessions. He enjoys learning new technology, and talks about tech even more than he writes.