Some clients know that their application needs to be performance tested, most do not.  Even fewer have any inkling of the approach needed to performance test their app.  There are 3 types of performance tests and it is up to you to inform the client on the best type of test to run.

  • A Load Test is a test that is engineered to reproduce documented or estimated production use.  This is the type of test my team find ourselves executing most often when testing cloud based systems such as Appian applications hosted on Appian Cloud.  Load Tests are best for instances such as this in order to determine if the sizing estimate for the cloud instance is appropriate for the application at the expected user load.  We find that the ideal sample time for such a test is usually 1 hour.
  • A Soak Test is designed much like a load test, with one major difference.  The sample time for soak tests range from 4-8 hours.  Soak tests allow the tester to spot issues that only arise from continued usage at a high volume.  These tests are perfect for uncovering memory leaks and other bits of nastiness that would not be visible in a standard load test.
  • This one is my personal favorite, the Stress Test.  A stress test is a load test scaled up anywhere from 4 to 8 times the expected peak usage.  These tests are meant to understand the upper bound of system’s capabilities, and potentially find the weakest link  These tests are the ones that can bring an application and in some instances an entire cloud environment to its knees.  Be prepared for your inbox and IMs to be flooded with by Cloud engineers, system admins, and an assortment of others scrambling to find out if you know anything about the activity on the system.  Finally, be sure that you have enough hardware resources on the test machines to accommodate this load, it is not uncommon for performance testers to overload the test machines and return false positives for poor performing requests.

It is important to choose the correct test to meet the client’s needs, so be sure to plan appropriately.  In theory the design and execution of a performance test is pretty straightforward.  In reality, there are many pitfalls that need to be avoided during planning and preparation.  This is where having an experienced performance tester or consultant on the team can help avoid the pitfalls on the road to a successful performance testing project.

Author

Justin Harrison

Today, as the Senior Manager of Performance Testing Services at VPS, I lead a team of people dedicated to performance testing, and I am recognized as a leading subject matter expert in performance testing applications on the Appian platform.

See author's posts