By Chandan Mukherjee
From internal stakeholders to clients, everyone involved with payment gateways expects maximum performance, stability and uptime. But how does one quantify this performance? How do we know if the gateway is indeed performing at the most optimal level? To provide answers, this article ventures into aspects of payment gateway testing from a performance perspective.
Performance testing for payment gateways has multiple objectives. The most obvious is to ascertain that the gateway will perform as expected under certain volumes anticipated. Other objectives to address include:
Also, code changes are often done to speed up and optimize transaction processing. To measure the effect of such changes, performance testing is the most commonly used and reliable tool. Comparing before and after results gives a clear view of the impact code changes have on the overall system.
A fundamental question asked is, How many transactions per second does the payment gateway handle? This is an indication of the number of concurrent transactions the gateway can process without reaching a break point. The real question is, What is the gateway's break point?
Generally, the break point for a payment gateway is not the moment when the gateway software stops, freezes or crashes. A well-built gateway is unlikely to reach that stage. The point where gateway performance is too slow and becomes unacceptable is the break point.
The slowdown occurs when transactions are taking too long to process inside a gateway. Some relevant questions to ask during a performance testing exercise are:
Performance test-case formation is a complex task involving many variables. Here are major points to consider during this process:
Payment gateways process varied sets of transactions and might handle each differently. To make the performance test reflect the reality of the production environment, the test must include a transaction mix that is similar to the production environment's mix. This includes testing specifics such as ratios of transaction types, bank identification number ratios, routing scenarios, etc.
While designing the test, it is necessary to determine if the test needs to run from one node or multiple nodes. Both scenarios are important and give performance data from two different angles. Single node offers understanding of scalability from a single point of acquisition; multiple nodes stress the gateway's ability to handle many points of acquisition simultaneously. An ideal performance test suite will contain test cases for both single and multiple nodes.
Performance test cases can employ a local loopback inside the gateway, a stub representing a processor or an actual processor. The most realistic scenario is to run the test cases with a processor, but that isn't always possible. Many processors throttle the performance in their test systems to enable all gateways to test without performance degradation.
An approach to resolve this issue is to use a processor stub that will use the same network connectivity, accept incoming transactions and send back a response. Advanced stubs can even vary the response at random or on a predefined basis – denying every fifth transaction, for example.
If writing a stub isn't an option, some developers use a local loopback method via which a transaction response is generated inside the gateway itself. While this method is simplest to implement, it does not reflect the reality of transaction processing, except for stand-in processing scenarios.
While performance testing can generate valuable data, interpreting that data requires some knowledge of statistical concepts. Arithmetic mean is useful to find out average values such as transactions per second, but a mean does not reflect the spread of the data points.
The spread of data points is better reflected in variance or standard deviation. This reveals the spread of the data. An example is the system's total throughput. If the data points have low variance, it will imply that most transactions have similar performance.
For most performance-related measurements, the value should remain within a narrow band; that is, low standard deviation. For example, it may be acceptable that 95 percent of the transactions complete their round-trip journey inside the gateway within one second. Though 5 percent do not fall in this boundary, it does show a high degree of gateway performance.
Statistical mode represents the most common value in the collection of data points. This is the most likely value that occurs, and hence most data in the collection have this value. For example, if the mode of transaction round trip is 0.8 seconds in a collection of values for transaction round trip, it indicates most merchant transactions are handled in 0.8 seconds. Some transactions may be handled with even better performance; others may be worse.
A test platform built for payment gateway performance testing should capture quite a few metrics, some of which I have just been mentioned. While test strategy is governed by the overall expectations and goals, here are four best practices to follow:
Gateways are complex systems, and businesses have high expectations for them. Evaluating performance requires precision. There are a number of moving parts in the evaluation process, test methods and analysis of the methods. A repeatable, automated performance test suite should be a mandate for any quality assurance team responsible for payment gateway testing. A planned approach to performance testing with clarity on strategy, goals and methods can lead to enhanced payment gateway quality and stability.
Chandan Mukherjee is the co-founder of PayCube Inc., which is a San Francisco Bay Area, Calif.-based payment consulting and IT services company providing custom software solutions and custom gateways for acquirers, ISOs, retailers and varied organizations in the world of payments and consumer transaction acquiring and management, including prepaid and gift card program, loyalty and promotion, payment start-up, POS solution, mobile payment and e-commerce players. For more information, email firstname.lastname@example.org, call 510-545-6854 or visit www.paycubeinc.com.
Notice to readers: These are archived articles. Contact names or information may be out of date. We regret any inconvenience.Prev Next