GS Logo
The Green Sheet, Inc

Please Log in

A Thing
View Archives

View flipbook of this issue

Care to Share?

Table of Contents

Lead Story

How to keep tabs on reps, merchants - Part 2


Industry Update

Report tells acquirers how to stay in the game

Name for ATM payment hub

Trouble for UC in the U.K.?

Bitcoin kiosks enter marketplace

'Hands-free' payments beckon

Selling Prepaid

When Big Data meets fulfillment


Opportunities on mobile horizon


Visa's new third-party registration fees – unintended consequences

Ken Musante
Eureka Payments LLC

Tackle risk vectors to improve portfolio performance

Ross Federgreen and Ed Barton
CSR / G2 Web Services


Street SmartsSM:
It's not what you say

Dale S. Laszig
Castles Technology Co. Ltd.

Busting myths about change

Jeff Fortney
Clearent LLC

Evaluating payment gateway performance

Chandan Mukherjee
PayCube Inc.

Company Profile

Cutter LLC

New Products

Funding approval in minutes

CAN Mobile Funder

Seamless QuickBooks integration



Roll out the welcome mat at work


Letter From the Editors:

Readers Speak

Resource Guide


A Bigger Thing

The Green Sheet Online Edition

September 23, 2013  •  Issue 13:09:02

previous next

Evaluating payment gateway performance

By Chandan Mukherjee

From internal stakeholders to clients, everyone involved with payment gateways expects maximum performance, stability and uptime. But how does one quantify this performance? How do we know if the gateway is indeed performing at the most optimal level? To provide answers, this article ventures into aspects of payment gateway testing from a performance perspective.

Testing challenges

Performance testing for payment gateways has multiple objectives. The most obvious is to ascertain that the gateway will perform as expected under certain volumes anticipated. Other objectives to address include:

Testing metrics

A fundamental question asked is, How many transactions per second does the payment gateway handle? This is an indication of the number of concurrent transactions the gateway can process without reaching a break point. The real question is, What is the gateway's break point?

Generally, the break point for a payment gateway is not the moment when the gateway software stops, freezes or crashes. A well-built gateway is unlikely to reach that stage. The point where gateway performance is too slow and becomes unacceptable is the break point.

The slowdown occurs when transactions are taking too long to process inside a gateway. Some relevant questions to ask during a performance testing exercise are:

Test-case solutions

Performance test-case formation is a complex task involving many variables. Here are major points to consider during this process:

Statistical concepts

While performance testing can generate valuable data, interpreting that data requires some knowledge of statistical concepts. Arithmetic mean is useful to find out average values such as transactions per second, but a mean does not reflect the spread of the data points.

The spread of data points is better reflected in variance or standard deviation. This reveals the spread of the data. An example is the system's total throughput. If the data points have low variance, it will imply that most transactions have similar performance.

For most performance-related measurements, the value should remain within a narrow band; that is, low standard deviation. For example, it may be acceptable that 95 percent of the transactions complete their round-trip journey inside the gateway within one second. Though 5 percent do not fall in this boundary, it does show a high degree of gateway performance.

Statistical mode represents the most common value in the collection of data points. This is the most likely value that occurs, and hence most data in the collection have this value. For example, if the mode of transaction round trip is 0.8 seconds in a collection of values for transaction round trip, it indicates most merchant transactions are handled in 0.8 seconds. Some transactions may be handled with even better performance; others may be worse.

Testing strategy recommendations

A test platform built for payment gateway performance testing should capture quite a few metrics, some of which I have just been mentioned. While test strategy is governed by the overall expectations and goals, here are four best practices to follow:

  1. Generally, do a performance test repeatedly for comparison purposes. To accommodate this, I suggest creating an automated test harness at the beginning and maintaining it with every change to the code base, updating as needed. Test cases must be predefined and upgraded from time to time to reflect changing business scenarios.
  2. A clear understanding of the transaction mix is essential. Design the performance test so that it reflects the target transaction mix.
  3. Capture and store multiple data points for future comparisons. Raw data points must be retained so that future calculations can be handled.
  4. Apply appropriate statistical calculation to the data to draw accurate conclusions from the testing.

Gateways are complex systems, and businesses have high expectations for them. Evaluating performance requires precision. There are a number of moving parts in the evaluation process, test methods and analysis of the methods. A repeatable, automated performance test suite should be a mandate for any quality assurance team responsible for payment gateway testing. A planned approach to performance testing with clarity on strategy, goals and methods can lead to enhanced payment gateway quality and stability.

Chandan Mukherjee is the co-founder of PayCube Inc., which is a San Francisco Bay Area, Calif.-based payment consulting and IT services company providing custom software solutions and custom gateways for acquirers, ISOs, retailers and varied organizations in the world of payments and consumer transaction acquiring and management, including prepaid and gift card program, loyalty and promotion, payment start-up, POS solution, mobile payment and e-commerce players. For more information, email, call 510-545-6854 or visit

Notice to readers: These are archived articles. Contact names or information may be out of date. We regret any inconvenience.

previous next

Spotlight Innovators:

North American Bancard | USAePay | Impact Paysystems | Board Studios