Scaling a Payment Orchestration System from 100 to 10,000 TPS on AWS

In real life, hardly any chance to do this project; if it really comes, it’s more than gold. This post is just a skeleton, such a project will be more complex

Background

This post walks through a real design exercise: taking a payment + virtual account (VA) system from 100 TPS to 10,000 TPS on AWS, balancing cost and performance at each stage. Before diving in, I want to call out something worth keeping in mind throughout.

10,000 TPS for a payment system is an almost fictional number.

Stripe, 2025 annual letter, one of the most sophisticated payment infrastructure companies in the world, processes roughly 250 million transactions per day --- that’s around 2,900 TPS averaged across 24 hours, with peak periods perhaps 3—5× that. So when we say “10,000 TPS payment core,” we’re describing something in the same weight class as Stripe at full peak, or larger. In practice, most fintech products --- even well-funded, mature ones --- operate comfortably in the 50—500 TPS range. A Series B payments company hitting 1,000 TPS is doing serious volume.

Why design to 10,000 TPS then? Because the architecture you’d build to get there is instructive at every scale below it. The strategies that take you from 100 to 1,000 TPS are the same ones that take you to 10,000 --- just applied more aggressively. Think of this as a scaling roadmap, not a literal deployment target.

Baseline: ~190 TPS

The starting point is deliberately modest --- the kind of stack a small engineering team would deploy for a real production system:

• Backend: 2 × (2 vCPU / 4 GB) application servers (ECS tasks or EC2)
• Database: Single RDS PostgreSQL instance (4 vCPU / 8 GB)
• No caching layer, no queue, no read replicas
• Provider layer (bank APIs / payment rails): avg 100—400ms, p99=300ms—1s depending on rail
• Payment API (end-to-end): avg 150—450ms, p99=400ms—1s --- dominated by provider rail latency
• VA account API (read): avg 5—15ms, p99=50ms --- no provider dependency, DB/cache only

Benchmarked on 2 × 2-vCPU backend nodes + single Postgres, no Redis or queue, with a log-normal connector simulator (p50=120ms, p99=380ms):

Clean ceiling: ~190/s. The bottleneck is not CPU --- both backend nodes sat at 151% of 200% capacity and Postgres at 272% of 400% when the system collapsed. The actual wall is the DB connection pool: with 40 connections per node and a ~420ms mean hold time (dominated by the provider call), Little’s Law caps throughput at 80 connections ÷ 0.42s ≈ 190/s. Every in-flight payment holds a DB connection for its entire duration, including the wait on the external payment rail.

One counterintuitive finding: adding Redis + Kafka raised the clean ceiling from ~190/s to ~250/s --- because Redis offloads provider-health and routing reads off Postgres, reducing per-payment DB connection pressure. The infra layer is a net throughput win, not overhead. The first bottleneck then shifts to backend CPU, with Postgres still having headroom.

The path past this ceiling is PgBouncer in transaction pooling mode --- it releases the real Postgres backend connection during the external provider call, breaking the pool ÷ hold-time ceiling entirely. That’s exactly what Strategy 2 addresses.

The Seven Scaling Strategies

1. Multi-layer Cache: Absorb Read Traffic Before It Hits the DB

At moderate TPS, read traffic usually grows faster than write traffic --- especially in a virtual account system where customers and merchants frequently query balances, transaction status, and configuration.

In most production payment systems today, “multi-layer cache” means two layers: JVM local cache + Redis. That combination covers the vast majority of read traffic. A gateway/edge cache layer exists as an option, but it is far less common than it used to be.

Client
  |
API Gateway
(authentication, rate limit — caching optional, see note below)
  |
Payment Service
  |
+------------------+
| L1: JVM Local    |
|     Cache        |
|     (Caffeine)   |
+------------------+
  |  miss
+------------------+
| L2: Redis        |
|     Cluster      |
+------------------+
  |  miss
Database
(Source of Truth)

A Note on Gateway / CDN Caching --- Optional, and Less Common Than Before

Gateway-level response caching used to be a standard recommendation. In modern payment stacks it has become a niche optimization, for a few reasons:

• Almost every payment API response is per-user and authenticated. Responses vary by API key, merchant, and account, which defeats shared HTTP caching.
• Config data moved to config systems. The classic gateway-cache candidates --- merchant config, fee rules, feature flags --- are now typically pushed to services via a config center or feature-flag system and held in the JVM cache, rather than cached at the edge.
• Redis already absorbs the load. With a warm L1 + L2, the marginal DB offload from a gateway cache is small, while the staleness and invalidation complexity it adds is not.

Where it still earns its place: truly static, non-personalized endpoints ---

GET /payment-methods
GET /supported-currencies
GET /country-rules

It remains unsuitable for:

• account balance
• ledger data
• transaction state

because stale financial information may create business problems.

Treat gateway caching as a targeted optimization for a handful of static endpoints --- not as a required layer. The two layers below are the ones that actually carry the system.

Layer 1 --- JVM Local Cache

The application cache is often the fastest cache layer.

Typical technology:

• Caffeine Cache
• Guava Cache

Good candidates:

Example:

Payment Request

        |
        v

JVM Cache
(microsecond latency)

        |
        miss

Redis

Advantages:

• extremely low latency
• no network hop
• reduces Redis pressure

Trade-off:

Each application instance owns an independent copy.

Therefore JVM cache should not store the source of truth for financial state.

Layer 2 --- Redis Distributed Cache

Redis provides shared high-speed state across application instances.

Typical usage:

Idempotency keys are especially important in payment systems:

idempotency:req_12345

{
 status:"SUCCESS",
 transaction_id:"txn_888"
}

TTL=24h

A client retry becomes a lookup instead of executing another payment.

Cache Consistency Model

Redis should not replace the database.

The ownership model should be:

Ledger Database
        |
        v
Source of Truth

Redis Balance Snapshot
        |
        v
Fast Read Model

Avoid assuming database and Redis can be updated atomically.

A safer pattern:

Database Transaction Commit
          |
          v
Payment Event
          |
          v
Redis Update

or:

Database
   |
CDC/Event Stream
   |
Redis Materialized View

The database owns correctness. Redis owns performance.

2. Read/Write Splitting: Multiply DB Read Capacity Linearly

Once you’ve maximized cache hit rate, the remaining DB read traffic needs to be distributed. PostgreSQL read replicas let you scale read capacity roughly linearly by adding nodes.

The most important thing to add before replicas: PgBouncer in transaction pooling mode.

Without connection pooling, 40 ECS tasks × 10 DB connections each = 400 real PostgreSQL connections --- and that number grows with every new task you spin up. PostgreSQL becomes unstable above a few hundred connections. PgBouncer sits in front of RDS and multiplexes application connections onto a much smaller pool of actual server connections:

pool_mode = transaction        ; critical — session mode doesn't help
max_client_conn = 2000         ; what your app sees
default_pool_size = 25         ; actual server connections per db/user
server_idle_timeout = 30

AWS RDS Proxy is worth considering as a managed alternative --- it handles connection pooling transparently, integrates with IAM for credentials, and automatically routes failovers. For teams that don’t want to operate PgBouncer themselves, it’s a compelling choice.

Read replica routing rules:

• Writes and reads-after-writes → primary only. A payment that has just completed must read its own write.
• Balance queries, transaction history → replicas, routed via PgBouncer or RDS Proxy.
• Reports and dashboards → replicas or (better) Redshift.

At 10,000 TPS, you’ll want a db.r7g.2xlarge primary (8 vCPU, 64 GB RAM) with 2—3 read replicas across availability zones.

3. Microservice Split: Payment Core and VA Account on Separate Databases

At high TPS, co-locating payment ledger writes and VA account reads on the same database is a liability. They have fundamentally different workload profiles:

• Payment Core --- write-heavy, strict ACID requirements, sequential consistency on the ledger. Every row matters. No eventual consistency.
• VA Account Service --- read-heavy, high-frequency balance queries, tolerates a few seconds of staleness for reads (backed by the Redis cache).

Sharing a database means their connection pools compete, their buffer caches fight each other, and their IOPS budgets overlap. Splitting them gives each service independent vertical scaling headroom and means a slow query in one can’t block the other.

Service boundaries:

Payment Core API                  VA Account Service
────────────────────              ────────────────────
POST /payments                    GET  /va-accounts/{id}
POST /refunds                     GET  /va-accounts/{id}/balance  
GET  /payments/{id}               POST /va-accounts/{id}/topup
Internal: ledger write            Internal: balance cache write

Inter-service communication should be asynchronous. The Payment Core should never call the VA Account Service synchronously during a request path --- that creates coupling and a latency chain. Instead, Payment Core emits a payment.completed event; VA Account Service consumes it and updates the balance. Both services scale independently, and neither blocks the other.

4. Async Decoupling via SQS/SNS: The Shock Absorber

Message queues are a default consideration for any payment system --- they’re what lets you decouple your API throughput from your processing throughput.

Two core patterns:

Pattern A --- Async settlement

Payment Core validates the request, writes an intent record to the DB, and immediately returns 202 Accepted. A background consumer picks up the SQS message and executes the actual settlement with the payment rail (bank API, card network, etc.). This means your public API TPS ceiling is no longer tied to the latency of an external HTTP call to a bank.

Pattern B --- SNS fan-out

A single payment.completed SNS topic fans out to multiple SQS queues:

• VA Account Service → update balance
• Notification Service → send receipt email/push
• Audit Service → append to compliance log
• Fraud Service → run async risk scoring

Each consumer scales independently. Adding a new downstream consumer means creating a new SQS subscription --- no changes to Payment Core.

Use SQS FIFO with MessageGroupId = account_id for payment events. This guarantees ordering per account, which matters: if two payments arrive for the same account, you don’t want the second processed before the first.

Dead-letter queues are non-negotiable --- any failed message after N retries should land in a DLQ with an alarm on queue depth. In a payment system, a silently dropped message is a worse outcome than a loud failure.

5. Sharding: Split Tables First, Then Databases

Read replicas can scale reads, but they cannot solve the final bottleneck:

• ledger writes
• account balance mutations
• payment state transitions
• idempotency records

When one database primary becomes the limiting factor, sharding becomes necessary. But “sharding” is really two separate techniques that are often conflated:

• Table sharding (分表): splitting one oversized logical table into many physical tables inside the same database. Solves single-table problems (index depth, VACUUM cost, DDL lock time).
• Database sharding (分库): distributing data across multiple database instances. Solves single-instance problems (write IOPS, CPU, connection limits, storage ceiling).

Payment systems should not shard too early. The typical evolution is:

Single Database
        |
        v
Table Partitioning (native, e.g. PARTITION BY RANGE)
        |
        v
Read Replicas
        |
        v
Database per Service
        |
        v
Table Sharding (split large tables, same instance)
        |
        v
Database Sharding (multiple instances)

Table Sharding: Splitting Large Tables Before Splitting Databases

A single instance often still has plenty of CPU and IOPS headroom while one table has already become the problem. In a payment system the usual suspects are transactions and ledger_entries --- append-heavy tables that can pass 100M+ rows within a year at even moderate TPS.

Single-table pain points:

• B-tree indexes get deeper; every lookup and insert pays for it
• VACUUM / ANALYZE cycles take longer and lag behind write volume
• ALTER TABLE and index rebuilds lock or churn for hours
• The table’s working set no longer fits the buffer pool, so cache hit rate on the hottest data drops

Table sharding splits the logical table into N physical tables in the same database, routed by the same key you would later use for database sharding:

Logical: transactions

Physical:
  transactions_00
  transactions_01
  ...
  transactions_63

Routing: hash(account_id) % 64 -> table suffix

How to choose the table count:

• Use a power of two (16 / 32 / 64 / 128) so a future split is a clean doubling.
• Over-provision deliberately. Creating 64 small tables on day one costs almost nothing, but it buys you a cheap migration path: when you later add database shards, you move whole tables between instances instead of re-hashing rows. E.g. 64 tables → 4 databases means each database hosts 16 tables, and the routing function hash(account_id) % 64 never changes --- only the table -> database mapping does.
• Target roughly 10—50M rows per physical table as a comfortable steady state for OLTP access patterns.

Hash split vs time split:

For payment cores, hash-by-account is usually correct for accounts and ledger_entries; time-based splitting is better left to the cold/hot archival strategy (Strategy 6) via native partitioning.

The cross-table query problem: once a table is sharded, any query not keyed by the shard key --- “all transactions for merchant X”, “find by external reference number” --- can no longer use one index. Standard solutions:

• an index mapping table (merchant_id -> account_ids) queried first, then fan-out to the relevant tables
• a search sidecar (Elasticsearch / OpenSearch) fed by CDC for back-office and ops queries, keeping OLTP tables out of it

A PostgreSQL-specific note: manual table sharding is a habit inherited largely from the MySQL ecosystem, where native partitioning was weak for years. PostgreSQL’s declarative partitioning (PARTITION BY HASH / RANGE) covers many of the same cases with the routing handled by the planner --- so on RDS PostgreSQL, exhaust native partitioning first. Manual table sharding still wins when you want the table→database migration path described above, or when a middleware layer (ShardingSphere) will manage both table and database sharding under one routing rule.

Database Sharding Strategy

The most common sharding key is a business identity.

For virtual account systems, account_id is usually a strong candidate.

Example:

account_id % 16

Shard 0
 ├── accounts
 ├── ledger_entries
 └── transactions

Shard 1
 ├── accounts
 ├── ledger_entries
 └── transactions

...

Shard 15

A given account always belongs to one shard.

Routing:

Payment Request

        |
        v

Global Router

        |
        v

account_id = 12345

        |
        v

Shard-5 Database

Ledger Sharding Challenges

Financial ledgers are harder than normal business tables.

A payment may involve:

Customer Account
        |
        v
Merchant Account
        |
        v
Platform Revenue Account
        |
        v
Settlement Account

One transaction may touch multiple financial accounts.

Therefore blindly distributing ledger records across shards creates distributed transaction problems.

Common approaches:

Option 1 --- Co-locate Related Accounts

Keep related financial entities together:

merchant_id -> shard_id

All merchant-related accounts remain on the same shard.

Option 2 --- Dedicated Ledger Service

A dedicated ledger service owns accounting consistency:

Payment Core

      |
      v

Ledger Service

      |
      +---- Ledger Shard 1
      +---- Ledger Shard 2
      +---- Ledger Shard 3

Other services consume ledger events asynchronously.

Sharding Implementation Options

Application-level Sharding

Spring Boot

     |
Shard Router

     |
+----+----+----+
DB1  DB2  DB3

Possible technologies:

• Apache ShardingSphere
• Vitess (mainly MySQL)
• custom routing layer

For payment systems, relational databases remain dominant because ACID guarantees are usually more important than unlimited horizontal scale.

6. Cold/Hot Data Split: Keep OLTP Lean, Move Analytics to OLAP

As transaction volume grows, your transactions table becomes the single most dangerous object in your database. Reporting queries against it compete with live payment writes. Historical data inflates table size, slows down index scans, and adds cost to every VACUUM cycle.

The split:

• Hot data (OLTP, RDS): Last 90 days of transactions, all active account states. Use PostgreSQL table partitioning (PARTITION BY RANGE (created_at)) with monthly partitions. The query planner skips irrelevant partitions entirely, and old partitions detach cleanly for archival.
• Cold data (S3/Glacier): Transactions older than 90 days, archived via a nightly Lambda job. Keep a thin transaction_archive pointer table in RDS ({id, s3_path, period}) for lookup without pulling data back.
• OLAP (Redshift/Athena): Use AWS DMS with CDC (Change Data Capture) to stream new transactions from RDS to Redshift in near-real-time. Finance reports, reconciliation, and fraud analytics all query Redshift --- they never touch the OLTP primary. For ad-hoc queries on the S3 archive, Athena is pay-per-query and requires no warehouse infrastructure.

This split is primarily a cost and reliability move. You protect OLTP performance, reduce RDS storage costs, and give your data team a warehouse they can query without asking you to add read replicas.

7. Connection Pooling and Compute Right-Sizing

Two things kill payment systems at high TPS before the architecture even gets stressed: connection exhaustion and over-provisioned compute that still can’t handle bursts.

ECS auto-scaling configuration that actually works:

Payment services are I/O-bound, not CPU-bound --- the application is mostly waiting on DB or Redis responses. CPU-based auto-scaling will scale too late. Use ALB RequestCountPerTarget as the scaling metric instead:

• Scale out when requests/tasks> 200 (sustained 2 min)
• Scale in when requests/task < 80 (sustained 10 min --- conservative, avoids thrash)
• Min tasks: 2, Max tasks: 40

Instance sizing: 2 vCPU / 4 GB per Fargate task is typically sufficient for payment API tasks --- profile before going larger. The throughput ceiling for a payment task is almost always the DB connection or downstream I/O, not the CPU.

Aurora PostgreSQL vs RDS PostgreSQL at scale: At 5,000+ TPS, Aurora becomes meaningfully better. Its shared storage architecture means read replicas are available almost immediately (no replication lag ramp-up), its buffer pool is shared across replicas, and it supports up to 15 read replicas vs 5 for RDS. The cost premium is real, but the operational simplicity at large scale pays for it.

The Full Architecture at 10,000 TPS

Putting it all together, the request path looks like this:

Client
  └─ CloudFront + WAF (TLS, DDoS, geo-routing, static edge cache)
      └─ API Gateway (auth, rate-limit; optional cache for static config only)
          └─ ElastiCache Redis (L2: balance, account state, idempotency keys)
              │   (L1 = per-task JVM Caffeine cache inside each service)
              ├─ Payment Core API (ECS Fargate, 2–40 tasks)
              │    └─ PgBouncer / RDS Proxy
              │        ├─ RDS Primary (writes)
              │        └─ RDS Replicas ×2 (reads)
              │            └─ SQS FIFO → async settlement, SNS fan-out
              └─ VA Account Service (ECS Fargate, 2–40 tasks)
                   └─ PgBouncer / RDS Proxy
                       ├─ RDS Primary (writes)
                       └─ RDS Replicas ×2 (reads)

Cold path: RDS → S3/Glacier (nightly archive, >90 days)
Analytics: DMS CDC → Redshift | Athena on S3

Cost vs Performance Levers

At 10,000 TPS, the biggest cost traps are over-provisioned RDS and under-utilized ElastiCache. Tune in this order:

1. Maximize Redis cache hit rate first. Every 1% improvement in cache hit rate directly reduces RDS load. Measure cache_hits / (cache_hits + cache_misses) per endpoint family, not globally --- a 90% hit rate on balance reads masking a 10% hit rate on account state reads will mislead you.

2. Commit to reserved instances on DB primaries. At 10,000 TPS, the primary DB instance is always-on infrastructure. A 1-year reserved instance saves 40—50% over on-demand. This is the single highest ROI infrastructure cost decision.

3. Right-size Fargate tasks before scaling out. Profile memory and CPU utilization on a single task under load before deciding to add tasks. Payment API tasks are frequently under-utilizing CPU at 20—30% while waiting on I/O. Adding vCPUs doesn’t help.

4. Use Athena for historical queries instead of keeping history in RDS. Storing 2 years of transactions in RDS to support “download statement” features is expensive. S3 + Athena handles this at a fraction of the cost with acceptable latency for a non-realtime operation.

The Hidden Topology Driver: 🤦Data Residency Law

Here’s something that doesn’t show up in most scaling guides, but completely reshapes this architecture in the real world.

If this payment system is global --- and any system at 10,000 TPS almost certainly is --- regulation decides your database topology before engineering does.

GDPR mandates that EU residents’ financial data must be stored and processed within the EU. It cannot legally transit to or rest in US infrastructure. US financial regulations have their own requirements. Some countries go further --- India’s RBI requires payment data to be stored exclusively on Indian soil. You don’t architect around this. You comply with it first, then engineer within the constraints.

The practical consequence: you cannot have a single global payment-core database. You run independent stacks per region, each fully isolated at the data layer.

EU Region (Frankfurt / Ireland)       US Region (us-east-1 / us-west-2)
──────────────────────────────        ──────────────────────────────────
API Gateway (EU)                      API Gateway (US)
Payment Core API                      Payment Core API
VA Account Service                    VA Account Service
RDS PostgreSQL (EU-resident data)     RDS PostgreSQL (US-resident data)
ElastiCache Redis                     ElastiCache Redis
SQS / SNS                             SQS / SNS

No cross-region DB replication. No shared primary. Completely independent stacks, by law.

And this is actually good news for the TPS math.

If your business splits 40% EU / 60% US --- a typical global fintech distribution --- you’re no longer designing a single system for 10,000 TPS. You’re designing two systems: one that needs to handle ~4,000 TPS peak, and one that handles ~6,000 TPS peak. Neither needs to be Stripe-scale on its own. The architecture becomes more tractable, and the cost drops proportionally.

What can be shared across regions:

The one engineering problem this creates: a German customer traveling to the US and making a payment. The request hits the US API gateway, but the account is EU-domiciled. You need a lightweight global routing layer --- essentially a small DynamoDB Global Table or Route 53 Latency Routing rule --- that maps account_id prefix to home region, then proxies or redirects accordingly. This routing table contains no financial data, just region mappings, so it can legally live anywhere.

The pattern is: one global router, N fully isolated regional stacks. The router knows where your account lives; the regional stack handles everything else.

This also means your disaster recovery story is cleaner than a single-region design. An EU outage doesn’t affect US customers at all --- they’re on a separate stack. You get fault isolation as a compliance side effect.

A Reality Check on the Numbers

10,000 TPS payment processing is an engineering thought experiment more than a deployment target for most teams. For context:

• Stripe processes ~250M transactions/day → ~2,900 TPS average, peak perhaps 3—5× that
• A healthy Series B fintech might see 100—500 TPS peak
• A major regional bank’s core payment processing might peak at 1,000—2,000 TPS

The architecture described here handles 10,000 TPS --- but the same layered approach is what you’d use at 500 TPS or 2,000 TPS, just with smaller instances and fewer replicas. The progression is incremental. You don’t build the 10,000 TPS stack on day one; you build the 500 TPS stack well, instrument it thoroughly, and add layers as the data tells you to.

The most important rule in payment system scaling: don’t add infrastructure ahead of measured bottlenecks. A well-tuned 2-node setup with PgBouncer and a warm Redis cache can handle far more than its raw specs suggest. Profile first, scale second.

Summary

The architecture isn’t about reaching 10,000 TPS. It’s about building a system that degrades gracefully under load, scales predictably as volume grows, and doesn’t require a full rewrite at each order of magnitude. That’s the goal worth engineering toward.

Written based on practical AWS architecture design for financial systems. Infrastructure numbers are illustrative; always benchmark your specific workload before committing to instance types and cluster sizes.