Insights22 January 2026Updated 25 October 20264 min read

Scalable Data Architecture: Measure Capacity Before Changing the Design

A glowing geometric cityscape forms a symmetrical star pattern against a dark background, illustrating process logic with intricate illuminated structures.

Scalable data architecture needs a capacity test, not a promise of unlimited growth. We recommend starting with the work the system must finish, the time available and the behaviour that remains acceptable when demand exceeds capacity. Treat a proposed redesign as a hypothesis to test against that workload.

A queue illustrates why this matters. Amazon's account of avoiding queue backlogs describes how delays grow when messages arrive faster than workers process them. Accepting a request is therefore not the same test as completing its work. Set an acceptance target for the finished business action, rather than celebrating a fast acknowledgement.

Define the workload before choosing the architecture

Start with a named workflow, such as creating an order and reserving its stock. Describe what counts as success and which record proves it. Separate the customer response deadline from any later work, such as generating a document or updating a reporting view.

Build the test from representative records. Include large orders, existing customers with long histories and simultaneous changes to the same stock item. Label these as test cases, not forecasts of what every business will experience. Record the assumptions so another engineer can repeat the exercise without guessing what the original result meant.

Specify the demand pattern as well as its volume. Test a steady arrival rate and a short burst, then measure recovery after the burst ends. Use the business's actual operating needs to choose the duration and acceptable delay. Do not turn an arbitrary growth multiplier into a contractual capacity commitment.

Keep the starting measurement. For each candidate change, retain the workload, software version, database size and resource allocation. Compare completed operations and unsuccessful requests alongside response times. Reject a claimed improvement if it only moves unfinished work somewhere the test no longer measures.

Give queued work a completion deadline

For asynchronous work, measure elapsed time from submission to the required result. Amazon explains that message age reveals processing delay, whereas producer-side latency largely reflects the queue service. Use that distinction when choosing an alert. A healthy submission endpoint should not be your only evidence that orders are progressing.

Define an overload policy before the test. Decide which requests may wait, which need an explicit rejection and which require an operator's attention. Make the user-facing status match that decision. Do not show an order as completed merely because its message entered a queue.

Test the recovery policy by slowing a dependency and then restoring it. Keep new work arriving during recovery. Check whether older work finishes within its deadline without preventing urgent work from progressing. Amazon's queue-backlog analysis discusses workload isolation and rejecting excess incoming work, but also explains that rejection is unsuitable for some order-processing systems. Choose the policy from the business consequence, rather than copying a cloud example unchanged.

Give failed messages an owner and a replay procedure. Include the operator's handling time in the acceptance exercise. Describe any manual approval as a deliberate control with a measured waiting time, rather than assuming that every human decision should disappear.

Make repeat submissions safe

Design a test where an operation succeeds but the caller never receives its response. Ask what should happen when the caller submits it again. For an order, our recommended acceptance condition is one intended order, with a way for the caller to discover the result.

HTTP's idempotent-method rules in RFC 9110 distinguish repeated requests whose intended effect stays the same. The specification warns against automatically retrying non-idempotent methods unless the client knows the operation is idempotent or can detect that the first request was never applied. Do not infer safe order creation from the existence of a retry button.

Specify how your application recognises a repeated operation. Test the same operation identifier twice, including simultaneous submissions. Also test a reused identifier carrying different order details and define the expected rejection. Treat these as application requirements to implement and verify, not a guarantee supplied by a queue or an HTTP method.

Inspect the database before splitting services

Use a representative dataset to inspect the queries involved in the slow workflow. PostgreSQL's version 17.0 documentation archive explains that indexes can accelerate row retrieval but also add overhead. An index is a candidate change to measure, not an instruction to index every column.

The same archive explains that EXPLAIN ANALYZE actually executes the query. Run potentially modifying statements only in a suitable test environment with controlled data. Save the plan and timings before and after a change, and repeat the workflow under concurrent load rather than relying on an isolated query result.

Ask what a service split is meant to improve. Name the work that needs independent capacity, the data it owns and the failure behaviour at its boundary. Require a test that demonstrates the intended improvement. If the proposal cannot identify those things, keep investigating before committing to a migration.

We recommend changing the smallest part that the measurements identify. That might be a query, a worker limit or the scheduling of a report. Keep the option of a larger redesign, but give it the same acceptance standard as a smaller change.

Update, October 2026

For teams extending this capacity work to AI pipelines, our engineering article on concurrency benchmarking provides a worked comparison of aggregate throughput and individual response speed under concurrent requests. Use it to frame the test, not as a capacity prediction for a different model or dataset.

Before approving any scaling change, ask for the repeatable workload and the results against its completion deadlines. Make the next investment answer a measured constraint.

Sources

Want this in your team?

Tell us what you're building and where you want to get to, and we'll think it through with you.

Let's have a chat Follow by RSS →

All insights →