Insights1 April 2026Updated 26 October 20265 min read

What to Test Before Spending £4,000 on Local AI Hardware

Dark neon edged box with cyan and magenta light threads streaming out from both sides

A £4,000 local AI budget should buy a tested capability, not a promise that owning the machine makes inference cheap, fast or sovereign. Treat that figure as a spending limit for a procurement exercise, rather than a verified retail price or a complete installation cost. Our recommendation is to approve a purchase only after a representative workload passes agreed quality, waiting-time and data-handling checks.

The hardware makes that exercise worth considering: NVIDIA's October 2025 announcement describes DGX Spark with 128GB of unified memory and support for running inference locally. That is a hardware capability statement, not evidence that a particular pair of models will serve your team acceptably.

Start with a model that fits the job

There is a credible large-model candidate to evaluate: OpenAI's gpt-oss repository describes gpt-oss-120b as fitting on a single 80GB GPU and identifies MXFP4 quantisation of its mixture-of-experts weights. Do not turn that vendor statement into a claim that the complete service, with your chosen context lengths and simultaneous requests, has a known memory footprint on a Spark. For procurement, require a successful run on the proposed hardware and serving software, rather than subtracting advertised model requirements from advertised system memory.

Use a small, approved evaluation set containing ordinary inputs, difficult inputs and examples where the correct response is to refuse or ask for clarification. For extraction, score the returned fields against checked answers and count invalid output as failure rather than excluding it from the results. For an engineering assistant, include questions whose answers depend on supplied repository evidence, then check the answers and citations against that evidence. Record the model revision, quantisation, serving version and evaluation inputs alongside the result so the next configuration can be compared fairly.

Begin with one serving model unless the trial establishes a reason to add another. If a smaller utility model is proposed alongside a larger reasoning model, test them together under the expected request mix before reserving any memory for the second service. Use the separate discussion of workload routing when deciding which requests deserve which model, rather than treating two models as a hardware requirement.

Measure useful capacity, not a launch demonstration

A compact glowing machine with many steady parallel streams, beside a single flashy spike of light that fades

Define the acceptance scenario before tuning the server: who submits work, how much arrives, how long an acceptable answer may take and what happens when the queue grows. Bass, Clements and Kazman's Software Architecture in Practice, fourth edition, chapter 9 includes a performance general scenario and tactics for performance. Apply that scenario approach by testing interactive requests separately from background work, then repeating the test with both active.

For a human-facing service, record the wait until the response begins and the wait until a usable answer is complete. For a background extraction service, count accepted records completed within the allowed window, including retries and rejected outputs in the report. Repeat with the input lengths and overlapping requests your team expects, not only a short prompt sent while the machine is idle. Make the purchase decision against those agreed limits rather than a token-rate comparison with a different model on a different workload.

Give the operator a bounded queue and an explicit response when capacity is exhausted. Agree which work can wait, which work must fail visibly and who can authorise another execution route. Treat these as service requirements to prove during the trial, not details to postpone until the machine is busy.

Prefix caching has a specific job

Several streams of light share one bright common segment at the start before branching apart

vLLM's versioned prefix-caching documentation explains that a query can reuse cached computation when it shares a prefix with an earlier query. The same documentation limits the benefit to prompt processing, not the generation of new tokens, so it does not support a blanket claim of free throughput. Where your workload repeats substantial prompt content, compare runs with and without caching and report the input shape as part of the result. Do not promise a saving until that comparison shows an improvement in the service measure you actually care about.

Keep the first trial configuration small and documented. Choose a supported deployment route, record its versions and retain the configuration that produced the accepted result. Require a repeatable restart and a recovery exercise before treating the installation as a service the team can rely on.

Specify what sovereign means for this service

For this procurement, define sovereignty through explicit requirements about who operates the service, where approved information may go and who can change those rules. Require a data-flow review covering prompts, retrieved documents, outputs, operational logs, backups and any tools the model can call. For every external destination, ask the owner to identify what leaves the machine and whether that destination is permitted for the proposed information.

Do not accept local execution as the evidence that this wider review has passed. Test the intended restrictions with synthetic data, including an attempted external fallback, and record whether the request is blocked or explicitly authorised. Set access rules for the inference endpoint and administrative functions, then verify that an unauthorised caller is refused. Keep this acceptance process separate from any claim of legal compliance, which this hardware assessment does not establish.

Cost the service before approving the box

Ask for a dated supplier quotation covering the proposed machine, applicable taxes, delivery and support before deciding whether it fits the £4,000 limit. Keep an ownership-cost worksheet for electricity, engineering time, maintenance, backups and the contingency for an unavailable machine. Compare that with the alternative service at the same accepted output quality and workload volume, rather than comparing a hardware invoice with an assumed API bill. Do not count unoccupied capacity as a saving or describe local requests as cost-free.

Our recommendation is to proceed when the trial meets the agreed service limits, the data-flow checks pass and a named operator accepts responsibility for recovery and maintenance. If one of those conditions remains unproven, retain the budget and resolve the missing evidence before purchasing.

Update, October 2026

Our DGX Spark concurrency engineering article, published on 22 April 2026, provides a later worked example of evaluating concurrent inference requests. Use it as a separate test-design reference, not as a performance guarantee for the installation you are considering.

Sources

Want this in your team?

Tell us what you're building and where you want to get to, and we'll think it through with you.

Let's have a chat Follow by RSS →

All insights →