Local Inference: Securing Data Flow Without False Guarantees

Running a model on your own hardware does not, by itself, establish a chain of custody for the information it processes. NIST's forensic guidance describes custody in terms of recorded handling, secure storage and integrity verification, not the location of an inference server. We would evaluate local inference and evidence handling as separate design decisions, then test how they work together.

Here, local metal inference means running the model on hardware under your organisation's control rather than sending the inference request to an external model service. It is a deployment description, not a security certification. For a compliance workflow, our starting question is what happens to a record before, during and after the model reads it.

Define the boundary before claiming privacy

We recommend drawing the complete route from the original document to the model response and any subsequent action. Include document extraction, retrieval, temporary files, monitoring, backups and any external fallback in that review. Treat each component as a separate access decision rather than assuming that a local model makes the entire workflow private.

This follows NIST's zero trust guidance, which rejects implicit trust based solely on physical location, network location or ownership. Our recommendation is to authorise the application to read only the records needed for its task, and to test that restriction at retrieval time. A useful acceptance test is a request for a record the current user is not allowed to see, with an expected result of denied access rather than an answer.

For sensitive workflows, we would make the response to local model failure explicit. If external processing is not approved, the acceptance criterion should be a visible failure or queued work, not an unannounced remote retry. Test that behaviour with the local inference endpoint unavailable, and inspect the destinations of outgoing requests.

Preserve evidence independently of the model

A model's account of what happened should not substitute for the underlying record. NIST's generative AI risk profile describes confidently presented false content and warns that generated citations or explanations can also be misleading. We therefore recommend keeping a generated summary separate from the source material and any authorised decision about it.

Consider a hypothetical construction inspection workflow in which a model extracts an observation from a report. We would retain the received report, associate the extraction with that version, and record whether a named reviewer accepted or corrected it. If a replacement report arrives, our proposed design creates a new version and preserves the relationship instead of silently overwriting the earlier evidence. These are design recommendations for the example, not claims about a deployed Dendro product or a particular statutory process.

For material that may become evidence, NIST SP 800-86 recommends a documented custody process, including who handled it, their actions and timing, secure storage, and integrity checks on originals and copies. Our application of that guidance is to record transfers and transformations alongside the source, rather than ask the model to reconstruct them later. The same guidance describes comparing message digests to verify copied data, a narrower test than proving the truth of the document's contents.

Protect the audit trail as a separate system concern

An audit trail needs a defined purpose and protection of its own. NIST SP 800-53 Revision 5, control AU-3, identifies event type, time, location, source, outcome and associated identities as audit record content. For the inspection example, we would map that structure to receipt of the report, the extraction attempt, the reviewer action and any export.

We use this control catalogue as an engineering reference, not as a compliance certificate or a statement of the reader's legal duties. Our recommendation is to agree the required events and retention policy with the people responsible for the workflow before implementation. We would also decide which references are sufficient to investigate an event without copying every report or prompt into the log.

That last decision matters because AU-3 explicitly discusses privacy risks in audit records, including records that capture inputs. AU-9 addresses protection against unauthorised access, modification and deletion, with enhancements covering separate storage and cryptographic protection. We would assess those protections against the actual threat, including an administrator or application credential that can alter business records.

Do not describe an ordinary application log as immutable without testing the stated protection. Our proposed test is to attempt alteration and deletion using the application's normal credentials, then check both the access result and the alerting behaviour. For a separate logging repository, include a test of what remains accessible after the application host becomes unavailable.

Specify what happens when recording fails

A successful inference response is not our proposed acceptance criterion for a completed compliance action. We would require the necessary evidence and audit event to be recorded before the interface reports that action as complete. Where that cannot happen, the interface should distinguish pending work from an accepted record.

Control AU-5 calls for alerts on audit logging failures and leaves additional responses to the organisation. For this example, we recommend testing exhausted log storage and loss of access to the audit repository, then checking the operator alert and the user-visible result. Choose whether to stop, queue or restrict work according to the consequences of proceeding without its evidence.

This is an architectural requirement, not a model setting. In Continuous Architecture in Practice, chapter 2, Murat Erder, Pierre Pureur and Eoin Woods explain why a quality label alone is insufficient and recommend scenarios with a stimulus, response and measurement. We would apply that approach to local inference by specifying the failure, the expected system behaviour and the evidence needed to accept the test.

Local processing is one deployment choice within that design. Our recommendation is to approve the workflow only when its data boundary, access decisions, record history and failure behaviour have each been demonstrated. Do not replace those checks with a promise that local hardware ensures compliance.

Update, October 2026

Our engineering article on team context infrastructure describes source-linked retrieval and citation verification in an agent workflow. It adds an implementation discussion of checking generated answers against their sources, which we would evaluate separately from custody and access controls.

Sources

• NIST, "Guide to Integrating Forensic Techniques into Incident Response", SP 800-86, August 2006.
• NIST, "Security and Privacy Controls for Information Systems and Organizations", SP 800-53 Revision 5, 23 September 2020, including updates of 10 December 2020.
• NIST, "Zero Trust Architecture", SP 800-207, 11 August 2020.
• NIST, "Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile", AI 600-1, 26 July 2024.
• Murat Erder, Pierre Pureur and Eoin Woods, "Continuous Architecture in Practice", chapter 2, Addison-Wesley, 21 May 2021.