Vector Databases on Edge Devices: Search Is Not the Record

Vector search can run locally on a small device. Alex Garcia's sqlite-vec demonstration used a Raspberry Pi Zero, with an embedding model and database on the device and no data leaving it. For field applications, that makes searching local observations a practical design option. It does not establish whether an inspection record was saved correctly or whether another device has received it.

We recommend treating the vector index as a way to find records, rather than as the record itself. Keep the original observation, its identity and its revision separately identifiable. Judge the design by whether an engineer can retrieve the right evidence offline, then reconcile the work when the connection returns.

What vector search contributes

An embedding is a numerical representation used to compare content. The Sentence-BERT paper describes sentence embeddings compared through cosine similarity. That supports searching for related wording, rather than requiring the query and the stored text to match exactly.

Consider a hypothetical maintenance app. An engineer describes a recurring vibration in their own words and wants to find earlier observations about the same symptom. Offer similarity search for that question. Keep a separate exact lookup for the asset identifier, and show which original observation each search result represents. Do not ask a similarity score to decide whether two records belong to the same physical asset.

Mozilla's June 2024 announcement of support for sqlite-vec describes adding vector search to an embedded SQLite database for local AI applications. A separate hosted vector service is therefore not the only design worth evaluating. Start with the information the field worker needs on the device, rather than choosing a server product first.

Save the observation before enriching it

For the maintenance example, we would save the entered text and asset identifier before scheduling embedding work. Make the record's save status visible independently of its search status. If embedding fails, the acceptance criterion should be that the engineer can still reopen the original entry and retry indexing it.

Vaughn Vernon and Tomasz Jaskuła distinguish command and query models in Strategic Monoliths and Microservices, chapter 9. Their discussion allows logical separation within one database, rather than requiring separate database servers. Applied here, the useful distinction is between the saved observation and the representation built for searching it.

We would give each indexed passage a reference to its source record and revision. Define a rebuild procedure that starts from retained source content, then test it on a disposable copy. Ask the test to prove that deleting and rebuilding the search index leaves the observations unchanged. Do not make recoverability an assumption hidden inside the choice of database.

Check the whole offline path

Garcia's local demonstration combines a model, SQLite, sqlite-vec and an embedding extension. The database is one component of the path, not a substitute for the model that produces embeddings.

For a proposed field deployment, disable connectivity and test the complete user task. Install the required model in advance, open the app from a cold start and search the documents assigned to that device. Include the first query after installation, not only repeated queries after everything has loaded.

Specify the expected behaviour when no embedding can be produced. For our example, we would retain exact asset lookup and the original observations, then label similarity search as unavailable. The engineer should not have to infer whether an empty result means no relevant history or an unfinished indexing job.

Measure usefulness on the target device

The sqlite-vec v0.1.0 release describes brute-force vector search, rather than promising an approximate nearest-neighbour index. Treat that as a description of the release, not a universal performance result for mobile applications.

Build a trial using the intended device and a representative collection of field records. Ask experienced users to identify useful results for real search questions before comparing search approaches. Include similar wording about different assets, outdated instructions and queries for which the collection contains no answer.

Measure the complete task, including embedding the query and opening the source record. Record memory use, storage use and battery consumption during the same trial. Set acceptance limits with the operational team instead of importing a benchmark from different hardware. Keep exact search as a comparison, and retain it if semantic search does not improve the task enough to justify its cost.

Make synchronisation explicit

For this design, we would synchronise source records using their identifiers and revisions, then update each device's search representation. Write down what happens when two engineers edit the same observation offline. Test interrupted transfers and repeated submissions before describing the workflow as complete.

Show the user when the device last received its assigned records. In search results, expose the source revision and distinguish local work from centrally accepted work. In the hypothetical maintenance example, an engineer should be able to see that a matching observation is awaiting synchronisation without mistaking it for an approved maintenance instruction.

Choose a model-version policy before deployment. Record which embedding model produced each searchable representation, and test the migration procedure when that model changes. We would rebuild the affected index from source records and rerun the relevance checks before enabling the replacement.

Local storage still needs security controls

NIST's mobile-device guidance addresses device loss, insecure access and data leakage through synchronisation. It also discusses encryption and application sandboxing. Keeping a search index on a tablet does not remove those security concerns.

Define which records each device may hold and how long they may remain available offline. Test result access with a user who must not see another team's records. Include downloaded source text, search previews and backups in the review, rather than checking only the main record screen. Decide how the app should behave when offline access expires, and make that limit visible before a field visit.

Update, October 2026

For teams extending this pattern into an assistant, our engineering article on grounding packs and deterministic context explains a related design: supplying dated evidence and explicit gaps before an agent acts. For a field-search pilot, start with a narrower acceptance test: retrieve the relevant observation offline, open its source revision, and preserve it through a failed synchronisation attempt.