Federated Learning on Edge Devices: Training Local LLMs Privately

Federated learning can train a shared model without collecting the raw training records centrally. That is a useful architectural choice, not a complete privacy guarantee. Google's description of the training cycle explains the local work and shared updates. Research on leakage from gradients shows why those updates still deserve scrutiny.

For a field application, we would begin with a narrower question than whether a phone can train an LLM: what should improve from local examples, and what information must never leave the device? Define that before choosing the training protocol or mobile framework.

Separate local predictions from shared training

In Google's federated learning example, a phone downloads a model, learns from its local data and sends an update for aggregation with other devices' updates. The records used for that local training stay on the phone. Google distinguishes this from running an existing model locally to make predictions.

Use that distinction in the product brief. Treat offline inference, local adaptation and participation in shared training as separate requirements. Do not make a technician's ability to complete an inspection depend on contributing to a training round. Decide independently when a device may accept a new model and when it should continue using an approved version.

For example, a proposed pilot might try to improve the classification of short maintenance notes. Define the allowed inputs, the output categories and the mistakes that matter. Keep this as a testable proposal, rather than claiming that every note should become training data or that every field task needs an LLM.

Keeping records local does not make updates anonymous

The authors of Deep Leakage from Gradients demonstrated reconstruction of private training data from shared gradients in their experiments, including language tasks. This is evidence of a possible leakage route. It does not establish that every federated system exposes every record.

The practical recommendation is to write down who can observe individual updates, who operates aggregation and what the resulting model is allowed to reveal. Include those questions in the security review. Reject a design explanation that stops at “we never upload the original notes”.

Google's work on formal differential privacy guarantees makes the distinction explicit: federated learning alone does not address a final model memorising information unique to an individual. Differentially private training adds randomness and provides a mathematical way to bound the effect of changes in training data on the output model.

Ask a proposed implementation to state its guarantee and its assumptions. Specify whether the protected unit is one example or all the examples from one user. Google's account distinguishes these levels and notes that limiting contributions and adding noise can affect accuracy. Treat that as a privacy and model-quality trade-off to evaluate, not an encryption setting to tick off.

Adaptation still needs a device budget

LoRA offers a different approach from updating every model weight. It freezes the pre-trained weights and introduces trainable low-rank matrices, reducing the number of parameters trained for a downstream task. That is the mechanism described in the original LoRA paper.

Do not turn that finding into a promise that a chosen LLM will train comfortably on the organisation's phones. Require a measurement on the actual model, training configuration and supported devices. Record peak memory, elapsed training time and the effect on the application while training is active. Set acceptance limits before a demonstration makes the choice feel settled.

The FwdLLM research addresses federated LLM adaptation on mobile devices specifically. It identifies memory consumption and slow convergence as problems, then proposes training based on perturbed forward passes combined with parameter-efficient methods. Its experimental results support investigating this approach. They do not certify a particular field deployment.

For a pilot, compare the proposed adaptation with an unchanged model on the same held-out tasks. Include a simpler non-LLM option where it can answer the question. Our recommendation is to justify the extra training machinery with a useful improvement, rather than adopting it because a research prototype exists.

Plan participation around the work

Google's early mobile deployment scheduled training when a device was idle, plugged in and on a free wireless connection. This is a concrete example of eligibility rules, not a universal schedule for every application.

For field teams, write explicit rules for training during a shift, using mobile data and resuming after interruption. Test a device that misses rounds and one that reconnects with an older model. Define what should be discarded and what may be retried. Keep operational records and training participation as separate concerns in those tests.

We would also require a named owner for model approval. Propose a release process that records the model version, evaluation results and privacy configuration together. Test how to stop participation and return to an earlier approved model before rolling the pilot out more widely. These are acceptance criteria we recommend, not capabilities that federated learning supplies automatically.

Make the decision on evidence

Start with one bounded task and a written reason for learning from local examples. Agree which records are eligible, which outputs require a person to check them and what result would justify continuing. Review privacy assumptions alongside device measurements, rather than treating them as separate sign-offs after implementation.

At Dendro Logic, we recommend keeping the first decision reversible: a small pilot, a fixed evaluation set and an explicit stop condition. Choose federated learning when its local-data training design answers a real requirement and the measured implementation meets it. Do not use the label as proof that the system is private, offline-ready or useful.

Update, October 2026

For the wider data-flow and jurisdiction review around an AI deployment, our Data Sovereignty and AI Security playbook provides a related planning resource. Keep that deployment review distinct from evidence about the training protocol itself.

Sources

• Google Research, "Federated Learning: Collaborative Machine Learning without Centralized Training Data", 6 April 2017.
• Zhu, Liu and Han, "Deep Leakage from Gradients", 19 December 2019, revised preprint.
• Hu and colleagues, "LoRA: Low-Rank Adaptation of Large Language Models", 16 October 2021, revised preprint.
• Google Research, "Federated Learning with Formal Differential Privacy Guarantees", 28 February 2022.
• Xu and colleagues, "FwdLLM: Efficient FedLLM using Forward Gradient", 20 January 2024, revised preprint.