#On-Device AI

4 articles tagged with On-Device AI

On-Device Federated Learning with Differential Privacy on iOS and Android

An enterprise privacy-preserving AI architecture guide. We analyze FedAvg gradient aggregation, Gaussian differential privacy noise injection, Secure Enclave local training, and training mobile ML models without centralizing user data.

9/2/202624 min read

On-Device NPU Power Consumption: Measuring Battery Drain of Local LLMs

A hardware engineering benchmark measuring the electrical current draw of running 3B and 7B models on mobile NPUs. We evaluate Apple A19 Pro, Qualcomm Snapdragon 8 Gen 5, and Google Tensor G6 energy efficiency.

9/2/202622 min read

On-Device Vector Search: SQLite + AI Embeddings in Flutter Apps

Build private, offline-capable semantic search in Flutter using SQLite vector extensions and local embedding models — no cloud API required.

7/19/202614 min read

Running Llama 3 on Mobile: The Ultimate Guide to Local LLMs with Flutter

The future of AI is offline. In this 4,500-word tutorial, we compile Llama 3 to run on iOS and Android using MLC LLM and Flutter. We benchmark token speed, memory usage, and battery drain.

2/8/202632 min read