Email is where most of us spend the first hour of the working day, and it is also where some of our most sensitive information lives: contracts, health details, family news, job offers. Cloud AI email assistants make that work faster, but they do it by sending your messages to a server. Offline AI on your phone offers a different deal: the same help with drafting, summarizing and tidying email, with nothing ever leaving the device.
Why email is the perfect job for on-device AI
Not every AI task suits a phone. Generating a feature-length video or training a model does not. Email, though, is an ideal match for small on-device models:
- The inputs are short. A typical email is a few hundred words. Even a long thread is a few thousand. Small models handle this easily, with no need for the huge context windows that cloud models advertise.
- The outputs are short too. A reply, a summary, a reworded paragraph. Short outputs mean fast responses, often in a couple of seconds.
- The work is private by nature. Email contains the things you least want uploaded: financial details, medical appointments, private conversations. Keeping the processing on the device is not a nice extra here, it is the whole point.
- It works anywhere. On a plane, in a dead zone, on a train with flaky Wi-Fi — your inbox helper keeps working because it never needed the network in the first place.
What offline AI can do with your email
Draft replies in your tone
The most common use is also the simplest. Paste an incoming email and a one-line brief — "polite decline, suggest next Tuesday" or "accept, confirm the 2pm start" — and the model drafts a reply. The trick is to give it enough context: who you are writing to, the relationship, and the tone. "Formal, to a client" produces a very different draft from "casual, to a colleague I know well."
A good habit is to treat the draft as a first version, not a finished message. Read it, fix anything that sounds off, add the detail only you know, and send. The AI saves you the blank-page problem; you keep the judgement.
Summarize long threads
Paste a long thread into your offline AI app and ask for three things: a two-sentence summary, the decisions made, and the open action items with owners. For very long threads, summarize in chunks and then ask the model to combine the summaries — smaller models are more accurate this way.
Turn rough notes into a clean email
Many emails start life as a hurried note: "tell sarah delivery delayed 2 days, apologize, offer discount code." An on-device model turns that into a properly structured message with a greeting, a clear explanation and a polite close. This is especially useful on a phone, where typing a long polished email on a small keyboard is slow.
Rewrite and polish
Paste a draft you wrote yourself and ask for a clearer, shorter, or more formal version. This is useful for important emails — a job application, a complaint, a sensitive message — where wording matters. Ask the model to explain what it changed, so you can decide whether each edit is an improvement or a change in meaning you did not intend.
Extract action items
Ask the model to read an email and list what you need to do, by when, and who is waiting on you. It is a quick way to turn an overflowing inbox into a to-do list without reading every message twice.
Translate messages privately
If you receive email in another language, an on-device translator can translate it without sending the message to a translation server. This matters when the email contains personal or business details you would rather not hand to a third party. On-device translation quality for common language pairs is now good enough for everyday email, though you should still be careful with contracts or anything legally binding.
A practical offline email workflow
You do not need to replace your email app. Offline AI works alongside it:
- Download a model once. Open an on-device AI app such as LLM Hub, pick a small chat model — Gemma, Llama or Phi models in the 3-4B range are a good starting point — and download it over Wi-Fi. This is the only step that needs internet.
- Save a reusable prompt. For example: "You are my email assistant. Draft replies in a friendly professional tone, under 150 words. Never invent facts, dates or names — only use what is in the email I paste."
- Copy, brief, paste back. Copy the email into the AI app, give a one-line brief (draft a reply, summarize, extract actions, translate), review the output, then paste it into your reply in the mail app.
After a few days this takes seconds per email. Some on-device apps support sharing text directly from your mail app, which removes the copy-and-paste step entirely.
Which model should you use for email?
Email is light work, so you do not need the largest model your phone can run. A 3-4B parameter model is the sweet spot for most people: fast, small enough to keep alongside other apps, and capable of clean, coherent email drafts.
A few practical pointers:
- Prefer an instruction-tuned model. Models tuned for following instructions draft better emails than raw base models, which are more likely to ramble or continue your text instead of answering it.
- Test with your real emails. Try two or three models on the same draft reply and see which one's tone you prefer. Model quality is partly a matter of taste.
- Mind the storage. A 3-4B model takes roughly 2-3 GB. If storage is tight, try a smaller 1-2B model first — it will still handle short replies and summaries well.
- Keep it to one or two models. You do not need a shelf full of models for email. Pick one you like and learn how it responds to your prompts.
The privacy advantage, honestly stated
The privacy case for offline email AI is simple: your messages never leave your phone. There is no server log, no account to breach, no training run that might memorize your text, and no advertiser or data broker downstream. That is a real difference from cloud AI email tools, which must by definition receive your messages to process them.
There are still things offline AI does not protect you from. It cannot stop your email provider from scanning messages on its own servers, and it cannot make a message you send to someone else private once it leaves your phone. What it does is remove one entire category of exposure: the AI assistant itself. The tool that reads your email to help you draft a reply is no longer a third party holding a copy of your inbox.
That matters most for the emails you would hesitate to paste into a cloud chatbot: medical correspondence, financial negotiations, legal matters, job applications, and anything involving other people's private information. With on-device AI, the hesitation goes away.
Where offline AI email falls short
Honesty matters here. Offline AI is not better than cloud AI at everything:
- It does not read your inbox directly. Because it lives outside your mail app, you copy and paste. Cloud tools built into Gmail or Outlook can draft from your whole inbox automatically. The privacy gain costs a little convenience.
- It cannot search your email history. Asking "what did Sarah say about the budget in March?" requires you to find and paste the relevant messages yourself. On-device document chat is improving, but it is not yet a full inbox search engine.
- Very long or complex threads can confuse small models. Summaries of enormous threads may miss a detail or mix up who said what. Chunking helps, but for a high-stakes thread, read the originals yourself.
- It will not know your calendar, contacts or files. Cloud assistants can check your diary and suggest meeting times. Offline AI only knows what you paste into it. That is the privacy trade-off, and it is worth understanding before you commit to it.
For most everyday email, none of these limits is a dealbreaker. For a small number of complex cases, they are worth knowing about.
Prompts that work well for email
Good results come from specific prompts. A few templates to adapt:
- Reply: "Draft a reply to this email. Tone: friendly and professional. Key points: [your points]. Keep it under 120 words."
- Decline: "Write a polite decline to this invitation. Thank them, give a brief reason, and suggest we reconnect next month."
- Summary: "Summarize this thread in three bullet points: what was decided, what is still open, and what I need to do."
- Simplify: "Rewrite this email so a non-expert can understand it. Keep all facts and numbers the same."
- Translate: "Translate this email into English. Keep names, dates and numbers exactly as they are."
- Tone check: "Read my draft below and tell me if it sounds too blunt. Suggest a softer version of any harsh sentences."
The pattern is always the same: what you want, the tone, the constraints, and the content. The more specific the brief, the less editing the draft needs.
Setting it up in LLM Hub
If you use LLM Hub, the setup takes a few minutes. Download a small chat model such as a Gemma or Llama 3-4B model over Wi-Fi, then open a chat and paste your email text with one of the prompts above. Because LLM Hub runs fully on-device, with no accounts and no tracking, your email text stays on your phone throughout. The translator and transcriber built into the app cover the other two common email jobs: translating a message in another language, and turning a voice memo about an email into text you can paste into a draft.
For a regular email routine, keep one chat thread for drafting and one for summaries, so the model keeps the context of each job separate. And remember the one rule that applies to every AI draft, on-device or not: read it before you send it.
The bottom line
Offline AI turns your phone into a private email assistant: drafting, summarizing, translating and tidying, with no connection and no copy of your messages sitting on a server. It asks for a little more effort than a built-in cloud assistant — a copy here, a paste there — and in return it gives you something cloud tools cannot: the certainty that your inbox stays yours.
If you write a lot of email and care about who reads it, that is a trade worth making.

