Search any app store for "offline AI" and you will find plenty of apps wearing the badge. Tap the description and it says "works without internet." Install it, open it, ask it a question in airplane mode — and watch it fail.
The word "offline" has become loose marketing. Some apps use it to mean "you can open the app without internet." Some mean "one tiny feature works offline, the rest needs the cloud." And some genuinely run AI models on your phone, no connection required. If you care about privacy, travel, battery, or just honest software, it is worth knowing how to tell the difference. Here is a practical guide to verifying an AI app's offline claims — with tests you can run in a few minutes on the phone you already own.
Why it matters more than it sounds
An app that quietly phones home is not just a nuisance. It decides:
- Your privacy. A prompt sent to a server is data you no longer control. A prompt processed on your phone never leaves.
- Whether it works where you need it. Planes, subways, rural areas, roaming abroad, outages — offline means the app works wherever you are.
- What it costs you. Cloud AI costs someone money per query, which is why it usually arrives with subscriptions, ads, or data collection. On-device AI costs nothing after the download.
- Your battery. Constantly streaming data to a server drains your phone faster than computing locally, which the NPU on modern phones is designed for.
Test 1: The airplane-mode test
This is the fastest, most honest test, and it takes two minutes:
- Open the app and use its main AI feature while online, so you know what "working" looks like.
- Turn on airplane mode (both Wi-Fi and cellular off).
- Try the exact same feature again.
Watch what happens. A truly offline AI app behaves exactly the same — it generates the answer, image, or transcription at the same speed, with no complaints. Anything else is information:
- "No internet connection" error — the app's AI runs in the cloud. It is not an offline app, whatever the description says.
- Long spinner then a cached or generic response — the app may be fetching and only pretending.
- Works, but slower — sometimes the app is downloading the model the first time and then caching it. Turn airplane mode back off, wait, then test again after a while to see if it was a one-time download.
- Works identically — this is the real deal. On-device processing.
One subtlety: give the app one chance to download its models first. Honest on-device AI apps (like LLM Hub) download their AI models on first setup — that download needs the internet. The test is about what happens after the models are on your phone.
Test 2: Check the app's data usage
Airplane mode tells you whether the app needs the internet. Data-usage stats tell you whether it uses it even when it claims not to.
- On Android: Settings → Network & internet → SIMs → App data usage (the exact path varies by manufacturer). Look at the app's background and foreground data over the past week. An honest offline app shows data used during the initial model download and updates, and essentially nothing during your daily use.
- On iPhone: Settings → Cellular (or Mobile Data) → scroll to the app. You can also disable cellular data for just that app, keep Wi-Fi off, and test whether the AI feature still works.
If you are technically inclined, a firewall or network-monitoring app on Android (many are open source) shows per-app traffic in real time. Open the app, use the AI feature, and watch whether any data leaves your phone. Any upload while you chat with the "offline" AI means your prompts are going somewhere.
Note: a small amount of traffic during startup — a version check, a crash report — is normal even in honest apps. What matters is traffic that correlates with you using the AI feature.
Test 3: Look at the model download size
Genuinely on-device AI apps have something cloud-only apps never do: model files stored on your phone. You can check:
- On Android: Settings → Apps → the app → Storage. An offline AI app usually shows hundreds of megabytes (sometimes gigabytes) of data — those are the downloaded AI models. An "offline AI" app that takes up 80 MB total cannot possibly contain a real language model.
- On iPhone: Settings → General → iPhone Storage → the app. Same logic.
Also look at what the app itself tells you. Honest on-device apps show you which models are installed, their sizes, and let you download or delete them (LLM Hub does exactly this with its 15+ models). If an app never mentions models, downloads, or storage — yet claims to work offline — be skeptical.
Test 4: Read the listing and permissions critically
Before you even install, the app's store listing and permission requests tell a story:
- Permissions. An offline chat app has no business asking for network-heavy permissions paired with an account requirement. Basic network access is fine (apps need it to download models and update), but be wary of the combination: mandatory account + location + contacts + "offline AI."
- Login walls. A fully on-device app has no server, so it has no technical reason to demand an account. When an "offline" app forces sign-in before the AI works, the login is usually there to identify you, meter your usage, or both.
- The description's fine print. Honest apps say which features work offline and what needs downloading. Vague claims like "AI works anywhere" without any mention of on-device models, downloads, or sizes are a yellow flag.
Test 5: Test after a fresh install
Some apps work offline on day one because they cached cloud answers, then degrade. Try this: uninstall and reinstall the app, skip creating an account if possible, deny network permission temporarily (or keep airplane mode on from the start), and see what works. A genuinely offline app will ask you to download its AI models and then work. A fake one will fail at the login screen.
Red flags vs green flags
Put it all together, and the pattern is clear:
Red flags:
- Fails the airplane-mode test, or works only with a connection
- Tiny storage footprint with no model downloads
- Mandatory account for "offline" features
- Data usage spikes every time you use the AI
- Store description says "offline" but never mentions on-device models
Green flags:
- Works identically in airplane mode after a one-time model download
- Shows you installed models, sizes, and lets you manage them
- No account required; no tracking
- Open-source code you (or anyone) can audit
- Honest about which features need a connection, if any
A quick note on hybrid apps
Not every honest app is 100% offline, and that is fine — as long as it is upfront. A reasonable hybrid might run chat on-device but need the internet for image generation, or download models on demand. The line between honest and dishonest is not "does it ever use the internet" but "does it tell you, and can you verify it." The tests above give you that verification.
The bottom line
"Offline" is a claim you can check in about five minutes: airplane mode, data usage, storage size, the login wall. Most fake-offline apps fail at least one of them. Once you know what to look for, you will spot the real ones quickly — they are the apps that show you their models, ask for no account, and keep working with the radios off.
LLM Hub is built to pass every test on this list: its AI models run entirely on your phone, the code is open source, there is no account and no tracking, and airplane mode changes nothing. If you want to see what genuinely offline AI feels like, that is the bar to compare everything else against.
