Real-device QA
Your app went offline. Did your user’s work survive?
A practical test plan for dropped connections, interrupted uploads, and retries that do not duplicate a user’s work.
A connection failure should not become a loss of work. Test the transition into failure, the state your user sees, and the recovery—not just whether an offline banner appears.
Start with the promise your app makes
A tester fills out a long inspection form, attaches a photo, and taps Submit. The connection drops. When it returns, the app shows an empty form. That is a more important finding than a slow loading animation: the user has lost work and cannot tell whether the submission exists. This guide uses that fictional inspection app to build a repeatable test plan.
Before testing, decide which actions must work offline, which can wait for a connection, and which must stop with an explanation. A cached task list may be available without a network; a server-verified submission may need to wait. Write the expected behaviour down for each action. Otherwise two testers can report opposite results and both be reasonable.
Android’s offline-first architecture guidance discusses local and network data sources, queued work, retries, and conflict handling. Those design concerns give you useful test boundaries. The scenarios below are our proposed QA exercises, not a platform certification checklist.
- Name one core journey and the exact build to test.
- Record what is saved locally, what is confirmed remotely, and what the user should see.
- Prepare disposable test data and a known starting state.
- Choose one representative phone and one contrasting device for the first pass.
Change the connection at the moment it matters
A cold start in airplane mode tests one state. It does not tell you what happens when a request has left the phone but its response never arrives. Exercise both. For each case, record the device, Android version, connection type, timing of the interruption, and the final saved record.
Use system connectivity controls for an initial manual pass. A controlled test network or debugging proxy can help the engineering team reproduce specific delays and response loss. Label how you simulated the failure; turning off Wi-Fi while mobile data remains active is not an offline test.
| Scenario | Action | Evidence to collect |
|---|---|---|
| Cold start offline | Open the app without a usable connection. | Available cached data and a clear explanation of unavailable actions. |
| Disconnect before submitting | Fill the form, disconnect, then submit. | The draft survives and the submission state is understandable. |
| Interrupt an upload | Disconnect while a photo is uploading, then reconnect. | A recoverable state, retained file, and one final attachment. |
| Lose the response | Have engineering simulate a saved request whose response is lost. | A retry does not create a second inspection. |
| Switch networks | Move between Wi-Fi and mobile data during the task. | No silent loss of edits or permanent loading indicator. |
| Resume later | Leave pending work, close the app, and reopen it. | Pending and completed work remain distinguishable. |
Follow a failed action all the way back to safety
After each interruption, ask the tester to explain the screen in their own words. Has the inspection been saved, is it waiting, or does it need another action? If the UI says only ‘Something went wrong’, the user may tap Submit again or abandon the task. Record the misunderstanding as well as the technical failure.
Retry the action once and inspect the resulting record through an authorised test environment. Count records and attachments, check field values, and compare the visible status with the stored result. A toast that says ‘Success’ is not enough evidence. If the tester cannot inspect the server, ask the developer to supply a redacted verification result.
Repeat with two quick retry taps and with an app restart between the failure and retry. Whether the implementation uses a queue, an idempotency key, or another mechanism is an engineering decision. Your acceptance condition is concrete: the intended operation completes once and the user can understand its state.
Make conflicts visible instead of guessing the answer
Suppose the inspection is edited on two devices. One saves an updated address online while the other keeps an older offline draft. Reconnecting now asks a product question: should the app preserve both edits, ask the user, or apply a documented rule? Do not decide that rule inside a bug report. Have the product owner define it, then test the implementation against it.
Also test what happens when an account session expires while work is pending. The app should avoid presenting unsent work as complete. After reauthentication, confirm that queued data is associated with the correct account. Use test accounts and fictional inspection details; recordings and diagnostic logs should not expose customer information.
Write the finding so someone else can reproduce it
A useful title is ‘Photo upload stays pending after reconnecting on build 42’, followed by the smallest sequence that reproduces it. Include the starting account state, selected file size, interruption timing, frequency, and expected recovery. Separate your observation from a guessed cause: ‘spinner remains for two minutes’ is evidence; ‘the queue is broken’ is a hypothesis.
For the inspection example, a release gate could require that drafts survive every listed interruption, retry produces a single inspection, and the UI matches the stored state. This is an example gate for one product, not a universal rule. Agree your own gate before the final retest.
Retest the repaired build on the original device and interruption sequence, then run the normal connected journey and one contrasting failure. Keep the original finding open until that evidence is attached. If the fix changes persistence or retry behaviour, add a focused regression scenario to the next release cycle.