From a pile of beta feedback to a release decision

Start by separating bugs, questions, and ideas

A closed beta ends with 38 comments. Some are duplicates, some describe an unclear label, and one says a saved draft disappeared. Treating every comment as the same kind of ticket hides the decision that matters: can the intended users finish their essential work safely? The numbers in this guide are illustrative, not customer results.

Make a first pass without assigning priority. Separate observed defects, usability friction, product suggestions, and setup or support problems. Keep the original tester wording, then add a short neutral summary. A request for dark mode and a crash on launch may both be valid feedback, but they need different decisions and different evidence.

Group reports that appear to describe the same failure while preserving their device and account context. Do not close a report as a duplicate solely because the title sounds similar. Two ‘cannot sign in’ reports may involve a delayed OTP and a separate keyboard layout problem. Ask for the missing detail before merging their histories.

Get enough evidence before deciding the cause

For each suspected defect, request the app build, device, Android version, starting state, shortest steps, expected result, and actual result. Include frequency as an observation: ‘three failures in five attempts’ is more useful than ‘always broken’ when the tester has tried it once. If a recording contains personal information, replace it with a redacted version or a reproduction using synthetic data.

Ask a second person to repeat the minimal sequence. If they cannot reproduce it, keep the original evidence and record what differs. A network, permission, account, or device-specific failure is not invalid simply because it does not happen on the developer’s phone. Use an explicit ‘needs more evidence’ state and name who will gather that evidence.

Separate observed results from explanations. ‘The app returned to the sign-in screen after reopening’ is a result. ‘The token refresh is broken’ is a proposed cause. Prematurely assigning a cause can send the engineer toward the wrong component and make a useful tester report harder to trust.

Keep severity and priority as different decisions

Severity describes the effect on the user. Priority describes when the team will address it. A crash in a frequently used onboarding flow can be both severe and urgent. A severe issue in an optional, disabled feature still needs a documented decision, but its release exposure may differ. Agree the vocabulary before the final day of testing.

Use a compact framework rather than an unexplained numerical score. Ask whether the defect loses work, exposes information, blocks the core task, or has a practical workaround. Then consider who is affected, how often the state occurs, what changed recently, and how confidently the team understands the fix. The table is a suggested team convention, not a Google Play severity scale.

Make the release meeting an evidence review

Keep a register with one row per distinct finding. Include issue ID, affected journey, observed impact, severity, priority, owner, current status, target build, and a link to evidence. Add the release decision and its reviewer. This is enough to replace an unstructured chat thread without introducing a large process.

For a fictional draft-loss issue, a useful entry is: ‘QA-017; inspection editing; draft missing after process restart; high impact; assigned to mobile engineer; fix in build 43; awaiting original-device retest.’ That row shows what happened and what remains unresolved. ‘Done’ or ‘developer checked’ would hide the missing verification.

Discuss unknowns explicitly. If the team accepts an unresolved issue, record who made the decision, the affected users and scope, the workaround, and when it will be revisited. A risk acceptance should not silently become a claim that the defect passed. Keep private diagnostics and tester details out of any public release notes.

Close a finding only after a targeted retest

Start with the original reproduction on the build containing the fix. Confirm the precise expected outcome and attach new evidence. Then test a nearby normal path and an adjacent failure path. For the draft-loss example, verify ordinary save and reopen, interrupted save, and account switching with disposable records, rather than merely checking that the screen opens.

Use separate states for fixed awaiting retest, verified, cannot reproduce with current evidence, and accepted risk. They answer different questions. If a fix is delivered near the end of the cycle, report the amount of testing actually completed on that build. Do not imply that a fresh build has the same coverage as a candidate tested for two weeks.

Write the release gate before looking at the final score

For a small app, an example gate might require verified completion of the main journey on the agreed device matrix, no unresolved data-loss or account-isolation defects, and named decisions for every remaining high-impact finding. Tailor it to the product. A calculator and a field-work app do not have identical failure consequences.

Record what you tested, what you learned, what changed, and which limitations remain. Google’s production-access process asks developers about testing and readiness; retain truthful evidence that can support those answers. This triage process does not replace Google’s eligibility requirements or guarantee production approval.

The final output should be a short decision note linked to the register: the candidate build, tested scope, resolved blockers, remaining risks, and responsible decision maker. That gives the next release a starting point and gives everyone a shared reason for shipping—or for testing a little longer.