Blog · Reports and retests
Blocked is not passed: how to read open checks in a QA report
Short answer
A check that was blocked or not run tells you nothing about whether that part of the site works. It must stay open and never count toward a green result. Sybqa lists it under Not a pass yet, and a required one keeps the verdict from reading Selected checks passed.
On this page
A QA report with no failures can still be unfinished. If the checkout check never reached checkout, the report has no news about checkout, good or bad. This post explains the open states in a Sybqa report, why they never turn green and what to do about each one. The website QA report example shows where they appear in a report.
By Sybqa · First published · Updated
Three states that are not a pass
| State | What happened | What it tells you about the site |
|---|---|---|
| Blocked | The check could not run. Something was in the way, such as a login or protection page, a request the read-only policy stopped, or a reference that was unavailable. | Nothing yet. The browser never reached what it was meant to check. |
| Not run | The check was skipped, for example because the time budget ran out, the run was stopped, or no journey was set up for it. | Nothing yet. The check did not happen. |
| Needs review | Something may be wrong, but the evidence does not settle it. | A possible problem that a person should look at. |
Passed is the only state that counts as passed. Failed means a real browser saw something break.
Why an open check never counts as green
Counting a blocked check as passed turns “we could not look” into “it works”. Those are different facts. A signup flow blocked by a login screen may be broken, may work, or may not be finished yet. The report cannot say, so it should not pretend to.
Counting it as passed also hides work. Once a blocked check disappears into the pass total, nobody goes back to finish it, and the pages behind a login or a payment step stay untested.
How open checks set the verdict
Every Sybqa report opens with one of three verdicts:
- Issues found: at least one check that decides the verdict failed (advisory leads do not count).
- Acceptance incomplete: no check that decides the verdict failed, but at least one required check was blocked, not run or needs review.
- Selected checks passed: every required check passed.
Some checks are advisory, such as an AI review lead. They stay visible under Not a pass yet but do not decide the verdict, and the report says how many of the open checks are advisory. The command-line runner follows the same rule: it exits with 0 when the required checks pass, 1 for an observed failure and 2 when required checks are still blocked, not run or awaiting review, so a pipeline can stop on an unfinished run.
What to do with each open check
| Reason in the report | Next step |
|---|---|
| A login, password or protection page was in the way | Test a URL a browser can reach without signing in. Hosted Sybqa does not sign in with test accounts or fill password fields, so cover pages behind a login with your own end-to-end suite and record that result separately. Never use a real customer account. |
| Requests were blocked by the read-only policy | Hosted runs block writes by default. Check signup, payment or saving in a test environment with test-mode keys, and record that result separately. |
| The time budget ran out | Narrow the plan to fewer pages or journeys and run it again. |
| No journey was set up for the task | Add a journey with an expected result, such as a heading that must appear. A crawl alone cannot show that a task can be completed. |
| The run was stopped | Completed evidence stays in the report. Start a new run for the unfinished checks. |
| Needs review | Open the evidence, decide whether it is a real problem and write down your decision. |
If a fix was supposed to close the check, retest it the same way it ran the first time. How to retest a fix without writing test code explains what has to stay the same.
Say what was not checked when you share a result
When you tell a client or teammate that QA passed, say which checks passed and which stayed open. “Signup and pricing passed at desktop and phone widths; checkout was blocked at the payment step and still needs a test-mode run” is a result someone can act on. “QA passed” is not.
The sample report shows an illustrative report with open items, recorded against a synthetic fixture. The recorded demo store run lists every open check of a real run on a fictional store next to the checks that failed. The smoke test guide shows how to keep a small check you can finish on every release.
Common questions
Is a blocked check a bug in my site?
Not necessarily. Blocked means the check could not run. The cause can be in the site, such as a broken login, or in the setup, such as missing test access. Find the reason before you decide.
Why does my report say Acceptance incomplete when nothing failed?
Because at least one required check was blocked, not run or needs review. Open Not a pass yet to see which ones and why.
Sources
Related
- How to smoke-test a published website and retest a fix
- AI testing false positives: a finding triage worksheet
- What is website QA? A plain definition with examples
- How to retest a website fix without writing test code
- AI exploratory testing: treat every finding as a lead
- Example run: a planted checkout bug on a demo store
Try Sybqa on your site
Explore bugs, UI consistency and user journeys from a reviewed plan. Inspect browser evidence and visual review leads in the report. The free plan gives 10 runs a month without AI review after you sign up. See pricing.
Add your site link here and it will be waiting in the console after you create an account. Then describe one journey to check and review the test plan before any run starts.