Sybqa

Blog · Testing basics

AI exploratory testing: treat every finding as a lead

Short answer

An AI explorer drives a real browser toward a goal and reports what looked wrong. Treat each report as a review lead until a deterministic browser check reproduces it. Sybqa marks an AI flag failed only when a browser measurement and a fresh replay confirm it; flags the measurement contradicts are listed as Dropped and never counted.

On this page

AI exploratory testing gives a model a goal and a browser and lets it look around, the way a tester explores a site without a script. It can reach paths nobody wrote down. It can also be confidently wrong. This post explains how to use it well, and how Sybqa keeps what an AI thought it saw apart from what a browser confirmed.

By Sybqa · First published · Updated

What an AI explorer does

Exploratory testing means learning a product by using it: you choose the next action from what you just saw instead of following a fixed script. An AI explorer does the same with a model in the loop. It reads the page, picks an action toward a goal, such as “find the cancellation policy, then start a booking”, and notes anything that seems wrong on the way.

In Sybqa, AI exploration is part of AI review on the paid plan. A run can use a team of 2 to 5 AI bots, each with its own mission (customer journeys, input validation, state and recovery, mobile layout or keyboard access), its own browser session and its own evidence. The first bot runs the baseline browser checks (accessibility, site integrity, UX and launch checks) on each page it captures. Specialists spend their time exploring, and the report names any page that only a specialist reached. One combined report keeps failures, blocked work and missing coverage.

What it is good at, and where it goes wrong

Useful for

  • Finding paths and edge cases nobody wrote a test for.
  • Trying awkward inputs: a very long name, an empty field, the back button halfway through a flow.
  • Noticing that something looks off: overlapping text, a button that seems disabled, a missing error message.

Weak at

  • Judging measurable things from a screenshot, such as text contrast or tap-target size.
  • Staying accurate about the page: a model can describe an element that is not there.
  • Telling a real defect from a loading state or a limit of the test environment.
  • Proving an outcome: a confirmation screen does not show that an order was saved.

Turn an AI report into a confirmed bug

  1. Split the claim. “The Pay button is invisible because of a CSS bug” is an observation plus a guess. Keep “the Pay button was not visible at 390 px on /checkout” and leave the cause until someone checks it.
  2. Name the expected result. Write what should be true, for example “the Pay button is visible and enabled”.
  3. Reproduce it in a fresh browser. Use the same URL, screen size, starting state and steps.
  4. Measure what can be measured. Contrast, labels and target size have rules. Check the rendered page against the rule instead of judging the screenshot.
  5. Record a decision. Confirmed, refuted with the measurement, or still needs review. Keep the evidence either way.

The AI testing false positives worksheet has a copyable record with these fields.

How Sybqa decides which AI flags count

In a Sybqa run with AI review, vision models look at the captured screens and flag possible problems. Flags about measurable things, such as contrast, labels and target size, are checked against a measurement of the rendered page with no model involved, then re-measured in a fresh browser replay; only those can be marked failed. A second AI model scores the other flags, and a low score lists a flag as Dropped. The rest are replayed in a fresh browser session when time allows and compared by a verifier model, and they stay Needs review whatever that comparison says.

What happens to an AI flag in a Sybqa reportScroll sideways for more columns.
What the browser checks foundStatus in the reportDecides the verdict?
A measurement confirms the flag and the fresh replay shows it againFailed, labelled confirmed by browser measurementYes
The flag cannot be measured, or the replay showed it only some of the time or not at allNeeds review, listed under Not a pass yetNo. It stays open for a person to judge
A measurement contradicts the flag, or a second review step rates it unlikelyDroppedNo. It is listed so you can see what was checked

Free runs have no AI review, so their reports have no AI flags and no Dropped list. Their results come from browser checks alone: journey steps and assertions, layout and overflow, accessibility rules, and console and network errors.

What AI exploration does not tell you

For a step-by-step workflow built on this idea, read the vibe testing guide. To turn a flow you have confirmed into a repeatable assertion, read AI end-to-end testing.

Common questions

Is AI exploratory testing the same as vibe testing?

They overlap. Both use natural-language goals and AI in a browser. The vibe testing guide uses the term for a whole workflow: a bounded goal, an expected result, browser evidence and a retest. Exploration is the looking-around part of it.

Why does Sybqa drop some AI findings?

Because a browser measurement contradicted them, or a second review step rated them unlikely. They are listed under Dropped so you can see what was checked, and they never count as failures.

Does the free plan include AI exploration?

No. Free runs use browser checks without AI review. AI review, including AI bots, is part of the paid plan; see pricing.

Sources

Subscribe to new posts (Atom feed)

Try Sybqa on your site

Explore bugs, UI consistency and user journeys from a reviewed plan. Inspect browser evidence and visual review leads in the report. The free plan gives 10 runs a month without AI review after you sign up. See pricing.

Add your site link here and it will be waiting in the console after you create an account. Then describe one journey to check and review the test plan before any run starts.

Use a site you own or may test. Nothing runs until you review the plan.