To hire an AI QA agent, treat it as a hire, not a tool purchase. Write a one-page job description (what it tests, what it must not touch, how it reports), then ask every vendor specific questions: does it test in a real browser, do I approve the plan before it starts, can I edit the test cases, does it re-test after a fix? Then trial it for five days on a staging copy of your own site, and score it on the real bugs it found and how clear the evidence was, not on the number of tests.
- how to hire an AI QA agent
- how to hire a QA tester
- questions to ask a QA testing vendor
- how to evaluate AI testing tools
- QA tester job description for a small business
- AI QA agent free trial what to test
What job are you actually hiring for?
Most people looking for a 'testing tool' start in the wrong place: they open vendor sites and compare features. Start from the job. For a human tester you would write an ad, interview candidates, then put them on probation. Hiring an AI QA agent follows the same order; the probation just takes days instead of months.
IllustrationThe job in one line: someone who checks, before your customers do, that the site works, tells you clearly what broke, and follows each bug until the fix is proven. New to the idea? Read what is an AI QA agent first.
- Title
- QA engineer for a website or web app
- Reports to
- The product owner, copy to the developer
- Core duty
- Test customer journeys before each release
- Success measure
- Real bugs caught before a customer
- Not in the role
- Load, security, design taste
What should the job description say?
One page is enough, and it doubles as the brief you later hand the agent. Write it in business language:
| Item | What to write | Example |
|---|---|---|
| What is tested | The site address, and its staging copy if any | The perfume store, staging copy |
| Key journeys | Three to five journeys the business earns from | Find a product, buy, track the order |
| What it must not touch | Clear limits | No real payment, no deletion, no messages to customers |
| Screens | Phone, desktop or both | Phone first; most visitors use one |
| Language | Interface languages to check | Arabic and English |
| Rhythm | When it works | Before every release, quick check daily |
| Delivery | Report format and who gets it | Report with screenshots, to owner and developer |
Which questions should you ask any vendor?
This is the interview. The questions fit any tool on the market; beside each is the reassuring answer and the red flag:
| Question | Reassuring answer | Red flag |
|---|---|---|
| Does it test in a real browser? | Yes, and I can watch it work | It 'analyzes the code' or 'reviews the page' without running anything |
| Do I review the plan before it runs? | It never starts before I approve | It starts clicking around my site as soon as I enter the address |
| Can I read and edit the test cases? | Readable steps I can edit and save | A black box |
| How does it prove a bug? | Screenshot, steps, expected versus actual | A general description, no evidence |
| How does it tell a site bug from a test problem? | It classifies the cause explicitly | Every failure counts as a bug |
| What happens after a fix? | It re-tests; the bug closes only on a pass | I close it by hand |
| Will it change my data? | Only with permission, within limits I write | There is no setting for that |
| How is the test account password stored? | Encrypted, never shown in reports | Visible in screenshots or logs |
| What does it not do? | A clear list of limits | 'It tests everything' |
How do you run a fair trial before committing?
A demo shows you the vendor's site, not yours. A real trial runs on your own site with a task from your job description. Five working days are enough:
- Day 1Setup
Add the staging address and prove ownership, and create a dedicated test account. Write the 'do not touch' list.
- Day 2First plan
Ask for a full review of your key journey. Read the proposed plan: did it understand the business? Did it miss something obvious?
- Day 3Run and report
Start the run and watch part of it. Then read the report with whoever will fix the bugs, not alone.
- Day 4The planted bug
Ask your developer to break something on purpose on staging, then rerun. Was it found?
- Day 5Re-test and decide
Fix one bug and request a re-test. Then fill in the scorecard and decide.
Why staging? Because the trial creates fake bookings and orders you do not want among customer data. The difference is explained in staging versus production.
How do you score the trial?
Do not be impressed by big numbers. Fifty passing cases on pages that do not matter are worth less than one real bug on the payment page. Use this card:
Seven ticks out of nine or more: rely on it. Fewer than five: either the tool does not fit your site, or your brief was vague. Rewrite the description and try once more before you judge.
What does an AI QA agent cost, and what drives the cost?
We give no figures, because pricing differs between vendors and changes. The cost drivers in this category are similar, though:
- Runs per month: a daily check uses more than testing only before a release.
- Browser minutes: a run on phone and desktop together uses more than one screen.
- Saved test cases: the bigger the system, the bigger the library.
- Number of sites: an agency testing several client sites needs a wider plan than one store owner.
- Extras: automatic schedules and report export or sharing usually sit on higher plans.
Compare that with the alternatives: your own time on manual checks, a freelance tester by the hour, or a full-time employee. The full reasoning is in what bugs cost and what QA costs.
An AI QA agent, a freelance tester, or both?
Pick the one that sounds like you to see the advice.
Start with the agent. It covers your core journeys this week and gives you a safety net to rerun after every change. Add a person later if you need one.
That is a problem of taste and judgement. Bring in a human tester, or five real customers trying it in front of you. An agent will not tell you a page is confusing.
Both: the agent reruns the full check after every change, and a person explores the new feature freely once.
Printers, barcode scanners, payment terminals? You need a person on site. The agent covers only the part that runs in the browser.
The detailed comparison, task by task, is in AI QA agent or a human tester.
How do you hire the QA Engineer in Fanann?
Put the interview questions above to Fanann's QA Engineer and these are its answers: it tests in a real browser you can watch live; it never starts before you approve the plan; it writes cases you can edit; it attaches screenshots, steps to reproduce and a severity; it says whether a failure comes from the app, the test or the setup; and it treats a bug as fixed only after the re-test passes. It tests any site you own, even one built outside Fanann, and is part of the paid plans. It does not test mobile apps installed from the stores.
You are the QA engineer for the clinic booking platform. Test the full patient journey at phone size: choosing a doctor, choosing a slot, entering a name and mobile number, then the confirmation message. Make sure the same slot cannot be booked twice and that a patient cannot see other patients' appointments. Try an incomplete mobile number and a date in the past. Use data that is obviously test data, do not cancel any existing booking and do not message real patients. Order the findings by severity and tell me whether the platform is ready.
Try it in FanannAfter the decision comes an organized rollout: a 30-day plan for putting an AI QA agent to work.
Key takeaways
- Start with a written job description; without it you judge tools by the demo, not by your need.
- The four key questions: real browser? plan approval before running? editable cases? re-test after the fix?
- Trial it on your own site, on a staging copy, and plant a bug you know about to see whether it finds it.
- Score real bugs, clear evidence and few false alarms, not the count of test cases.
- An agent does not remove the need for a person to judge usability; many teams combine both.
Frequently asked questions
What is the most important question before choosing an AI QA agent?
How long should an AI QA agent trial take?
Do I need technical skills to hire an AI QA agent?
Should I trial the agent on my live site?
Should I hire a human QA tester or an AI QA agent?
Can an agency use it to test client sites?
Ask your AI assistant about this
Copy this prompt into the AI assistant you use, then add the details of your own case.
I am reading this guide: “How to hire an AI QA agent: what to ask and what to expect” https://fanann.com/en/blog/how-to-hire-an-ai-qa-agent/ My question: how to hire an AI QA agent Use that page as a source and explain the answer step by step for my situation, tell me what to check before I start, and cite the page link.