That patience with unreliable systems is about to become very useful. AI systems are, in a sense, permanently flaky: ask the same question twice, get two different answers. Teams building with them need people who already know how to make sense of results that aren't perfectly repeatable. That's you.
Search interest is steady: "qa engineer jobs" about 880 a month in India and 1,600 in the US; "qa automation engineer jobs" 260 / 590; "qa jobs remote" 1,000 / 5,400.
Flaky Test Fortune Teller
A crystal ball floats on the page. Type the name of your most annoying flaky test (any name; nothing is stored). The ball swirls and "predicts" its cause from a list of real, common culprits: timing waits, shared test data, order dependence, network calls, animations. Then it reveals the twist: an LLM (large language model) answer behaves like a flaky test on purpose, and shows the three techniques that handle both — retries with limits, tolerance thresholds, and scoring instead of exact matching. A small joke, a real lesson.
The flaky-test crystal ball
Four routes, side by side
| AI test automation | AI evaluator | LLM eval engineer | AI red-teamer | |
|---|---|---|---|---|
| What you do | AI-assisted suites, self-healing tests, testing AI features in apps | Rate and compare AI answers | Build eval datasets, scoring and CI checks for AI features | Attack AI systems safely and report findings |
| Code level | Medium | Low | Medium–high | Low–medium |
| Closest to your current job | Very close | Further | Close | Medium |
| Free tools to start | Playwright, Selenium, Katalon Free Studio | A rubric and a spreadsheet | Python, open-source eval frameworks | garak, PyRIT, OWASP LLM Top 10 |
| Bridge page | Open | Open | Open | Open |
Slide to your code comfort
The slider only suggests. Many people combine two.
What your suite taught you
| Habit | Where it goes |
|---|---|
| Page objects and clean frameworks | Reusable eval harnesses |
| CI pipelines | Running AI regression checks on every change |
| Test data management | Golden datasets |
| Flaky-test triage | Variance in model outputs |
| Reporting dashboards | Eval reports leaders can read |
What's new in test tooling (checked 23 Oct 2026)
- Playwright documents test agents — planner, generator and healer — that work with an AI tool to draft and repair tests.
- Katalon offers a Free Studio; Professional is a paid per-seat plan (checked 25 Oct 2026).
- mabl has a 14-day free trial.
- garak (NVIDIA) and PyRIT (Microsoft, open source) scan LLM systems for weaknesses.
- OWASP Top 10 for LLM Applications (2025) is the common risk vocabulary.
Short-term income platforms
From our platform checks (21 Oct 2026): uTest and Testlio are listed. Turing is listed with caution (3+ years' experience, contract, long vetting, payment delays reported) — your profile may fit. Outlier, Alignerr and Test IO are also listed with caution: irregular work, confirm pay model first, Test IO describes its payouts as small and irregular. Not for India: DataAnnotation accepts only US, Canada, UK, Ireland, New Zealand and Australia applicants, so we don't recommend it here. Free to join; no VPN. See AI training jobs.
Where your years of automation point
2–5 years: AI test automation. 5–10 years: LLM evaluation engineering or leading AI QA. 10+ years: QA strategy for AI products — deciding what "tested enough" means for systems that vary.
Automation testers in India and abroad
India: services firms are adding GenAI testing to delivery; product companies want QA engineers who can test AI features. Global: remote QA roles stay competitive; AI-specific testing skills help you stand out. Visa and tax questions are outside our scope.
Your realistic 90 days
You maintained Selenium or Playwright suites, fought flaky tests, and kept CI green. Teams now ship AI features that your assertions can't check, and AI tools are reshaping test automation itself. This plan tests four routes against the framework you already know.
Assumes 2–3 hours a day alongside job applications, a laptop and a phone, and free plans only. Every step has a smaller version and a way to pause. Use them. It runs in three blocks of 30 days, each ending with a signal that you're on track. A signal is not a promise of a job or income.
What AI changed in your old tasks
| You used to… | Now you… |
|---|---|
| Asserted exact text on a page | Score AI outputs with rubrics and evals |
| Stabilised flaky tests | Measure and report the variance of AI behaviour |
| Wrote page objects by hand | Use AI assistants to draft tests, then review them like code |
Your first 7 days
| Day | Task | Done when |
|---|---|---|
| 1 | List your framework, languages, CI tool and the flakiest test you fixed | Notes |
| 2 | AI evaluator try-out: score 15 chatbot answers with a rubric | 15 rows |
| 3 | Red-team try-out: 10 adversarial prompts against a public bot, logged | 10 rows |
| 4 | AI test automation try-out: have an AI assistant write a Playwright test; review and fix it | Fixed test passes |
| 5 | LLM eval try-out: run Ragas on 10 RAG answers | Scores |
| 6 | Pick the route that felt most like 'your' kind of problem | Choice written |
| 7 | Save 10 postings | 10 links |
Days 1–30: Proof built on your framework
| Step | Time | Done when | Smaller version | If you need to pause |
|---|---|---|---|---|
| Build the chosen route's proof inside a repo structured like your old framework | about 14 hours over 3 weeks | repo with CI | Without CI. | Save or commit what works now and stop adding. A finished small version beats an unfinished big one. |
| Share with 5 SDET or QA-lead ex-colleagues | about 2 hours | 5 messages | Share with 2 SDET or QA-lead ex-colleagues. | Optional. If reaching out feels like too much this week, skip it and put the time into your proof. |
Signal by day 30: A QA lead asks about your approach.
Days 31–60: Depth
| Step | Time | Done when | Smaller version | If you need to pause |
|---|---|---|---|---|
| Add failure analysis: what broke, why, how you'd prevent it | about 8 hours | analysis in README | Half page. | Stop at the end of the step you're on and write two lines about where you stopped. Pick it up from those notes later. |
| Apply to 10 roles where your automation tester years count | about 5 hours | 10 logged | Apply to 5 roles. | Stop sending for now. Keep a list of who you've contacted so you can follow up later without doubling up. |
Signal by day 60: A screening call.
Days 61–90: Interviews
| Step | Time | Done when | Smaller version | If you need to pause |
|---|---|---|---|---|
| Prepare a 10-minute walkthrough of your framework design | about 2 hours | recorded once and watched back | Bullet notes for the design choices. | Optional. Skip it if the week is full; nothing later depends on it. |
| Ask 2 former QA leads to refer you | about 1 hour | 2 asks sent with your repo link | One message to your closest ex-lead. | Optional. If reaching out feels like too much this week, skip it and put the time into your proof. |
Signal by day 90: A technical round.
Job-hunt moves that matter for this route
- Search 'SDET + AI', 'AI QA engineer', 'LLM evaluation', 'Playwright' on LinkedIn and Naukri.
- Your CI and flakiness experience is rare among AI-eval newcomers. Lead with it.
- Check platform eligibility for India before any contract signup.
Where the openings are, and what to type
Also worth a weekly look: Instahyre, Cutshort, Wellfound, and iimjobs or hirist for senior roles. Upwork and Contra if you want freelance work.
Services firms, product companies and testing vendors. Naukri and LinkedIn carry most automation roles; AI QA and eval posts appear mainly on LinkedIn.
- LinkedIn search string:
("automation tester" OR SDET) AND (AI OR LLM OR evaluation OR "red team") - Naukri / foundit keywords: automation tester, SDET, Playwright, Selenium
Worries people bring to this route
- Over 40 in automation? Framework judgement is senior work.
- A gap. A dated public test repo answers it.
- Degree. Skills tests decide most interviews.
- English. Repos and reports speak.
- Family. SDET is an engineering title.
- Runway. Standard automation roles still hire.
Your 30/60/90 quests
XP: learning 20 · proof 60 · outreach 40. Ranks: Wayfinder → Builder → Pathmaker. Progress stays in your browser only.
On automation pay
We haven't verified pay data for these routes.
Offers that are really scams
Fake QA "projects" that require a deposit, or "recruiters" on Telegram. Never share OTPs or UPI PINs. Official domains only. Report: cybercrime.gov.in / 1930 (India), ReportFraud.ftc.gov (US).
Will AI automate the automation tester?
AI writes test drafts faster. Someone still decides what matters, reviews the assertions and owns quality. That's a QA job, reshaped. The testers most at risk are the ones who only maintain scripts; the ones who shape test strategy, read results critically and now test AI features too are in a stronger position. Nobody can promise outcomes, but that direction is within your control.
Take the quiz · Related: Manual tester · Productised QA
Newer step-by-step plan: Automation Tester → Productised QA, each with one proof project and a 90-day schedule.
Quick quiz: does this path fit?
Answer each, then press Check. Every answer explains itself.
Make your Path Card
A shareable reminder of where you're heading. The day count starts the first time you open this page; quest XP comes from your log above.
Scam check before you apply anywhere
- Real job and gig platforms are free to join. Never pay a "joining" or "training" fee.
- Never share an OTP or UPI PIN — no genuine employer needs them.
- Use official domains only; type the address yourself.
- India: report at cybercrime.gov.in or call 1930.
- US: report at ReportFraud.ftc.gov.
Small steps, every day.
Each quest has a smaller version for hard weeks, and a pause button. Nothing is lost.
Keep exploring
Sources
Checked between 21 Oct 2026 and 23 Oct 2026. We opened every link below ourselves. Search volumes on this site come from OpenSEO keyword research (Oct 2026).
| What we checked | Where |
|---|---|
| Platform verdicts | AINewPath platform checks (Oct 2026) |
| Playwright test agents | playwright.dev |
| garak | github.com |
| PyRIT | github.com |