Bayt.com

Submitting more applications increases your chances of landing a job.

Here’s how busy the average job seeker was last month:

Opportunities viewed

Applications submitted

Keep exploring and applying to maximize your chances!

Looking for employers with a proven track record of hiring women?

Click here to explore opportunities now!
We Value Your Feedback

You are invited to participate in a survey designed to help researchers understand how best to match workers to the types of jobs they are searching for

Would You Be Likely to Participate?

If selected, we will contact you via email with further instructions and details about your participation.

You will receive a $7 payout for answering the survey.


User unblocked successfully
https://bayt.page.link/afeXLWBEeAXxNZDN9
Back to the job results

Senior AI QA Automation Engineer

30+ days ago 2026/08/16
Other Business Support Services
Create a job alert for similar positions
Job alert turned off. You won’t receive updates for this search anymore.
Undo

Job description

About the Company
CodeNinja is a global technology and AI firm with 10+ years in the industry, helping enterprises modernize systems and embed AI into their workflows. We work across enterprise software, financial services, defense, health, logistics, and more, with 240+ clients across 15 countries. We partner with organizations to deliver AI infrastructure, dedicated engineering teams, and forward-deployed engineers who work directly inside client environments to own outcomes.


About the role


We are hiring an AI Software Development Engineer in Test II (AI SDET II) — an experienced quality engineer who owns feature-level test strategy and automation for AI-enabled products. In this role, you will design and maintain automated tests and AI evaluation suites, integrate quality signals into CI/CD pipelines, and partner with Scrum teams to prevent defects and deliver reliable AI capabilities.


Key Responsibilities


  • Own test strategy and execution for assigned AI features across unit, API, UI, and end-to-end layers, aligned to Definition of Done and release criteria.
  • Create and maintain automated AI evaluation suites (including golden datasets, scoring, and regression baselines) covering key user journeys and failure modes.
  • Validate prompt, retrieval, and orchestration changes using repeatable test harnesses and defined acceptance thresholds.
  • Build and extend automation frameworks, reusable utilities, and improve test stability, speed, and signal-to-noise.
  • Integrate automated tests and AI evaluations into CI/CD, triage pipeline failures, and reduce flaky tests.
  • Partner with engineers to improve testability (contracts, logging, observability) and drive shift-left quality practices.
  • Analyze defects and AI output issues, perform root cause analysis, and implement regression coverage to prevent recurrence.
  • Provide guidance to junior SDETs on automation, test design, and AI validation patterns; communicate quality risks to stakeholders.
  • Support production readiness by validating guardrails, monitoring signals, and performance requirements for AI features.
This job post has been translated by AI and may contain minor differences or errors.
Apply on company site Email to Friend Complete Questionnaire
You’ve reached the maximum limit of 15 job alerts. To create a new alert, please delete an existing one first.
MANAGE
Job alert created for this search. You’ll receive updates when new jobs match.
Manage alerts
Are you sure you want to unapply?

You'll no longer be considered for this role and your application will be removed from the employer's inbox.