Senior AI QA Engineer

Job Description:

We are seeking a detail-oriented QA Engineer to test and assure the quality, performance, reliability, and compliance of solutions built using Microsoft Power Platform and Copilot Studio, including data integrations and AI/ML-enabled features. This role will lead test planning and execution across low-code applications, automated workflows, APIs/custom connectors, and conversational AI experiences, partnering closely with developers, data engineers/scientists, and business stakeholders.

Key Responsibilities:

• Define and maintain QA strategy, test approach, and coverage for Power Platform and Copilot Studio solutions.
• Create, review, and execute test plans, test cases, and test scripts across SDLC phases.
• Perform functional, smoke, regression, integration, end-to-end testing.
• Ensure traceability between requirements, test cases, defects, and releases.
• Validate Dataverse/Databricks tables/entities, Power Automate flow, relationships, security roles, and data integrity rules.
• Design and execute conversation test scenarios for multi-turn context, ambiguity handling, and error recovery.
• Perform regression testing for AI  bot behaviour changes due to configuration updates, connector changes, or model/version updates.
• Perform API testing for REST services, Azure ML endpoints, and custom integrations using tools such as Postman/JMeter.
• Test data integrations and pipelines involving Azure Databricks, Azure Storage and Dataverse.
• Plan and execute performance and load tests for workflows and APIs
• Use JMeter to simulate real-world traffic patterns and produce performance reports
• Identify, document, and track defects using Azure DevOps, Jira, or equivalent tools.
• Communicate test status, risks, and release readiness to stakeholders.
• Collaborate with developers, data scientists, and product owners to clarify acceptance criteria and non-functional requirements.
• Contribute to QA best practices, reusable test assets, and automation frameworks.

Required Qualifications:

• 3+ years of QA/testing experience, including 1–2 years testing Microsoft Power Platform solutions.
• Hands-on experience testing Power Automate workflows and data-centric applications.
• Working knowledge of Dataverse, connectors/custom connectors, Azure Databricks, and Azure Storage.
• Familiarity with Copilot Studio or chatbot testing methodologies (NLU, multi-turn flows, fallback handling).
• Strong understanding of software testing principles (test design, execution, defect lifecycle, reporting).
• Experience with API testing (REST) using Postman and/or scripted validation (Python preferred).
• Hands-on performance testing experience using JMeter or LoadRunner, including analysis and reporting.
• Ability to validate AI/ML outputs and collaborate with data teams on statistical checks and model behaviour.
• Basic understanding of designing prompts and edge cases to evaluate AI agent behaviour.
• Develop and maintain automation scripts for API testing, Regression test suites, Chat flow validation
• Build automated validation for chatbot responses using Pattern matching, Similarity scoring, Semantic validation (where applicable)

Preferred Qualifications:

• Microsoft Certifications in Power Platform or Data Analytics/AI.
• Experience in cloud-based testing environments (Azure preferred).
• Knowledge of accessibility, usability, and security testing.
• Exposure to CI/CD pipelines and integration of performance testing.
• Agile/Scrum project experience with Azure DevOps pipelines for Power Platform or Copilot deployments.
• Understanding of RAI principles
• Understanding of LLM-based systems, Prompt engineering basics, RAG architecture, Multi-agent workflows

Tools & Technologies:
• Azure DevOps
• Postman
• Selenium (Python automation)

Our Commitment to a Culture of Inclusion & Belonging
Ecolab is committed to fair and equal treatment of associates and applicants and furthering the principles of Equal Opportunity to Employment. We will recruit, hire, promote, transfer and provide opportunities for advancement based on individual qualifications and job performance in all matters affecting employment, compensation, benefits, working conditions, and opportunities for advancement. Ecolab will not discriminate against any associate or applicant for employment because of race, religion, color, creed, national origin,citizenship status, sex, sexual orientation, gender identity and expressions, genetic information, marital status, age, or disability.