AI Software Development Engineer in Test (AI-SDET)

This position is posted by Jobgether on behalf of a partner company. We are currently looking for an AI Software Development Engineer in Test (AI-SDET) in India.

This role sits at the intersection of software quality engineering and modern AI systems, focusing on how testing evolves in an AI-native world. You will help redefine QA practices by introducing AI-powered validation frameworks, automation strategies, and intelligent testing tools across engineering teams. The position involves deep work on LLM evaluation, RAG systems, and autonomous agent workflows, ensuring reliability, safety, and performance of next-generation AI applications. You will act as both a hands-on engineer and a change agent, enabling QA teams to adopt advanced AI testing methodologies. The environment is highly innovative, fast-evolving, and centered on continuous experimentation and improvement. This is a unique opportunity to shape how quality engineering is performed in AI-driven software ecosystems.

• Drive the adoption of AI-powered testing tools and modern QA practices across engineering teams, acting as a change agent for quality transformation.
• Design and implement evaluation frameworks for LLMs, including hallucination detection, bias assessment, grounding, safety validation, and regression testing.
• Build automated pipelines to validate AI outputs using semantic scoring, embeddings, structured assertions, and benchmarking datasets.
• Test and validate RAG systems, including chunking strategies, retrieval accuracy, embedding quality, and vector database performance.
• Develop and maintain scalable automation frameworks for UI, API, microservices, and event-driven systems using Python and modern testing tools.
• Integrate automated testing and quality gates into CI/CD pipelines across cloud-native environments and distributed architectures.
• Validate AI agent workflows, including tool use, orchestration, memory, and multi-agent systems across leading AI frameworks.
• Contribute to observability, monitoring, and reliability improvements for AI systems deployed at scale.
• Collaborate with engineering and product teams to embed AI quality standards into the SDLC and improve overall software reliability.

Requirements:

• 4–10+ years of software engineering or QA experience, including 2–4+ years focused on AI/ML, LLMs, or AI-based systems.
• Strong programming skills in Python with hands-on experience in test automation and software engineering.
• Deep understanding of QA principles, software testing methodologies, and quality engineering practices.
• Hands-on experience with LLM testing, prompt engineering, RAG systems, and AI evaluation frameworks.
• Experience working with APIs, microservices, and UI automation frameworks such as Playwright, Selenium, or Robot Framework.
• Familiarity with CI/CD pipelines and cloud environments (AWS, Azure, GCP, or OCI).
• Knowledge of Docker, Kubernetes, and distributed system testing.
• Experience working with vector databases and AI frameworks such as LangChain, LangGraph, or similar tools.
• Strong analytical, debugging, and problem-solving skills with attention to system reliability and performance.
• Ability to influence engineering teams and drive adoption of new AI-driven QA practices.
• Preferred experience in AI red teaming, security testing, MLOps, or observability tools for AI systems.
• Bachelor’s or Master’s degree in Computer Science, Engineering, AI/ML, or equivalent practical experience.

Benefits:

• Competitive compensation with performance-based growth opportunities.
• Flexible work arrangements including hybrid, remote, or office-based models depending on role alignment.
• Comprehensive healthcare and wellness support programs.
• Learning and development support, including certifications, training, and AI upskilling resources.
• Exposure to global projects and cutting-edge AI technologies.
• Inclusive and diverse workplace culture with active employee communities.
• Career growth opportunities through mentorship, innovation initiatives, and cross-functional collaboration.
• Paid time off, holidays, and programs supporting work-life balance.