Quality Assurance Automation Engineer

Title: Sr./Lead Generative & Agentic AI Tester (Airline/Travel Domain exp. is a must)

Location : - Abu Dhabi (Onsite)

Exp.: 8-9 + Yrs.

Note: Looking for immediate joiners or candidates who are currently serving their notice period. It is acceptable to consider candidates from outside of UAE (like India), provided they have a genuine interest in the opportunity and are willing to relocate.

Key Skills: Python | Generative AI | LLMs | AI Testing | LLM Evaluation | AI Agent Testing | Prompt Testing | Hallucination Testing | AI Agents | Agentic AI | Airline Domain | QA Automation | Responsible AI.

Role Overview

We are seeking an experienced AI Engineer with strong expertise in Generative AI, LLMs, AI Agents, and AI Testing . The primary focus of this role is to ensure the quality, reliability, accuracy, security, and performance of AI-powered solutions used within the airline industry. The ideal candidate will have hands-on experience developing AI applications and establishing robust testing frameworks for LLMs, AI agents, and conversational AI systems.

Key Responsibilities

• Develop and support AI applications using Python, Generative AI, and Large Language Models (LLMs) .
• Design, execute, and automate comprehensive AI test strategies across the AI application lifecycle.
• Validate AI models, LLMs, AI agents, and agentic workflows for accuracy, consistency, reliability, and safety.
• Perform functional, integration, regression, performance, security, and user acceptance testing for AI solutions.
• Conduct LLM evaluation, prompt testing, hallucination testing, bias and fairness assessments, and guardrail validation .
• Develop automated frameworks for testing AI agents, chatbots, copilots, and autonomous workflows.
• Define and track AI quality metrics, benchmarks, and acceptance criteria.
• Identify, analyze, and resolve defects related to AI outputs, workflows, and model behavior.
• Collaborate with business stakeholders and development teams to ensure production-ready AI solutions for airline operations.

Required Skills

• Strong experience in Python .
• Hands-on experience with LLMs, Generative AI, RAG architectures, Vector Databases, and AI Models .
• Experience with AI frameworks such as LangChain, LangGraph, CrewAI, AutoGen, LlamaIndex, or Semantic Kernel .
• Experience developing and testing AI Agents, Agentic AI solutions, Chatbots, and Conversational AI platforms .
• Strong expertise in AI Testing , including:
• LLM Evaluation & Validation
• AI Agent Testing
• Prompt Testing & Optimization
• Hallucination Testing
• Bias & Fairness Testing
• Functional Testing
• Integration Testing
• Regression Testing
• Performance & Load Testing
• Security & Responsible AI Testing
• Test Automation for AI Applications
• Experience creating AI quality frameworks, test cases, evaluation datasets, and benchmarking methodologies.
• Familiarity with Azure OpenAI, AWS AI, Google AI services, or equivalent platforms.

Domain Experience

• Experience in the Airline, Aviation, Travel, or Transportation domain is required.
• Knowledge of airline operations, reservations, customer service, crew management, airport operations, or passenger experience solutions is an advantage.

Preferred Qualifications

• Bachelor's or Master's degree in Computer Science, Artificial Intelligence, Data Science, Engineering, or related field.
• Certifications in AI, Machine Learning, Software Testing, or Cloud Technologies are preferred.