Frontend Automation QA Engineer

Description Position at LeoVegas Spain ABOUT THE ROLE: At the core of LeoVegas Group is Team Leo. Our culture is our foundation and is what enables us to innovate, build, and lead as we trailblaze our way through the igaming industry. We’re a team of over 2000 innovators, initiators, and groundbreakers working in a fast-paced and agile environment across 19 offices worldwide. Are you an experienced Automation QA Engineer looking to make an impact in a fast-paced, dynamic environment? Join our Sportsbook Domain and become part of the Sports Betting team at LeoVegas Group, where you’ll help shape and enhance our sports product capabilities. You will take end-to-end ownership of your domain quality, working hands-on with automation and end-to-end validation. Your work will ensure a secure, seamless, and enjoyable experience for customers across all our brands. In this role, you will contribute directly to strengthening our sportsbook platform by designing effective tests and defining clear testing scopes that align with our business needs. Working in an Agile environment, you will collaborate with highly skilled frontend and backend developers, along with dedicated Quality Assurance Engineers, to deliver high-quality solutions that support a reliable and engaging customer experience. YOU WILL BE RESPONSIBLE FOR: Drives high-impact outcomes by leading complex, ambiguous projects and resolving critical issues that directly influence business quality and performance Shapes testing strategy and architecture by leading test design for complex problems, ensuring reliability, observability and alignment with business priorities Communicates risks, insights and technical considerations clearly to both technical and non technical stakeholders, contributing meaningfully to roadmaps and planning Uplifts QA practices across squads by promoting best practices, anticipating issues through strong processes and enabling others to improve testing for performance, scale and security OUR SUCCESSFUL CANDIDATE WILL HAVE THE FOLLOWING: ESSENTIAL SKILLS Experience with frontend testing including UI automation using Typescript (Playwright) Typescript/Javascript programming experience Experience with visual testing Experience with backend testing Proficiency with test management tools such as TestRail Understanding of modern software development practices in Agile/Scrum self-organizing teams and DevOps environments NICE TO HAVES ISTQB certification Familiarity with microservice architectures Experience with accessibility testing Practical experience improving CI/CD pipelines Experience with Grafana, Datadog or any other monitoring tool Interest/experience in Gambling domain WHO WE ARE At the core of LeoVegas Group is Team Leo. Our culture is our foundation and is what enables us to innovate, build, and lead as we trailblaze our way through the igaming industry. We’re a team of over 2000 innovators, initiators, and groundbreakers working in a fast-paced and agile environment across 21 offices worldwide. BENEFITS Hybrid work policy (2 days a week of working from home) 4 weeks of Workation (T&C apply) 300 EUR wellness contribution annually Cobee - benefits app with flexible compensation and discounts Health insurance (we use Alan) Life insurance Employee Assistance Program (free emotional, legal, and financial support) Short Fridays - we work until 16:00 Complimentary snacks and drinks in our offices, Monday breakfast and Wednesday fika (Swedish break for coffee and something sweet) Team and office social events throughout the year JOIN US! In our pride, we empower our teammates to find their roar and run with their wildest ideas. We don’t wait for things to happen; we pounce and make it happen! Would you be a good fit for the Leo Pride - give us a roar! **As our company working language is English, we’d like to see your CV in English, please**

Job Position : Senior AI Quality Assurance (QA) Engineer Location : Bengaluru Experience : 7+ Years  Required Skills: We are looking for an experienced Senior AI Quality Assurance (QA) Engineer to ensure the quality, reliability, and production readiness of enterprise-grade AI applications. This role extends beyond traditional software testing and focuses on the evaluation and quality engineering of LLM applications, RAG systems, and AI agents. You will own automated testing, AI evaluations, benchmark creation, observability, prompt regression testing, and end-to-end validation of AI workflows. Working closely with AI Engineers, Product Managers, and Platform teams, you will establish measurable quality standards and ensure every release meets enterprise-grade expectations for accuracy, reliability, performance, and scalability. Key Responsibilities: AI Evaluation & Benchmarking Design benchmark (golden) datasets and build automated evaluation pipelines for LLM applications. Define quality gates and continuously evaluate prompts, models, retrieval pipelines, and agent behaviour using metrics such as hallucination rate, tool selection accuracy, execution accuracy, precision, recall, latency, and cost. RAG & Agent Quality Validation Validate Retrieval-Augmented Generation (RAG) pipelines and AI agent workflows by testing retrieval quality, context relevance, tool invocation, reasoning flow, memory, and end-to-end task completion. Design evaluation scenarios covering ambiguous queries, multi-turn conversations, retrieval failures, and edge cases. Python Automation & API Testing Develop and maintain scalable automation frameworks using Python and Pytest for unit testing, integration testing, API testing, regression testing, and end-to-end validation. Build reusable test utilities and integrate automated quality checks into CI/CD pipelines. Frontend Automation Develop automated UI test suites using Playwright to validate AI-powered user journeys, conversational interfaces, workflow execution, and end-to-end application behaviour across releases. Observability & Root Cause Analysis Use OpenTelemetry, tracing platforms, and AI observability tools to analyse execution traces, latency, model responses, API calls, and workflow behaviour. Perform root cause analysis to identify regressions, hallucinations, bottlenecks, and production issues. Performance & Enterprise Readiness Validate AI application performance by monitoring latency, throughput, reliability, and scalability. Ensure production readiness through regression testing, API validation, workflow testing, and enterprise quality standards. Required Skills: 7+ years of experience in Software QA, Test Automation, or AI Quality Engineering. Strong Python programming skills. Hands-on experience with Pytest for: Unit testing Integration testing API testing Regression testing Experience testing REST APIs and backend services. Experience evaluating LLM-powered applications. Hands-on experience evaluating Retrieval-Augmented Generation (RAG) systems. Understanding of AI agent evaluation methodologies. Experience measuring AI quality using metrics such as: Hallucination Rate Tool Selection Accuracy Execution Accuracy Precision / Recall Latency (P50/P95/P99) Token Usage Experience with Playwright or similar frontend automation frameworks. Experience with OpenTele metry, tracing, or observability platforms. Strong debugging and root cause analysis skills. Experience integrating automated tests into CI/CD pipelines. Nice to Have Experience with Lang Smith, Lang fuse, MLflow, Arize Phoenix, or similar AI observability platforms. Experience evaluating multi-agent systems and orchestration frameworks such as LangGraph, CrewAI, Google ADK or AutoGen. Experience with vector databases such as Pinecone, Milvus, Weaviate, pgvector, or Vertex AI Vector Search. Exposure to OpenAI, Anthropic, Gemini, or Azure OpenAI. Experience with performance and load testing tools. Prior experience in Banking, Financial Services, or Insurance (BFSI).