Hoppa till huvudinnehållet
WeQuel AB logo

AI Evaluation & Quality Engineer

WeQuel AB
  • OrtStockholm
  • OmfattningHeltid
  • Sista ansökan30 dagar kvar
  • Publicerad20 nov.

Om tjänsten

WeQuel is looking for an AI Evaluation & Quality Engineer for an assignment with one of our clients. You will make sure agentic

workflows are measured honestly before and after they reach production – so the organisation knows what works, what does not, and

why.

About the assignment

Workload: 100%
Location: Hybrid
Language: English

Your responsibilities

Design evaluation strategies and quality metrics for LLM agents (accuracy, task success, safety, cost, latency)
Build and maintain evaluation datasets, test cases and automated regression tests
Set up LLM-as-judge and human review processes where needed
Run red-teaming and robustness tests, including prompt injection scenarios
Monitor agent quality in production and feed findings back to development teams
Report results clearly to product owners and stakeholders

Requirements

At least 3 years of experience in test automation, QA engineering or data science
Hands-on experience evaluating LLM or ML solutions
Strong Python skills
Understanding of statistics and experiment design
Fluent English, written and spoken

Meritorious

Experience with evaluation and observability tools (e.g. Langfuse, LangSmith, Ragas, DeepEval, promptfoo)
Experience with CI/CD pipelines for ML or LLM applications
Knowledge of AI safety and the EU AI Act
Background in industrial or manufacturing environments
Swedish

Who you are

You are analytical, sceptical in a constructive way and thorough. You enjoy finding where things break, explain results in plain

language, and help teams make better decisions based on evidence.

Applications are reviewed continuously – apply today!

Inte redo att söka direkt? Spara annonsen så skickar vi en påminnelse 3 dagar före sista ansökningsdag.

Synka i iPhone-appen →

Restid & pendling

Krav- & nyckelordscheck

Efterfrågade kompetenser – klicka på det du kan:

Anpassa ett gratis CV direkt mot annonsens kravprofil.Matcha & bygg CV för tjänsten

Representerar du WeQuel AB? Lyft fram annonsen i Stockholm och dölj konkurrerande jobb.