HUMAN DATA QUALITY ENGINEER (FOUNDING TEAM)

Prolific

Prolific isn’t just enabling AI innovation – we’re redefining it. While foundational AI technologies are becoming commoditized,

Prolific’s human data infrastructure provides the high-quality, diverse data required to train the next generation of AI models.

Through our platform, we empower researchers and companies to access a global, ethically curated participant base, ensuring

cutting-edge AI research and training grounded in inclusivity and precision.

The Role

As one of the founding members of Prolific's newly formed AI Data Services team, you'll help build the quality systems behind some

of the world's most advanced AI models. Data quality is a strategic priority for Prolific, so this is a high-visibility role with

direct exposure to senior stakeholders. This isn't a traditional QA role. We are not looking for someone to review data against a

predefined checklist. We are looking for an innovative thinker that can leverage their expertise to define what good means where

no definition exists yet.

Acting as a strategic thought partner, you’ll work at the intersection of human data, machine learning, evaluation across frontier

use-cases that define what high-quality human data looks like for the next generation of advanced AI. This means that much of the

work involves novel problems with no established answer, so you’ll be comfortable working through ambiguity.. .

Your primary focus is working directly with clients and alongside frontier AI labs, translating what their models need into robust

human data and evaluation strategies. Rather than checking quality at the end of a project, you'll engineer quality into every

stage of the lifecycle, from study design and participant strategy through to evaluation, launch readiness and client delivery.You

will also work alongside our product engineering, and supply teams to define and build the quality infrastructure that will enable

us to deliver high quality human data at scale.

Much of the work you'll tackle won't have an existing playbook. You'll help create it.

What You’ll Be Doing

Design the quality frameworks that underpin complex human data programmes, from evaluation rubrics through to launch readiness.
Work with clients as a strategic thought partner, challenging annotation schemas and data requirements when they won't produce

the signal the model needs.

Advise on project design and how the choice of schema can impact data quality.
Build quality upstream across the operational workflow, from recruitment, screening, and training through to writing guidelines

and running calibration sessions.

Build scalable quality systems, measurement frameworks and automated checks using Python and SQL.
Partner with product and engineering to build the quality infrastructure that delivers high-quality human data at scale.
Investigate data quality and integrity issues, identifying root causes and turning insights into scalable improvements.
Architect and build dashboards, monitoring and reporting that provide clear visibility into quality and operational

performance.

Raise the quality capability across the company, upskilling operations and acting as a thought mentor to junior analysts.
Help define how Prolific approaches quality across new AI domains, shaping best practice as the team grows.

What You’ll Bring to the Role

5+ years of experience in building quality, evaluation or annotation systems within AI, machine learning, LLMs or human data

environments.

Strong Python and SQL skills, with a passion for using data to solve complex quality problems.
A solid understanding of machine learning pipelines and how human data impacts model performance.
Strong analytical and statistical thinking, with experience designing scalable quality frameworks.
The confidence and credibility to interact with stakeholders at frontier labs and act as a partner.
The ability to leverage your experience and expertise to influence and guide stakeholders at every level, both client side and

internally. in

The ability to turn your own data analysis and quality methodology into requirements that product and engineering can build

into systems.

The ability to explain yout data analysis and findings clearly to non-technical stakeholders, so they can act on them.
A proactive, builder's mindset -you enjoy creating new systems, navigating ambiguity and improving how things work.

Even Better if you have:

Experience with LLM evaluation, RLHF, AI safety or red teaming.
Experience translating vague model or evaluation goals into clear annotation specifications.
Experience working with human annotation programmes or human data operations.
Familiarity with calibration, inter-rater agreement, drift detection or other evaluation methodologies.

Why Prolific is a great place to work

We've built a unique platform that connects researchers and companies with a global pool of participants, enabling the collection

of high-quality, ethically sourced human behavioral data and feedback. This data is the cornerstone of developing more accurate,

nuanced, and aligned AI systems.

We believe that the next leap in AI capabilities won't come solely from scaling existing models, but from integrating diverse

human perspectives and behaviors into AI development. By providing this crucial human data infrastructure, Prolific is positioning

itself at the forefront of the next wave of AI innovation – one that reflects the breath and the best of humanity.

Working for us will place you at the forefront of AI innovation, providing access to our unique human data platform and

opportunities for groundbreaking research.

Join us to enjoy a competitive salary, benefits, and remote working within our impactful, mission-driven culture. At Prolific, our

compensation packages for eligible roles include base salary, equity, and benefits. Many roles also include the opportunity to

earn a cash variable element, such as a bonus or commission. Each job posting shows a salary range that reflects the minimum and

maximum target for new hires, based on the role’s location as well as your skills, experience, and relevant education or training.

Your recruiter will also be happy to share the specific salary range for your preferred location during the hiring process.

LINKS TO MORE INFORMATION

Website

Youtube

Benefits

External Handbook

Privacy Statement

By submitting your application, you agree that Prolific may collect your personal data for recruiting and global organisation

planning. Prolific's Candidate Privacy Notice explains what personal information Prolific may process, where Prolific may process

your personal information, its purposes for processing your personal information, and the rights you can exercise over Prolific

use of your personal information.

Role

Human Data Quality Engineer (Founding Team)

Tags for this job

Services

Similar Jobs

Founding engineer- Mo team (AI Health Companion)

Alan•Paris

🤖 WHO WE ARE We’re building Mo — a health companion for everyone. We’re a small team within Alan, on a mission to redefine the future of care by democratizing access to medical expertise. We are building Mo, an intelligent, empathetic, multimodal LLM-based conversational AI assistant able to prescribe, triage, and proactively follow up on care. Imagine giving every person in the world instant, 24/7 access to personalised medical expertise - through chat, voice, or video. That’s what we are building 🤖. We kicked off the project in 2023, and we have launched Mo to all our french members (1M) in Q1 2026, making it the first safe AI health companion live at scale in Europe. 🚀 WHAT KEEPS US UP AT NIGHT We’re building a medical agent that works — reliably, safely, and at scale. You’ll work at the intersection of product engineering, ML engineering, and medical expertise. Our mission is to build the most accurate and reliable medical agent in the world, while delivering an outstanding conversational experience. Things we’re working on: * 💬 Deliver the most memorable chatting experience — You’ll work on making Mo the obvious choice when it comes to asking a health question. It means having a world-class snappy and delightful chatting experience, blazing fast time to answer, and rich content shared. * 🧱 Integrate Mo with Alan services and our healthcare system — You’ll work on integrating Mo with our other services (digital chat with HP, map of physical location of HPs, tailored prevention programs, …) to help members get from words to actions. * ➕ Extend our architecture to support the next chapters of Mo — You’ll work on our architecture, and agentic framework, to support Mo’s next needs (get more context, use new skills, be accessible in other countries we operate in) If you’re excited by the idea of building the future of healthcare, this is the team for you - we're at the start of the journey. If you’re curious, here is a paper we published recently. 🎯 IS IT YOU WE’RE LOOKING FOR? We’re looking for Product engineers with 3+ years of experience who get excited about building state-of-the-art AI systems and applying them to real-world, high-stakes problems in healthcare. * You’ll work at the frontier of LLMs, multi-agent architectures, and human-in-the-loop learning — and see your work deployed in production, helping real people. * You’ll implement new capabilities for Mo, working across engineering, data, and medical expertise * You'll have deep product ownership over the use cases supported in Mo, making sure we actually help our members get and stay better. * You’ll have significant influence on system architecture and code quality * You’ll mentor engineers, contribute to our product & engineering culture, and help build the team (including participating in hiring, shaping our interview loops, etc.). You would be a good fit in Mo if most of this sounds like you: * Applied background in software engineering and systems architecture * Experience with traditional ML, or LLMs, or ML dev ops is a big plus. * Product-minded: you want your solutions to be used, and impactful * Self-directed, pragmatic, driven, hard working * Humble and still willing to grow (no matter your seniority!) * Collaborative, enthusiastic about teamwork, learning, and teaching * Fluent in English (no need to speak French!) * Ex-founders are typically great fit in our team But above all, we are looking for high potential and curiosity: make sure to show us this when you apply! Everything else is a bonus. 🤘 HOW WE DO IT ? We benefit from Alan’s platform, infrastructure, distribution, and health expertise — but operate with the speed and focus of a startup. If you’re curious about Alan’s broader vision, our Shareholder Letters are worth a read. People joining Alan are often surprised and delighted by our innovative working method. We have a set of cultural values that guide our approach to work, such as: HOW WE WORK: OUR LEADERSHIP PRINCIPLES * Mission is the Boss — We think long-term and are methodical optimists who take risks, seeking our mission's success above all else. * Distributed Ownership — Accountable enlightened despots: everyone owns their decisions and results. * Radical Transparency — All information is accessible and written-first, so everyone can make the best decisions asynchronously. * Always Growing — Direct, positive, and caring feedback, combined with self-growth ownership. 🙌 PERKS & BENEFITS At Alan, we believe that being in good health is a basic need, and it starts with our employees. This is why Alaners are provided with a stimulating environment and perks ensuring they are happy, efficient and spend only high-quality time with co-workers. Therefore, we offer: * Fair rewards. Generous equity packages complement your base salary. * Flexible Office. Amazing office space at our HQ, sponsored co-working hubs or a full-remote experience with home office equipment sponsorship, we want you to live where you’re the happiest. * All the tools you need. Top of the range equipment: Macbook Pro, keyboard, laptop stand, monitor, and noise-canceling headphones. * Flexible vacation policy and flexible working hours. Organize your time as you wish. * Delightful healthcare insurance: Extremely comprehensive health insurance - 100% for you and your children, 90-100% for partners depending on your country (permanent contracts only). * Transport. Country-specific commuter benefits. * Learning & Training opportunities. A highly flexible Training policy free books and budget to attend and speak at conferences if the opportunity arises. * Personal growth through coaching: At Alan, coaching isn't just a perk - it's core to who we are. Every Alaner is paired with a dedicated coach from day one, who helps maximize their impact, nurture engagement, and navigate Alan's values to develop their full potential. Learn more about our coaching culture. * Parental leave. Extended parental leave for all new parents. IMPORTANT NOTE: WE HIRE PEOPLE, NOT ROLES. If you're excited about this opportunity but don't check every box, we'd love to hear from you. Everyone, no matter how underrepresented, should feel free to apply as it can only bring learnings or success. If you identify yourself as a woman: Did you know that research shows women often apply only when meeting 100% of requirements? Remember, this is just a guide, not a checklist. We'll be thrilled to receive your application! 🔖 Check out our About Alan and Career pages, as well as our Medium, blog and Glassdoor page for more info. #LI-DNP

Deadline: Aug 16Posted: Apr 21

Software Engineer II - Data Platform

May Mobility•Remote, USA

May Mobility is transforming cities through autonomous technology to create a safer, greener, more accessible world. Based in Ann Arbor, Michigan, May develops and deploys autonomous vehicles (AVs) powered by our innovative Multi-Policy Decision Making (MPDM) technology that literally reimagines the way AVs think. Our vehicles do more than just drive themselves - they provide value to communities, bridge public transit gaps and move people where they need to go safely, easily and with a lot more fun. We’re building the world’s best autonomy system to reimagine transit by minimizing congestion, expanding access and encouraging better land use in order to foster more green, vibrant and livable spaces. Since our founding in 2017, we’ve given more than 500,000 autonomous rides to real people around the globe. And we’re just getting started. We’re hiring people who share our passion for building the future, today, solving real-world problems and seeing the impact of their work. Join us. JOB SUMMARY As a Software Engineer II, you will contribute to the design, development, and maintenance of scalable data pipelines that support our data-driven initiatives. You will work independently to define the approach for assigned tasks, applying standard engineering procedures and techniques to solve complex problems. You will be expected to handle ambiguous requirements, exercise full agency over your own work, and ensure that your technical solutions are compatible with systems and platforms in adjacent areas. You will balance technical execution with a clear understanding of business and product outcomes. ESSENTIAL RESPONSIBILITIES * Contribute to the design, implementation, and maintenance of real-time and historical data pipelines, ensuring adherence to standard procedures and best practices. * Build and maintain well-organized, documented libraries and APIs to manage, search, and analyze vehicle datasets. * Collaborate on the development of front-end interfaces to visualize data and improve fleet operations. * Partner with cross-functional autonomy and platform teams to address software needs, ensuring system compatibility and synchronization. * Independently determine technical approaches to assigned tasks, actively discussing improvements and potential alternatives with the team. * Apply rigorous testing and standard procedures to optimize data processing workflows for performance, scalability, and cost-effectiveness. * Implement data quality checks and monitoring tools, maintaining awareness of the impact of these systems on product and business outcomes. * Proactively invest in professional growth and technical mastery through research and application of industry standards. SKILLS AND ABILITIES Success in this role typically requires the following competencies: * Technical Proficiency: Strong software development skills with proficiency in object-oriented programming (Python, Rust, Java, etc.) and a commitment to applying industry-standard techniques. * Problem Solving: Demonstrated ability to identify potential alternatives to solve technical problems and the ability to work independently with ambiguous requirements. * Communication: Effective, clear, and concise communication of both technical and non-technical information to teammates and peers across different domains. * Collaboration: Active contribution to design discussions and a collaborative approach to identifying and resolving dependencies. * Ownership: A high degree of agency in executing assigned specs and a proactive mindset toward system compatibility. QUALIFICATIONS AND EXPERIENCE Candidates most successful in this role typically hold the following qualifications or comparable knowledge or experience: * Required * B.S. Degree in Computer Science, Computer Engineering, or an equivalent degree or experience. * 2+ years of industry experience in software engineering. * Working knowledge of data structures, algorithms, and object-oriented programming. * Experience with cloud platforms (AWS, Azure, GCP) and deployment practices. * Experience with agile development methodologies and CI/CD practices. * Desirable * Experience building and maintaining data-processing pipelines in a cloud environment. * Knowledge of data warehousing or data lake concepts. * Familiarity with containerization (Docker, Kubernetes) and telemetry systems. * Understanding of real-time data processing and distributed systems. PHYSICAL REQUIREMENTS * Standard office working conditions which includes but is not limited to: * Prolonged sitting * Prolonged standing * Prolonged computer use * Travel required? - Minimal: 1%-10% ---------------------------------------------------------------------------------------------------------------------------------- Benefits and Perks * Comprehensive healthcare suite including medical, dental, vision, life, and disability plans. Domestic partners who have been residing together at least one year are also eligible to participate. * Health Savings and Flexible Spending Healthcare and Dependent Care Accounts available. * Rich retirement benefits, including an immediately vested employer safe harbor match. * Generous paid parental leave as well as a phased return to work. * Flexible vacation policy in addition to paid company holidays. * Total Wellness Program providing numerous resources for overall wellbeing Don’t meet every single requirement? Studies have shown that women and/or people of color are less likely to apply to a job unless they meet every qualification. At May Mobility, we’re committed to building a diverse, inclusive, and authentic workforce, so if you’re excited about this role but your previous experience doesn’t align perfectly with every qualification, we encourage you to apply anyway! You may be the perfect candidate for this or another role at May. Want to learn more about our culture & benefits? Check out our website [https://maymobility.com/careers/]! May Mobility is an equal opportunity employer. All applicants for employment will be considered without regard to race, color, religion, sex, national origin, age, disability, sexual orientation, gender identity or expression, veteran status, genetics or any other legally protected basis. Below, you have the opportunity to share your preferred gender pronouns, gender, ethnicity, and veteran status with May Mobility to help us identify areas of improvement in our hiring and recruitment processes. Completion of these questions is entirely voluntary. Any information you choose to provide will be kept confidential, and will not impact the hiring decision in any way. If you believe that you will need any type of accommodation, please let us know. Note to Recruitment Agencies: May Mobility does not accept unsolicited agency resumes. Furthermore, May Mobility does not pay placement fees for candidates submitted by any agency other than its approved partners. Salary Range $110,000—$157,000 USD May Mobility uses automated tools to support — but not replace — human judgment in our recruiting process, to find out more please click here [https://drive.google.com/file/d/1eGgErDdZ99HVoyxBrVpVUjhdUcmKogki/view?usp=drive_link]

Deadline: Aug 29Posted: Jul 29

Other

Growth Engineer

Extend•New York City

Extend is building a modern document processing platform. We're on a mission to transform how the world works with unstructured data. We’re hiring a Growth Engineer to join our founding team and own technical distribution. This role sits at the intersection of engineering, product, and GTM, and is focused on shipping real technical artifacts that earn developer attention and drive adoption. This is a hands-on engineering role. You will be writing code. WHY YOU SHOULD CONSIDER JOINING * We grew revenue 400% last year, with growth continuing to accelerate * Extend is used in mission-critical workflows at companies like Chime, Flatiron Health, Brex, Opendoor, Checkr, and more * The document processing market has expanded 1000× due to LLMs — and existing solutions are brittle and low-trust * We punch far above our weight with a lean team — every hire has outsized leverage * We’re solving novel problems at the intersection of LLMs, evals, complex data pipelines, and human-in-the-loop systems * Backed by world-class investors (YC, Homebrew, investors from OpenAI, and more) * You’ll work directly with the CEO, CTO, and founding team with real ownership IMPACT Document processing is exploding. Developers are actively evaluating tools and asking: what actually works? Your job is to answer that question publicly and credibly. This role exists to: * Own technical distribution for Extend * Build artifacts developers trust because they can inspect and test them * Shape how the market understands document processing, OCR, and extraction Success looks like shipping projects that drive real usage, pipeline, and category leadership — not vanity metrics. ROLE You will build and ship technical growth experiments end-to-end. Responsibilities include: * Build public-facing technical projects (e.g. OCR Arena (ocrarena.ai), benchmarks, demos, microsites) * Design and run benchmarks for document processing (accuracy, latency, cost, tradeoffs) * Ship technical demos, reference implementations, and eval harnesses * Write production-quality code across the stack * Own projects from idea → build → launch → iteration * Instrument experiments and tie them back to adoption and revenue impact * Partner closely with Engineering, Product, and GTM * Move fast, test aggressively, and kill ideas that don’t work WHAT WE’RE LOOKING FOR You likely have: * Strong engineering fundamentals and experience shipping real systems * Comfort working across the stack (frontend, backend, infra, data) * Experience with APIs, datasets, evals, or developer tooling * Background in high-growth startups or highly ambiguous environments * Ability to communicate clearly with a technical audience You might have: * Built tools, benchmarks, or demos that became externally useful * Written technical content grounded in systems you actually built * Shipped side projects that attracted real developer usage or attention * Worked at the intersection of product, engineering, and growth You definitely have: * High agency and velocity * Strong technical taste (signal > noise) * A founder mentality — no task is beneath you * Obsession with outcomes, not activity BENEFITS Beyond the below, we care deeply about each of our candidates on a personal level. We take the time to understand your needs to craft a package designed for you: * Insurance - 90% health insurance premium coverage * Relocation - for candidates based outside of NYC * Unlimited PTO policy - we trust everyone like owners, take what you need * Daily lunch & snacks covered (plus dinner if you're in the office late) - you'll never go hungry and you help pick the snacks * Unlimited token / tooling access - no handicaps on your productivity * Learning and development investment - we invest heavily in our team and support your growth Extend is an Equal Opportunity Employer. We consider all qualified applicants for employment without regard to race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, disability, genetic information, citizenship status, marital status, pregnancy, protected veteran status, or any other characteristic protected by applicable federal, state, or local law.

Deadline: Aug 17Posted: Jan 12

AIGoRust