Research Engineer (Agentic Models) på JetBrains i Amsterdam
JetBrains · Amsterdam
Research Engineer (Agentic Models)
At JetBrains, code is our passion. Ever since we started, back in 2000, we’ve been striving to make the strongest, most effective
developer tools on earth. Toda...
At JetBrains, code is our passion. Ever since we started, back in 2000, we’ve been striving to make the strongest, most effective
developer tools on earth. Today, AI-powered assistance and agents are becoming a core part of how developers work in our IDEs.
We’re building multi-step coding agents that can understand large codebases, plan changes, call tools, and iterate with the user.
As a Research Engineer in the Agentic Models team, you’ll be responsible for the models, training loops, and evaluation pipelines
that power these agents.
You’ll work at the intersection of SFT and RL-style post-training, and product-driven evaluation, using our distributed GPU and
MapReduce clusters to ship models into JetBrains products.
AS PART OF OUR TEAM, YOU WILL:
Design, implement, and maintain SFT and RL post-training pipelines for multi-step coding agents.
Train and adapt LLMs for agent workflows, including planning, tool use, and multi-step interactions inside JetBrains IDEs.
Build and develop evaluation and simulation environments where coding agents can act, be measured, and compared on realistic
developer tasks.
Design evaluation frameworks and metrics for agent behavior, analyze traces and logs, and close the loop from evaluation back
into training, data, and reward design.
Analyze training and evaluation results to propose and implement improvements to model architectures, training recipes, and
datasets.
Work with large-scale infrastructure, including distributed training on GPU clusters and large MapReduce-style data processing
for pre-training and fine-tuning datasets.
Collaborate closely with research, product, and infrastructure teams to turn high-level product visions into concrete models,
experiments, and shipped features.
WE’LL BE HAPPY TO BRING YOU ON BOARD IF YOU HAVE:
Extensive hands-on experience training LLMs (pre-training, fine-tuning, or post-training) in a research or production setting.
Deep expertise in modern deep learning frameworks such as PyTorch, and specialized LLM training stacks (e.g. Megatron, NeMo,
verl, or similar).
Strong theoretical and practical understanding of LLM fundamentals: architectures, tokenization, data pipelines, batching,
mixed precision, distributed training, and debugging unstable runs.
The ability to own projects end to end, starting from a high-level problem or product pain point and overseeing it through the
design, experimentation, implementation, and iteration phases.
A product-aware mindset – you care about how developers actually use agents and can translate product needs and failure modes
into modeling and evaluation work.
At least 3 years of Python experience writing clean, maintainable code in modern ML codebases.
OUR IDEAL CANDIDATE WOULD HAVE EXPERIENCE WITH:
ML orchestrators and workflow tools such as Kubeflow, Dagster, Airflow, ZenML, and/or job schedulers like Kubernetes or SLURM.
Large-scale data and training pipelines, e.g. MapReduce-style clusters, multi-node GPU training, or workloads on the order of
1M+ CPU/GPU hours.
Designing and maintaining evaluation pipelines for LLMs or agents, including metrics, dashboards, experiment tracking, and
automated regression checks.
AI agent development, such as tool-using agents, planners, or multi-step coding workflows, and familiarity with agentic
frameworks or patterns.
Experiment tracking and observability using tools like Weights & Biases, MLflow, Langfuse, or similar.
Inference optimization and serving optimized models in production.
#LI-KP1
We are an equal opportunity employer
We know great ideas can come from anyone, anywhere. That’s why we do our best to create an open and inclusive workplace – one that
welcomes everyone regardless of their background, identity, religion, age, accessibility needs, or orientation.
We process the data provided in your job application in accordance with the Recruitment Privacy Policy.
Amsterdam•Publicerad: 15 juni•Sista ansökan: 16 aug.
At JetBrains, code is our passion. Ever since we started back in 2000, we have been striving to make the strongest, most effective
developer tools on earth. By automating routine checks and corrections, our tools speed up production, freeing developers to grow,
discover, and create.
We’re looking for a Research Engineer who will own the training stack and model architecture for our Mellum LLM family. Your job
is easier said than done: make training faster, cheaper, and more stable at a large scale. You’ll profile, design, and implement
changes to the training pipeline – from architecture to custom GPU kernels, as needed.
AS PART OF OUR TEAM, YOU WILL:
* Be responsible for improving end-to-end performance for multi-node LLM pre-training and post-training pipelines.
* Profile hotspots (Nsight Systems/Compute, NVTX) and fix them using compute/comm overlap, kernel fusion, scheduling, etc.
* Design and evaluate architecture choices (depth/width, attention variants including GQA/MQA/MLA/Flash-style, RoPE scaling/NTK,
and MoE routing and load-balancing).
* Implement custom ops (Triton and/or CUDA C++), integrate via PyTorch extensions, and upstream when possible.
* Push memory/perf levers: FSDP/ZeRO, activation checkpointing, FP8/TE, tensor/pipeline/sequence/expert parallelism, NCCL tuning.
* Harden large runs by building elastic and fault-tolerant training setups, ensuring robust checkpointing, strengthening
reproducibility, and improving resilience to preemption.
* Keep the data path fast using streaming and sharded data loaders and tokenizer pipelines, as well as improve overall throughput
and cache efficiency.
* Define the right metrics, build dashboards, and deliver steady improvements.
* Run both pre-training and post-training (including SFT, RLHF, and GRPO-style methods) efficiently across sizable clusters.
WE’LL BE HAPPY TO BRING YOU ON BOARD IF YOU HAVE:
* Strong PyTorch and PyTorch Distributed experience, having run multi-node jobs with tens to hundreds of GPUs.
* Hands-on experience with Megatron-LM/Megatron-Core/NeMo, DeepSpeed, or serious FSDP/ZeRO expertise.
* Real profiling expertise (Nsight Systems/Compute, nvprof) and experience with NVTX-instrumented workflows.
* GPU programming skills with Triton and/or CUDA, and the ability to write, test, and debug kernels.
* A solid understanding of NCCL collectives, as well as topology and fabric effects (IB/RoCE), and how they show up in traces.
OUR IDEAL CANDIDATE WOULD HAVE EXPERIENCE WITH:
* FlashAttention-2 and 3, CUTLASS and CuTe, TransformerEngine and FP8, Inductor, AOTAutograd, and torch.compile.
* MoE at scale (expert parallel, router losses, capacity management) and long-context tricks (ALiBi/YaRN/NTK scaling).
* Kubernetes or SLURM at scale, placement and affinity tuning, as well as AWS, GCP, and Azure GPU fleets.
* Web-scale data plumbing (streaming datasets, Parquet and TFRecord, tokenizer perf), eval harnesses, and benchmarking.
* Safety and post-training methods, such as DPO, ORPO, GRPO, and reward models.
* Inference ecosystems such as vLLM and paged KV.
#LI-KP1
We are an equal opportunity employer
We know great ideas can come from anyone, anywhere. That’s why we do our best to create an open and inclusive workplace – one that
welcomes everyone regardless of their background, identity, religion, age, accessibility needs, or orientation.
We process the data provided in your job application in accordance with the Recruitment Privacy Policy.
Amsterdam•Publicerad: 15 juni•Sista ansökan: 16 aug.
At JetBrains, code is our passion. Ever since we started back in 2000, we have been striving to make the world’s most robust and
effective developer tools. By automating routine checks and corrections, our tools speed up production, freeing developers to
grow, discover, and create.
We are working on an ambitious new platform that provides AI capabilities to all JetBrains products. Our platform is based on
models developed in-house for writing and coding assistance, as well as integration with our strategic partners.
We are looking for a Research Engineer who can contribute to training foundation models for coding tasks. You’ll be working on
developing Large Language Models from scratch and deploying them into production environments where they will be accessible by end
users across the globe.
WE VALUE ENGINEERS WHO:
* Can plan projects and make decisions independently, consulting with others if needed.
* Identify customer needs and prioritize their tasks accordingly.
* Start with the simplest solutions and gradually add complexity as needed.
* Take sole responsibility for an entire subsystem.
* Have a passion for learning and a desire to stay up to date with the latest developments in the LLM field.
IN THIS ROLE, YOU WILL:
* Work with stakeholders to convert business requirements into technical specifications.
* Train LLMs from scratch on a large GPU cluster.
* Collect and process pre-training and fine-tuning datasets.
* Support and improve existing subsystems.
WE’LL BE HAPPY TO HAVE YOU ON OUR TEAM IF YOU HAVE:
* Experience in design, deployment, and support of production ML systems.
* A strong theoretical background in NLP and transformer-based approaches.
* Proficiency with modern deep learning frameworks such as PyTorch and common libraries for NLP.
* Experience in distributed training of multi-billion parameter models.
* Attention to detail in everything you do and great communication skills.
WE’D BE ESPECIALLY THRILLED IF YOU HAVE EXPERIENCE WITH:
* LLM inference frameworks such as vLLM, DeepSpeed, TensorRT.
* LLM alignment techniques such as RLHF/RLAIF.
* MLOps tools and practices, including CI/CD for ML.
* K8s and Kubeflow.
* Scientific publications in the NLP field.
HOW WE DEVELOP JETBRAINS AI:
* A cluster of hundreds of NVIDIA GPUs as training infrastructure.
* Git for source control management.
* Python, PyTorch, and HuggingFace as an ML stack.
* Kubeflow and Weights & Biases for experiment tracking.
* TeamCity as a CI Automation system.
#LI-KP1
We are an equal opportunity employer
We know great ideas can come from anyone, anywhere. That’s why we do our best to create an open and inclusive workplace – one that
welcomes everyone regardless of their background, identity, religion, age, accessibility needs, or orientation.
We process the data provided in your job application in accordance with the Recruitment Privacy Policy.
Amsterdam•Publicerad: 15 juni•Sista ansökan: 16 aug.
At JetBrains, code is our passion. Ever since we started, back in 2000, we've been striving to make the strongest, most effective
developer tools on earth. Today, AI-powered coding agents are becoming a core part of how developers write Kotlin – and we want to
make sure they write it well.
The Kotlin AI Value Stream team is responsible for how AI agents understand, generate, and improve Kotlin code across all
platforms: Android, Kotlin Multiplatform, server-side, web, desktop, and others. We build the evaluation infrastructure, error
analysis tools, and post-training pipelines that measure and improve agent behavior on real Kotlin developer tasks.
As a Research Engineer on this team, you'll own the end-to-end loop: Analyze how agents fail on Kotlin → build evals that capture
those failures → research and implement methods to fix them → measure the improvement. Your work will directly shape how millions
of developers experience Kotlin through AI coding agents.
AS PART OF OUR TEAM, YOU WILL:
Build tools for agentic error analysis
* Design and implement tooling to systematically capture, classify, and analyse errors that AI coding agents make when generating
Kotlin code.
* Build observability pipelines over agentic traces – mining patterns from agent sessions in JetBrains IDEs, Junie, Claude Code,
Cursor, and other coding agents.
Build evaluation pipelines
* Design, implement, and maintain evaluation pipelines that measure Kotlin code generation quality across dimensions, including
correctness, idiomaticity, build success, framework usage, and test coverage.
* Build simulation environments where coding agents can be measured on realistic Kotlin developer tasks – from greenfield KMP
projects and Gradle dependency management to migrating Spring applications from Java to Kotlin.
* Own evaluation infrastructure: metrics, experiment tracking, automated regression checks, and reproducible benchmarking.
Research methods for improving agent and model behavior on Kotlin
* Experiment with post-training techniques (SFT, DPO, GRPO) to improve how models handle Kotlin-specific patterns, idioms, and
frameworks.
* Investigate context engineering approaches: CLAUDE.md/AGENTS.md files, compiler-as-verifier feedback loops, Kotlin LSP
integration, and MCP-based tooling.
* Run experiments to measure impact: A/B comparisons, benchmark suites, and before/after analyses on real codebases.
* Collaborate with model providers (Anthropic, OpenAI, and Google) to translate Kotlin-specific findings into model improvements.
Build public Kotlin benchmarks
* Design and build open-source benchmarks that measure AI coding agent performance on Kotlin tasks and eventually become the
standard reference for the ecosystem.
* Create task datasets covering the breadth of Kotlin usage: the server side (Spring, Ktor), multiplatform projects (KMP), build
systems (Gradle), Android, library development, and others.
* Include both mined real-world tasks and carefully designed synthetic tasks that test specific Kotlin capabilities.
* Maintain and evolve benchmarks as models improve, ensuring they remain challenging, relevant, and contamination-resistant.
WE'LL BE HAPPY TO HAVE YOU ON BOARD IF YOU HAVE:
* Hands-on experience building evaluation or analysis pipelines for LLMs or AI coding agents in a research or production setting.
* Strong Python engineering skills (at least three years), with the ability to write clean, maintainable code in data-heavy and
ML-adjacent codebases.
* Experience with data analysis at scale: querying large datasets (SQL/Athena), building data pipelines, and performing
statistical analysis of experimental results.
* The ability to own projects end to end – from identifying a problem in agent traces to designing an eval, running experiments,
and shipping a fix.
* A product-aware mindset: You care about how agents are actually used by developers and can translate real failure modes into
evaluation and training work.
* Familiarity with Kotlin or a strong willingness to develop deep Kotlin expertise (you'll be living in Kotlin codebases daily).
OUR IDEAL CANDIDATE WOULD ALSO HAVE EXPERIENCE WITH:
* Post-training LLMs: SFT, RLHF, DPO, GRPO – either hands-on training or designing the data and reward pipelines that feed into
training.
* Modern deep learning frameworks (PyTorch) and LLM training stacks (TRL, verl, Megatron, or similar).
* AI agent development: tool-using agents, multi-step coding workflows, agentic frameworks.
* Evaluation frameworks and tools: Inspect AI, Promptfoo, LM-evaluation-harness, or custom eval pipelines.
* Experiment tracking and observability: Weights & Biases, MLflow, Langfuse, or similar.
* The Kotlin ecosystem: Android, Gradle, KMP, Spring, Ktor – with an understanding of the developer workflows that agents need to
support.
* Contributing to or maintaining open-source projects, especially benchmarks or evaluation tools.
Don't check every box? That's okay – if you're excited about this work and bring strong fundamentals, we'd love to hear from you.
We're happy to talk and provide the training you need to grow into the role.
WHY JOIN JETBRAINS?
* Strong base salary. We offer competitive pay that reflects your skills and experience.
* Flexible work location. Enjoy the freedom to work from home or from the office.
* Remote work. Spend up to 30 days per year working remotely from abroad.
* Extra time off. More days to relax, recharge, and do the things you love.
* Medical insurance allowance. Enjoy peace of mind for you and your family
* Learning and development opportunities. Access to conferences, courses, and language classes.
* Relocation support. We help make your move as smooth and stress-free as possible.
* Language classes. Pick up the local language or sharpen your English skills.
* Fuel your day. Enjoy a hot meal or receive a lunch allowance on workdays.
* Mental health support. To help you feel your best, we provide easy access to professional mental health services.
* Sports benefit. Enjoy an on-site gym or sports club stipend.
* Internal events. Join company-wide celebrations and team gatherings.
*Some benefits may vary depending on location.
#LI-KP1
We are an equal opportunity employer
We know great ideas can come from anyone, anywhere. That’s why we do our best to create an open and inclusive workplace – one that
welcomes everyone regardless of their background, identity, religion, age, accessibility needs, or orientation.
We process the data provided in your job application in accordance with the Recruitment Privacy Policy.
