Principal Infrastructure Engineer

PhysicsX

ABOUT US PhysicsX is a deep-tech company with roots in numerical physics and Formula One, dedicated to accelerating hardware innovation at the speed of software. We are building an AI-driven simulation software stack for engineering and manufacturing across advanced industries. By enabling high-fidelity, multi-physics simulation through AI inference across the entire engineering lifecycle, PhysicsX unlocks new levels of optimization and automation in design, manufacturing, and operations — empowering engineers to push the boundaries of possibility. Our customers include leading innovators in Aerospace & Defense, Materials, Energy, Semiconductors, and Automotive. NOTE: WE ARE CURRENTLY RECRUITING FOR MULTIPLE POSITIONS, HOWEVER PLEASE ONLY APPLY FOR THE ROLE THAT BEST ALIGNS WITH YOUR SKILLSET AND CAREER GOALS. THE ROLE The Principal ML Infrastructure Engineer will extend and operate the infrastructure that powers our research model training, fine-tuning, and serving pipelines. You will be embedded within our Research function, partnering directly with ML engineers and research scientists to ensure they can train Large Physics Models efficiently and reliably at scale. TEAM CONTEXT In this role, you will be vertically embedded in Research, working daily with: * Research Scientists who determine the model architectures and methods * ML Engineers who implement and develop the models * Simulation Data Engineers who are accountable for upstream data pipelines You will have end-to-end responsibilities over the research infrastructure, with the autonomy to make architectural decisions and the responsibility to keep data flowing reliably. Horizontally, you will be part of an infrastructure engineering group responsible for infrastructure across the company. WHAT YOU WILL DO TRAINING INFRASTRUCTURE * Design and operate distributed training infrastructure for neural operator architectures (Transolver, Point Cloud Transformer, etc.) on our large NVIDIA DGX B200 platform. * Optimize training pipelines for throughput, fault tolerance, and cost efficiency, including checkpointing strategies, gradient accumulation, and multi-node synchronization. * Build and maintain experiment tracking and observability systems that give researchers clear visibility into training runs, hyperparameter sweeps, and model performance. DATA I/O AND PERFORMANCE * Solve data loading bottlenecks for large-scale mesh datasets. * Optimize data pipelines for efficient I/O from cloud storage, including prefetching, caching, and format optimization. * Work with heterogeneous data sources of varying formats and resolutions. MODEL SERVING AND DEPLOYMENT * Build serving infrastructure for pre-trained LPMs, supporting both zero-shot inference and uncertainty quantification (Monte Carlo Dropout). * Design and implement model packaging pipelines for customer deployment. Models must run reliably in customer environments with fine-tuning capabilities. * Ensure reproducibility: any model checkpoint should be deployable with consistent behaviour. PLATFORM AND TOOLING * Improve developer experience for the Research team with fast iteration cycles, reliable CI/CD, clear debugging tools. * Collaborate with the broader Infrastructure team on shared patterns and standards. WHAT YOU BRING TO THE TABLE * Ability to scope and effectively deliver projects, prioritising activity as needed. * Problem-solving skills and the ability to analyse issues, identify causes, and recommend solutions quickly. * Excellent collaboration and communication skills, especially in a research setting. You can translate "the model isn't converging" into infrastructure hypotheses and solutions, and can bridge technical abstractions with implementations. * 5+ years of experience building and operating ML infrastructure at scale: * Deep expertise in distributed training: you've debugged NCCL hangs, optimized collective communication, and know when to use FSDP vs. DDP vs. pipeline parallelism * Strong systems fundamentals: Linux, networking (including domain specific NVLink and InfiniBand), storage I/O, profiling and performance optimization * Production experience with Kubernetes and SLURM for job orchestration on GPU clusters * Proficiency in Python and ML frameworks (PyTorch strongly preferred) * Experience with cloud GPU infrastructure; ideally CoreWeave or similar GPU/HPC-focused clouds IDEALLY * Experience with geometric deep learning or neural operators, ****architectures that operate on meshes, point clouds, or graphs * Background in HPC for simulation engineering, familiarity with how CFD/FEA workflows generate and consume data * Experience building model serving infrastructure with latency and throughput requirements * Familiarity with experiment tracking tools (Weights & Biases, MLflow) and observability stacks (Prometheus, Grafana) * Experience packaging models for deployment into customer environments (containers, model registries, versioning)   What we offer Build what actually matters Help shape an AI-native engineering company at a formative stage, tackling problems that genuinely matter for industry and society. This is work with real-world impact - and something you can be proud to stand behind. Learn alongside exceptional people Work with a high-caliber, collaborative team of engineers, scientists, and operators who care deeply about doing great work, and about helping each other get better. We come from diverse backgrounds, but we share a commitment to operating at the highest level and addressing some of the most complex challenges out there. If you’re ambitious, thoughtful, and driven by impact, you’ll feel at home. Influence over hierarchy We operate with a flat structure: good ideas win - wherever they come from. Questioning assumptions and challenging the status quo isn’t just welcomed, it’s expected. Sustainable pace, long-term ambition Building meaningful technology is a marathon, not a sprint. We believe in balancing focused, ambitious work with a life beyond it. Our hybrid model blends time together in our Shoreditch office with work-from-home days, giving you the flexibility to work sustainably while staying connected in person. And it doesn’t stop there … 🚀 Equity options - share meaningfully in the company you’re helping to build. 🏦 10% employer pension contribution - because investing in future matters. 🍽️ Free office lunches - to keep you energised and focused. 👶 Enhanced parental leave - 3 months full pay paternity and 6 months full pay maternity leave, to provide extra flexibility during the moments that matter most. 🍼 YellowNest nursery scheme - to help working parents manage childcare costs. ☀️ 25 days of Annual Leave (+ Public Holidays) - because taking time to rest matters. 🏥 Private medical insurance - 100% employee cover, giving you complete peace of mind. 💪 Wellhub Subscription - gain access to thousands of gyms, classes and wellness apps, supporting both physical and mental wellbeing. 👀 Eye tests - because good work depends on good health. 📈 Personal development - dedicated support for learning, development, and leveling up over time. 💛 Employee Assistance Programme (EAP) - confidential wellbeing support, available whenever you need it. 🚲 Bike2Work scheme and 🚆 Season ticket loan - to make getting to work easier and greener. 🚗 Octopus EV salary sacrifice - for a simpler, more sustainable way to drive electric. 🔎 Watch this space, we’re continuing to build this as we grow…   We value diversity and are committed to equal employment opportunity regardless of sex, race, religion, ethnicity, nationality, disability, age, sexual orientation or gender identity. We strongly encourage individuals from groups traditionally underrepresented in tech to apply. To help make a change, we sponsor bright women from disadvantaged backgrounds through their university degrees in science and mathematics.    We collect diversity and inclusion data solely for the purpose of monitoring the effectiveness of our equal opportunities policies and ensuring compliance with UK employment and equality legislation. This information is confidential, used only in aggregate form, and will not influence the outcome of your application.

Implico

DEINE AUFGABEN Your mission As Principal Infrastructure & Release Lead, you will be responsible for the Group-wide infrastructure, deployment and release governance of the Implico Group. You will ensure that our product architectures are not only conceptually strong, but can also be delivered in a scalable, stable and cost-efficient manner. You will build the structural foundations for professional CI/CD processes, modern cloud infrastructures and a controlled release organization. Your responsibility CI/CD & Deployment Governance * Definition and introduction of group-wide CI/CD standards * Design and further development of deployment architectures (private cloud, public cloud, SAP BTP) * Standardization of release processes and environmental strategies Cloud Infrastructure & Scalability * Build scalable cloud infrastructures across all product lines * Definition of monitoring, observability and security standards in the delivery process * Ensuring technical delivery capability in close coordination with Technical Architects Cost & Platform Responsibility * Active cloud cost management and visibility into infrastructure costs * Establish cost-conscious architecture and deployment principles * Member of the Architecture Board with responsibility for total cost and delivery perspective Establishment of the Platform Engineering Team * Design and build a scalable DevOps structure * Technical leadership and development of the DevOps team Organizational classification * Group-wide responsibility across all product lines * Principal-level role• Member of the Architecture Board * Close cooperation with CTO, Enterprise Architects, Technical Architects and Director AI Engineering DEIN PROFIL Your profile • Several years of experience in setting up and operating modern cloud infrastructures• Deep expertise in CI/CD, DevOps methods and release management• Experience with containerization and scalable deployment architectures• Strong understanding of cloud cost models and infrastructure optimization• Experience in standardizing technical processes• High technical credibility as well as structured, assertive way of working

Moniepoint

Who we are Moniepoint Inc. is Africa’s all-in-one financial platform, helping 20 million businesses and individuals access seamless payments, banking, credit, cross-border, and business management tools each month.  As Nigeria’s largest merchant acquirer, we power most of the country’s point-of-sale (POS) transactions. Through our subsidiaries, Moniepoint Inc. processes over $250 billion in digital payment transaction value annually. About the role Engineering at Moniepoint is an inspired, customer-focused community dedicated to crafting solutions that redefine our industry. Our infrastructure runs on some of the cool tools that excite infrastructure engineers - kubernetes, docker etc. We also make business decisions based on the large stream of data we receive daily, so we work daily with big data, perform data analytics and build models to make sense of the noise and give our customers the best experience. Curious about what makes Moniepoint an incredible place to work? Check out posts on how we cultivate a culture of innovation, teamwork, and growth. Position Overview We are seeking an experienced Cloud Engineer to design, implement, and manage our multi-cloud infrastructure. The ideal candidate will have deep expertise in cloud platforms, container orchestration, infrastructure automation, CI/CD pipelines, and observability solutions, ensuring scalable, reliable, and cost-effective cloud operations across multiple cloud providers. Principal Duties and Responsibilities Cloud Infrastructure Management * Design, deploy, and manage multi-cloud infrastructure across Google Cloud Platform (GCP), Amazon Web Services (AWS), Azure, and Oracle Cloud Infrastructure (OCI) * Architect and implement highly available, fault-tolerant, and scalable cloud solutions * Manage cloud resources including compute instances and networking components * Design and implement disaster recovery and business continuity plans for cloud workloads * Migrate on-premises applications and services to cloud environments with minimal disruption * Optimize cloud resource utilization and implement auto-scaling policies * Maintain comprehensive documentation of cloud architectures, configurations, and runbooks Kubernetes & Container Orchestration * Design, deploy, and manage production-grade Kubernetes clusters across multiple cloud providers * Implement and maintain container orchestration strategies for microservices architectures * Configure and manage Kubernetes resource objects * Manage Kubernetes cluster upgrades, scaling, and performance optimization * Troubleshoot complex container and orchestration issues in production environments * Implement multi-cluster and multi-region Kubernetes deployments for high availability Service Mesh & Advanced Networking * Design, deploy, and manage Istio service mesh for microservices communication and observability * Configure Istio traffic management, including virtual services, destination rules, and gateways * Implement advanced traffic routing (canary deployments, A/B testing, traffic splitting) using Istio * Deploy and manage Istio observability components (telemetry, distributed tracing, service graphs) * Implement circuit breaking, retries, timeouts, and fault injection for resilience testing * Configure Istio ingress and egress gateways for external traffic management * Monitor and optimize service mesh performance and resource utilization * Implement multi-cluster service mesh architectures across different cloud providers Reverse Proxy & Load Balancing * Deploy, configure, and manage HAProxy for high-performance load balancing and reverse proxy * Implement HAProxy ACLs, backend routing, health checks, and session persistence * Design and implement Nginx as reverse proxy for web applications and API gateways * Configure Nginx for rate limiting and request filtering * Implement Nginx load balancing algorithms and upstream health monitoring * Manage Nginx Plus features for advanced traffic management and monitoring * Optimize HAProxy and Nginx performance for high-throughput environments Infrastructure as Code & Configuration Management * Develop and maintain infrastructure as code using Terraform * Create reusable, modular Terraform configurations for various cloud resources and Implement Terraform state management and remote backends * Design and implement configuration management solutions using Ansible * Develop Ansible playbooks and roles for automated server provisioning and configuration * Integrate Terraform and Ansible workflows for end-to-end infrastructure automation * Implement infrastructure version control, code review processes, and GitOps practices * Manage infrastructure drift detection and remediation * Create and maintain infrastructure documentation and architecture diagrams * Implement policy-as-code using tools like OPA (Open Policy Agent) or Sentinel CI/CD Pipeline Management * Design, implement, and maintain continuous integration pipelines using Jenkins and Harness * Optimize build times and pipeline efficiency * Integrate security scanning (SAST, DAST, container scanning) into CI/CD pipelines * Configure Jenkins jobs, pipelines, and shared libraries for automated build, configure build agents, runners, and execution environments * Implement Harness deployment pipelines for cloud-native applications * Integrate CI/CD pipelines with version control systems (Git, GitHub, GitLab) * Implement continuous deployment workflows using ArgoCD for Kubernetes-based applications * Design and implement GitOps workflows with ArgoCD for declarative application delivery * Manage ArgoCD application definitions, sync policies and multi-cluster deployments * Implement progressive delivery strategies (blue-green deployments, canary releases) using ArgoCD Message Streaming & Event-Driven Architecture * Deploy and manage Apache Kafka clusters for real-time data streaming and event-driven architectures * Configure Kafka topics, partitions, replication factors, and retention policies * Implement Kafka Connect for data integration with various sources and sinks * Monitor Kafka cluster health, performance metrics, and consumer lag * Optimize Kafka performance for high-throughput and low-latency use cases * Troubleshoot Kafka producer and consumer issues Database & Proxy Management * Deploy, configure, and manage ProxySQL for MySQL load balancing and high availability * Implement query routing, caching, and connection pooling strategies using ProxySQL * Optimize database performance through ProxySQL query analysis and optimization * Implement database failover and disaster recovery using ProxySQL * Monitor ProxySQL metrics and troubleshoot connection and performance issues * Integrate ProxySQL with database clusters and replication topologies * Implement database access security and audit logging through ProxySQL Cloud Networking * Design and implement cloud networking architectures, including VPCs, subnets, and network segmentation * Configure and manage cloud load balancers (Application Load Balancers, Network Load Balancers, Cloud Load Balancing) * Implement VPN connections, Direct Connect/Interconnect, and hybrid cloud networking solutions * Implement network security controls, including security groups, network ACLs, and firewall rules * Implement network monitoring and traffic analysis * Troubleshoot complex networking issues across multi-cloud environments * Design and implement private connectivity between cloud providers Secrets Management & Security * Configure and manage HashiCorp Vault for centralized secrets management across multi-cloud environments * Configure Vault secret engines (KV, database, PKI, AWS, GCP, Azure dynamic secrets) * Manage Vault high availability clusters and disaster recovery procedures * Implement dynamic database credentials and secret rotation strategies * Manage Vault encryption as a service for application-level encryption * Implement Vault agent and sidecar injectors for Kubernetes workloads * Migrate secrets from legacy systems to Vault   Qualifications, Competency & Skills Required Education & Experience * Bachelor's degree or diploma in Computer Science, Information Technology, Engineering, or related field * Minimum of 5 years of proven experience in cloud engineering, DevOps, or platform engineering roles * Hands-on experience managing production workloads across multiple cloud platforms * Relevant cloud and technology certifications are highly desirable  Technical Skills Cloud Platforms (Required) * Google Cloud Platform (GCP): Deep expertise in Compute Engine, GKE, Cloud Storage, Cloud SQL, VPC, Cloud Functions, Cloud Run, IAM * Amazon Web Services (AWS): Proficiency in EC2, EKS, S3, RDS, VPC, Lambda, ECS, CloudFormation, IAM * Microsoft Azure: Experience with Virtual Machines, AKS, Blob Storage, Azure SQL, Virtual Networks, Azure Functions, ARM templates * Oracle Cloud Infrastructure (OCI): Familiarity with Compute, OKE, Object Storage, networking, and OCI-specific services * Multi-cloud architecture design and implementation experience * Cloud migration strategies and execution (lift-and-shift, re-platforming, re-architecting) Container & Orchestration (Required) * Expert-level Kubernetes knowledge, including cluster architecture, networking, storage, and security * Hands-on experience with managed Kubernetes services (GKE, EKS, AKS) * Proficiency in Docker containerization, image optimization, and registry management * Experience with Helm charts for application packaging and deployment * Knowledge of container runtime environments (containerd, CRI-O) Service Mesh & Microservices (Required) * Istio: Deep expertise in Istio architecture, deployment, and operations * Istio traffic management (virtual services, destination rules, gateways, service entries) * Istio security features (mTLS, authorization policies, peer authentication, request authentication) * Istio observability and telemetry configuration * Multi-cluster and multi-mesh deployments * Service mesh troubleshooting and performance optimization * Understanding of sidecar proxy patterns and Envoy proxy * Experience with other service mesh solutions (Linkerd, Consul Connect) is a plus Reverse Proxy & Load Balancing (Required) * HAProxy Advanced configuration and management for load balancing and high availability * Nginx Expert-level configuration as reverse proxy and API gateway * Nginx rate limiting, and performance tuning * Nginx load balancing algorithms and upstream configurations * Experience with Nginx modules and custom configurations * High availability configurations using keepalived, VRRP, or similar * Integration with Kubernetes ingress controllers (Nginx Ingress, Istio Ingress) Infrastructure as Code (Required) * Advanced Terraform skills for multi-cloud infrastructure provisioning * Terraform module development, state management, and workspace strategies * Proficiency in Ansible for configuration management and automation * Ansible playbook development, roles, and inventory management * Experience with version control systems (Git) and GitOps workflows * Infrastructure testing frameworks (Terratest, Kitchen-Terraform) CI/CD Tools (Required) * Jenkins: Pipeline development (declarative and scripted), shared libraries, plugin management * Harness: Deployment pipeline configuration, workflow creation, approval gates * ArgoCD: GitOps workflows, application synchronization, multi-cluster management * Integration of CI/CD tools with Kubernetes and cloud platforms * Automated testing and deployment strategies * Artifact repository management (Nexus, Artifactory, cloud-native registries) Messaging & Streaming (Required) * Apache Kafka architecture, cluster management, and operations * Kafka topic design, partitioning strategies, and performance tuning * Kafka Connect experience * Experience with Kafka management tools (Kafka Manager, Cruise Control) * Understanding of event-driven architectures and patterns Database & Proxy Technologies (Required) * ProxySQL configuration, management, and optimization * MySQL database administration basics * Understanding of database replication and clustering Observability & Monitoring (Required) * Prometheus metrics collection, PromQL, and alerting rules * Grafana dashboard design and visualization techniques * Log aggregation and analysis Networking (Required) * Deep understanding of TCP/IP, DNS, HTTP/HTTPS, and network protocols * Cloud networking concepts (VPC, subnets, routing tables, NAT, VPN) * Load balancing strategies and implementations * Service discovery and DNS-based routing * Network security and firewall configuration * Software-defined networking (SDN) concepts Scripting & Programming * Proficient in scripting languages: Python, Bash * Go or python programming basics for tooling development * YAML and JSON for configuration management * Understanding of software development best practices Secrets Management (Required) * HashiCorp Vault: Advanced knowledge of Vault architecture, deployment, and operations * Vault authentication methods and integration with cloud providers and Kubernetes * Vault secret engines (KV v1/v2, database, transit, cloud dynamic... What we can offer you * Culture -We put our people first and prioritize the well-being of every team member. We’ve built a company where all opinions carry weight and where all voices are heard. We value and respect each other and always look out for one another. Above all, we are human. * Learning - We have a learning and development-focused environment with an emphasis on knowledge sharing, training, and regular internal technical talks. * Compensation - You’ll receive an attractive salary, pension, health insurance, annual bonus, plus other benefits. What to expect in the hiring process * A technical interview with the Hiring Manager * A behavioural and technical interview with a member of the Executive team.