# Machine Learning Infrastructure Engineer, Orchestration

Canonical page: https://efference.ai/careers/ml-infrastructure-engineer-cluster-orchestration/

Machine Learning Infrastructure Engineer, Orchestration at Efference. San Francisco · Full time · On-site. You'll build the compute platform that schedules and runs robot models on shared GPUs.

You'll build the compute platform that schedules and runs robot models on shared GPUs. You'll own placement, model lifecycle, and service reliability across inference, training, and evaluation workloads, accounting for the latency and continuity requirements of active robot sessions.

## What you’ll do

- Develop scheduling and admission control based on model memory, accelerator capacity, session requirements, and measured network and inference latency.
- Build model loading, warm pools, caching, and container lifecycle management to reduce startup delays and avoid disruption during capacity changes.
- Coordinate inference-serving behavior with performance engineers, balancing batching and GPU utilization against observation freshness and response deadlines.
- Isolate customer workloads and manage quotas, resource contention, and priority so background training or evaluation cannot silently degrade live robot sessions.
- Implement health checks, failure recovery, staged model rollouts, and rollback, preserving the session and version information needed to diagnose incidents.
- Instrument queue time, GPU utilization, memory pressure, deadline misses, and serving cost; use workload replay and load tests to guide capacity planning.

## What we are looking for

- Experience building distributed infrastructure, schedulers, runtimes, or ML platforms that run production workloads.
- Strong systems programming in Go, Rust, C++, or Python and sound judgment about concurrency, failures, and consistency.
- Understanding of Linux, containers, networking, storage, and how resource limits affect application performance.
- Experience operating services with clear reliability targets and debugging problems across application and infrastructure boundaries.
- Ability to translate workload measurements into scheduling and architecture decisions, including tradeoffs between utilization and latency.

## Nice to Have

- Experience with Kubernetes, Ray, Slurm, custom schedulers, or GPU-aware container runtimes.
- Work on inference serving, checkpoint/restore, model caching, multi-GPU workloads, or accelerator isolation.
- Experience with latency-sensitive robotics, multimodal models, or scheduling mixed training and inference workloads.