<div class="content-intro"><p>Dive in and do the best work of your career at DigitalOcean. Journey alongside a strong community of top talent who are relentless in their drive to build the simplest scalable cloud. If you have a growth mindset, naturally like to think big and bold, and are energized by the fast-paced environment of a true industry disruptor, you’ll find your place here. We value winning together—while learning, having fun, and making a profound difference for the dreamers and builders in the world. </p></div><p>We are seeking a Senior Engineering Manager to lead our Inference Orchestration team, driving the strategy, execution, and scaling of our Kubernetes-based AI infrastructure. You will be responsible for balancing business needs with technical excellence, ensuring high throughput, optimal GPU utilization, and robust fault tolerance for our next-generation disaggregated inference, fine-tuning, and training workloads.</p> <h2><strong>What You'll Do:</strong></h2> <ul> <li><strong>Team Leadership & Development:</strong> Recruit, mentor, and coach engineers on the team, fostering a culture of ownership, technical excellence, and continuous improvement.</li> <li><strong>Execution & Delivery:</strong> Own the team's project execution, translating high-level business goals into clear technical roadmaps, measurable milestones, and successful, on-time delivery.</li> <li><strong>Cross-Functional Partnership:</strong> Collaborate with Product Management, other engineering teams, and key stakeholders to align priorities, manage dependencies, and communicate progress and risks.</li> <li><strong>Operational Health:</strong> Ensure the production health, stability, and on-call rotation of all services owned by the Inference Orchestration team.</li> <li><strong>Strategic Architecture & Planning:</strong> Define the technical roadmap and oversee the architecture of high-throughput scheduling systems for massive Kubernetes clusters (1,000+ nodes, 10,000+ pods), focusing on scalability techniques like multi-scheduler architectures and batch dispatching.</li> <li><strong>Maximize GPU Utilization</strong>: Eliminate GPU waste in multi-tenant environments by implementing fractional GPU allocation, leveraging mechanisms like KAI-Scheduler's Reservation Pods or hard-isolation tools like HAMi, and configuring time-based fairshare scheduling to balance over-quota pool access.</li> <li><strong>Orchestrate Complex Inference</strong>: Implement and manage disaggregated AI inference pipelines using frameworks like NVIDIA Grove, coordinating multicomponent deployments (e.g., prefill leaders, decode workers, KV routers) with multilevel autoscaling and explicit startup ordering.</li> <li><strong>Optimize Placement & Topology</strong>: Deploy topology-aware scheduling to align pod placement with physical hardware dimensions, such as NVLink connections, PCIe lanes, and NUMA nodes, minimizing communication latency for multi-GPU operations.</li> <li><strong>Platform Performance & Reliability:</strong> Drive initiatives to enhance overall cluster performance, including optimizing scheduling latency, API server load, and implementing fault tolerance mechanisms like Checkpoint/Restore for long-running AI training jobs.</li> <li><strong>Manage AI Storage & Fault Tolerance:</strong> Orchestrate efficient model weight distribution using OCI Image Volumes and implement Checkpoint/Restore capabilities (via CRIU and NVIDIA cuda-checkpoint) for long-running training fault recovery.</li> <li><strong>Security and Isolation:</strong> Define and enforce security best practices for AI workloads, ensuring multi-layered isolation environments and agent sandboxes are deployed to safely execute untrusted code (e.g., using Kata Containers, gVisor, or microVMs).</li> </ul> <h2><strong>What You’ll Bring:</strong></h2> <ul> <li><strong>Engineering Leadership Experience:</strong> Proven track record of managing and growing high-performing engineering teams, preferably within a distributed systems or infrastructure domain.</li> <li><strong>Kubernetes and AI Infrastructure Domain Knowledge:</strong> Deep expertise in Kubernetes at scale and a strong foundational understanding of the core challenges in AI workload orchestration, scheduling, and resource management.</li> <li><strong>Hardware-Aware Optimization:</strong> Strategic knowledge of GPU architectures (NVIDIA and/or AMD), interconnects (like NVLink), and hardware topology and their direct impact on AI training and inference performance.</li> <li><strong>Resource and Cost Management:</strong> Experience in balancing performance against cost, applying principles like Dominant Resource Fairness (DRF), and directing strategies for maximizing cluster efficiency.</li> <li><strong>Systems Engineering & Security:</strong> Familiarity with concepts in container runtime internals, system isolation, and security contexts to manage risk in shared infrastructure.</li> <li><strong>AI/ML Serving Architectures:</strong> Strong understanding of modern LLM serving architectures, disaggregation patterns, and common serving engines (e.g., vLLM, Triton, SGLang).</li> <li><strong>Observability and SLOs:</strong> Expertise in defining, tracking, and operationalizing deep infrastructure and inference metrics (e.g., TTFT, TPOT) to drive performance improvements and meet service level objectives.</li> </ul> <h2><strong&g