<h3><strong>About the Role</strong></h3> <p>In this role, you will operate, scale, and optimize multi-petabyte storage systems purpose-built for the world’s largest AI training and inference workloads. Storage now sits on the inference critical path as well, so you’ll also build the caching and data-movement layers that deliver KV cache and model weights directly into GPU memory at fabric speed. You’ll manage and scale high-performance parallel filesystems and object stores, evaluate and integrate cutting-edge technologies such as Vast, Weka, Ceph, and Lustre, and solve the complex engineering challenges of operating at extreme throughput, low-latency data paths, and massive cluster-scale storage operations. </p> <p>You will also build Kubernetes-native storage operators and self-service platforms that provide automated provisioning, strict multi-tenancy, performance isolation, and quota enforcement at cluster scale. Day-to-day, you’ll optimize end-to-end data paths for 10-50 GB/s per node, design multi-tier caching architectures, implement intelligent prefetching and model-weight distribution, build KV-cache offload and reuse tiers for inference, and tune parallel filesystems for AI workloads. </p> <p><strong>Responsibilities</strong></p> <ul> <li>Architect and implement the technical strategy and storage roadmap for Together AI, driving high-performance architectural decisions as we scale our GPU fleet.</li> <li>Engineer and scale multi-petabyte AI/ML storage systems by integrating Vast, Weka, and Ceph while executing deep cost optimization through automated tiering and lifecycle policies.</li> <li>Develop intelligent caching and tiered storage architectures to achieve extreme IOPS and cluster-wide throughput at GPU scale for training and inference workloads, including GPU-direct (RDMA) delivery of model weights for fast cold starts and autoscaling.</li> <li>Design and build KV-cache offload and reuse for inference: tiering KV blocks across GPU HBM, host memory, local NVMe, and distributed cache, enabling prefix reuse across requests and nodes, and integrating with inference engines (vLLM, SGLang, TensorRT-LLM) to cut time-to-first-token.</li> <li>Tune storage isolation at the L2/L3 network layers to ensure secure, production-grade multi-tenancy for storage clients.</li> <li>Code Kubernetes storage operators and controllers to enable automated provisioning, self-service abstractions, and quota enforcement.</li> <li>Engineer end-to-end data paths to achieve 10+ GB/s per GPU node; architect multi-tier caching for model weights, KV cache, and datasets; tune parallel filesystems using advanced profiling; and scale storage infrastructure across thousands of nodes.</li> <li>Optimize end-to-end data paths through advanced benchmarking and profiling, contributing high-impact code to open-source storage and inference projects and internal tooling.</li> </ul> <h3><strong>Requirements</strong></h3> <ul> <li>8+ years in storage engineering, managing distributed storage at multi-petabyte scale</li> <li>Proven track record building, deploying and operating high-performance storage for GPU/HPC clusters, for training and/or inference</li> <li>Experience scaling storage for inference: KV-cache offload or disaggregated KV storage, prefix caching, model-weight distribution, or similar latency-critical data paths</li> <li>Deep Kubernetes and cloud-native storage experience in production environments</li> <li>Strong coding skills in Go and Python with demonstrated ability to build production-grade systems and tooling</li> <li>BS/MS in Computer Science, Engineering, or equivalent practical experience</li> <li>History of technical leadership: designing systems that significantly improved performance, reliability (99.999%+ uptime), or cost efficiency</li> <li>Distributed Storage Systems: Deep expertise in either of Ceph, WekaFS, Lustre, Vast, GPFS, or similar parallel filesystems at multi-petabyte scale</li> <li>Object Storage: Production experience with S3, MinIO, Ceph, or R2 including performance optimization and cost management</li> <li>Kubernetes Storage: CSI drivers, StatefulSets, PersistentVolumes, storage operators, and custom controllers</li> <li>Storage optimization for GPU workloads, RDMA/InfiniBand networking, parallel filesystem optimization (TB/s aggregate cluster throughput - line saturation)</li> <li>Programming: Go and Python for automation, operators, and tooling; C/C++ or Rust for performance-critical data paths</li> <li>Infrastructure as Code: Terraform, Ansible, Helm, GitOps (ArgoCD)</li> <li>Linux Storage Stack: Advanced knowledge of filesystems (ext4, xfs), LVM, NVMe optimization, RAID configurations</li> <li>Observability: Prometheus, Grafana, Thanos architecture and operations</li> </ul> <h3><strong>Nice to Have Skills</strong></h3> <ul> <li>GPU Direct Storage (GDS), NVMe-oF, storage networking, RDMA implementations</li> <li>GPUDirect RDMA and zero-copy data movement (ibverbs, NIXL, UCX), and KV-cache transfer frameworks (e.g. LMCache, Mooncake, Dynamo)</li> <li>ML/AI storage patterns (model weights, checkpointing, dataset caching, KV-cache offload and prefix caching)</li> <li>Storage benchmarking and profiling tools (fio, iperf3, iostat, blktrace)</li> <li>Patents, publications (FAST, OSDI, SOSP, SC, ATC), or significant open-source contributions in storage or systems</li> </ul> <h3><strong>About Together AI</strong></h3> <p>Together AI is a research-driven artif