<h3><strong>About the Role</strong></h3> <p>In this role, you will design and deliver multi-petabyte storage systems purpose-built for the world’s largest AI training and inference workloads. You’ll architect high-performance parallel filesystems and object stores, evaluate and integrate cutting-edge technologies such as WekaFS, Ceph, and Lustre, and drive aggressive cost optimization-routinely achieving 30-50% savings through intelligent tiering, lifecycle policies, capacity forecasting, and right-sizing. </p> <p>You will also build Kubernetes-native storage operators and self-service platforms that provide automated provisioning, strict multi-tenancy, performance isolation, and quota enforcement at cluster scale. Day-to-day, you’ll optimize end-to-end data paths for 10-50 GB/s per node, design multi-tier caching architectures, implement intelligent prefetching and model-weight distribution, and tune parallel filesystems for AI workloads. </p> <p>Hybrid Working 2 days a week at our offices in Amsterdam</p> <p><strong>Responsibilities</strong></p> <ul> <li>Design multi-petabyte AI/ML storage systems; integrate WekaFS, Ceph, etc.; lead capacity planning and cost optimization (30-50% savings via tiering, lifecycle policies, right-sizing).</li> <li>Design/optimize RDMA, InfiniBand, 400GbE networks; tune for max throughput/min latency; implement NVMe-oF/iSCSI; troubleshoot bottlenecks; optimize TCP/IP for storage.</li> <li>Build Kubernetes storage operators/controllers; enable automated provisioning, self-service abstractions, multi-tenant isolation, quotas; create reusable Helm/Terraform patterns.</li> <li>Deliver 10-50 GB/s per GPU node; optimize caching (weights/datasets/checkpoints), parallel filesystems, and data paths; troubleshoot with profiling tools; scale to thousands of nodes.</li> <li>Build multi-tier caches (local NVMe, distributed, object); optimize data locality and model-weight distribution; implement smart prefetching/eviction.</li> <li>Implement monitoring, alerting, SLOs; design DR/backups with runbooks; run chaos engineering; ensure 99.9%+ uptime via proactive/automated remediation.</li> <li>Partner with ML/SRE teams; mentor on storage best practices; contribute to open-source; write docs, postmortems, and public learnings.</li> </ul> <h3><strong>Requirements</strong></h3> <ul> <li>8+ years in storage engineering with 3+ years managing distributed storage at multi-petabyte scale</li> </ul> <ul> <li>Proven track record deploying and operating high-performance storage for GPU/HPC clusters</li> </ul> <ul> <li>Deep Kubernetes and cloud-native storage experience in production environments</li> </ul> <ul> <li>Strong coding skills in Go and Python with demonstrated ability to build production-grade tools</li> </ul> <ul> <li>BS/MS in Computer Science, Engineering, or equivalent practical experience</li> </ul> <ul> <li>History of technical leadership: designing systems that significantly improved performance (>3x), reliability (99.9%+ uptime), or cost efficiency</li> </ul> <ul> <li>Distributed Storage Systems: Deep expertise in WekaFS, Lustre, GPFS, BeeGFS, or similar parallel filesystems at multi-petabyte scale</li> </ul> <ul> <li>Object Storage: Production experience with S3, MinIO, Ceph, or R2 including performance optimization and cost management</li> </ul> <ul> <li>Kubernetes Storage: CSI drivers, StatefulSets, PersistentVolumes, storage operators, and custom controllers</li> </ul> <ul> <li>Storage optimization for GPU workloads, RDMA/InfiniBand networking, parallel filesystem optimization (100+ GB/s aggregate cluster throughput)</li> </ul> <ul> <li>Programming: Go and Python for automation, operators, and tooling</li> </ul> <ul> <li>Infrastructure as Code: Terraform, Ansible, Helm, GitOps (ArgoCD)</li> </ul> <ul> <li>Linux Storage Stack: Advanced knowledge of filesystems (ext4, xfs), LVM, NVMe optimization, RAID configurations</li> </ul> <ul> <li>Observability: Prometheus, Grafana, Thanos architecture and operations</li> </ul> <h3><strong>Nice to Have Skills</strong></h3> <ul> <li>GPU Direct Storage (GDS), NVMe-oF, storage networking (100GbE/400GbE)</li> </ul> <ul> <li>ML/AI storage patterns (model weights, checkpointing, dataset caching)</li> </ul> <ul> <li>Kubernetes operator development (controller-runtime, kubebuilder)</li> </ul> <ul> <li>Storage snapshots, cloning, and thin provisioning</li> </ul> <ul> <li>Backup and disaster recovery (Velero, Restic, cross-region replication)</li> </ul> <ul> <li>Storage encryption (at-rest and in-transit), security and compliance</li> <li>Storage benchmarking and profiling tools (fio, iperf3, iostat, blktrace)</li> </ul> <p><strong>About Together AI</strong></p> <p>Together AI is a research-driven artificial intelligence company. We believe open and transparent AI systems will drive innovation and create the best outcomes for society, and together we are on a mission to significantly lower the cost of modern AI systems by co-designing software, hardware, algorithms, and models. We have contributed to leading open-source research, models, and datasets to advance the frontier of AI, and our team has been behind technological advancement such as FlashAttention, Hyena, FlexGen, and RedPajama