Senior / Staff Machine Learning Engineer, ACAI Training & Inference Platform
Apple · Cupertino
- Senior / Staff Machine Learning Engineer, ACAI Training & Inference Platform
- Cupertino, California, United States
- Software and Services
- Submit Resume
- Summary
- Posted:
- Oct 16, 2026
- Role Number:
- 200689942-0836
- The ACAI Training & Inference Platform team is seeking a Senior / Staff Software Engineer with strong distributed systems expertise and deep experience in AI training and inference. As part of Apple Cloud AI (ACAI) within Apple’s Data Platform (ADP), you will design and build core capabilities for large-scale model training, post-training, fine-tuning, reinforcement learning, and high-performance inference.
- You will develop scalable AI execution frameworks, developer-facing APIs, and runtime optimizations that make it easier to train, adapt, and serve advanced models efficiently across Apple’s AI ecosystem. You will partner closely with AI researchers and engineers to translate evolving model architectures and workload requirements into reliable, high-performance training and inference solutions. This role is ideal for an engineer who combines deep distributed-systems expertise with hands-on knowledge of modern LLM training, post-training, and inference optimization.
- Description
- ADP brings together the Apple Cloud Data Platform (ACDP), Apple Cloud AI (ACAI), and Apple Governance Platform (AGP) to provide foundational capabilities for data, analytics, AI, and governance at Apple scale. Within ACAI, the Training & Inference Platform team enables scalable model development and production inference across Apple’s AI ecosystem.
- As a Senior / Staff Software Engineer, you will build capabilities spanning the model lifecycle, from distributed training and post-training to optimized inference and serving.
- On the training side, you will develop frameworks and APIs for fine-tuning, reinforcement learning, and post-training workflows, enabling AI teams to efficiently adapt foundation models using techniques such as supervised fine-tuning (SFT), preference optimization, and reinforcement learning from feedback or verifiable rewards. You will simplify the execution of complex training pipelines, improve experimentation and iteration speed, and enable reusable, production-ready training workflows.
- On the inference side, you will drive deep runtime and LLM-serving optimizations to improve throughput, reduce latency, and lower the cost of serving large models. You will optimize execution across heterogeneous hardware and model architectures, leveraging techniques such as continuous batching, KV-cache optimization, quantization, speculative decoding, disaggregated prefill and decode, and efficient distributed inference.
- Across both areas, you will improve the performance, scalability, reliability, and developer experience of AI training and inference workloads. You will work closely with AI practitioners to identify bottlenecks across model execution, scheduling, communication, and resource utilization, translating these insights into end-to-end improvements in model development and serving efficiency.
- Responsibilities
- Design and build scalable distributed AI execution frameworks for large-scale model training, fine-tuning, post-training, reinforcement learning, and inference using Ray and related technologies. Develop high-level APIs, SDKs, and reusable workflows that simplify model training and adaptation, including supervised fine-tuning, preference optimization, and reinforcement learning workflows.
- Enable end-to-end post-training workflows, from dataset preparation and experiment orchestration through distributed execution, evaluation, and iterative model improvement.
- Optimize LLM inference runtimes for cost, latency, throughput, and resource efficiency across diverse model architectures and hardware configurations.
- Drive deep runtime optimizations spanning GPU execution, memory management, kernel utilization, communication, batching, KV-cache management, and distributed inference.
- Evaluate and integrate modern inference and training engines, including vLLM, SGLang, and other relevant frameworks, to deliver efficient and extensible AI execution capabilities.
- Design and optimize distributed execution strategies for large models, including parallelism, workload placement, resource allocation, and communication efficiency.
- Improve training and inference reliability through fault tolerance, checkpointing, workload recovery, observability, and performance diagnostics.
- Partner with AI researchers, model developers, and product teams to translate model and workload requirements into reusable capabilities that accelerate model development and production adoption.Establish performance benchmarks and optimization strategies that quantify improvements in training efficiency, post-training iteration speed, inference latency, throughput, and cost per token.
- Contribute architectural improvements to Ray and related open-source AI systems where appropriate.
- Minimum Qualifications
- 6+ years of experience building distributed systems, AI platforms, model execution frameworks, or high-performance software systems.
- Strong hands-on experience in at least one of the following areas: large-scale AI training, LLM post-training and reinforcement learning, or LLM inference and serving optimization.
- Proficiency in Python and experience with a systems-level language such as C++, Rust, Go, or Java.
- Experience with PyTorch and modern foundation model architectures, including the execution and performance requirements of large language models.
- Understanding of distributed execution and parallelism strategies for large models, including data, tensor, pipeline, or expert parallelism.
- Experience profiling and optimizing AI workloads across GPUs or other accelerators, with the ability to identify and resolve performance bottlenecks across compute, memory, and communication.
- Strong understanding of at least one of the following: distributed training and post-training workflows, reinforcement learning systems,