Machine Learning Engineer

About The Company

Red Hat is a globally recognized leader in enterprise open source software solutions, renowned for its community-powered approach to delivering high-performing Linux, cloud, container, and Kubernetes technologies. With a presence in over 40 countries, Red Hat fosters a flexible, inclusive, and innovative work environment that encourages employees to bring their best ideas forward regardless of their role or tenure. The company is committed to open source principles of transparency, collaboration, and inclusion, creating a culture where diverse perspectives are valued and innovation thrives. Red Hat’s dedication to making a positive impact in the tech industry is reflected in its extensive product portfolio and its mission to drive technological advancement through open source solutions.

About The Role

Red Hat is seeking a highly skilled Principal Machine Learning Engineer specializing in model optimization algorithms to join our AI Inference team. In this pivotal role, you will work closely with product and research teams to develop state-of-the-art deep learning software, focusing on the optimization and deployment of large language models (LLMs). Your expertise will contribute to designing, developing, and testing inference optimization algorithms, including quantization, sparsification, and parallelization techniques, to enhance AI model performance across diverse hardware platforms. You will play a vital role in creating and managing inference serving deployment pipelines, benchmarking, profiling, and evaluating various model acceleration approaches. Additionally, you will stay abreast of the latest advancements in open source LLM architectures, hardware features, and inference techniques, ensuring Red Hat remains at the forefront of AI innovation. This role offers an exciting opportunity to solve complex technical challenges, mentor fellow engineers, and contribute to impactful open source projects such as vLLM, llm-d, and LLM‑compressor.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or a related field; PhD is a strong plus
• Proven experience in machine learning and deep learning fundamentals
• Extensive experience with LLM inference optimization, including quantization, sparsification, and parallelization techniques
• Proficiency with tensor math libraries such as PyTorch and NumPy
• Strong programming skills in Python, with a track record of implementing ML solutions
• Knowledge of mathematical software, especially linear algebra, gradients, probability, and graph theory
• Ability to develop innovative research ideas and algorithms
• Excellent communication skills for collaboration with technical and non-technical team members
• Experience with hardware architecture, including CPU and GPU features, to optimize inference performance

Responsibilities

• Design, develop, and test inference optimization algorithms within the vLLM and related projects
• Create and manage scalable inference serving deployment pipelines for enterprise applications
• Benchmark, profile, and evaluate different parallelization, quantization, and sparsification approaches to optimize model performance on various hardware platforms
• Participate in technical design discussions, providing innovative solutions to complex problems
• Stay current with the latest advancements in open source LLM architectures, inference techniques, and hardware features
• Conduct thorough code reviews, ensuring code quality and best practices
• Mentor and guide junior engineers, fostering a culture of continuous learning and innovation
• Collaborate with internal teams and external open source contributors to enhance project development and community engagement

Benefits

• Comprehensive medical, dental, and vision coverage
• Flexible Spending Account (FSA) for healthcare and dependent care expenses
• Health Savings Account (HSA) for high deductible medical plans
• Retirement 401(k) plan with employer matching contributions
• Paid time off and holidays to support work-life balance
• Paid parental leave for new parents
• Leave benefits including disability, family medical leave, and military leave
• Additional perks such as employee stock purchase plans, tuition reimbursement, transportation expense accounts, and employee assistance programs

Equal Opportunity

Red Hat is an equal opportunity employer committed to creating an inclusive environment for all employees. We do not discriminate based on race, color, religion, sex, sexual orientation, gender identity, national origin, ancestry, citizenship, age, veteran status, genetic information, physical or mental disability, medical condition, marital status, or any other protected characteristic under applicable law.