Senior Software Engineer, Quantized Inference

- Implement quantized and sparse recipes in inference engines (vLLM, TRT-LLM, SGLang)

- Own model export pipelines (ModelOpt, Megatron-LM HuggingFace), ensuring quantized checkpoints serialize correctly for downstream serving

- Build prototypes and benchmarking harnesses to evaluate recipe throughput/interactivity before full optimization

- Develop data analysis tooling and visualizations for numerics debugging

- Improve developer productivity across the team: CI, build systems, training infrastructure, pipeline friction

- Participate in code reviews and incorporate feedback

- Proficient in Python; familiarity with C++

- Strong software engineering fundamentals: concise, well-tested code; fluent with AI-assisted tooling

- Experience with ML accelerators with a basic understanding of how certain ML layers affect execution time

- Familiarity with PyTorch internals (custom ops, autograd, export) or equivalent framework

- Experience reading, modifying, or contributing to a large open-source codebase

- MS/PhD in Computer Science or related field, or equivalent experience.

- 4+ years in a relevant software engineering role

- Demonstrated ability to move fast with ambiguous requirements, with strong written and verbal communication

- Experience contributing to inference serving frameworks (vLLM, TRT-LLM, SGLang) or Triton kernel development

- Track record of debugging numerical issues across mixed-precision boundaries

- Deep experience with model compression techniques: PTQ, QAT, structured/unstructured sparsity

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 152,000 USD - 241,500 USD for Level 3, and 184,000 USD - 287,500 USD for Level 4.

You will also be eligible for equity and benefits .

Applications for this job will be accepted at least until July 26, 2026.

This posting is for an existing vacancy.

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.