Senior Machine Learning Test Engineer

As a Senior Machine Learning Test Engineer in the Research Enablement team, you will work side-by-side with researchers, Machine Learning developers and developers, and software developers and developers to define and uphold quality standards for ML systems. You are a quality-focused developer who is passionate about reliable, repeatable evaluation of ML models and data.

Your skills span test strategy, automation, and a little MLOps, with a strong software engineering base. You are excited to collaborate across research and product to ship ML capabilities with clear quality gates. You are comfortable working at the intersection of research and product and are competent in using Autodesk CAD software.

Reporting Structure: You will report to an Engineering Manager in Research Enablement.

We are a global team, located in London, San Francisco, Toronto, and remotely. Autodesk is a hybrid-first company, allowing workers to work remotly, in an office, or a mix of both.

- Define ML quality strategy and acceptance criteria across data, model, and system levels

- Design and maintain model evaluation suites, metrics, and test datasets

- Evaluating CAD RL model outputs for geometric validity or policy stability

- Defining structured rubrics that translate qualitative findings into measurable evaluation gates

- Automate ML QA workflows using Python and CI/CD (e.g., GitHub Actions, Jenkins)

- Create and maintain test harnesses for ML services and APIs

- Mentor teams on ML QA best practices and consistent evaluation standards

- Build quality gates for training and deployment pipelines (e.g., regression checks, drift detection)

- Contribute to multi-team projects and codebases, ensuring code quality and consistency

- Participate in code reviews and provide constructive feedback to peers

- Document and present findings and ideas across the company

- Bachelor's degree in Computer Science, Engineering, or equivalent experience

- 7+ years of professional experience in software engineering or QA for ML/AI systems

- Strong programming skills in Python, with experience in test automation

- Experience designing QA frameworks or platforms used by multiple teams

- Understanding of software architecture and design patterns

- Experience with data validation tooling (e.g., Great Expectations) or labeling workflows

- Familiarity with ML frameworks (e.g., PyTorch, TensorFlow)

- Experience with data pipelines and orchestration tools (e.g., Airflow, Metaflow)

- Familiarity with MLOps practices (model monitoring, drift, deployment checks)

- Experience with ML evaluation methods, metrics, and benchmarking

- Passion for learning new technologies and improving existing systems

- Experience with cloud providers (e.g., AWS, Azure, Google Cloud Platform)

- Knowledge of experiment tracking tools (e.g., Comet, MLflow, Weights & Biases)

- You demonstrate initiative to provide solutions and to learn and develop new technologies

- Comfortable building QA systems from scratch and writing maintainable automation

- You enjoy learning and collaborating across global locations

- You are comfortable working in newly forming ambiguous areas

- You are comfortable building scalable and maintainable systems that will be relied on by others

Are you an existing contractor or consultant with Autodesk? Please search for open jobs and apply internally (not on this external site). If you have any questions or require support, contact Autodesk Careers .