Sr. Staff Machine Learning Engineer - Data Lake, Anomaly Detection

We are looking for a Sr. Staff Software Engineer to join our team. This is a hybrid, based in San Jose, CA 3 days a week role, reporting to the Sr.

Manager, Software Engineering in the Zscaler Digital Experience (Core Intelligence and Data) department. You will join the team responsible for building the world’s largest cloud security platform, helping us enhance services and increase our global footprint. You will play a pivotal role in enabling organizations worldwide to harness speed and agility through a cloud-first strategy, leveraging our multitenant architecture that serves over 15 million users.

Own agentic troubleshooting framework, framing high-impact use cases, designing workflows and playbooks, and building processes for all products

Evaluate and integrate state-of-the-art GenAI advances to deliver reliable and cost-efficient production features, utilizing LLMs, various machine learning models, data processing, fine-tuning, and inference optimization

Work with the world class cloud platform and data lakes for feature exploration and generation

Handle volume data with the real time pipeline for data processing and aggregation

Design, implement, and operate scalable production systems, specifically focusing on microservices, data pipelines, orchestration, and caching

You thrive in ambiguity. You're comfortable building the path as you walk it. You thrive in a dynamic environment, seeing ambiguity not as a hindrance, but as the raw material to build something meaningful.

You act like an owner. Your passion for the mission fuels your bias for action. You operate with integrity because you genuinely care about the outcome.

True ownership involves leveraging dynamic range: the ability to navigate seamlessly between high-level strategy and hands-on execution.

You are a problem-solver. You love running towards the challenges because you are laser-focused on finding the solution, knowing that solving the hard problems delivers the biggest impact.

You are a high-trust collaborator. You are ambitious for the team, not just yourself. You embrace our challenge culture by giving and receiving ongoing feedback—knowing that candor delivered with clarity and respect is the truest form of teamwork and the fastest way to earn trust.

You are a learner. You have a true growth mindset and are obsessed with your own development, actively seeking feedback to become a better partner and a stronger teammate. You love what you do and you do it with purpose.

Foundational understanding of AI/ML technologies and experience leveraging, securing, or positioning AI-driven solutions to optimize outcomes within your functional domain

BS in Computer Science with 8+ years of experience, or MS/PhD with 5+ years of experience solving real-world problems using AI/ML and distributed systems

Proficiency in programming, data structures, algorithms, and machine learning, with exceptional problem-solving skills driven by first-principles thinking

Hand-on experience with AI modeling, including feature generation, prompt engineering, evaluations and productionization

Experience in the full lifecycle of ML models (building, deployment, monitoring, optimization) alongside expertise in designing and operating distributed microservices using Kubernetes and Docker in Python, Go, or Java

Experience designing and scaling autonomous AI agents, agentic orchestration workflows, and advanced large language model (LLM) tooling to automate complex cloud infrastructure troubleshooting or incident response

Experience fine-tuning and deploying proprietary SLMs/LLMs at scale, with a focus on optimizing latency, cost, safety, and evaluations

Experience delivering production-ready AI systems, including expertise in anomaly detection, event correlation, incident investigation, and resilient systems with well-defined service-level objectives