Data Architect, Databricks

- Own end-to-end architecture for the Databricks-on-Azure lakehouse platform , including workspace design, Unity Catalog governance, storage topology, and cluster policies

- Define and scale medallion architecture standards (Bronze, Silver, Gold) across domains

- Establish platform standards and reusable patterns to accelerate Databricks adoption across teams

- Design and implement secure, compliant data access frameworks (RBAC, ABAC, row/column-level security)

- Lead CI/CD and DevOps patterns for Databricks deployments, ensuring audit-ready pipelines

- Optimize performance and cost across Delta Lake pipelines (partitioning, clustering, Photon, optimization strategies)

- Build reusable ingestion frameworks for structured and streaming data (API, Kafka, CDC, batch pipelines)

- Partner with Data Governance, Security, SRE, and Legal teams to align platform design with healthcare compliance requirements

- Provide architectural guidance on PHI/PII data handling and lifecycle management

- Evaluate and recommend platform tooling (build vs. buy decisions)

- Mentor engineers and architects; establish design reviews and documentation practices

- Bachelor's degree in Computer Science, Information Systems, or a related field, or relative experience, and typically requires 7+ years of experience in data engineering and data architecture

- Proven experience building and scaling Databricks platforms from the ground up , including workspace provisioning, Unity Catalog, and governance frameworks

- Strong hands-on expertise in Databricks and Azure (ADLS Gen2, networking, identity, security)

- Demonstrated ability to remain technically hands-on while leading architecture decisions

- Deep expertise in Delta Lake, including performance optimization and data lifecycle management

- Experience designing enterprise data platforms in regulated environments (e.g., PHI/PII handling)

- Strong understanding of data governance, access controls, and compliance frameworks

- Proven ability to establish standards, drive adoption, and influence multiple engineering teams

- Experience in healthcare or other regulated industries (HIPAA, PHI, compliance-driven environments)

- Experience scaling early-stage Databricks platforms and building enterprise-wide standards

- Familiarity with streaming and event-driven architectures (Kafka, Structured Streaming)

- Experience with Infrastructure-as-Code tools (Terraform, Azure Bicep)

- Knowledge of Databricks advanced features (Delta Live Tables, Unity Catalog federation, Delta Sharing)

- Experience supporting AI/ML data readiness and feature engineering pipelines