Lead Data Engineer

Description:

Mastercard is seeking a highly skilled and experienced Lead Software Engineer (Data Engineer) to join its Identity Graph & Ingestion team in Vancouver, BC. This hybrid, full-time role is integral to Mastercard’s mission to empower people and economies by enabling secure, scalable, and intelligent data processing systems. The selected individual will lead efforts to build and maintain large-scale data pipelines that power Mastercard’s Identity Verification and machine learning platforms.

The Lead Software Engineer will work within a globally distributed Scrum team to design, implement, and optimize big data systems, with a strong emphasis on Apache Spark and Scala. This position involves close collaboration with internal stakeholders across Data Science, Security, and Product, ensuring robust, efficient, and compliant data processing workflows that enable fraud prevention and risk mitigation at global scale.

Key Responsibilities:

• Architect, develop, and maintain large-scale data processing systems using Apache Spark, Scala, SQL, and related tools.

• Lead the creation of technical design documentation, diagrams, and review sessions with cross-functional stakeholders.

• Drive improvements in performance and capability of existing data pipelines and data transformation systems.

• Collaborate with geographically diverse teams across North America and Europe as part of an agile Scrum team.

• Participate in backlog prioritization, project scoping, and stakeholder communication to ensure alignment with business goals.

• Serve as a mentor and technical leader, guiding software engineers, conducting interviews, and elevating engineering standards.

• Work closely with teams responsible for data governance, privacy, and machine learning to meet evolving compliance and performance standards.

• Help evolve the engineering culture by identifying emerging technologies and streamlining processes.

Required Skills & Qualifications:

• Solid foundation in computer science fundamentals—data structures, algorithms, and software design principles.

• Expert-level proficiency in Apache Spark, Scala, and SQL.

• Proven experience using Databricks and Airflow for orchestrating ETL pipelines and data transformations.

• Deep understanding of cloud infrastructure, ideally AWS, and experience with REST APIs and third-party data integration.

• Exposure to machine learning workflows and understanding of how to support Data Science teams with robust data platforms.

• Demonstrated ability to communicate complex technical topics clearly to both technical and non-technical stakeholders.

• Deep understanding of Agile methodologies and ability to drive cross-team collaboration.

• Bachelor’s degree in Computer Science, Software Engineering, or a related field; equivalent practical experience also considered.

Why Join Mastercard?

• Be part of a global leader in digital payments, powering commerce in over 200 countries.

• Work on cutting-edge data engineering and ML-enabling platforms that have real-world financial impact.

• Collaborate within a diverse, global team in a company that values inclusion, merit, and continuous innovation.

• Enjoy the support of a mission-driven company that invests in professional development, career growth, and technical excellence.