Lead Data Engineer

Description:

E-Solutions is seeking a highly skilled and experienced Lead Data Engineer to join their dynamic team in a fully remote position across Canada. The ideal candidate will be a seasoned data engineering professional with strong technical skills, leadership capabilities, and a passion for building scalable data systems. You will be responsible for designing, developing, and maintaining complex data pipelines and architectures while leading a team of data engineers.

This role requires hands-on expertise in SQL, Python, AWS services, and Snowflake, along with a solid foundation in data warehousing concepts, data modeling, and large-scale data handling. The ideal candidate should be a proactive problem-solver who thrives in a fast-paced and evolving environment, especially one that aligns with start-up culture dynamics.

Key Responsibilities:

• Design, develop, and optimize robust data pipelines and workflows for ingesting, processing, and transforming large volumes of data

• Lead the end-to-end architecture and implementation of scalable data platforms using AWS, Snowflake, SQL, and Python

• Apply advanced data engineering practices, including data modeling, ETL development, and data governance standards

• Manage code repositories using GIT and integrate workflows using CI/CD pipelines

• Ensure data quality and consistency across multiple sources and destinations

• Serve as a technical mentor and leader for junior team members, providing guidance, best practices, and reviews

• Collaborate closely with cross-functional teams including data scientists, analysts, and business stakeholders

• Adapt quickly to shifting priorities and requirements, maintaining a customer-focused mindset

• Ensure proper documentation of processes, code, and data pipelines

• Contribute to architecture decisions and long-term planning of data infrastructure

Required Skills and Qualifications:

• 8 to 12 years of overall experience with a strong focus on data engineering

• Hands-on expertise in SQL, Python, AWS services, and Snowflake

• Proficiency in data warehousing, data modeling, and working with large datasets

• Solid knowledge of Python libraries and efficient data structure implementations

• Experience working with version control tools such as GIT and implementing CI/CD pipelines

• Demonstrated experience in building and managing complex data pipelines

• Proven ability to manage time, prioritize multiple projects, and work independently in a remote environment

• Excellent communication skills and a collaborative mindset

• Must be a self-starter and a problem-solver, comfortable working in high-energy, fast-paced environments

• Leadership experience with a demonstrated ability to guide and support team members technically

Nice to Have:

• Knowledge and experience with Apache Airflow and DBT (Data Build Tool)