Description:
Senior Data Engineer
Location: Toronto, Ontario
Key Responsibilities:
- Lead the migration of enterprise data platforms from Netezza and SSIS to AWS-native data architectures using Glue, PySpark, S3, Redshift, and Lambda.
- Design, develop, and validate scalable ETL/ELT migration patterns, ensuring functional parity and performance optimization between legacy and target platforms.
- Provide hands-on architecture leadership with active involvement in data engineering, coding, troubleshooting, and performance tuning across AWS services.
- Own end-to-end data migration and reconciliation strategy, including source-to-target validation, data completeness checks, and exception management.
- Design and implement robust data quality, auditability, lineage, and regulatory reporting controls to support enterprise risk and compliance requirements.
- Optimize Amazon Redshift data models, schema design, workload management, and query performance for large-scale reporting and batch processing workloads.
- Develop and enhance AWS Glue and PySpark pipelines for data ingestion, transformation, enrichment, and orchestration across multiple source systems.
- Lead the modernization of legacy Netezza SQL, nz_unload processes, and SSIS workflows, ensuring efficient conversion to cloud-native solutions.
- Define and enforce PII protection, encryption standards, data governance policies, and secure data handling practices throughout the migration lifecycle.
- Collaborate with risk, analytics, and data science teams to enable data consumption for risk models, scoring engines, model inference pipelines, and reporting platforms.
- Implement and support CI/CD, deployment automation, and workflow orchestration using Airflow, Glue Workflows, or equivalent tools to ensure reliable production operations.
- Provide technical leadership, code reviews, architecture governance, and best-practice guidance while ensuring successful delivery of large-scale Netezza-to-AWS migration initiatives.