Description:
As ATB’s next Data Developer you will work closely with data developers and data analysts to build, maintain and upgrade ATB’s enterprise data platform. You will be responsible for the design, development and operational stability of our cloud-based data infrastructure, ensuring seamless data processing and analysis. As the ideal candidate, you will have experience in the banking or financial services industry and be comfortable working with various stakeholders. You are passionate about efficiency, automation, and innovation, and have a strong aptitude for Platform as a Service (PaaS) solutions. This role offers the exciting opportunity to work with a wide range of technologies and contribute to the continuous improvement of our data platform.
Accountabilities
- Design, develop and maintain data pipelines and infrastructure using Google Cloud Platform (GCP) services such as BigQuery, Dataplex, Dataflow, Cloud Functions, Airflow, Cloud Logging, Vertex, etc. and integrations with other technologies such as Splunk, Collibra and other on-prem and cloud based platforms
- Implement and manage Infrastructure as Code (IaC) using Terraform / OpenTofu to automate the provisioning and management of GCP resources
- Build and optimize Gitlab CI/CD pipelines to automate code deployment and ensure the continuous integration and delivery of platform enhancements
- Implement observability and monitoring capabilities for the platform and data pipelines, optimizing performance and cost
- Collaborate with data developers and analysts to understand their needs and translate them into robust and scalable solutions.
- Troubleshoot and resolve operational issues related to the data platform, ensuring its high availability and performance.
- Maintain and enhance existing processes, APIs, and integrations to improve platform efficiency and functionality.
- Stay abreast of industry best practices and emerging technologies in cloud computing, data engineering, data governance and DevOps.
- Strong communication skills to work effectively with cross-functional teams and stakeholders, ensuring alignment and collaboration
- Participate in on-call rotation to provide 24/7 support for critical platform components.
Skills, Experience & Requirements
- Technical Skills:
- Proficiency with GCP services such as BigQuery, Dataplex, Dataflow, Composer Airflow, PubSub, Cloud Storage, Cloud Functions, Dataproc and Kubernetes Engine.
- Experience leveraging AI and ML capabilities to proactively monitor the health of the platform and data pipelines, and detect potential issues before they impact operations.
- In-depth understanding of containerization technologies (e.g. Kubernetes, Docker), including orchestration, deployment, and management of containerized applications
- Strong proficiency in Python and SQL, complemented by hands-on experience in backend programming and API development, showcasing the ability to build robust and scalable data solutions
- Extensive hands-on experience with Infrastructure as Code (IaC) tools, particularly Terraform/OpenTofu, for efficiently managing and provisioning cloud infrastructure
- Demonstrated expertise in source code management (e.g., Git) and CI/CD technologies (e.g., GitLab, Jenkins) for building, maintaining, and optimizing automated pipelines, ensuring efficient and reliable, continuous delivery
- Experience with Splunk, Cloud Monitoring, Cloud Logging or similar log management and monitoring tools, enabling effective troubleshooting and proactive issue resolution.
- Familiarity with data security best practices and compliance standards.
- Experience:
- 5+ years of experience in a data engineering or platform development role.
- 3+ years of experience in the banking or financial services industry
- Proven track record of designing, building, and maintaining cloud-based data platforms
- Experience working with large datasets and data-intensive applications
- Experience with Collibra development and support would be an asset
- Accreditation:
- Recognized post-secondary Information Technology university/ college/ technical school or industry certification
- Working knowledge in one or more general purpose programming languages, including but not limited to: Java, C/C++, C, Objective C, Python, JavaScript, etc.
- Google Cloud Certification in a relevant field (Cloud Engineer, Data Engineer, Cloud Architect) would be an asset