Data Engineer

Description:

Role Responsibilities

If you are a data engineer who enjoys working with a small business team, and you want to make a difference, this job might be for you!

Reporting to the Manager – Data Operations, the Data Engineer builds enterprise-wide data infrastructure to enable Department digital services and business intelligence. This includes developing and optimizing ETL pipelines, managing data warehouse and orchestration platforms, and cloud storage to reliably move data from multiple source systems, assessing existing infrastructure, and developing a roadmap for infrastructure improvement.

The Data Enginer also works with colleagues and management to improve Advanced Education's data management capacity and maturity by teaching skills, establishing processes and standards, and advising on best practices.

The Data Engineer possesses a combination of skills, from technical acumen to well-developed leadership and communication skills. This skill combination is essential for an emerging area of maturity in Advanced Education. The Data Engineer retains an open-minded, collaborative, and listening attitude to help lead this area of change.

These are your goals:
Cloud Data infrastructure is developed to meet AE's operational and analytic needs.

• Identify and document requirements for new data infrastructure, working with data scientists, data architets, data modellers, and business users to ensure outcomes are clear and shared.
• Build data pipelines from one or more sources to cloud storage and data warehouse in a customized way, including orchestration, data masking, testing, reviewing, troubleshooting, scheduling, and documenting.
• Organize structured and unstructured data into analyst-friendly tables in the centralized data cloud.
• Develop messaging and alerting systems so that failure and error reporting happen in an automated, trusted way.
• Maintain the tooling libraries that enable automation and quality of the data warehouse: custom operators and scripts for orchestration, continuous integration and deployment, code deployment process, and linting; maintaining source to target mapping documents, metadata and data dictionary.
• Enable analysts and modellers to work effectively by supporting the tools they use, including Data Build Tools (DBT) for data transformations inside the centralized data cloud.