Data Engineer

Description:

VTRAC Consulting Corporation seeks a highly skilled Data Engineer to join their collaborative, remote team. The role focuses on designing, building, and maintaining large-scale data infrastructure critical to business operations. This includes developing data pipelines, data warehouses, and data lakes that enable efficient data storage, processing, and accessibility.

You will work closely with data architects to implement robust data models and architectures, collaborate with data scientists to deploy machine learning models, and ensure data quality and integrity by implementing validation and cleansing processes. This position offers a dynamic environment where staying updated on cutting-edge data technologies and trends is encouraged.

Key Responsibilities:

• Design, build, and maintain large-scale data systems and infrastructure

• Develop and implement data warehouses with platforms like Amazon Redshift, Google BigQuery, and Snowflake

• Create and maintain data pipelines using Apache Beam, Apache Spark, and AWS Glue

• Manage data lakes utilizing Apache Hadoop, Apache Spark, and Amazon S3

• Collaborate with data architects to design data models and architectures

• Work with data scientists to support machine learning model development and deployment

• Ensure high data quality by implementing data validation and cleansing procedures

• Coordinate with cross-functional teams to align data systems with business needs

• Keep abreast of emerging technologies and recommend new tools and practices for adoption

Qualifications:

• Minimum 5 years of experience in data engineering or a closely related discipline

• Proficient in programming languages including Python, Java, and Scala (5+ years)

• At least 3 years experience in data modeling and data architecture

• Strong hands-on experience with Apache Beam, Apache Spark, AWS Glue, Amazon Redshift, Google BigQuery, and Snowflake (3+ years)

• Proven expertise with data warehousing and data lake solutions

• Solid experience in data validation and cleansing techniques

• Excellent communication and collaboration skills

• Bachelor’s degree in Computer Science, Engineering, or related field

Preferred / Nice to Have:

• Experience in machine learning and data science workflows

• Familiarity with cloud platforms such as AWS, Google Cloud Platform, or Azure

• Knowledge of containerization technologies like Docker and Kubernetes

• Experience working within Agile development frameworks (Scrum, Kanban)

• Understanding of data governance and security best practices