Senior Data Engineer

Description:

Millennium Software and Staffing Inc. is seeking a highly skilled Senior Data Engineer to work on a project with Capgemini in Montreal, Canada. This role is a contract-based position with an initial duration of 4 months, with the potential to transition to a Contract-to-Hire after 90 days. The selected candidate will play a critical role in designing, building, and maintaining large-scale data pipelines and distributed systems, ensuring the delivery of high-performance and scalable big data solutions.

Key Responsibilities include:

• Leading and managing large-scale data processing initiatives with strong focus on instrumentation and optimization.

• Writing high-quality, efficient, and maintainable code in Java, Python, or Scala.

• Implementing engineering best practices in big data ecosystems.

• Designing and developing batch and real-time data pipelines for production environments.

• Working with relational and non-relational databases such as DB2 and MongoDB.

• Handling multiple data formats including Parquet, CSV, JSON, XML, and relational datasets.

• Utilizing big data tools and technologies such as Databricks, Elasticsearch, Snowflake, Spark, Kafka, Hadoop, Iceberg, and Airflow.

• Applying strong knowledge of data modeling, relational and columnar databases.

• Leveraging software engineering principles such as OOP, functional programming, design patterns, and code quality practices.

• Collaborating with cross-functional teams and communicating technical concepts to non-technical stakeholders effectively.

Required Skills & Qualifications:

• Minimum of 4+ years of proven experience in data engineering and large-scale distributed systems.

• Strong coding expertise in Java, Python, or Scala.

• Hands-on experience with big data products and tools such as Databricks, Elasticsearch, Snowflake, Spark, Kafka, Hadoop, Iceberg, and Airflow.

• Experience with multiple databases, both relational and non-relational (DB2, MongoDB).

• Familiarity with multiple data formats and data modeling approaches.

• Strong communication and collaboration skills to work effectively across teams.

• Ability to manage projects independently while maintaining high-quality deliverables.

Additional Notes:

• Selected candidates must be willing to meet Capgemini team members in person at the closest client office before starting the assignment.

• For non-local candidates, travel expenses will be covered by Capgemini.