GCP Senior Data Engineer Grade

GCP Senior Data Engineer Grade

Work Location: Canada (Remote)

Job Summary

The GCP Senior Data Engineer will lead a team of 10 data engineers in designing, developing, and optimizing scalable data pipelines and cloud-based data infrastructure on Google Cloud Platform (GCP). This is a leadership-focused role requiring experience managing onshore and offshore teams, establishing engineering standards, and delivering reliable data solutions to support analytics and machine learning use cases.

Key Responsibilities

• Pipeline Development: Build and optimize ETL/ELT pipelines using Dataflow (Apache Beam), Dataproc (Spark), Pub/Sub, and Cloud Functions.
• Data Infrastructure: Architect and manage BigQuery, Cloud Storage, and Data Fusion for efficient data ingestion, transformation, and storage.
• Team Leadership: Lead a team of 10 engineers, conduct code reviews, and establish data engineering standards.
• DevOps & Monitoring: Implement CI/CD practices, infrastructure-as-code, and monitoring using Cloud Monitoring and Cloud Logging.
• Performance & Cost Optimization: Improve pipeline performance and query efficiency while managing GCP costs through techniques such as slot reservations and partitioning.
• Collaboration: Work with data architects, data scientists, and analysts to deliver self-service data platforms and support advanced analytics and machine learning use cases.
• Data Quality & Governance: Implement data validation, lineage, and metadata management using tools such as Data Catalog and dbt.

Required Qualifications

• Leadership: Proven experience leading data engineering teams across onshore and offshore locations. Candidates must have leadership experience; this is not an individual contributor-only role.
• Programming & Distributed Systems: Proficiency in SQL, Python, Java/Scala, and distributed processing frameworks such as Spark and Beam.
• GCP Expertise: Hands-on experience with BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Composer (Airflow), and Terraform.
• DevOps: Familiarity with CI/CD, Git, and infrastructure-as-code practices.
• Data Engineering: Experience building and optimizing data pipelines, managing cloud data infrastructure, and implementing data quality and monitoring practices.

Nice to Have

• GCP certifications, such as Google Cloud Professional Data Engineer.
• Experience with real-time data processing.
• Familiarity with Vertex AI and machine learning workflows.
• Experience working in multi-cloud environments.