Skip to Content

GCP Data Engineering Lead

--Canada (Remote)--

Apply Now!

GCP Data Engineering Lead

Location: Canada (Remote)

Duration: 4 Months

Employment Type: Contract

Job Summary

We are seeking a highly experienced GCP Data Engineering Lead to serve as the senior technical and delivery operations lead for a large-scale data engineering engagement. This role combines hands-on GCP pipeline architecture expertise with QA strategy ownership and Agile delivery management responsibilities.

The ideal candidate will be responsible for ensuring technical quality, sprint execution, delivery governance, and successful knowledge transfer while leading cross-functional and offshore engineering teams.

Key Responsibilities

Technical Leadership & Architecture

  • Lead pipeline architecture design for Netscout and Sandvine ingestion workflows including:
    • Source extraction
    • GCS landing
    • Dataflow transformations
    • BigQuery loading across Bronze/Silver/Gold layers
  • Design and implement Medallion Architecture data flows and layer transition rules
  • Define data quality check specifications for each Medallion layer
  • Lead data reconciliation strategy including UDP vs. Datahub parity validation
  • Provide hands-on support for complex pipeline transformation logic
  • Coordinate technical decisions, environment readiness, and access requirements with client Data Engineering teams

QA & Data Quality Ownership

  • Define and own end-to-end QA strategy including:
    • Test types
    • Test coverage
    • Tooling
    • Defect management
    • Quality gates
  • Architect reusable and parameterized testing templates for:
    • Batch pipelines
    • Streaming pipelines
    • Schema validation
    • Transformation validation
    • Business logic testing
    • Governance compliance checks
  • Review and quality assure all pipeline and QA deliverables produced by offshore Data Engineers

Agile Delivery & Team Management

  • Lead sprint planning, sprint reviews, and retrospectives using Agile methodology
  • Maintain RAID logs (Risks, Assumptions, Issues, Dependencies) and drive mitigation actions
  • Produce weekly project status reports and delivery trackers
  • Manage offshore Data Engineers and Observability Engineers including:
    • Task allocation
    • Delivery oversight
    • Quality management
  • Support mentoring and knowledge transfer initiatives

Required Skills & Experience

  • Advanced GCP Data Engineering expertise including:
    • BigQuery
    • Dataflow (Apache Beam)
    • Pub/Sub
    • Google Cloud Storage (GCS)
  • Strong experience with Medallion Architecture design and implementation
  • Expertise in QA framework design for data pipelines using:
    • Great Expectations
    • dbt tests
    • Equivalent validation frameworks
  • Strong Python and SQL development skills
  • Agile delivery management experience including:
    • Sprint planning
    • RAID management
    • Status reporting
  • Advanced GCP Certifications preferred

Preferred Experience

  • Telecom data or high-volume telemetry processing experience
  • Experience leading distributed/offshore engineering teams
  • Strong stakeholder communication and mentoring skills
  • Ability to design scalable, reusable, and governed data platform solutions

 

Apply Now!