GCP Data Engineering Lead

Location: Toronto, ON (Onsite)

Duration: 4 Months

Employment Type: Contract

Job Summary

We are seeking a highly experienced GCP Data Engineering Lead to serve as the senior technical and delivery operations lead for a critical data engineering engagement. This role combines hands-on GCP pipeline architecture expertise with QA strategy ownership and day-to-day Agile delivery management responsibilities.

The ideal candidate will be accountable for technical quality, sprint execution, delivery governance, and successful knowledge transfer outcomes while leading a distributed engineering team.

Key Responsibilities

Technical Leadership & Architecture

• Lead pipeline architecture design for Netscout and Sandvine ingestion workflows including:
  • Source extraction
  • GCS landing
  • Dataflow transformations
  • BigQuery loading across Bronze/Silver/Gold layers
• Design Medallion Architecture data flows and layer transition rules
• Define data quality check specifications per Medallion layer for implementation by the Observability Engineer
• Lead data reconciliation strategy including UDP vs. Datahub parity validation
• Provide hands-on development support for complex pipeline transformation logic
• Coordinate technical decisions with the client Data Engineering team

QA & Data Quality Ownership

• Define and own the end-to-end QA strategy including:
  • Test types
  • Coverage
  • Tooling
  • Defect management
  • Quality gates
• Architect reusable and parameterized test templates for:
  • Batch pipelines
  • Streaming pipelines
  • Schema validation
  • Transformation accuracy
  • Business logic validation
  • Governance compliance
• Review and quality assure all pipeline and QA deliverables produced by offshore engineers

Delivery & Agile Management

• Lead sprint planning, sprint reviews, and retrospectives using Agile methodologies
• Maintain RAID logs (Risks, Assumptions, Issues, Dependencies) and drive mitigation actions
• Produce weekly status reports and maintain project delivery trackers
• Manage day-to-day task allocation and quality oversight for offshore Data Engineers and Observability Engineers
• Support mentoring and knowledge transfer activities

Required Skills & Experience

• Advanced GCP Data Engineering expertise with:
  • BigQuery
  • Dataflow (Apache Beam)
  • Pub/Sub
  • Google Cloud Storage (GCS)
• Strong experience designing and implementing Medallion Architecture
• Expertise in QA framework design for data pipelines using:
  • Great Expectations
  • dbt tests
  • Equivalent validation frameworks
• Strong Python and SQL development skills
• Agile delivery management experience including:
  • Sprint planning
  • RAID management
  • Status reporting
• Experience leading distributed or offshore engineering teams
• Advanced GCP Certifications preferred

Preferred Experience

• Telecom data or high-volume telemetry processing experience
• Strong mentoring and stakeholder communication skills
• Experience building scalable, reusable, and governed data platforms