Job Summary
The Senior GCP Data Engineer leads the design and implementation of enterprise grade data platforms. They translate complex business requirements into technical blueprints, ensuring data integrity across high volume batch and real time streaming architectures
Required Technical Expertise
• GCP Core: Deep mastery of BigQuery (partitioning/clustering), Dataflow (Apache Beam), Cloud Composer (Airflow), and Pub/Sub.
• Data Engineering Tools: Expertise in dbt for modeling, Spark/PySpark for distributed processing, and Docker/Kubernetes for containerized data services.
• Modern Data Stack: Proficiency with Delta Lake, Iceberg, or Hudi in Lakehouse architectures.
• Programming: Senior level proficiency in Python, SQL, and occasionally Java or Go.
Key Design & Leadership Responsibilities
• Architectural Blueprinting: Designing end to end data architectures using Medallion Architectures (Raw/Bronze, Curated/Silver, Gold) and domain driven design.
• Framework Development: Building reusable ingestion and transformation frameworks rather than one off jobs to accelerate pipeline delivery.
• Infrastructure as Code (IaC): Architecting cloud environments using Terraform to ensure reproducible and automated infrastructure management.
• AI & ML Readiness: Designing feature stores and specialized pipelines to deliver clean, ML ready datasets for Vertex AI or Generative AI workloads.
• Technical Mentorship: Leading daily scrums, performing detailed code reviews, and mentoring junior engineers on best practices for performance tuning and cost optimization.
• Data Governance & Security: Architecting solutions for data lineage, metadata management, and column level security using Dataplex and IAM.
Qualifications & Experience
• Experience: Typically 6‿10+ years in data engineering, with at least 3+ years specifically leading GCP projects.
• Certifications: GCP Professional Data Engineer or GCP Professional Cloud Architect is strongly preferred.
• Education: B.Tech/B.E. in Computer Science or related fields.