Sanghavi Yelmelwar

About

Data engineer who has owned end-to-end production pipelines ingesting ~500GB/day from APIs/databases/Kafka into an S3 data lake (Glue/Spark) with Airflow-orchestrated Great Expectations quality gates. Built resilient external data collection systems with idempotent jobs, exponential-backoff retries, raw data capture, and backfills; also shipped Snowflake-backed APIs with caching, versioned endpoints, and backward-compatible data contracts. Led an early-stage Azure data platform build with phased delivery and GitHub Actions CI/CD, resolving schema-mismatch incidents quickly without downstream corruption.

Key Strengths

• Owned end-to-end data pipeline ingesting ~500GB/day (APIs, databases, Kafka) into S3 data lake
• Implemented automated data quality gates in Airflow using Great Expectations (schema, null/duplicate, reconciliation) to prevent bad data propagation
• Reduced pipeline latency by 30% via optimized incremental processing
• Designed resilient external data collection with monitoring, retries (exponential backoff), idempotency, raw data capture, and parameterized backfills
• Handled upstream schema-change incidents quickly using early validation, centralized logs, parser patching, and backfills with no downstream data corruption
• Built and operated a Snowflake-backed data service with query optimization, caching, versioned endpoints, and backward-compatible schemas

Experience

Skills

Languages