Hyndhavi Sibbala

About

Data platform engineer with hands-on ownership of high-volume financial data pipelines (millions of transactions/day) on Azure (ADF, Databricks, Delta Lake, Synapse), emphasizing schema-drift protection and automated data-quality gates. Also built resilient web scraping pipelines with anti-bot and backfill strategies, and shipped a versioned FastAPI + Redis data API with autoscaling, testing, and CI/CD via GitHub Actions.

Key Strengths

• Owned end-to-end financial transactions data pipeline processing millions of records daily
• Built automated, modular data-quality framework (schema validation, null checks, metadata-driven rules) with halt-and-alert safeguards
• Handled schema drift via automated Python comparisons of incoming vs expected schemas
• Optimized lakehouse/storage and query performance via partitioning and compression tradeoffs to reduce processing time and cloud cost
• Designed resilient web scraping system with monitoring, exponential-backoff retries, anti-bot tactics, validation gates, and backfill+dedupe process
• Shipped versioned FastAPI data service with async performance, Redis caching, strict schema validation, and strong test coverage
• Delivered in ambiguous early-stage environment using managed services and GitHub Actions CI/CD with iterative improvements from launch feedback

Experience

Education

Skills

Languages