Shabari Vignesh

About

Data/Backend engineer who has owned end-to-end merchant analytics systems on AWS: orchestrated multi-source ingestion (FISERV/Shopify/Clover) with Step Functions/Lambda, enforced strong data quality gates, and served curated datasets via Redshift and a FastAPI layer. Also built an early-stage Merchant Insights AI agent that converts natural language questions into SQL using OpenAI models, with full CI/CD and observability.

Key Strengths

• Owned end-to-end AWS-based data pipeline (ingestion to Redshift serving) processing thousands of merchants and millions of transaction records daily
• Built robust data quality framework (schema consistency, critical-field null checks, volume spike/drop detection) with gating before downstream impact
• Designed scalable parallelized ingestion/orchestration using Step Functions and per-merchant job fan-out
• Implemented production monitoring/alerting with CloudWatch and custom metrics/logging to quickly localize failures
• Achieved <1% pipeline failure rate with rapid issue resolution via targeted retries (retry only failed merchant jobs)
• Designed fault-tolerant external data collection with exponential backoff retries, throttling, and schema-change detection/versioning
• Shipped a Python/FastAPI data service with caching, optimized/partitioned tables, and Swagger docs for strong developer usability
• Stood up an ambiguous early-stage AI agent (natural language to SQL) with AWS serverless deployment, CI/CD, and iterative accuracy improvements post-launch

Experience

Education

Awards

• Ranked among National Toppers in AI: Knowledge, Representation & Reasoning - IIT Madras (NPTEL)
• Performer of the Milestone - ITC Infotech
• Nominated for Dashing Debut - ITC Infotech

Skills

Languages

Certifications