Divyam Agrawal

About

Internship experience building a production RAG+LLM pipeline to map messy card transaction descriptions to merchant brands, including a custom modified-ROUGE evaluation approach for weak/variant ground truth. Improved scalability and cost by moving from a managed LLM endpoint (e.g., Bedrock) to self-hosted vLLM, and orchestrated massive embedding backfills (5,000+ files, 10B+ rows) using an Airflow-triggered SQS + ECS worker architecture with robust retry/DLQ handling.

Key Strengths

• Built and deployed a production RAG+LLM system for transaction-to-merchant brand labeling
• Designed pragmatic evaluation for messy/noisy labels (modified ROUGE aligned to human judgment)
• Cost/performance optimization by migrating from managed LLM endpoint to self-hosted vLLM with batching
• Scaled large data/embedding pipelines using Airflow + SQS + ECS worker pattern
• Reliability engineering for AI workflows (retries, idempotency, backpressure, DLQ testing)
• Pragmatic model selection via baselines and measurement (smallest model that meets accuracy/latency/cost)
• Effective collaboration with non-technical stakeholders to define business-critical errors and confidence thresholds

Experience

Education

Awards

• First place - UWash Data Science Hackathon

Skills

Languages

Publications

1 publication