Prateek Pravanjan

About

LLM/agent engineer who built a production LangGraph multi-agent orchestrator connecting GitHub and APM/observability signals with a chain-of-verification loop for root-cause analysis. Emphasizes pragmatic architecture (start simple with state summaries), performance tuning (async LLM calls, Docker), and rigorous evaluation (LLM-as-judge, adversarial testing, hallucination/instruction adherence metrics, tool-call tracing) while iterating with non-technical stakeholders via A/B testing.

Key Strengths

• Built and deployed a LangGraph-based multi-agent orchestrator integrating GitHub and APM/observability for iterative issue analysis/RCA
• Pragmatic architecture decisions to avoid overengineering (state summaries first; evaluate RAG/knowledge graph only if needed)
• Improved scalability via dockerization
• Reduced latency by making LLM calls asynchronous
• Strong agent evaluation practice: LLM-as-judge, adversarial comparisons vs zero-shot frontier models, and quantitative metrics (F1/ROUGE/cosine)
• Reliability/faithfulness focus through tool-call tracing and hallucination/instruction-adherence tracking
• Effective collaboration with non-technical stakeholders using iterative A/B testing and transparent communication of constraints
• Built end-to-end customer-facing expert workflow UI and data pipeline supporting ~800 experts
• Iterative delivery with feature lock-in and isolated production environment releases
• Operational reliability via model endpoint fallbacks when client APIs failed under high throughput
• Performance optimization through denormalized storage to handle extremely high write volumes
• Designed modular, dockerized services for scalability (microservices-style architecture)
• Async-first backend design with testing suite and fallbacks for reliability
• Implemented RabbitMQ-based queuing across specialized ranking services/agents with message contracts
• Created internal zero-shot LLM analytics tool adopted across multiple teams; enabled NL queries to generate Altair charts
• Drove adoption with onboarding docs, Loom walkthroughs, and early-user testing/A-B UI experiments

Experience

Education

Awards

• Research under Spring ’24 ECE Scholarship Program - Stevens Institute of Technology

Skills

Languages