Manjory saran

About

LLM infrastructure engineer who built a production-critical real-time personalization and memory retrieval system for a user-facing product, adding <100ms P99 latency while improving relevance ~20–25% and holding SLA through 3x traffic. Experienced designing tiered retrieval backends (Redis + vector store), deploying on Kubernetes with autoscaling/circuit breakers, and running rigorous observability, incident response, and agent evaluation (shadow traffic, A/B tests, regression/replay).

Experience

Education

Key Strengths

• Built and deployed production-critical real-time LLM personalization/memory retrieval with <100ms added P99 latency
• Improved response relevance by ~20–25% while maintaining SLA under 3x traffic growth
• Strong system design for scale (tiered hot/cold retrieval paths; stateless prompt-assembly gateway; offline embeddings/backfills)
• Pragmatic tradeoff-making (prompt-time retrieval vs fine-tuning; latency predictability over maximum recall; eventual consistency for context)
• Reliability/incident leadership: diagnosed cache TTL misconfiguration causing P99 spike (120ms→1.5s) and recovered within ~20 minutes via rollback, caps, resizing, and scaling
• Deep observability practice (golden signals, distributed tracing, custom LLM quality/coverage metrics, decision-point logging)
• Designed and evaluated agent-based workflows with structured loops, layered state, and online/offline evaluation (A/B, shadow runs, regression/replay)