Shayan Shokri

About

ML/LLM practitioner with experience at Truveta building an LLM-based evaluation framework; identified non-overlapping evaluator failure modes and proposed an ensemble approach that enabled scaling training data and drove ~5% performance gains across multiple internal projects. Strong focus on robustness to distribution shift (augmentation/domain adaptation/meta-learning) and production reliability via monitoring, drift detection, and safe fallbacks.

Experience

Education

Key Strengths

• Improved in-house LLM performance by ~5% via ensemble-based LLM evaluator framework (Truveta)
• Strong experimental rigor: error analysis to select complementary ensemble models
• Balances model quality with compute/budget constraints in architecture decisions
• Practical production mindset: monitoring for confidence and input drift with safe fallback behavior
• Applies research to practice by implementing papers/lectures (e.g., meta-learning to improve generalization across tasks)