Julian Lee

About

AI/ML infrastructure-focused engineer who has built production RAG systems from scratch (Supabase/pgvector + OpenAI embeddings) and iterated using formal eval metrics to improve retrieval quality. Also debugged real-time audio issues in a LiveKit-based pipeline by correlating packet loss with VAD behavior, and has deep experience building brittle, customer-specific financial platform integrations in Python/Playwright (2FA, redirects, token refresh, rate limits).

Key Strengths

• Built and deployed a production doc RAG pipeline end-to-end (ingestion, embeddings, vector DB, indexing)
• Improved retrieval quality by restructuring chunking around Markdown headers and adding section metadata
• Evaluation-driven iteration using relevancy/faithfulness metrics (DeepEval) to tune retrieval
• Systematic, layer-by-layer debugging of real-time audio issues using LiveKit diagnostics and network metrics
• Adapted Python/Playwright integrations to highly variable, customer-specific auth flows (2FA, redirects, token refresh, rate limits)
• Translates real user playtest observations into actionable engineering fixes; created automated dialogue testing based on observed failure patterns
• Built and deployed production RAG pipeline over product documentation (Supabase/pgvector HNSW, OpenAI embeddings)
• Strong retrieval quality debugging and iteration (chunking strategy, thresholds, top-k tradeoffs)
• Designs LLM app memory systems using structured fact extraction and vector retrieval (LanceDB, mem0-inspired)
• Hands-on orchestration across LLM, web, and real-time voice stacks (LangChain, LiveKit Agents)
• Specification-driven testing and evaluation of LLM workflows with measurable metrics (DeepEval, regression testing)
• Pragmatic model/retrieval/prompt selection based on constraints (latency vs accuracy; reranking; few-shot when needed)
• Translates non-technical user feedback into concrete AI system improvements (playtests, STT/LLM dialogue fixes)

Experience

Education

Skills

Languages

Certifications