Vetted Model Evaluation Professionals

“Software/cloud engineer with PwC experience deploying a nationwide Australian Government Salesforce labor licensing platform used by 200k+ professionals, emphasizing safe integration, CI/CD, and UAT-driven quality improvements (40% defect reduction). Also built a Python/FastAPI RAG system with the U.S. Army to convert CONOP documents into risk assessments, adding human-in-the-loop and provenance features to address operator trust concerns.”

“Built and shipped production improvements to a Paylocity RAG-based AI assistant, redesigning retrieval into a hybrid HNSW + keyword pipeline and using tuned RRF to fuse rankings—cutting latency by ~2s and reducing token usage by ~5000. Previously spearheaded Apache Airflow integration across ETL pipelines at Acuity Knowledge Partners, creating reusable templates and automated triggers to reduce manual job monitoring.”

“Asset Management Risk professional at Fidelity Investments who built and productionized an agentic RAG platform enabling compliance and analysts to query 10,000+ fund documents with cited answers in seconds. Implemented structure-aware semantic chunking (AWS Textract), hierarchical retrieval, and hybrid search to raise accuracy from 68% to 94%, and built an evaluation framework tracking accuracy/latency/cost/hallucinations—delivering 40+ hours/month saved and zero critical production failures.”

“LLM/agent engineer who has shipped production RAG chatbots in sustainability-focused domains, including a packaging recommendation assistant that standardized messy user inputs and used Pinecone-backed retrieval over product/regulatory data. Experienced orchestrating end-to-end ML workflows with Airflow and AWS Step Functions/Lambda, emphasizing reliability (property-based testing, circuit breakers, OpenTelemetry) and measurable performance (latency/cost). Partnered closely with non-technical leadership to ship 3 weeks early, driving adoption by 150+ businesses and ~20% reported waste reduction.”

“Amazon Fire TV Devices engineer who built and shipped a production LLM-powered lab triage and validation system that grounds recommendations in internal runbooks/known-issue data and pushes evidence-based actions via dashboards and Slack. Emphasizes safety and measurability with structured JSON outputs, replay-based evaluation on historical incidents, and production metrics (e.g., disagreement rate and time-to-first-action), plus cost/latency optimizations like caching, batching, and rule-based fast paths.”

“Built and shipped a production-grade GenAI Fraud & Compliance Investigation Copilot for a large US bank, integrating OCR docs, structured data, and prior case history to generate grounded, regulator-friendly summaries and red-flag highlights. Demonstrates strong end-to-end LLM systems engineering (LangGraph/LangChain, hybrid retrieval with FAISS+BM25, guardrails/citations, streaming/latency optimization) plus rigorous evaluation and close partnership with compliance stakeholders.”

“GenAI/LLM engineer and architect who built and deployed a production generative AI financial forecasting and scenario analysis platform at McKinsey, leveraging Claude (Anthropic), LangChain, Airflow, MLflow, and AWS SageMaker. Demonstrates strong LLMOps/MLOps rigor (monitoring, drift detection, automated retraining) and deep experience implementing global privacy controls (GDPR, differential privacy, audit trails) while partnering closely with finance executives and legal/IT stakeholders.”

“Built a production LLM/RAG-based “model excellence scoring” system at Uber to automatically evaluate hundreds of ML models, standardizing quality assessment and cutting evaluation time from days to minutes on GCP. Also delivered an NLP document classification solution for insurance claims at Globe Life, partnering closely with compliance/operations and improving routing accuracy from ~85% manual to 93% with the model.”

“Built and productionized a RAG-based analytics Q&A assistant for a financial analytics team, enabling natural-language querying across 200+ datasets (SQL tables, PDFs, compliance docs, wikis) and cutting turnaround time by 60%. Deep experience delivering regulated, audit-ready LLM systems on Azure (Azure OpenAI + LangChain) with strict grounding/citations, hybrid retrieval, and AKS-based low-latency deployment, plus strong collaboration with compliance analysts and auditors via iterative Gradio demos.”

“Built a hackathon "Patient Summary Assistant" backend focused on healthcare workflows, combining RAG-based summarization with HIPAA-minded privacy controls (NER redaction + encryption). Demonstrated strong infra skills by deploying on Kubernetes with Helm/HPA and GitOps (ArgoCD), plus migrating from OpenAI to an on-prem Llama 3 stack (vLLM, quantization, shadow-mode testing) and adding real-time Kafka ingestion for patient vitals/anomaly alerts.”

“Built an "AskMyVideo" system that turns YouTube videos into queryable knowledge graphs by transcribing audio (Whisper), chunking and embedding content, and enabling traceable answers back to exact timestamps. Strong in entity resolution (rules + fuzzy matching + TF-IDF/cosine with PR-curve thresholding) and modern retrieval stacks (FAISS, hybrid dense/sparse, domain fine-tuning with ~12% precision gain), with a production mindset using Airflow/Prefect, Docker/FastAPI, and LangSmith/Prometheus/Grafana observability.”

“New graduate with hands-on LLM work building a RAG pipeline (HNSW, lexical reranking/boosting, ReAct) and optimizing it through ablation to dramatically reduce latency. Also building a modular personal assistant with a custom wake word model, router-driven agent selection, and integrations like Spotify with secrets managed via .env.”