Vetted Model Evaluation Professionals

“Backend/ML engineer who built production LLM-enabled systems at PRGX, including an interpretable contract opportunity scoring engine (Bradley-Terry pairwise ranking) that reached 0.82 weighted Spearman agreement with SME auditors and was integrated into workflow. Also built a Duke student advisor chatbot and hardened it for real-world reliability/security with schema-driven tool calling, normalization, and off-domain defenses; led staged production rollouts with shadow testing and achieved 0.90 F1 on a new extraction field before shipping.”

“Built and deployed a production GenAI internal support agent at Bank of America (“Ask GPS/AskGPT”) using RAG on Azure, focused on reducing escalations and improving response quality for repetitive knowledge-based queries. Demonstrates strong production LLM engineering: custom LangChain orchestration, retrieval tuning to reduce hallucinations, rigorous offline/online evaluation, and model benchmarking with dynamic routing (e.g., GPT-4 vs Claude).”

“At Fileread, the candidate built and deployed an LLM-powered legal document classification and retrieval layer for an agentic extraction system that turns unstructured legal PDFs into structured tables with line-level citations. They productionized a RAG-style pipeline (ingestion, embeddings, retrieval, reranking, generation) and report 95%+ F1 across 70+ legal categories, emphasizing rigorous evaluation and close collaboration with legal domain experts for high-stakes precision.”

“Amazon engineer who built a production AI agent platform (Python/AWS Strands on Bedrock) that lets teams create tool-using, multi-agent workflows—e.g., agents that auto-triage and resolve customer support tickets by reading internal documentation and collaborating with a research agent. Previously worked in Deloitte on IAM using Ping Identity/Ping DaVinci orchestration, and applies orchestration thinking plus structured evaluation (LLM-as-judge, surveys, automated tests) to improve agent reliability.”

“Built ToolDreamer, a framework for selecting relevant tools for LLM agents by training a retriever on LLM-generated reasoning traces, and has hands-on experience building multi-agent systems in AutoGen (MAG-V) focused on question generation and tool-trajectory verification. Currently works as an AI-guides supervisor at Penn State, regularly communicating AI concepts to non-technical stakeholders.”

“Built and deployed a production LLM + vector search clinical decision support system at UnitedHealth Group, retrieving medical evidence and patient context in real time for prior authorization and risk scoring. Strong in end-to-end RAG architecture (Hugging Face embeddings, Pinecone/FAISS, SageMaker, Redis) plus orchestration (Airflow/Kubeflow) and rigorous evaluation/monitoring, with demonstrated ability to align solutions with clinical operations stakeholders.”

“Backend/AI engineer who built a real-time vector database system for high-frequency financial data using Kafka/Flink on Kubernetes, achieving sub-100ms similarity search at 10k+ concurrent load and resolving tricky duplication issues with idempotency/versioning. Also shipped an end-to-end LLM-based travel itinerary feature (profiling + prompt workflows + APIs) with a focus on quality consistency and low latency.”

“Edtech/education-focused engineer who took an accessibility-critical LLM/vision feature from concept to production: built an OpenCV-gated whiteboard capture pipeline feeding Gemini Vision for handwriting-to-LaTeX, improving math transcription 80% while cutting inference costs 60%. Also built RAG observability and retrieval fixes that stabilized inconsistent answers, and partnered directly with sales to reshape demos and open a new K-12 revenue pipeline aligned to California Digital Divide grant requirements.”

“Software engineer in the mortgage/financial services domain (Freddie Mac) who builds end-to-end loan origination and credit risk capabilities using Spring Boot microservices, Angular dashboards, and data pipelines. Delivered measurable impact (30% reduction in underwriting turnaround time) and emphasizes production reliability/compliance with strong guardrails, observability, and evaluation loops for risk scoring systems.”

“GenAI/ML engineer with production experience building multilingual LLM systems (English/Spanish) and RAG-based clinical documentation summarization at Walgreens, combining prompt engineering, structured output validation, and rigorous evaluation (ROUGE + pharmacist review). Also orchestrated end-to-end ML pipelines for demand forecasting using Apache Airflow, PySpark, and MLflow with scheduled retraining and production monitoring.”

“LLM engineer who built a production document intelligence/RAG pipeline to extract structured data from thousands of unstructured PDFs, cutting manual review time by 60%. Experienced with LangChain and Airflow orchestration plus rigorous evaluation (labeled datasets, prompt testing, HITL review, monitoring) to improve accuracy and reduce hallucinations while partnering closely with non-technical operations stakeholders.”

“ServiceNow engineer who built and launched a production LLM-powered ticket resolution/knowledge assistant using RAG (LangChain + Hugging Face embeddings + vector search) integrated into internal support dashboards via REST APIs. Optimized the system from ~6–8s to ~2–3s latency while improving usability with concise, cited answers and guardrails (grounding + similarity thresholds), delivering ~30–35% reduction in manual ticket investigation effort.”

“Built and shipped a production LLM/RAG risk-case summarization and triage system used by fraud/compliance analysts, with strong grounding controls (evidence-cited outputs and refusal on low confidence). Demonstrates end-to-end ownership across retrieval quality, Airflow-orchestrated indexing pipelines, and compliance-grade privacy (PII redaction, RBAC, encrypted redacted logging, and auditable prompt/model versioning) plus a tight feedback loop with non-technical domain experts.”

“GenAI/ML engineer from Lucid Motors who built and productionized an LLM-powered RAG diagnostic assistant for manufacturing and maintenance teams, deployed on AWS with Docker/Kubernetes and MLflow. Demonstrates end-to-end ownership from retrieval/prompt design to scalability, monitoring, and workflow integration via APIs, plus production ML pipeline orchestration with Kubeflow (Spark/Kafka + TensorFlow) for predictive maintenance use cases.”

“Backend engineer at Goldman Sachs who deployed internal LLM-powered utilities to summarize operational logs/tickets, with a strong emphasis on data sensitivity and reliability. Built deterministic workflows with template-based prompts, confidence checks, and rule-based fallbacks, and used monitoring plus failure-rate metrics to tune performance; also has hands-on Temporal orchestration experience for resilient async backend jobs.”

“Backend/infra engineer with experience across Nestle, J.P. Morgan, and Capgemini, combining ML systems work (YOLOv8/PyTorch object detection with TFLite edge deployment) with production-grade cloud/Kubernetes operations. Has delivered measurable impact via AWS migrations (25% cost reduction, 99.9% availability), microservice modernization (35% faster processing), and low-latency Kafka streaming for financial dashboards (<100ms) using DLQs and idempotent consumers.”

“AI/ML engineer with production experience building LLM-powered document intelligence and customer support systems in healthcare/insurance, emphasizing high-accuracy RAG, long-document processing, and robust monitoring/fallback mechanisms. Also automates and scales ML lifecycle workflows using Apache Airflow and Kubeflow, and partners closely with non-technical operations stakeholders to drive adoption.”

“Internship experience building production-oriented AI systems, including a real-time voice scam/spoof detector (RawNet + AASIST) hardened for noisy audio via aggressive augmentation and Zoom-based noise simulation, evaluated with EER on clean and wild datasets. Also built an LLM-driven UI automation agent using Playwright for apps like Linear/Notion with modular tool design, unit tests, and replayable scripted scenarios, and has AWS Step Functions experience orchestrating Lambda/Cognito workflows.”

“Built and deployed an AWS-based LLM/RAG ticket triage and knowledge retrieval system (Pinecone/FAISS + Step Functions + MLflow) that cut support resolution time by 20%. Demonstrates strong production focus on hallucination reduction, PII security, and low-latency orchestration, with measurable evaluation improvements (e.g., ~25% grounding accuracy gain via re-ranking) and proven collaboration with support operations stakeholders.”

“Red Hat ML/LLM engineer who designed and deployed a production LLM-powered customer support automation system using RAG, improving latency by 30% via PEFT and vector search optimization. Built security and governance into retrieval (access-level filtering, encrypted Pinecone/ChromaDB) and delivered SHAP-based explainability via a dashboard for non-technical stakeholders. Experienced orchestrating distributed ML/RAG pipelines across AWS SageMaker and OpenShift with Airflow/Prefect, plus multi-agent workflows using CrewAI and LangGraph.”

“Built and deployed a production LLM-powered RAG document intelligence/Q&A system for healthcare prior authorization, reducing manual medical document review time and improving decision efficiency. Strong in end-to-end LLM application engineering (LangChain/LangGraph), retrieval quality improvements (hybrid search, embedding tuning, chunking strategies), and rigorous evaluation/monitoring for reliability.”

“Built an AI journaling app at HackCU 2025 featuring a speaking AI avatar with long-term memory via RAG (ChromaDB) and low-latency microservices coordinated through Kafka, including deployment under AMD/non-CUDA constraints using a quantized Llama 8B model. Also has Goldman Sachs experience deploying a Trade UI on Kubernetes with CI/CD rollback automation, plus a healthcare AI internship at CU Anschutz collaborating closely with physicians on diagnostic reasoning and dataset annotation.”