Vetted A/B Testing Professionals

“Project lead who partners with PM and customers to gather requirements, adjust project plans, and deliver new functionality that drives customer satisfaction and revenue. Has experience building features end-to-end and presenting successful technical demos to engineering and management audiences; no stated experience with LLM/agentic systems.”

“Built and productionized a real-time enterprise RAG pipeline to improve factual accuracy and reduce LLM hallucinations by grounding responses in constantly changing internal knowledge bases (policies, manuals, FAQs). Experienced in orchestrating end-to-end ML workflows (Airflow/Kubernetes), handling messy multi-format data with schema enforcement (Pydantic/Hydra), and maintaining freshness via streaming incremental embeddings plus batch refresh. Also delivers applied ML solutions with non-technical teams (marketing/CRM) for segmentation and personalized engagement.”

“Backend engineer who helped redesign and refactor Forma’s backend during an app rewrite, emphasizing modularity, maintainability, and A/B testing support while delivering feature parity on a quarter-long timeline. Led a careful database migration using parallel databases with schema differences, validating integrity via staging and SQL checks, and has experience debugging subtle computer-vision overflow edge cases.”

“LLM/agentic-systems builder who has shipped production tools for investment research and procurement insights, including a company screener that processes thousands of conference-listed companies using FireCrawl + Google Search + Gemini. Demonstrates strong orchestration expertise (LangGraph multi-agent graphs), performance optimization (async/batching to sub-30s), and pragmatic reliability/evaluation practices with stakeholder-friendly UX (real-time cost tracking and model/parameter toggles).”

“Gameplay systems designer with shipped AAA experience on Overwatch, including end-to-end ownership of player progression and major systems like matchmaking (role delta system cutting uncertainty/volatility by 50%) and Stadium mode round/shop + MMR/balancing systems (second-highest engaging mode). Strong in KPI/retention-driven design, player research, and telemetry-based balancing, with a UE5 Blueprint prototyping workflow built for modularity, data-driven tuning, and handoff to engineering.”

“Performance marketing analyst at Meta supporting Amazon who identified Amazon app users as a high-value segment (2x ROAS), built and validated a retargeting audience via rigorous A/B testing, and helped drive a global rollout across thousands of campaigns resulting in ~$80M annual revenue. Strong in experiment design, SQL-driven insight generation, and translating performance learnings into cross-platform creative and catalog strategy.”

“LLM/Conversational AI engineer who built a production multi-turn dialogue system using LoRA fine-tuning on LLaMA, cutting training compute/memory by 90%+ while maintaining low-latency inference via quantization and streaming generation. Experienced in orchestrating end-to-end ML workflows with Prefect/Airflow/Kubeflow (including hyperparameter sweeps and W&B tracking) and improving agent reliability through benchmark-driven testing, shadow-mode rollouts, and stakeholder-informed guardrails.”

“Entrepreneurially minded builder focused on proving architecture concepts via minimal demo prototypes for marketing. Has hands-on experience improving an A/B experimentation framework by interviewing stakeholders, identifying system limits and bottlenecks, and defining success criteria to scale experimentation and speed up analysis.”

“Backend/data engineer with Amazon supply-chain experience building production serverless Python services and ETL pipelines on AWS (Lambda, API Gateway, S3, RDS, Glue). Has modernized legacy SAS jobs into Python with rigorous parity testing and phased migrations, and has delivered major SQL performance gains (minutes down to seconds) through indexing and partitioning.”

“Backend/data engineer from Oscar Health specializing in healthcare claims systems on AWS. Built HIPAA-compliant real-time services (FastAPI/Postgres/Kafka on EKS) and serverless ingestion pipelines, and led modernization of a legacy SAS claims pricing system to Python/Spark with rigorous parity validation. Demonstrated measurable impact with high uptime/low latency services and major Snowflake performance and cost reductions.”

“Built and shipped a production LLM-powered medical scribe that generates structured clinical visit summaries using RAG, strict JSON schemas, and post-generation validation to reduce hallucinations. Experienced in making LLM workflows deterministic and observable (structured logging/metrics/tracing) and in evaluation-driven iteration with metrics like schema pass rate and edit rate; collaborated closely with clinicians and policy stakeholders at Scale AI to drive adoption.”