Vetted Apache Spark Professionals

“Project lead who partners with PM and customers to gather requirements, adjust project plans, and deliver new functionality that drives customer satisfaction and revenue. Has experience building features end-to-end and presenting successful technical demos to engineering and management audiences; no stated experience with LLM/agentic systems.”

“Built and productionized a real-time enterprise RAG pipeline to improve factual accuracy and reduce LLM hallucinations by grounding responses in constantly changing internal knowledge bases (policies, manuals, FAQs). Experienced in orchestrating end-to-end ML workflows (Airflow/Kubernetes), handling messy multi-format data with schema enforcement (Pydantic/Hydra), and maintaining freshness via streaming incremental embeddings plus batch refresh. Also delivers applied ML solutions with non-technical teams (marketing/CRM) for segmentation and personalized engagement.”

“Built and shipped an LLM-powered data quality assistant that generates maintainable validation checks from metadata while executing validations via Great Expectations, exposed through FastAPI and integrated into Airflow-managed pipelines. Emphasizes production reliability (structured outputs, guardrails, monitoring, versioning, human review) and works closely with compliance/operations teams to deliver clear, auditable, user-friendly AI outputs.”

“Former Amazon intern who owned an end-to-end computer vision system to detect package anomalies in fulfillment centers, from data collection/labeling to production deployment on AWS (EC2/S3) with a Streamlit live-monitoring dashboard. Also has ML-in-production experience deploying and updating a recommendation model on Kubernetes (Minikube) with CI/CD via GitHub Actions, plus prior SDE experience with Jenkins-based pipelines and on-prem to AWS migration work using Glue.”

“Meta engineer who improved a critical revenue estimation dataset pipeline that was arriving ~6 days late—diagnosed via raw logs/lineage, redesigned legacy scans to only process the needed window, and shipped validation plus freshness/lag dashboards. Delivered ~50% latency reduction (to ~3 days) and regained adoption by running old/new pipelines in parallel with gated cutover and evidence-based customer communication. Applies incident-response rigor to real-time LLM/agentic workflow debugging and regularly runs developer demos/workshops.”

“Engineering leader who modernized and unified a fragmented product suite at Milestone via a multi-year cloud-native roadmap, delivering an MVP in three quarters and boosting team velocity by 40% through cross-functional squads. At Prometheum, led a trust-building hybrid architecture (AWS control plane + customer-hosted data plane) using Kubernetes to ensure sensitive enterprise data never left customer networks while remaining cloud-agnostic across providers.”

“Entrepreneurially minded builder focused on proving architecture concepts via minimal demo prototypes for marketing. Has hands-on experience improving an A/B experimentation framework by interviewing stakeholders, identifying system limits and bottlenecks, and defining success criteria to scale experimentation and speed up analysis.”

“Backend/data engineer from Oscar Health specializing in healthcare claims systems on AWS. Built HIPAA-compliant real-time services (FastAPI/Postgres/Kafka on EKS) and serverless ingestion pipelines, and led modernization of a legacy SAS claims pricing system to Python/Spark with rigorous parity validation. Demonstrated measurable impact with high uptime/low latency services and major Snowflake performance and cost reductions.”

“Built and shipped a production LLM-powered medical scribe that generates structured clinical visit summaries using RAG, strict JSON schemas, and post-generation validation to reduce hallucinations. Experienced in making LLM workflows deterministic and observable (structured logging/metrics/tracing) and in evaluation-driven iteration with metrics like schema pass rate and edit rate; collaborated closely with clinicians and policy stakeholders at Scale AI to drive adoption.”

“Built and shipped a real-time AI phone agent for small businesses that handles bookings/FAQs/messages using streaming ASR, an LLM with tool-calling, and TTS; deployed to production for multiple paying customers. Demonstrates strong applied LLM reliability practices (tool-first grounding, retrieval, hard-negative testing, and production monitoring) and experience orchestrating multi-step AI workflows with Airflow, Prefect, and AWS Step Functions.”

“Player-coach backend leader at PostLo who led a major backend architecture upgrade to enable AI-driven features by separating transactional systems from AI workloads (vector embeddings/image validation) and adding async processing for heavy jobs. Also owned production reliability improvements (query/index optimization, workload isolation, monitoring and load testing) and translated an ambiguous retention goal into a shipped cashback rewards feature with auditable transactions.”

“Data engineer focused on building and operating reliable Airflow-orchestrated pipelines into BigQuery, including daily billing ingestion (~1GB/day) and ad platform (Facebook/LinkedIn) data collection. Implemented end-to-end data quality checks plus org-wide incident response automation integrating PagerDuty, Slack, and Jira, and has experience executing large backfills (4–5TB) via time-window batching.”