Vetted PySpark Professionals

“Software engineer with experience at Google and the City and County of San Francisco building production AI systems, including a RAG-based internal support chatbot and ML-driven ticket priority tagging. Has scaled data/ML platforms with Airflow on GCP (1M+ records/day, 99.9% SLA) and deployed multi-component systems with Docker and Kubernetes (GKE), using modern LLM tooling (LangChain/CrewAI, Claude/OpenAI, Pinecone/ChromaDB, Bedrock/Ollama).”

“Lead Data Scientist (10+ years) with recent work in healthcare data: built production pipelines that unify EHR, genomics, and clinical notes using NLP (spaCy/BERT/BioBERT) and scalable Spark-based processing. Also led development of domain-specific LLM/NLP systems for chatbots and semantic search, deploying models via FastAPI/Flask and improving retrieval with FAISS-backed, fine-tuned clinical embeddings and RAG-style workflows.”

“GenAI/LLM engineer and architect who built and deployed a production generative AI financial forecasting and scenario analysis platform at McKinsey, leveraging Claude (Anthropic), LangChain, Airflow, MLflow, and AWS SageMaker. Demonstrates strong LLMOps/MLOps rigor (monitoring, drift detection, automated retraining) and deep experience implementing global privacy controls (GDPR, differential privacy, audit trails) while partnering closely with finance executives and legal/IT stakeholders.”

“ML/NLP engineer focused on production GenAI and data linking systems: built a large-scale RAG pipeline over millions of support docs using LangChain/Pinecone and added a LangGraph-based validation layer to cut hallucinations ~40%. Also built scalable PySpark entity resolution (95%+ accuracy) and fine-tuned Sentence-BERT embeddings with contrastive learning for ~30% relevance lift, with strong CI/CD and observability practices (OpenTelemetry, Prometheus/Grafana).”

“Backend/product engineer with Tesla experience building and operating a real-time OTA update monitoring and fleet analytics platform at massive scale (telemetry from 3M+ vehicles). Delivered end-to-end systems across Kafka-based ingestion, TimescaleDB/Postgres analytics modeling, FastAPI/GraphQL APIs, and React/TypeScript dashboards, and handled production scaling incidents on AWS EKS during major rollout spikes.”

“Owned and built Bloomberg’s end-to-end bitemporal dividend & dividend-forecast data platform powering BQL for 400k+ terminal users. Architected real-time Kafka ingestion (5k–10k msgs/sec) across 100k+ tickers with strong correctness guarantees (PIT/bitemporal time-travel, immutable history to avoid look-ahead bias) and achieved sub-100ms p95 query latency through indexing and caching, deployed with Kubernetes + DLQ and robust monitoring.”

“Built a production LLM/RAG-based “model excellence scoring” system at Uber to automatically evaluate hundreds of ML models, standardizing quality assessment and cutting evaluation time from days to minutes on GCP. Also delivered an NLP document classification solution for insurance claims at Globe Life, partnering closely with compliance/operations and improving routing accuracy from ~85% manual to 93% with the model.”