Vetted PySpark Professionals

“McKinsey & Company ML/NLP practitioner who builds production-grade AI systems across sectors (notably healthcare and finance), including RAG/LLM solutions, entity resolution pipelines, and embedding-powered search with vector databases. Demonstrated measurable impact (40% reduction in data duplication) and strong MLOps/data workflow practices (Airflow, MLflow, Spark, AWS/GCP, Prometheus, CI/CD).”

“NVIDIA engineer who built and shipped a production LLM-powered enterprise knowledge system (summarization, transcription, and Q&A) that cut document retrieval time ~30%. Deep hands-on experience with RAG (FAISS/Pinecone), GPU-accelerated microservices on AWS, and reliability/safety practices (Guardrails AI, prompt A/B testing, canary releases) plus strong MLOps orchestration across Airflow, Step Functions, and Kubernetes GitOps.”

“Built and productionized a real-time enterprise RAG pipeline to improve factual accuracy and reduce LLM hallucinations by grounding responses in constantly changing internal knowledge bases (policies, manuals, FAQs). Experienced in orchestrating end-to-end ML workflows (Airflow/Kubernetes), handling messy multi-format data with schema enforcement (Pydantic/Hydra), and maintaining freshness via streaming incremental embeddings plus batch refresh. Also delivers applied ML solutions with non-technical teams (marketing/CRM) for segmentation and personalized engagement.”

“Built and shipped an LLM-powered data quality assistant that generates maintainable validation checks from metadata while executing validations via Great Expectations, exposed through FastAPI and integrated into Airflow-managed pipelines. Emphasizes production reliability (structured outputs, guardrails, monitoring, versioning, human review) and works closely with compliance/operations teams to deliver clear, auditable, user-friendly AI outputs.”

“LLM/agentic-systems builder who has shipped production tools for investment research and procurement insights, including a company screener that processes thousands of conference-listed companies using FireCrawl + Google Search + Gemini. Demonstrates strong orchestration expertise (LangGraph multi-agent graphs), performance optimization (async/batching to sub-30s), and pragmatic reliability/evaluation practices with stakeholder-friendly UX (real-time cost tracking and model/parameter toggles).”

“Backend/data engineer with Amazon supply-chain experience building production serverless Python services and ETL pipelines on AWS (Lambda, API Gateway, S3, RDS, Glue). Has modernized legacy SAS jobs into Python with rigorous parity testing and phased migrations, and has delivered major SQL performance gains (minutes down to seconds) through indexing and partitioning.”

“Backend/data engineer from Oscar Health specializing in healthcare claims systems on AWS. Built HIPAA-compliant real-time services (FastAPI/Postgres/Kafka on EKS) and serverless ingestion pipelines, and led modernization of a legacy SAS claims pricing system to Python/Spark with rigorous parity validation. Demonstrated measurable impact with high uptime/low latency services and major Snowflake performance and cost reductions.”