Vetted PySpark Professionals

“Backend/ML-focused software engineer who built and evolved a Python/FastAPI backend for a large-scale conversational AI platform, decoupling API and inference services to improve stability and deployment velocity. Experienced in production hardening (timeouts/fallbacks/monitoring), secure multi-tenant systems (JWT/RBAC/RLS), and low-risk migrations using shadow deployments and incremental traffic ramp-ups.”

“Backend/data engineer with production experience building secure Django/DRF APIs (JWT RS256 + rotating refresh tokens), background processing with Celery, and strong reliability practices (timeouts, retries/backoff, structured logging, audit trails). Has delivered AWS solutions spanning Lambda + ECS with IaC/CI-CD and built Glue/PySpark ETL pipelines with schema evolution and data-quality quarantine patterns; also modernized a legacy SAS pipeline to Python/PySpark with parallel-run parity validation and phased rollout.”

“LLM/ML platform engineer with hands-on experience taking an LLM document summarization prototype into a production-grade service on AWS EKS, emphasizing low-latency inference, drift monitoring, and safe CI/CD rollouts (canary + rollback). Strong in real-time debugging of agentic/RAG systems (tracing, retrieval/index drift fixes) and in developer enablement through practical workshops (Docker/Kubernetes/FastAPI) plus pre-sales support via demos and benchmarks to close pilots.”

“Founding AI engineer (June 2024) at Talon Labs who built and productionized an LLM-powered chatbot for interacting with proprietary supply-chain documents, deployed at large scale (25–100,000 users). Experienced with RAG/LLM orchestration (LangChain, LlamaIndex, Groq AI) and production ops tooling (Kubernetes, Docker, Kubeflow, Airflow), with a metrics-driven approach to evaluation, observability, and stakeholder alignment.”

“Backend engineer who owned a Python task management API with JWT auth, async notifications, and performance work (DB optimization/caching) to handle high volumes. Led an on-prem to Azure private cloud migration at Morgan Stanley using GitOps and IaC (Terraform/ARM) with phased rollout and rollback planning. Also built a Kafka real-time streaming pipeline with exactly-once/idempotent consumers and Prometheus/Grafana monitoring.”

“Finance-domain ML/LLM engineer who has shipped production systems including a RAG-based financial insights assistant with a custom post-generation validation layer that verifies atomic claims against retrieved source text to prevent hallucinations in compliance-critical workflows. Also built large-scale MLOps automation on AWS using Kubeflow + MLflow + CI/CD for fraud detection and credit risk models processing 500M+ transactions/day with a 99.99% uptime goal, and partnered closely with JP Morgan risk/compliance stakeholders on NLP-driven compliance monitoring.”

“Backend engineer with strong Python/FastAPI microservices ownership, including an ML-serving service with embeddings, async DB access, and Redis caching to reduce latency under high load. Experienced deploying and operating containerized services on Kubernetes using GitOps (Argo CD/Helm) with automated CI/CD, plus hands-on Kafka streaming pipeline tuning and enterprise migration work (Infosys) using blue-green/active-passive strategies.”

“PayPal ML/AI practitioner who built and productionized a hybrid recommendation engine (BERT/LLM embeddings + collaborative filtering + XGBoost ranking) on AWS with end-to-end MLOps and orchestration. Addressed real-world issues like cold start and embedding latency (ONNX, clustering, caching, PySpark/Delta Lake) and drove a 27% lift in upsell conversion via A/B testing and stakeholder collaboration with marketing.”

“Healthcare data/ML practitioner with experience at UnitedHealth Group building production ETL and streaming pipelines (Python, BigQuery, Kafka) that unify EHR, IoT device, and lab data for patient risk prediction. Also implemented embedding-based semantic search/linking for noisy clinical notes via domain adaptation and rigorous validation with clinical stakeholders; previously built churn prediction at DirecTV using XGBoost.”

“Strategic sourcing professional specializing in seasonal apparel supply chains, combining Coupa/JD Edwards analytics with Excel/Python modeling and Power BI dashboards to drive cost reduction and OTIF gains. Notable for rapid mitigation of a 10-day factory delay affecting 12 holiday SKUs (preserved 95% of revenue) and for automating PO workflows to cut cycle time by 4.2 days and improve OTIF by 15%.”

“ML/LLM engineer who has deployed a production LLM-powered assistant for intent classification and query routing (order recommendation/support deflection), combining BERT fine-tuning with an embedding-based retrieval layer and optimizing for low-latency inference. Experienced with end-to-end reliability practices—Airflow-orchestrated ETL, data validation/alerting, MLflow experiment tracking, and iterative improvements driven by user feedback and monitoring.”

“Data/MLOps engineer (Cognizant background) who owned an AWS/Airflow/Snowflake healthcare transactions pipeline processing ~8–10M records/day and cut pipeline/data-quality incidents by ~33%. Also built and deployed a production FastAPI model-inference service on Kubernetes (Docker, HPA) with strong observability (Prometheus/Grafana), versioned endpoints, and resilient backfill/idempotent external data ingestion patterns.”

“Built and deployed a production RAG-based internal knowledge assistant that let analysts query company documents in natural language, using LangChain/LangGraph with Pinecone and a FastAPI service for integration. Emphasizes reliability in production through hallucination mitigation (retrieval tuning + prompt guardrails) and measurable evaluation/monitoring (accuracy, latency, task completion, hallucination rate), iterating based on user feedback.”

“Backend engineer with experience evolving a high-volume agricultural loan processing platform (APMS) at HDFC Bank, emphasizing transactional integrity, auditability, and modularity while integrating with credit bureaus, document management, and risk engines. Also improved automation/reporting robustness at Trend Micro by catching duplicate-event retry edge cases and adding idempotency safeguards.”

“At KPMG, built and productionized a secure RAG-based LLM assistant that lets business and risk stakeholders query data warehouses in natural language, reducing dependence on data engineers for ad-hoc analysis. Demonstrates strong production rigor (Airflow orchestration, CI/CD, containerization), retrieval/embedding tuning (rechunking, semantic abstraction for structured data), and reliability controls (confidence thresholds, refusal behavior, monitoring and canary evals).”

“Built and deployed a production agentic LLM knowledge assistant that answers complex questions over internal documents, APIs, and databases using a RAG architecture (FAISS/Pinecone) and LangChain/LangGraph orchestration. Emphasizes production-grade reliability and hallucination control through grounding, confidence thresholds, validation, retries/fallbacks, and full observability (logging/metrics/traces) with continuous evaluation and feedback loops.”

“Open-source JavaScript contributor focused on performance and maintainability in data visualization libraries—refactored legacy ES5 into modular ES6, added tests/docs, and delivered ~30% faster load times with positive community adoption. Also optimized a React dashboard (~40% load-time reduction) and took ownership in an ambiguous AI product initiative by setting milestones, standing up an initial ML pipeline, and shipping a prototype in ~6 weeks that became the basis for production.”

“Software engineer who owned end-to-end delivery of customer-facing agricultural forecast reporting (crop yield/health) and iterated quickly via rigorous edge-case testing and customer feedback. Also built an internal ML training platform (TypeScript/React + Flask/Python + MongoDB) used by every developer, with architecture designed to stay responsive under heavy compute load.”

“ML/GenAI engineer with production experience building an LLM-powered governance layer that summarizes verified drift/performance signals into validation reports and release notes, designed for regulated environments with de-identification and non-blocking fallbacks. Strong Airflow-based orchestration background across healthcare and finance, integrating Databricks/Spark and MLflow for scalable retraining/monitoring. Demonstrated ability to partner with non-technical healthcare operations teams to deliver actionable risk-scoring outputs via dashboards and automated reporting.”

“Backend/data engineering candidate specializing in Python/Flask services and ML-enabled systems, deploying containerized workloads on AWS ECS/EKS with strong observability (Prometheus/Grafana) and PostgreSQL performance tuning. Built multi-tenant architectures with row- and schema-level isolation and optimized a Kubernetes-based Airflow + Spark nightly ETL pipeline for an e-commerce client, improving performance by 250%+ and reliably beating morning reporting deadlines; also contributed to Apache Airflow (SQLAlchemy/PostgreSQL area).”

“Built production LLM systems including a real-time customer feedback analysis and workflow automation platform using RAG and multi-agent orchestration with confidence-based human escalation, addressing privacy and legacy integration challenges. Also automated ML operations with Airflow/Kubernetes (e.g., daily churn model retraining) cutting retraining time to under 30 minutes, and demonstrates a rigorous testing/monitoring approach plus strong non-technical stakeholder collaboration.”

“ML/GenAI engineer with recent CVS Health experience building a production RAG system over unstructured financial/research documents using LangChain, FAISS, and Pinecone, plus LoRA/PEFT fine-tuning of GPT/LLaMA for domain-aware summarization. Demonstrates strong applied MLOps and data engineering skills (Airflow/Prefect, Docker/Kubernetes, CI/CD, MLflow) and measurable impact (sub-second retrieval, ~40% better context retrieval, ~25% entity matching improvement).”

“Built a production "sentinel insight engine" to tame information overload from millions of product reviews and support transcripts, combining Azure OpenAI (GPT-3.5) zero-shot classification with a fine-tuned T5 summarizer to generate weekly actionable product insights. Demonstrated strong MLOps/production engineering by adding drift monitoring with embedding-based detection, integrating REST with legacy SOAP/queue-based CRM via FastAPI middleware, and scaling reliably on Kubernetes with HPA.”

“Healthcare-focused data engineer/ML practitioner with experience at Lightbeam Health Solutions and Humana building production entity-resolution and semantic similarity pipelines across EMR, lab, and claims data. Uses NLP/ML (spaCy, scikit-learn, BioBERT/LightGBM) plus Snowflake/Airflow and vector search (Pinecone) to improve linkage accuracy (reported 90%) and semantic match quality (reported +12–15%), while reducing manual cleanup by 40%+.”