Vetted PySpark Professionals

“Amazon data engineer who built a real-time fraud detection pipeline for AWS Lambda, tackling multi-region telemetry quality issues and scaling stream processing for billions of daily requests. Strong in production-grade data/ML workflows on AWS (EMR, Glue, Kinesis, SageMaker) with hands-on entity resolution and anomaly detection.”

“Backend/data engineer with production experience building FastAPI services with strong reliability patterns (circuit breaker, rate limiting, caching, graceful degradation) and JWT/OAuth2 auth. Has delivered AWS EKS deployments via Terraform with Secrets Manager/IRSA and HPA autoscaling, and built Glue/Spark ETL pipelines on S3 Parquet with schema-evolution and idempotent reruns; also demonstrated measurable SQL tuning impact (20–30s to <10s).”

“Backend/data engineer with hands-on production AWS experience spanning serverless APIs (Chalice/Lambda/API Gateway/Cognito) and data pipelines (Glue PySpark + Step Functions). Has modernized a legacy SAS reporting system into AWS microservices and implemented schema-drift detection and incident prevention for ETL workflows, plus measurable SQL tuning wins (30 min to <10 min runtime).”

“LLM/RAG engineer who built a production-style multi-agent orchestrator for resume-to-recommendation workflows (PDF ingestion through screening and recommendations), emphasizing prompt tuning and strict JSON output contracts. Currently building a RAG application for an NGO using Airflow (DAGs + embeddings) and tackling messy, missing/imbalanced data; has hands-on retrieval stack experience (FAISS/HNSW, bge embeddings) and uses rigorous evaluation metrics for groundedness and hallucination control.”

“Backend engineer with experience at Meta and Accenture building regulated-data systems (healthcare/financial) using Python/Flask and Postgres. Has scaled high-throughput services to millions of daily requests, delivering measurable latency wins (~40% API latency reduction; ~35% faster DB-backed endpoints), and has productionized ML inference services using Docker/Kubernetes and AWS (ECS/SageMaker).”

“ML/LLM engineer at Adobe who deployed a transformer-based personalization and campaign-targeting recommender system end-to-end, including PySpark/Airflow pipelines processing 12M+ events/day and containerized inference on AWS SageMaker (Docker/Kubernetes). Also has hands-on LLM workflow experience (RAG, semantic search, prompt optimization, hallucination mitigation) with a metrics-driven approach to reliability, drift monitoring, and reproducible retraining via MLflow.”

“Founder of Photosphere Labs (agentic AI for ecommerce data synthesis/analysis) who worked directly with customers to scope, build, demo, and iterate LLM-based solutions, including an AI chat product for brand owners. Previously at Block, built and explained a nuanced causal inference/propensity model tied to Square POS integrations, translating model specs and outputs into business impact for varied client contexts.”

“Built a production RAG-based natural-language-to-SQL system at Global Atlantic to replace slow, expensive manual analytics ticket workflows, focusing heavily on retrieval quality and measurable evaluation (200-question ground-truth set; recall@5 improved 0.65→0.78 via semantic chunking). Also built a custom MCP-style agent orchestrator for a personal project (arxiv-ai) to improve flexibility and Langfuse-aligned observability, and has hands-on experience with LangGraph, CrewAI, and n8n.”

“Backend/data engineer with Optum healthcare claims domain experience building high-reliability Python microservices (FastAPI/Kafka/Postgres) and AWS data platforms (EKS, Glue, Redshift). Demonstrated strong production ownership: fixed duplicate Kafka processing via transactional outbox/idempotency, scaled to millions of daily events, and delivered major SQL performance gains (40+ min to <5 min, ~60% CPU reduction). Seeking remote-only work; targets $130k base.”

“Backend/data engineer with experience building Kafka-driven real-time pipelines that support ML code deployment and downstream integrations. Currently migrating high-throughput mainframe (COBOL/assembly) processing to Java, using Spark/Databricks to preserve performance and employing rigorous A/B testing across dev/pre-prod/prod with years of historical data.”

“Backend/data engineer with production experience at Walmart and HealthSnap building Python services and data pipelines on AWS (EKS, Lambda, Glue, Airflow). Strong reliability and operations focus—implemented idempotency + circuit breakers for peak-traffic consistency issues, GitOps CI/CD, and observability. Demonstrated measurable performance wins (Postgres p95 45s to <5s, ~60% CPU reduction) and modernized SAS batch workflows to Python with parallel-run parity validation and feature-flagged rollout.”

“ML/LLM engineer with production experience building a multi-GPU LLM inference platform using TensorRT and vLLM, achieving ~40% p95 latency reduction through batching/KV caching, quantization, and CUDA/runtime tuning. Also has end-to-end orchestration experience (Kubernetes, Airflow) and has delivered real-time fraud detection systems at Accenture in close collaboration with non-technical risk and product stakeholders.”