Vetted PySpark Professionals

“AI/LLM practitioner who built a GPT-2-like language model from scratch at the University of Maryland using PyTorch and multi-GPU distributed training, with experiment tracking in Weights & Biases. As an AI Operations intern at ScaleUp360, delivered multiple production-style AI agent automations (Gmail classification and Fireflies-to-Claude workflows that extract and assign CEO tasks) and set up measurable evaluation using test cases and classification metrics.”

“Full-stack engineer with experience at Zoho and Amazon who has owned production systems end-to-end, including a monolith-to-microservices migration using Kafka and Cassandra that improved search latency ~25% and increased throughput without data loss. Also built a hackathon project (Buildwise) into a sold product for a construction company (AI-driven document compliance checks) and shipped an IoT-based parking availability MVP in 3 weeks.”

“AI/ML engineer who has deployed production LLM/transformer-based systems for merchant intelligence and fraud/support optimization, delivering +27% merchant engagement and +18% payment success. Deep experience in privacy-preserving, PCI DSS-compliant data/ML pipelines (Airflow, AWS Glue, Spark, Delta Lake) and scalable microservices on Kubernetes, plus proven cross-functional delivery in healthcare claims analytics at UnitedHealth Group (12% HEDIS claim reduction).”

“Backend engineer who built a low-latency real-time transaction API in Python/Flask, with strong depth in PostgreSQL/SQLAlchemy performance tuning (time-based partitioning, indexing, connection pooling). Has production experience integrating ML scoring and OpenAI-style APIs with safety/latency controls, and designing multi-tenant isolation strategies including per-tenant pooling/caching and premium-tenant isolation.”

“Software engineer focused on data platforms and applied LLM systems: built an internal data quality monitoring layer to catch silent data drift and iterated post-launch after finding ~30% false-positive alerts, reducing noise via dynamic baselines and improved structured logging. Also shipped a production RAG-based internal knowledge assistant over Jira/Confluence with citations, confidence-based fallbacks, and nightly automated evals to prevent regressions.”

“Backend/data engineer from ASML who modernized a legacy SAS-based statistical processing system into a cloud-native AWS platform (Lambda/FastAPI, Step Functions/EventBridge, Glue, S3/RDS) with strong reliability and data-quality practices. Demonstrated measurable performance wins (RDS query reduced from 90+ seconds to <5 seconds) and hands-on incident ownership for production ETL pipelines.”

“Backend engineer who built a real-time pipeline for recording, transcribing, and analyzing audio from 400+ news radio stations, scaling Whisper on an HPC cluster with 7 H100 GPUs. Has strong performance optimization experience (30% latency reduction via SQL/query design; 50% DB call reduction via Redis caching) and has implemented region-based data isolation and PII protections in a regulated environment (JP Morgan Chase).”

“Qualcomm engineer building Android applications that run on Qualcomm AI accelerators, with hands-on experience in C++ concurrency, chipset stress testing, and power/performance tuning. Has deployed on-device AI models and built deployment/log post-processing workflows using Docker/Kubernetes and CI/CD; interested in translating this embedded AI/performance background into robotics (perception/real-time systems).”

“Data engineer at Plaid who built greenfield, end-to-end real-time transaction pipelines and FastAPI data services for fraud detection and analytics, handling millions of events per day. Strong focus on reliability and data integrity via Great Expectations validation, Airflow-based monitoring/SLAs, quarantine/staging patterns, and robust external data ingestion with schema versioning and backfills (reported 50% fewer anomalies and ~40% fewer failures).”

“Backend/data engineer with production experience building credit/fraud enrichment services and checkout pipelines on AWS (EKS + Lambda) using FastAPI, Kafka, Postgres, and Redis, with a strong focus on reliability patterns (timeouts/retries/circuit breakers) and observability. Has also built AWS Glue/PySpark ETL into S3/Redshift with schema evolution and data quality controls, and modernized legacy credit decisioning into Java/Node microservices with parallel-run parity validation and feature-flag rollouts.”

“Robotics/computer-vision engineer with industrial safety monitoring experience, building real-time pose estimation (TRTPose) and 2D-to-3D localization and optimizing pipelines to sustain 30+ FPS under heavy multi-entity load. Also brings edge-to-cloud distributed systems work (HoloLens + Google Vision/Translation) and production ML deployment experience using Docker/CI/CD across finance and edge camera environments.”

“Engineering leader who managed a 20-person cross-functional team building customer-driven software solutions, delivering a 50% reduction in simulation/test lifecycle and securing a long-term strategic SLA. Strong in scalable data ingestion architectures (FastAPI + Kafka + multiprocess workers), operational diagnostics (correlation IDs/centralized logging), and microservice decoupling for analytics/visualization. Active open-source contributor who shipped a NATS bug fix and improved SDK onboarding with automation that cut ramp time by 30%.”

“Built and shipped production improvements to a Paylocity RAG-based AI assistant, redesigning retrieval into a hybrid HNSW + keyword pipeline and using tuned RRF to fuse rankings—cutting latency by ~2s and reducing token usage by ~5000. Previously spearheaded Apache Airflow integration across ETL pipelines at Acuity Knowledge Partners, creating reusable templates and automated triggers to reduce manual job monitoring.”