Vetted Apache Spark Professionals

“AI/ML engineer who has deployed production LLM/transformer-based systems for merchant intelligence and fraud/support optimization, delivering +27% merchant engagement and +18% payment success. Deep experience in privacy-preserving, PCI DSS-compliant data/ML pipelines (Airflow, AWS Glue, Spark, Delta Lake) and scalable microservices on Kubernetes, plus proven cross-functional delivery in healthcare claims analytics at UnitedHealth Group (12% HEDIS claim reduction).”

“Software engineer focused on data platforms and applied LLM systems: built an internal data quality monitoring layer to catch silent data drift and iterated post-launch after finding ~30% false-positive alerts, reducing noise via dynamic baselines and improved structured logging. Also shipped a production RAG-based internal knowledge assistant over Jira/Confluence with citations, confidence-based fallbacks, and nightly automated evals to prevent regressions.”

“Ticketmaster data engineer who built CDC-driven Kafka pipelines feeding Snowflake for analytics and data science teams. Hands-on in production operations—scaled Kafka during sudden playoff-driven transaction spikes and improved monitoring for preemptive scaling. Known for using small-batch experiments and quantitative metrics to align stakeholders and drive cost-saving architecture changes (e.g., buffering to reduce AWS Lambda invocation frequency).”

“Backend engineer with experience at UnitedHealth Group redesigning a high-traffic Spring Boot microservice from blocking to reactive architecture during peak season, cutting median latency by 47% for a service used by ~10M customers annually. Strong in Kubernetes-based deployment/scaling and pragmatic rollout strategies (blue-green/incremental traffic shifting) with performance and database troubleshooting.”

“Teradata Text Security intern who built a production LLM-powered planner agent that decomposes complex tasks into dependency-aware subtasks (DAG/topological graph) and executes them via a custom orchestrator with parallelism, status tracking, and error handling. Also contributed to an HR-facing internal document chatbot concept to streamline onboarding, showing cross-functional collaboration.”

“ML/AI engineer focused on production-grade model reliability: built a monitoring and validation framework to detect drift, trigger anomaly alerts/retraining, and maintain consistent performance for device intelligence workflows at scale. Strong MLOps background with Python pipelines, Docker/Kubernetes deployments, Airflow orchestration, and real-time monitoring dashboards; experienced partnering with product managers to deliver business-facing insights.”

“Built a production LLM/RAG assistant for insurance/health claims agents that ingests 100–200 page patient PDFs via OCR (migrated from local Tesseract to Azure Document Intelligence) and delivers grounded claim detail retrieval plus summaries with PII/PHI guardrails. Experienced orchestrating large workflows with Celery worker pipelines and AWS Step Functions (S3-triggered, Fargate-based batch inference/accuracy aggregation), and collaborates closely with non-technical SMEs (claims agents/nurses) through shadowing, iterative demos, and SME-defined evaluation.”

“AI engineer at Anote.ai who built and shipped a production multi-agent LangGraph/LangChain/Ray RAG platform for enterprise search and workflow automation, supporting 3 commercial products and 100+ developers. Drove measurable gains (30% accuracy improvement, lower latency) and improved reliability with Redis-based state checkpointing, message-queue synchronization, and Milvus retrieval optimizations, while partnering with PMs/clients to add transparency features like confidence scores and real-time logs.”

“LLM infrastructure engineer who built a production-critical real-time personalization and memory retrieval system for a user-facing product, adding <100ms P99 latency while improving relevance ~20–25% and holding SLA through 3x traffic. Experienced designing tiered retrieval backends (Redis + vector store), deploying on Kubernetes with autoscaling/circuit breakers, and running rigorous observability, incident response, and agent evaluation (shadow traffic, A/B tests, regression/replay).”

“Capital One engineer leading the architecture and delivery of a large-scale AWS Glue/Spark/Delta Lake batch messaging pipeline that decoupled batch from real-time flows, added multi-region failover and automated retries, and delivered ~40% AWS cost savings with ~3x performance gains. Currently building an LLM-powered Slack bot using RAG to automate message investigations by querying CloudWatch, Snowflake, and internal documentation with privacy-aware masking of NPI/PII.”

“Data engineer at Plaid who built greenfield, end-to-end real-time transaction pipelines and FastAPI data services for fraud detection and analytics, handling millions of events per day. Strong focus on reliability and data integrity via Great Expectations validation, Airflow-based monitoring/SLAs, quarantine/staging patterns, and robust external data ingestion with schema versioning and backfills (reported 50% fewer anomalies and ~40% fewer failures).”

“Backend/data engineer with production experience building credit/fraud enrichment services and checkout pipelines on AWS (EKS + Lambda) using FastAPI, Kafka, Postgres, and Redis, with a strong focus on reliability patterns (timeouts/retries/circuit breakers) and observability. Has also built AWS Glue/PySpark ETL into S3/Redshift with schema evolution and data quality controls, and modernized legacy credit decisioning into Java/Node microservices with parallel-run parity validation and feature-flag rollouts.”

“Backend-focused engineer with hands-on experience modernizing a legacy SNMP/PNM data collection system at CableLabs into a cloud-accessible Kubernetes pipeline, feeding Prometheus-formatted metrics into VictoriaMetrics and visualizing real-time network health in Grafana for 100+ modems. Also built a FastAPI + Supabase appointment booking portal for a clinic with encryption and phone-number-based auth, and has frontend experience debugging S3-based HEIF image rendering issues.”