Vetted Apache Spark Professionals

“Full-stack engineer with experience across Magna, C3.ai, and Amazon, building GenAI-enabled products and finance transaction systems. Has shipped Next.js (App Router) + TypeScript features backed by Go/Python RAG pipelines, and emphasizes production quality via load testing, Selenium regression coverage, LLM-aware integration testing, and Azure observability. Also built LangGraph-orchestrated multi-step content generation workflows with robust retry/idempotency strategies.”

“Meta engineer focused on reliability/operations tooling who built a unified real-time health dashboard and scalable telemetry pipelines (AWS + Datadog) for thousands of devices. Also shipped an internal LLM-powered knowledge assistant using RAG over wikis/runbooks/logs with strong guardrails and a rigorous eval loop that drove measurable accuracy improvements via automated doc ingestion and embedding updates.”

“Platform and distributed-systems leader (player-coach) who owned architecture and reliability for an Amazon analytics/data platform serving ~100K internal users at exabyte scale. Built an ML-driven “Lakeflow” optimization layer that cut pipeline completion times ~20–25% and reduced compute waste >15%, and led major incident response/redesign efforts (e.g., deletion storm) with strong rollout/observability/rollback practices.”

“CMU IoT coursework project builder who implemented an end-to-end TinyML gesture recognition system on a Particle Photon + ADXL345, streaming data via MQTT/Node-RED to a real-time Node.js frontend and deploying a quantized logistic regression model on-device. Also explored multi-drone coordination, implementing leader-follower offset control and a pivot/arc turning strategy to avoid collisions, and brings practical Docker/Kubernetes plus CI/CD workflow experience from internships.”

“Senior engineering leader in the Bay Area with experience spanning VMware, Hortonworks/Cloudera, Barracuda, and Palo Alto Networks, including leading open-source work (Apache Knox) and architecting large-scale security platforms. Has driven disaster recovery and cloud security products, designed Python microservices for Microsoft 365 security, and scaled teams (3x) while formalizing enterprise readiness practices with automated documentation using Notebook LLM.”

“Built and productionized a dual-agent LLM invoice-processing system for GFI Partners, adding guardrails and audit trails to earn stakeholder trust and drive adoption while cutting operational burden by 75%. Uses LangSmith observability to diagnose real-time workflow regressions and has experience teaching agentic AI concepts (e.g., at Carnegie Mellon) through hands-on, scaffolded demos.”

“Backend/data engineer with Optum healthcare claims domain experience building high-reliability Python microservices (FastAPI/Kafka/Postgres) and AWS data platforms (EKS, Glue, Redshift). Demonstrated strong production ownership: fixed duplicate Kafka processing via transactional outbox/idempotency, scaled to millions of daily events, and delivered major SQL performance gains (40+ min to <5 min, ~60% CPU reduction). Seeking remote-only work; targets $130k base.”

“Backend/data engineer with experience building Kafka-driven real-time pipelines that support ML code deployment and downstream integrations. Currently migrating high-throughput mainframe (COBOL/assembly) processing to Java, using Spark/Databricks to preserve performance and employing rigorous A/B testing across dev/pre-prod/prod with years of historical data.”

“Backend/data engineer with production experience at Walmart and HealthSnap building Python services and data pipelines on AWS (EKS, Lambda, Glue, Airflow). Strong reliability and operations focus—implemented idempotency + circuit breakers for peak-traffic consistency issues, GitOps CI/CD, and observability. Demonstrated measurable performance wins (Postgres p95 45s to <5s, ~60% CPU reduction) and modernized SAS batch workflows to Python with parallel-run parity validation and feature-flagged rollout.”

“Built and shipped a production LLM-driven multi-agent platform (ciATHENA) at CustomerInsights.AI to automate analytics/ML/compliance workflows in healthcare and life sciences. Implemented LangGraph/LangChain orchestration with strong backend-style rigor (schemas, Pydantic validation, retries, auditability) and optimized latency/cost while keeping the system usable for non-technical users via guided natural-language interactions and structured/visual outputs.”

“Former Xendit (YC-backed fintech) operator applying outcome-driven, scalable product-building experience to a new startup: a real-time, personalized real estate investment intelligence platform. Has begun prototyping, completed ROI analysis, and validated the space with US market research plus broker interviews in Chicago and Los Angeles, aiming to differentiate from Zillow/Redfin with goal-based ranked investment recommendations.”

“ML/LLM engineer with production experience building a multi-GPU LLM inference platform using TensorRT and vLLM, achieving ~40% p95 latency reduction through batching/KV caching, quantization, and CUDA/runtime tuning. Also has end-to-end orchestration experience (Kubernetes, Airflow) and has delivered real-time fraud detection systems at Accenture in close collaboration with non-technical risk and product stakeholders.”

“Built a production RAG-based "knowledge copilot" for support/ops using LangChain/LangGraph, implementing the full pipeline (ingestion, chunking, embeddings, vector DB retrieval/rerank, guarded generation with citations) and operating it as monitored microservices with CI/CD. Also designed an event-driven, streaming backend for real-time inventory ordering predictions that reduced stockouts by 25%, and has hands-on incident response experience stabilizing LLM API latency/5xx spikes using Datadog/APM and resilience patterns.”