Vetted Apache Spark Professionals

“AI/LLM-focused engineer with hands-on experience building RAG pipelines, prompt engineering workflows, and multi-agent systems using tools like LangChain. Stands out for combining AI-assisted development with production-grade validation and for leading the architecture/orchestration of agent-based recommendation systems that improved response time, accuracy, and scalability.”

“Engineering leader with hands-on platform, cloud, security, and data infrastructure experience who managed a 9-person cross-functional team spanning data engineering, infrastructure, SRE/DevOps, and DevSecOps. Notable impact includes completing a Mesos-to-ECS migration, cutting AWS spend by $1.6M annually, standing up a security engineering team, and building a no-code data lake/warehouse platform in partnership with sales, support, data science, and BI.”

“Built and productionized a dual-agent LLM invoice-processing system for GFI Partners, adding guardrails and audit trails to earn stakeholder trust and drive adoption while cutting operational burden by 75%. Uses LangSmith observability to diagnose real-time workflow regressions and has experience teaching agentic AI concepts (e.g., at Carnegie Mellon) through hands-on, scaffolded demos.”

“Backend/data engineer with Optum healthcare claims domain experience building high-reliability Python microservices (FastAPI/Kafka/Postgres) and AWS data platforms (EKS, Glue, Redshift). Demonstrated strong production ownership: fixed duplicate Kafka processing via transactional outbox/idempotency, scaled to millions of daily events, and delivered major SQL performance gains (40+ min to <5 min, ~60% CPU reduction). Seeking remote-only work; targets $130k base.”

“Backend/data engineer with production experience at Walmart and HealthSnap building Python services and data pipelines on AWS (EKS, Lambda, Glue, Airflow). Strong reliability and operations focus—implemented idempotency + circuit breakers for peak-traffic consistency issues, GitOps CI/CD, and observability. Demonstrated measurable performance wins (Postgres p95 45s to <5s, ~60% CPU reduction) and modernized SAS batch workflows to Python with parallel-run parity validation and feature-flagged rollout.”

“ML/LLM engineer with production experience building a multi-GPU LLM inference platform using TensorRT and vLLM, achieving ~40% p95 latency reduction through batching/KV caching, quantization, and CUDA/runtime tuning. Also has end-to-end orchestration experience (Kubernetes, Airflow) and has delivered real-time fraud detection systems at Accenture in close collaboration with non-technical risk and product stakeholders.”

Edward Yeh

Screened

Director-level Data Engineering & MDM leader specializing in enterprise data platforms

Hunt Valley, MD20y exp

“Former Xendit (YC-backed fintech) operator applying outcome-driven, scalable product-building experience to a new startup: a real-time, personalized real estate investment intelligence platform. Has begun prototyping, completed ROI analysis, and validated the space with US market research plus broker interviews in Chicago and Los Angeles, aiming to differentiate from Zillow/Redfin with goal-based ranked investment recommendations.”

greg farhadian

Screened

Senior Software Engineer specializing in cloud data platforms and Java microservices

Remote4y exp

“Backend/data engineer with experience building Kafka-driven real-time pipelines that support ML code deployment and downstream integrations. Currently migrating high-throughput mainframe (COBOL/assembly) processing to Java, using Spark/Databricks to preserve performance and employing rigorous A/B testing across dev/pre-prod/prod with years of historical data.”

Michael Kilgore

Screened

Mid-Level Software Development Engineer specializing in AWS data pipelines and forecasting systems

3y exp

“Built and deployed (via an Upwork contract) an LLM-powered agent for options trading that detects large options trade events, enriches them with market/filing data (price history, earnings transcripts, insider trading), and delivers recommendations via Telegram. Implemented schema-constrained outputs (Pydantic/Google GenAI), robust orchestration, logging, and error-notification handling, plus vector-DB-based reuse of prior outputs to improve consistency.”

Chaithanya Konda

Screened

Mid-level Data Engineer specializing in multi-cloud analytics platforms

Waltham, MA6y exp

“Data engineer with hands-on GCP platform experience spanning BigQuery, Cloud SQL, Dataflow, and Cloud Composer, including both production operations and cloud migration work. They led a migration from legacy SQL Server/Oracle systems to a cloud-native BigQuery architecture and cite measurable impact: processing reduced from hours to minutes, query latency improved 60%+, and ingestion time improved 40%.”

“Software engineer with practical, day-to-day experience embedding AI into development workflows across coding, testing, code review, and AWS data pipelines. Uses tools like Claude, Cline, JUnit, Mockito, and Amazon Bedrock, and stands out for having a realistic, mature view of agent limitations, hallucinations, and the need for strong prompting and human validation.”

“Senior software engineer candidate who uses AI and multi-agent workflows thoughtfully to speed up development while preserving engineering rigor for production-critical decisions. Stands out for a clear risk-based framework: leveraging agents for boilerplate, refactoring, testing, and debugging, while relying on fundamentals, metrics, and human review for system design and scalability.”

“Built and owned an LLM support copilot at Stripe focused on improving agent ticket resolution. Designed the backend and ML system end to end, using RAG, Redis caching, hybrid vector search, and LoRA fine-tuning to achieve 40% lower latency and 22% higher response accuracy, with continuous quality monitoring via Ragas and related evaluation frameworks.”

“Built a production RAG-based "knowledge copilot" for support/ops using LangChain/LangGraph, implementing the full pipeline (ingestion, chunking, embeddings, vector DB retrieval/rerank, guarded generation with citations) and operating it as monitored microservices with CI/CD. Also designed an event-driven, streaming backend for real-time inventory ordering predictions that reduced stockouts by 25%, and has hands-on incident response experience stabilizing LLM API latency/5xx spikes using Datadog/APM and resilience patterns.”