Vetted Apache Spark Professionals

“Machine learning/NLP engineer who built a production-oriented retrieval-based AI system at Morgan Stanley for healthcare use cases, combining RAG over unstructured patient records with deep-learning medical image segmentation (U-Net/Mask R-CNN). Strong in end-to-end pipelines and MLOps (Spark/MongoDB, AWS SageMaker, CI/CD, monitoring, automated retraining) and in entity resolution/data quality validation for noisy clinical data.”

“GenAI/ML engineer with production experience building multilingual LLM systems (English/Spanish) and RAG-based clinical documentation summarization at Walgreens, combining prompt engineering, structured output validation, and rigorous evaluation (ROUGE + pharmacist review). Also orchestrated end-to-end ML pipelines for demand forecasting using Apache Airflow, PySpark, and MLflow with scheduled retraining and production monitoring.”

“LLM engineer who built a production document intelligence/RAG pipeline to extract structured data from thousands of unstructured PDFs, cutting manual review time by 60%. Experienced with LangChain and Airflow orchestration plus rigorous evaluation (labeled datasets, prompt testing, HITL review, monitoring) to improve accuracy and reduce hallucinations while partnering closely with non-technical operations stakeholders.”

“Backend/founding-engineer-style builder who designed and evolved a near-real-time customer churn prediction platform (FastAPI + AWS SageMaker/Lambda + Redis + MLflow) to enable real-time retention actions, reporting ~18% churn reduction. Demonstrates strong production engineering in secure API design, incremental migrations with data integrity safeguards, and robustness improvements in async pipelines (idempotency, DLQs, retry visibility).”

“Built and deployed a production LLM-based real-time stance detection system for social media, fine-tuning LLaMA 3.1 on A100s with DeepSpeed ZeRO/FSDP and iteratively refining data to handle sarcasm and context-dependent meaning. Also has Kubernetes operations experience (Kafka/Logstash/Elasticsearch observability pipeline) and delivered an OCR automation project during a Worley India internship that saved 20+ hours/week for on-site energy safety stakeholders.”

“Built and shipped a production LLM/RAG risk-case summarization and triage system used by fraud/compliance analysts, with strong grounding controls (evidence-cited outputs and refusal on low confidence). Demonstrates end-to-end ownership across retrieval quality, Airflow-orchestrated indexing pipelines, and compliance-grade privacy (PII redaction, RBAC, encrypted redacted logging, and auditable prompt/model versioning) plus a tight feedback loop with non-technical domain experts.”

“GenAI/ML engineer from Lucid Motors who built and productionized an LLM-powered RAG diagnostic assistant for manufacturing and maintenance teams, deployed on AWS with Docker/Kubernetes and MLflow. Demonstrates end-to-end ownership from retrieval/prompt design to scalability, monitoring, and workflow integration via APIs, plus production ML pipeline orchestration with Kubeflow (Spark/Kafka + TensorFlow) for predictive maintenance use cases.”

“Computer science engineer and master’s graduate who independently built a mechatronics-heavy capstone prototype: a smartphone concept for deafblind users using micro-actuator arrays for braille reading. Also has platform engineering experience at Quantiphi, deploying webhooks to Kubernetes and implementing GitOps-based CI/CD using AWS CodeCommit/CodeBuild and ECR.”

“Built and deployed a production LLM-powered market intelligence and decision-support platform for noisy, real-time financial data, using a high-throughput embedding + vector DB RAG architecture to reduce hallucinations while keeping latency and cost low. Operated it at scale with GPU-backed inference (continuous batching/quantization), FastAPI on Kubernetes, and Airflow-orchestrated ingestion/embedding/retraining workflows, with strong schema-based reliability and monitoring.”

“AI/ML engineer with production experience building an LLM-powered resume-to-job matching and feedback product using RAG, with a strong focus on latency, hallucination control, and scalable deployment. Experienced orchestrating ML inference and backend services on Kubernetes and applying rigorous evaluation/guardrail practices; also partnered with business/product stakeholders at Walmart to improve an NLP-based supplier support system.”

“Built and deployed a production LLM-powered RAG assistant for semiconductor manufacturing failure analysis, reducing engineer triage effort by grounding outputs in retrieved evidence and gating responses with SPC + ML signals (LSTM anomaly scores, XGBoost probabilities). Experienced with LangChain/LangGraph to ship reliable, observable multi-step agents with branching/fallback logic, and evaluates impact using both technical metrics and business KPIs like mean time to triage and downtime reduction.”

“Built and deployed a production LLM/RAG chatbot at Wells Fargo for securely querying regulated financial and compliance documents, emphasizing low hallucination rates, explainability, and strict governance. Experienced with LangChain multi-agent orchestration plus Airflow/Prefect pipelines for ingestion, embeddings, evaluation, and retraining, and partnered closely with compliance/operations to drive adoption through demos and feedback-driven retrieval rules.”

“Backend engineer with strong AWS/CI/CD experience (multi-repo deployments, Lambda + core app, immutable ECR and image promotion) and a published master’s thesis building an ML framework for Solar PV energy prediction and CO2 reduction impact modeling using ensemble and meta-learning approaches benchmarked against SAM.”

“Internship experience building production-oriented AI systems, including a real-time voice scam/spoof detector (RawNet + AASIST) hardened for noisy audio via aggressive augmentation and Zoom-based noise simulation, evaluated with EER on clean and wild datasets. Also built an LLM-driven UI automation agent using Playwright for apps like Linear/Notion with modular tool design, unit tests, and replayable scripted scenarios, and has AWS Step Functions experience orchestrating Lambda/Cognito workflows.”

“Software Developer at NYU (GEMSS) focused on scaling and optimizing a data-heavy asset management web app, including migrating/optimizing data access via Google Sheets API and Firestore. Previously an SDE at Sainapse working on Spring Boot microservices POCs (Kafka, Hadoop at 2B+ record scale). Built an end-to-end Apple Wallet coupon generation/redemption system using PassKit + Google Apps Script with measurable ops impact (40% efficiency gain).”

“Built and deployed an AWS-based LLM/RAG ticket triage and knowledge retrieval system (Pinecone/FAISS + Step Functions + MLflow) that cut support resolution time by 20%. Demonstrates strong production focus on hallucination reduction, PII security, and low-latency orchestration, with measurable evaluation improvements (e.g., ~25% grounding accuracy gain via re-ranking) and proven collaboration with support operations stakeholders.”

“ML/NLP practitioner with experience building a transformer-failure prediction system that combines sensor signals with unstructured maintenance comments using LLM-based extraction and similarity validation. Strong emphasis on production readiness—data leakage controls, SQL-driven data quality tiers, and rigorous bias/fairness validation (including contract/spec evaluation across diverse company profiles).”

“Full-stack/AI-focused builder who shipped a production Career Advisor app using LLMs + RAG + vector DB (React/Node/MongoDB/Claude API) and grew it to 2000+ users, handling real deployment issues and CI/CD on Vercel/Render. Also developing an AI-powered iOS “3D World Explorer” (text-to-3D) and has cloud experience across Azure and AWS (S3/SageMaker/EC2).”

“Backend/data platform engineer who recently owned an end-to-end large-scale financial data platform delivering real-time decision support for finance and operations. Has hands-on experience modernizing legacy batch pipelines into AWS cloud-native ELT with parallel-run cutovers, strong data quality controls (dbt-style tests, reconciliation), and measurable improvements in runtime, cost, and SLA compliance. Also builds scalable, secure FastAPI microservices using Docker, ALB-based horizontal scaling, Redis caching, and managed auth with Cognito/Supabase plus Postgres RLS.”

“Backend/platform engineer who built a high-scale audience segmentation and real-time targeting system using Spark/Glue + S3/Hudi and low-latency API services backed by Redis/relational stores. Demonstrates strong production rigor: Spark performance tuning to eliminate OOM failures, API idempotency/caching to cut p95 latency ~40%, and careful dual-run/feature-flag migrations with reconciliation and rollback runbooks. Experienced implementing layered security with JWT/OAuth, RBAC/ABAC, and database row-level security to prevent privilege escalation.”

“AI/ML engineer who built a production agentic system to automate computational research experiments (simulation execution, parameter exploration, and numerical analysis) and mitigated context-window failures using constrained tool-calling/prompt-chaining patterns in LangChain with OpenAI tool-enabled models. Also has adtech/big-data pipeline experience at InMobi, orchestrating Spark jobs in Airflow to filter bot-like user IDs and publish clean IDs to an online NoSQL store for live serving, plus Apache open-source collaboration experience.”

“Built and deployed a production LLM-driven Dungeons & Dragons game where the model acts as a dungeon master, adding a structured combat system and a macro-state tree to ensure campaigns converge to a clear ending. Fine-tuned Gemini 2.5 Flash on Vertex AI and deployed on GCP with Kubernetes, using RAG over DnD rules/spells plus multi-agent orchestration (intent-based routing between narrative and combat agents) to reduce hallucinations and improve reliability.”

“Built and deployed a production LLM/RAG clinical decision support system that enables real-time semantic search over unstructured EHR notes and delivers patient risk insights. Strong in healthcare-grade MLOps and compliance (HIPAA, PHI handling, encryption, RBAC, audit logs) and scaled embedding/retrieval pipelines using Spark/Databricks and Airflow. Partnered with clinicians via Power BI dashboards and explainability, contributing to an 18% reduction in patient readmissions.”