Vetted PySpark Professionals

“Owned and built Bloomberg’s end-to-end bitemporal dividend & dividend-forecast data platform powering BQL for 400k+ terminal users. Architected real-time Kafka ingestion (5k–10k msgs/sec) across 100k+ tickers with strong correctness guarantees (PIT/bitemporal time-travel, immutable history to avoid look-ahead bias) and achieved sub-100ms p95 query latency through indexing and caching, deployed with Kubernetes + DLQ and robust monitoring.”

“Data engineer with experience at Moderna and Block owning high-volume (≈10TB/day) production pipelines on AWS, using Kafka/S3/Glue/dbt/Snowflake with strong data quality and observability practices (schema validation, anomaly detection, CloudWatch monitoring). Also built external financial API ingestion with Airflow retries, throttling/token rotation, and schema versioning, and helped stand up an early-stage biomedical data platform with CI/CD and incident debugging.”

“Built a production LLM/RAG-based “model excellence scoring” system at Uber to automatically evaluate hundreds of ML models, standardizing quality assessment and cutting evaluation time from days to minutes on GCP. Also delivered an NLP document classification solution for insurance claims at Globe Life, partnering closely with compliance/operations and improving routing accuracy from ~85% manual to 93% with the model.”

“Healthcare data platform builder with experience at Aetion delivering a rule-based EMR/EHR ingestion and validation framework that cut onboarding from 8–10 weeks to hours and unlocked $30M+ in revenue over ~3 years. Motivated to found an AI/agent-driven healthcare solution, with a specific interest in using PET scans, doctor notes, and treatment data with LLMs to help predict cancer progression and guide next-step treatments.”

“Built and shipped a production LLM-powered investor research feature for a fintech product, focused on grounded answers and minimizing hallucinations. Implemented retrieval-quality and evidence-coverage gating with clear refusal fallbacks, and evaluates systems with regression tests and metrics like correct-refusal rate, hallucination rate, and latency. Comfortable orchestrating workflows with LangChain or custom Python depending on production needs.”

“AI/LLM engineer at Capital One who deployed a production RAG-powered fraud analysis and document intelligence platform using LangChain, OpenAI, Pinecone, Kafka, and AWS. Focused on reliability in real-time investigations via hybrid retrieval, schema-validated outputs, and LLM verification loops, reporting review-time reduction from hours to minutes and ~99% fraud detection precision.”

“Built a hackathon "Patient Summary Assistant" backend focused on healthcare workflows, combining RAG-based summarization with HIPAA-minded privacy controls (NER redaction + encryption). Demonstrated strong infra skills by deploying on Kubernetes with Helm/HPA and GitOps (ArgoCD), plus migrating from OpenAI to an on-prem Llama 3 stack (vLLM, quantization, shadow-mode testing) and adding real-time Kafka ingestion for patient vitals/anomaly alerts.”

“Built and deployed a production RAG-based multi-agent chatbot during an internship to help consultants answer client questions and guide users through new IT systems with step-by-step instructions. Demonstrates hands-on experience with LangGraph/LangChain/Google ADK, unstructured document parsing and chunking for RAG, and a reliability-first approach to agent workflows (metrics, fallbacks, human-in-the-loop, guardrails).”

“ML/LLM engineer building production, real-time fraud detection for financial transactions using a two-tier architecture (fast ML + GPT) to deliver both low-latency decisions and analyst-friendly risk explanations. Experienced orchestrating end-to-end retraining, drift monitoring, and automated model promotion with Databricks Jobs/Workflows and MLflow, and partnering closely with fraud analysts to tune alerts, thresholds, and dashboards.”

“ML/AI engineer with production experience at Shopify and Intel, building a deep learning product ranking system that lifted add-to-cart ~14% and serving real-time similarity search via FAISS+Redis under <20ms latency at massive scale. Also deployed computer vision models to 100+ retail edge locations using Docker/Ansible/k3s with zero-downtime rollouts, and applies strong MLOps practices (A/B testing, canary/shadow, observability) plus performance optimization (OpenVINO, INT8).”

“Built an "AskMyVideo" system that turns YouTube videos into queryable knowledge graphs by transcribing audio (Whisper), chunking and embedding content, and enabling traceable answers back to exact timestamps. Strong in entity resolution (rules + fuzzy matching + TF-IDF/cosine with PR-curve thresholding) and modern retrieval stacks (FAISS, hybrid dense/sparse, domain fine-tuning with ~12% precision gain), with a production mindset using Airflow/Prefect, Docker/FastAPI, and LangSmith/Prometheus/Grafana observability.”

“Senior data engineer with experience in both healthcare (CVS Health) and financial services (Bank of America), building large-scale Azure lakehouse pipelines (30+ EHR sources, ~5TB) and real-time streaming services (Event Hubs/Kafka) for patient vitals. Strong focus on reliability and data quality (Great Expectations, monitoring/alerting, schema drift automation), with measurable outcomes like 50% runtime reduction and 99%+ uptime for regulatory reporting pipelines.”

“Built and deployed a production multi-agent RAG system at JPMorgan Chase to automate regulated credit analysis and compliance clause discovery across large internal policy/document libraries. Implemented LangGraph-based supervisor orchestration with structured state management (Azure OpenAI) to support long-running, resumable workflows, plus hybrid retrieval + re-ranking and guardrails for reliability. Strong at evaluation/observability (trace logging, LLM-judge, HITL) and at communicating results to non-technical stakeholders via Power BI embeds and Streamlit prototypes.”

“Built and shipped a production LLM-powered multi-agent RAG system to automate complex internal support workflows, integrating tool execution (SQL/APIs) with validation guardrails to reduce hallucinations. Optimized for real-world latency and cost via model routing, caching, and async parallel tool calls, and enforced reliability with CI-gated golden test sets derived from anonymized production queries.”

“Built a production LLM-powered RAG agent for healthcare/insurance operations that retrieves and summarizes patient medical documents with grounded citations, scaling to ~4.5M records. Addressed medical shorthand and terminology by fine-tuning ~120 lightweight DistilBERT models by specialty and validating entities against SNOMED/RxNorm, while using SHAP/LIME and human-in-the-loop review to make decisions explainable to stakeholders.”

“ML/NLP practitioner working across healthcare and business/finance use cases: currently fine-tuning a domain-specific Llama 3.1 model for safe reasoning over EHRs/clinical notes using RAG + RL/DPO and RAGAS-based evaluation. Has built UMLS-driven entity normalization pipelines with quantified quality gains and developed embedding/vector-DB systems (FAISS) for semantic matching and forecasting/recommendation applications at Aurora AI and Banxico.”

“Full-stack engineer with Cisco supply-chain and Wipro internal platform experience, focused on customer-facing UI performance and secure backend services. Built a bulk Excel inventory upload feature (Spring Boot/Apache POI) that cut manual effort ~80%, and delivered high-scale Angular/React dashboards with strong reliability/observability (FastAPI, JWT, Docker, AWS, AppDynamics).”

“TypeScript backend engineer in the robotics domain with hands-on experience building low-latency (20–40ms) production systems using RabbitMQ, Redis, and HA PostgreSQL (Patroni). Has owned end-to-end services supporting 15 clients via config-driven architecture, with strong CI/CD, automated testing, and observability (OpenTelemetry) practices, plus API versioning/deprecation using Keycloak auth.”

“Systems and infrastructure engineer pivoting toward robotics software; brings strong low-level debugging, multithreaded systems, and networking experience where correctness and timing matter. Has hands-on experience using Docker and CI/CD to build reproducible test/evaluation environments (thesis), and proposes a disciplined, contract-driven approach to distributed communication and real-time performance debugging.”

“Independently built a production-grade, containerized enterprise agentic AI platform (stateful orchestration + RAG) focused on real-world reliability—guardrails, citation-based outputs, reranking, query rewriting, and evaluation harnesses to reduce hallucinations. Hands-on with OpenAI SDK, CrewAI, and LangGraph, and has delivered AI solutions for non-technical NGO stakeholders via demos and practical POCs.”

“Software engineer with experience at Striim and Amazon who ships end-to-end production systems across UI, backend, ML, and operations. Built a real-time PII detection capability for a streaming data platform by integrating Python ML inference into a Java monolith via gRPC sidecars, achieving ~3M events/hour throughput and ~93% accuracy, and helped drive enterprise adoption (Fiserv, CVS). Also modernized internal Amazon tooling for multi-region scale with modularization and fully automated deployments.”

“Infrastructure/DevOps engineer with primary ownership in enterprise Linux and AWS/Azure production environments (including financial systems). Built secure, repeatable CI/CD pipelines deploying containerized workloads to EKS/ECS and implemented Terraform/CloudFormation IaC with drift detection and rollback practices; lacks direct IBM Power/AIX/PowerHA experience.”

“Walmart engineer who built and shipped a production LLM+RAG system to automate triage and analysis of computer support chats/tickets, producing grounded, schema-constrained JSON outputs for summaries, urgency, and routing recommendations. Emphasizes reliability (hallucination control, confidence thresholds, human-in-the-loop) and runs end-to-end pipelines with Airflow and AWS-native orchestration, plus rigorous evaluation and monitoring tied to business KPIs.”