Vetted PySpark Professionals

“LLM/AI engineer who has deployed production RAG conversational analytics and Text-to-SQL systems over Snowflake and curated data marts, emphasizing enterprise-grade guardrails for accuracy, security, and cost. Notable for a structured approach to reducing hallucinations (curated metric/table registry, SQL validation, RBAC, and citation-backed responses) and for building resilient, observable multi-step agent workflows using LangChain/LlamaIndex and Airflow.”

“Backend engineer with deep Kafka/FastAPI microservices experience who redesigned a notification pipeline to cut end-to-end latency from ~5s to ~3s (including custom partition assignment and consumer tuning). Led a high-stakes ClickUp-to-Oracle migration of 1M+ records using idempotent ETL, reconciliation, and shadow deployment to achieve >99% integrity with zero downtime, and has hands-on production security implementation with Django/DRF (JWT + RBAC).”

“Built a production-grade LLM evaluation and regression system that stress-tests models across hundreds of iterations, combining LLM-as-judge, semantic similarity, statistical metrics, and rule-based checks, with results delivered via stakeholder-friendly HTML reports and dashboards. Experienced orchestrating multi-agent RAG workflows using LangChain/LangGraph and event-driven GenAI pipelines in n8n integrating OCR, speech-to-text, and external APIs, with strong emphasis on reliability, observability, and explainable failures.”

“Applied LLMs to classify long nonprofit mission statements into 8 segments without labeled data, using an ensemble of clustering/embedding methods plus zero-shot RoBERTa/BART and a Tree-of-Thought prompting pipeline with LLM-as-judge evaluation (Gemma). Also built LangChain/LlamaIndex agentic RAG workflows including a text-to-SQL data analysis assistant grounded on DB schema with retries and performance optimizations on an HPC cluster.”

“Built a production-grade, semi-automated document recognition and classification system for large volumes of scanned PDFs, starting from little/no labeled data and handling highly variable scan quality. Deployed on AWS using SageMaker + Docker and orchestrated on EKS with a microservices design that scales CPU-heavy OCR separately from GPU inference, with strong reliability controls (validation, fallbacks, retries, readiness probes).”

“Software engineer who deployed and stabilized a real-time analytics platform at Senecio Software, focusing on production reliability, observability, and performance under load. Experienced debugging issues spanning distributed services and networking (e.g., tracing timeouts to packet loss from misconfiguration) and extending Python (FastAPI/Django) APIs for customer-specific analytics features in a configurable, maintainable way.”

“LLM/agentic AI builder who deployed a production ITSM automation agent on Google ADK integrating ServiceNow and FreshService, with strong safety guardrails (human-approval gating and runbook-only command execution) and rigorous evaluation (500 synthetic tickets; 80%+ false-positive reduction). Also partnered with finance to deliver an AI agent that automated invoice/SOW retrieval and monthly reporting to account managers, reducing manual back-and-forth.”

“Built and deployed a production-grade, multi-agent Text-to-SQL assistant that lets non-technical stakeholders query large enterprise databases in natural language. Uses Pinecone-based schema retrieval + LLM reasoning (Gemini/Claude/GPT) with a dedicated validation agent (schema/syntax checks and safe dry runs) to reduce hallucinations and improve reliability, while optimizing latency and cost via async execution and embedding caching.”

“Backend/applied AI engineer from Cloud Rack Systems who built production GenAI/RAG and data platforms on Azure/Databricks at enterprise scale (2.5M records/day). Known for making LLM systems behave like deterministic services via strict retrieval contracts, citation-based validation, and strong observability—shipping a knowledge assistant used daily by 50+ users while driving hallucinations near zero and materially improving latency and cost.”

“Data/ML practitioner who has built a production RAG-based knowledge assistant integrated into Microsoft 365/internal dashboards to help employees query internal documents in plain English. Experienced orchestrating and hardening ETL pipelines with Airflow and Azure Data Factory (validation, retries, monitoring) and running end-to-end model evaluation and production performance tracking via Power BI.”

“Robotics competition (ABU Robocon) team member who programmed two robots for a rugby-style game, integrating IoT sensors and real-time decision-making. Implemented low-latency, secure inter-robot communication by moving from Bluetooth to ESP8266/NodeMCU WiFi (with Bluetooth as backup) and used OpenCV plus CNN training workflows for vision-related tasks; no practical ROS/ROS2 experience.”

“Data-driven growth and partnerships professional with experience leading an analytics/reporting vendor rollout end-to-end (vendor selection via stakeholder interviews and PoC, then negotiating scope/pricing/support and tracking adoption/efficiency/accuracy KPIs). At PC Financial, built regression and segmentation models to optimize multi-channel targeting (in-app/email/push), driving +15% campaign engagement and +10% PC Optimum offer loads, and ran behavior-triggered lifecycle experiments that lifted upsell conversion by 20%.”

“Customer-facing software engineer who builds and deploys practical AI/RAG solutions (e.g., an AI assistant for searching billing PDFs) by deeply understanding support workflows and iterating with users. Demonstrates strong production instincts—quickly stabilizing peak-traffic API timeouts with caching/background jobs, then implementing durable fixes with proper monitoring and maintainable code practices.”

“Backend/AI engineer who has built production LLM orchestration and agentic workflow systems in Python/FastAPI on Kubernetes across AWS/Azure. Demonstrated strong reliability engineering by debugging a real-world memory retention issue that caused latency spikes/timeouts, and strong data/performance chops with a PostgreSQL optimization that cut query latency from ~1.2s to ~15ms. Targets roles building scalable, guardrailed AI-driven workflow automation with robust observability and human-in-the-loop controls.”

“Built and deployed a production RAG-based multilingual analytics assistant for healthcare operations, enabling non-technical teams to query claims/EHR and risk metrics with grounded explanations. Demonstrates strong end-to-end LLM system engineering (retrieval tuning, re-ranking, hallucination controls, verification layers) plus workflow orchestration (Airflow/Composer/Step Functions) and stakeholder-driven iteration via prototypes and dashboards.”

“Built an LLM-driven MLOps agent at SBD Technologies that automated an EV-charging prediction workflow end-to-end, integrating with real-time Kafka/FastAPI systems supporting 120K+ chargers at 99.99% event delivery. Addressed frequent schema drift by implementing SQLAlchemy/Flyway validation (60% reduction in drift issues) and deployed as Kubernetes microservices with GitHub Actions CI/CD; also has Airflow-based ingestion/crawling experience into Snowflake and stakeholder-facing delivery via a Fleetcharge PWA.”

“LLM/agent systems builder who deployed a production hybrid router for immerso.ai that dynamically selects retrieval vs reasoning vs generative pathways, achieving an 82% factual-accuracy lift. Deep hands-on experience optimizing local Mistral 7B inference (4–5 bit GGUF quantization, KV-cache reuse) and building reliable RAG/agent workflows with LangChain/LangGraph/AutoGen across GCP Cloud Run and AWS (ECS/Lambda).”

“Built and deployed a production generative-AI copilot at Tungsten that automates invoice/form extraction template creation, reducing weeks of manual model-building work. Combines fine-tuned LLMs (PyTorch/HuggingFace) with OpenCV layout grounding to reduce hallucinations, and runs an end-to-end Kubeflow-based MLOps pipeline with drift monitoring, canary releases, and automated retraining.”

“Backend-leaning full-stack engineer in fintech/payments who shipped an end-to-end Stripe payments + webhook system for a financial microservices platform, emphasizing ledger accuracy via idempotency, transactional writes, retries, and DLQs. Also delivered a real-time React/TypeScript payment status dashboard informed by user interviews, and improved production performance by 35% p95 latency through PostgreSQL tuning and Redis caching on AWS.”

“Product-minded engineer who has owned features end-to-end, including a full onboarding redesign that lifted completion ~25% and a production LLM/RAG report-generation system with strong guardrails (schema-constrained JSON, confidence gating, logging) and an automated eval/regression loop built from real user queries. Also built a scalable research data pipeline ingesting messy PDFs/JSON/CSVs with normalization, idempotent reruns, observability, and cost/latency tradeoffs.”

“Data engineer with experience at Morgan Stanley and Star Health owning production-grade lakehouse pipelines for credit risk and healthcare datasets. Built Azure/Databricks/Delta/Snowflake-based platforms processing millions of records per day with strong data quality, observability (Monte Carlo/Azure Monitor), and reliability practices, plus experience delivering curated data services with performance tuning and backward-compatible versioning.”

“Backend/data engineer focused on production AWS systems: builds multi-tenant FastAPI services on ECS behind API Gateway/ALB with serverless orchestration (Lambda, SQS, Step Functions) and strong reliability practices (JWT/JWKS auth, idempotency, backoff retries, structured logging). Also delivers AWS Glue/PySpark ETL pipelines with schema/data-quality controls and has modernized legacy analytics logic into Python with parity validation; improved a key dashboard SQL query from ~12–25s to ~2–3s.”

“Senior backend/data engineer who has built and maintained HIPAA-compliant, real-time clinical FastAPI services on AWS, orchestrating ML/LLM and vector DB calls with strong reliability patterns (auth, timeouts/retries, graceful degradation, idempotency). Also delivered AWS IaC/CI-CD (Terraform/Helm/GitHub Actions) across EKS/Lambda/SageMaker and built Glue/Spark ETL with schema evolution and data quality controls, plus demonstrated large SQL performance wins (15 min to <9 sec) and hands-on incident ownership.”