Vetted PySpark Professionals

Peeyush Dyavarashetty

Screened ReferencesModerate rec.

Intern AI/ML Engineer specializing in GenAI, LLMs, and agentic RAG systems

Miami, FL2y exp

“AI/LLM practitioner who built a GPT-2-like language model from scratch at the University of Maryland using PyTorch and multi-GPU distributed training, with experiment tracking in Weights & Biases. As an AI Operations intern at ScaleUp360, delivered multiple production-style AI agent automations (Gmail classification and Fireflies-to-Claude workflows that extract and assign CEO tasks) and set up measurable evaluation using test cases and classification metrics.”

“AI/ML engineer who has deployed production LLM/transformer-based systems for merchant intelligence and fraud/support optimization, delivering +27% merchant engagement and +18% payment success. Deep experience in privacy-preserving, PCI DSS-compliant data/ML pipelines (Airflow, AWS Glue, Spark, Delta Lake) and scalable microservices on Kubernetes, plus proven cross-functional delivery in healthcare claims analytics at UnitedHealth Group (12% HEDIS claim reduction).”

“Backend engineer who built a low-latency real-time transaction API in Python/Flask, with strong depth in PostgreSQL/SQLAlchemy performance tuning (time-based partitioning, indexing, connection pooling). Has production experience integrating ML scoring and OpenAI-style APIs with safety/latency controls, and designing multi-tenant isolation strategies including per-tenant pooling/caching and premium-tenant isolation.”

“Data engineer focused on large-scale ETL/ELT pipelines across cloud stacks (GCP and AWS), including Spark-based transformations and orchestration with Airflow. Has experience loading up to ~2TB per BigQuery target table and designing atomic loads to multiple downstream systems (Elasticsearch + Kafka), with Kubernetes deployment and Jenkins CI/CD.”

“Data engineer with experience at Bloomberg, UBS, and Bank of America building high-volume financial data platforms and services. Owned an end-to-end pipeline processing ~150–200M records/day (Kafka/Cassandra/S3 → Spark/PySpark → Snowflake) with strong data quality controls and Airflow reliability practices, reporting ~99% reliability and major performance gains. Also built large-scale external API ingestion with compliance-minded rate limiting, schema versioning, and quarantine/validation layers.”

“Platform/infra engineer with strong ownership of Kubernetes on VMware and day-to-day hybrid on-prem-to-AWS operations. Has hands-on experience automating infrastructure delivery with Terraform/Ansible/CI-CD, and has resolved real production issues spanning CSI storage reattachment during upgrades, vSphere storage-latency performance degradation, and hybrid connectivity/routing failures with improved validation, monitoring, and failover.”

Vignesh Shanmugasundaram

Screened

Junior Software Engineer specializing in full-stack development and applied ML

New York, NY2y exp

“Full-stack engineer with experience at Zoho and Amazon who has owned production systems end-to-end, including a monolith-to-microservices migration using Kafka and Cassandra that improved search latency ~25% and increased throughput without data loss. Also built a hackathon project (Buildwise) into a sold product for a construction company (AI-driven document compliance checks) and shipped an IoT-based parking availability MVP in 3 weeks.”

Poorna Pedapudi

Screened

Mid-Level Software Engineer specializing in distributed backend systems and cloud-native microservices

Seattle, WA5y exp

“Software engineer focused on data platforms and applied LLM systems: built an internal data quality monitoring layer to catch silent data drift and iterated post-launch after finding ~30% false-positive alerts, reducing noise via dynamic baselines and improved structured logging. Also shipped a production RAG-based internal knowledge assistant over Jira/Confluence with citations, confidence-based fallbacks, and nightly automated evals to prevent regressions.”

Abhay Murjani

Screened

Director-level Data Science Manager specializing in ML forecasting, experimentation, and MLOps

New York, NY6y exp

“Data/ML engineer with experience at American Express and Amazon, owning an end-to-end rewards redemption/liability ML pipeline (~200GB) with rigorous regulatory/audit validation and quarterly executive reporting. Also built web-scraped product datasets with anti-bot protections at a startup and helped modernize an authn/authz service using AWS, plus led early-stage migration work from an internal warehouse to GCP with CI/CD and cloud observability.”

“Built a production GenAI support agent at Amazon for FBA on-call operations, using Bedrock, Lambda, RAG, and confidence-based human fallback to safely automate ticket triage. The system materially reduced ticket volume and manual workload while improving MTTR, showing strong depth in reliable LLM agent architecture under real operational constraints.”

“Analytics candidate with strong product and growth analytics experience across SQL, Spark, Python, and Tableau. They have built clickstream funnel pipelines, automated Bayesian experiment evaluation, and used Markov chain journey modeling to uncover onboarding friction that led to a 5% conversion improvement. They also show strong cross-functional influence by standardizing churn definitions across product and marketing teams and operationalizing adoption in shared dashboards.”

“Backend/data engineer from ASML who modernized a legacy SAS-based statistical processing system into a cloud-native AWS platform (Lambda/FastAPI, Step Functions/EventBridge, Glue, S3/RDS) with strong reliability and data-quality practices. Demonstrated measurable performance wins (RDS query reduced from 90+ seconds to <5 seconds) and hands-on incident ownership for production ETL pipelines.”

Sarthak Gupta

Screened

Mid-level AI/ML Engineer specializing in LLMs, NLP, and real-time AI systems

New York, NY4y exp

“Backend engineer who built a real-time pipeline for recording, transcribing, and analyzing audio from 400+ news radio stations, scaling Whisper on an HPC cluster with 7 H100 GPUs. Has strong performance optimization experience (30% latency reduction via SQL/query design; 50% DB call reduction via Redis caching) and has implemented region-based data isolation and PII protections in a regulated environment (JP Morgan Chase).”

“Data engineer at Plaid who built greenfield, end-to-end real-time transaction pipelines and FastAPI data services for fraud detection and analytics, handling millions of events per day. Strong focus on reliability and data integrity via Great Expectations validation, Airflow-based monitoring/SLAs, quarantine/staging patterns, and robust external data ingestion with schema versioning and backfills (reported 50% fewer anomalies and ~40% fewer failures).”

Rojin Bakhti

Screened

Junior Software Engineer specializing in Edge AI and ML deployment

San Diego, CA3y exp

“Qualcomm engineer building Android applications that run on Qualcomm AI accelerators, with hands-on experience in C++ concurrency, chipset stress testing, and power/performance tuning. Has deployed on-device AI models and built deployment/log post-processing workflows using Docker/Kubernetes and CI/CD; interested in translating this embedded AI/performance background into robotics (perception/real-time systems).”

“Data engineer/backend developer with experience owning end-to-end, high-volume data pipelines for ML/analytics using Python, Airflow, SQL, and PySpark, reporting ~30% error reduction through improved reliability and data quality checks. Has also built Django-based REST APIs with caching/pagination and strong versioning practices, and operated external data collection/web scraping pipelines with anti-bot measures, monitoring, retries, and idempotent backfills.”