.png){kind=logo}
ZZ
Zhiwen Zhao
Junior Data Engineer specializing in cloud ETL and big data platforms
New York, NYData Engineer Associate 23 years experienceJuniorFinancial ServicesTransportation & LogisticsGovernment
ScreenedIdentity Verified
Connect with Zhiwen
Zhiwen already has a relationship with Reval, so a warm intro from us gets a much better response than cold outreach.
Recommended
Already have an account?
About
Data engineer focused on transit/transportation datasets, building Spark-based pipelines that ingest from Oracle/APIs, apply PySpark data-quality fixes, and publish star-schema fact tables to Azure Data Lake. Experienced troubleshooting complex Spark failures (using checkpointing to manage long lineage) and operating Airflow-driven backfills and GitLab CI deployments for production DAGs.
Experience
Data Engineer Associate 2Bank of China America Data Center
Azure Data Engineer AssociateMetropolitan Transportation Authority
Teaching Assistant of Big Data Management & AnalyticsNYU Tandon
Data Scientist InternShanghai Big Data Center
Education
New York Universitymaster, Computer Science (2025)
Shanghai University of Finance and Economicsbachelor, Data Science and Big Data Technologies (2022)
Key Strengths
- Owned end-to-end data pipelines from Oracle ingestion through PySpark transformations to Azure Data Lake serving layer
- Hands-on data quality validation with route coverage and record-count checks (e.g., routes 1–6, 300–500 records per route)
- Debugged Spark execution issues (apply for Java error) by isolating root cause and using local checkpointing to break long lineage and reduce recomputation
- Designed/ran backfills via Airflow DAGs and manual Spark submit with parameterized date conditions; enabled parallel backfill execution
- Pragmatic technology selection in ambiguous projects (chose GraphFrames over NetworkX for Spark DataFrame compatibility)
- Implemented daily freshness checks using last-modified timestamps to ensure timely updates
Browse Similar
Discover more candidates like Zhiwen
Search across thousands of pre-screened, high-quality, high-intent candidates on Reval.
Search Talent