Taipei City, Taiwan
2018 - 2020
2012 - 2016
2022/03 - Present
• Built Spark-based ELT pipelines through AWS eco-systems.
• Project owner of dbt who is responsible for:
-- building dbt infrastructure on AWS Athena and defining the development process.
-- designing CI/CD pipelines and integrating dbt core with Airflow.
-- collaborating with analysts to build transformation pipelines.
Skills: Redash · Apache Superset · dbt · PySpark · Amazon Athena · Apache Airflow
2020/07 - 2022/02
• Implemented ETL and ELT from MongoDB & MySQL databases to Google BigQuery, & managed data pipelines through Airflow. Encapsulated BigQuery API for monitoring and better practices.
• Launched A/B/n testing data modeling & built a dockerized interactive interface by Streamlit to create or edit experiment info.
• Coordinated with interdepartmental personnel to implement customer relationship management.
-- Deployed customer lifetime value prediction model which applies to finding high-value customers.
-- Built position-based attribution model to analyze contributions from media channels.
Skills: Google Data Studio · Google BigQuery · MongoDB · Apache Airflow
Taipei City, Taiwan
2018 - 2020
2012 - 2016
2022/03 - Present
• Built Spark-based ELT pipelines through AWS eco-systems.
• Project owner of dbt who is responsible for:
-- building dbt infrastructure on AWS Athena and defining the development process.
-- designing CI/CD pipelines and integrating dbt core with Airflow.
-- collaborating with analysts to build transformation pipelines.
Skills: Redash · Apache Superset · dbt · PySpark · Amazon Athena · Apache Airflow
2020/07 - 2022/02
• Implemented ETL and ELT from MongoDB & MySQL databases to Google BigQuery, & managed data pipelines through Airflow. Encapsulated BigQuery API for monitoring and better practices.
• Launched A/B/n testing data modeling & built a dockerized interactive interface by Streamlit to create or edit experiment info.
• Coordinated with interdepartmental personnel to implement customer relationship management.
-- Deployed customer lifetime value prediction model which applies to finding high-value customers.
-- Built position-based attribution model to analyze contributions from media channels.
Skills: Google Data Studio · Google BigQuery · MongoDB · Apache Airflow