CakeResume 找人才

and collaborate with a dynamic team. New Taipei City, Taiwan Work Experience Data Engineer • Groundhog Technologies Inc. JulyPresent - Built and maintained data piplines (through which several hundred millions rows of data flow through daily) using Scala Spark/ Hadoop - Managed cron jobs and performed regular data recovery using Apache Airflow - Performed regular Extract, transform, load (ETL) operations through Hive and HDFS command line interfaces - Utilized streaming technologies such as Kafka to store data into various pools and lakes alike Graduate Hourly employee • University of Illinois at Urbana-Champaign SeptemberDecember 2020 Designed and developed student assessments/assignments for

import traditional CSV files. Utilized Databricks for large-scale data processing, leveraging its Spark capabilities to efficiently transform and aggregate incoming data streams. With the combined power of Databricks and AWS Lambda, ensured unparalleled data consistency, quality, and preparedness for sophisticated analytics and reporting. Utilized Databricks and Airflow to run extensive data profiling tasks, analyzing data patterns and identifying potential quality issues before they reached the Databricks Delta Lake. Established robust guardrails using the combined might of AWS Lambda, Apache Airflow and Databricks, ensuring that data stored in the DataBricks Delta Lake consistently met the highest

stable api under high qps with load testing / concurrent testing. Software engineer • Gogoro/Goshare • 2019/11~2022/01 ．Implement Tarjan's strongly connected component algorithm and topological sort algorithm in the task management system. ．Develop a customized synchronizer system by combining core concepts from Apache Airflow and Apache Kafka Connecter . ．Experience in performance test , load test and stress test and unit test. ．Experience in solving java native memory leak in JIT c2 level 4 compiler issue. Software Engineer • Lang Live • 2019/03 ~ 2019/10 ．Deal with 10 million+

料抓取與邏輯計算，最後將資料以網頁呈現至前端‧ 這整套系統自建於Linode的雲端主機，使用UbuntuLTS系統環境，安裝包括nginx, mysql, phpmyadmin, python, apache airflow ‧ Apache airflow 曾經的踩雷: 1. 每次跑task時都會讓Scheduler與Mysql Crash，查 Mysql log之後發現好像是主機記憶體 不足 的問題，把linode的主機從RAM

. # Air quality real-time monitoring mapUtilized the ASP.NET MVC framework for building a low-cost PM2.5 sensing platform for the Environment Protection Administration Executive Yuan R.O.C. 2. Integrated Google Maps API to display GIS base data, and leveraged Apache EChart to present historical data. 3. Developed a Messenger Bot to provide convenient data querying services. EducationAppWorks School Blockchain Program國立暨南國際大學(National Chi Nan University) 資訊管理學系國立暨南國際大學

operation, moderating model training results, and designing SLIs/SLOs for EKS Clusters. More responsibilities/details as below. Optimize music (UMG) pipeline with queries and memories for Elasticsearch and PostgreSQL, the pipeline saving 90% execution time from 10+ hours to 40 mins. Migrate service from apache spark, AWS Data Lake Formation to AWS MWAA, EKS airflow environment. Design, and deliver distributed system for Ray Serve with AI team. Design, and implement a modern machine learning pipeline for a recommendation, and moderation pipe. Design SLA and implement alert log reporting system (history logs

Chih-Wei Chang  Data Engineer | BI | Web3 • [email protected] •Aiming to forge a progressive career in business intelligence, employing my analytical and technical prowess in data engineering to effectively further business objectives. Having BI/DATA Engineer experience with different industry; substantial experience designing and executing solutions for business problems involving large scale data warehousing Design and implement data warehouse solutions for different stage of project Familiar with design, implement and maintain ETL process Abilities Programming Python SQL Dataflow Cloud Pubsub ETL MSBI Hive Airflow Tableau Docker Trinity DW Platforms AWS GCP People/Personal Project