許博淳
Over four years of experience in data handling as a Data Engineer and Data Analyst.
Solid analytical skills with SQL and Python.
Proficient in contributing to and maintaining the data pipeline.
TOEIC: 890
Email: [email protected]
Data Pipeline
- Azure Synapse Analytics
- Azure Automation Runbooks
- SQL Server Management System
- SQL Server Analysis Service
Trend Micro(趨勢科技), Data Engineer, 2022/10 ~ NOW
Construction and Management of Data Pipelines
- Regularly fetch and update data from external platforms using APIs, establishing retry mechanisms and data quality checks.
- Ensure data quality; investigate the cause of anomalies during data updates and verify if data quality alerts are false alarms.
- Integrate cloud and on-premise pipelines to guarantee that data users have access to accurate information.
Cloud Migration of Data Pipelines
- Migrated the on-premises pipeline to Azure Synapse Analytics and redesigned it, eliminating unnecessary dependencies.
- Refactored the on-premises stored procedure for migration. Accelerated the stored procedure execution by 6-10x.
- Implementing a daily data validation mechanism. If any abnormal data changes are detected, recording and halting the pipeline to prevent data contamination.
Junyi Academy(均一平台教育基金會), Data Analytical Engineer, 2020/02
~ 2022/07
Maintenance and Refactoring of the Data Pipeline
- Refactor and decouple the SQL procedure of weekly update.
- Optimize the structure and the operator of Airflow DAGs.
Leading the Interns
- Help them finding out the career path by trying different kind of works. Let the interns learn what they want and prevent the trap of curiosity. Finally it increase the willingness of them to become the formal employee.
- Automatize the routinely report. Free 10% of the labor hour and prevent the human error.
-
learn the domain knowledge and apply to business analysis.
Creation and Maintenance of the Dashboard
Produce the Technical Document
- The importance of the technical document is not only record, but experience inheritance. Whenever we need, we can easily find the document for help. Furthermore, we continuously update it. As time goes by, the document still be correct.
National Cheng Chi University
Management Information System
2017 ~ 2019
National Chiao Tung University
Transportation and Logistic Management
2012 ~ 2016