Construction and Management of Data Pipelines 

1. Regularly fetch and update data from external platforms using APIs, establishing retry mechanisms and data quality checks.
   
2. Ensure data quality; investigate the cause of anomalies during data updates and verify if data quality alerts are false alarms.
   
3. Integrate cloud and on-premise pipelines to guarantee that data users have access to accurate information.

1. Migrated the on-premises pipeline to Azure Synapse Analytics and redesigned it, eliminating unnecessary dependencies.
2. Refactored the on-premises stored procedure for migration. Accelerated the stored procedure execution by 6-10x.
   
3. Implementing a daily data validation mechanism. If any abnormal data changes are detected, recording and halting the pipeline to prevent data contamination.
   

• Help them finding out the career path by trying different kind of works. Let the interns learn what they want and prevent the trap of curiosity. Finally it increase the willingness of them to become the formal employee.  
• Automatize the routinely report.  Free 10% of the labor hour and prevent the human error. 
• learn the domain knowledge and apply to business analysis.

• Let the mass society understand the situation of Junyi, e.g. https://www.junyiacademy.org/statistics.
• Let the colleagues of Junyi can easily monitor the key indicator and correct the problems. 

• The importance of the technical document is not only record, but experience inheritance. Whenever we need, we can easily find the document for help. Furthermore, we continuously update it. As time goes by, the document still be correct.