Profile 02 00@2x 71843ef6a0df47d6255a9c0436c409dcd5cd81f6514c51a6b2a93339d82bbff6

linsam

data engineer、data scientist • 0972724528 •  台灣  •  [email protected]

Experience with data mining, machine learning, distributed system, and web crawling. Hopes to focus more on data science and data engineer in future career.

Work Experience


SinoPac Holdings - Data Engineer,Nov. 2019 年 11 月 - now

• Develop python Api (shioaji) for stock/option/future place orde and account. 

• Develop C# Api for stock/option/future place orde and account. 

• Collecting Distributed System Log by elk, grafana and prometheus.

• Monitor Distributed System and alert chatbot.


Tripresso - Data Engineer,Oct. 2018 年 10 月 - Nov. 2019 年 10月

• Analysis travel data and build a machine learning model. Estimating increase 4% orders (revenue). 

• Maintain and develop an ETL distributed queuing system with 20 machines. 

• Optimize the ETL system reduced more than 50% execution time. 

• Develop new product crawler let product volume increase 1.5%. 

• Making analysis charts provide for other departments.

Mandatory Military Service,Oct. 2017 - Oct. 2018

NDHU - RA,Mar. 2016 - Aug. 2017

Analysing G7 financial data. Model validation and parameter estimation by regression models ( SUR, MLE, Bootstrapping ). And comparing single equation estimators and confidence interval with system equation.

NDHU - TA,Sep. 2015 - Jul. 2017

Calculus, Linear Algebra, Statistics.

Projects


FinMind Open data Api


Open source financial data, more than 50 dataset, provide Api. 

 Automatic update daily by docker swarm, distributed queue system rabbitmq and celery ( 8 cloud machines ). 

1000 stars on github.



Bosch Production Line Performance - Kaggle Post-competition analysis, top 6% rank.

Highly imbalance data, ratio is 1000 : 1, 10 GB dataset size. And the data is 50% missing value. More than 4000 variables, but I build models by only 50 features.


Rossmann Store Sales - Kaggle 

Post-competition analysis, top 10% rank.

Time series problem. Building models predict sales after 48 days.


Grupo Bimbo Inventory Demand - Kaggle

Post-competition analysis, top 8% rank. 

Time series problem, eighty millions data size. Building models predict inventory demand after 2 weeks.


Instacart Market Basket Analysis - Kaggle

Real competition, top 25% rank. 

Predicting which products will an consumer purchase again.



 Verification code to text

Create python package of Taiwan Train Verification Code to text.

The model is made by keras-CNN.

Skills


Cloud Cluster Server

1. Using swarm, portainer, manager backend service. Include crawler, backend api, finmind api, db and web. 

2. Using traefik, on swarm, manager router and DNS. Include DNS of backend api, finmind api and web. 

3. Using traefik auto register SSL let's encrypt.

4. Linode Cloud.


Financial Api

1. Including more than 50 Taiwan stock datasets.

2. Github 900 stars.

3. Using python and fastapi develop.

4. Document.

http://api.finmindtrade.com/docs


CI/CD

1. Create automated tests and automated deploy for the FinMind team. 

2. Using gitlab runner. 

3. CD for auto publish python package. 

4. CD for auto update and deploy new version service.


Data Mining

Python - numpy, pandas, sklearn, multiprocessing, joblib. 

R - parallel, dplyr, data.table, mice.


Machine Learning

Python - xgboost-gpu. 

R - xgboost, svm, random forest, knn.


Deep Learning

Python - kears-CNN.


Statistical Model

R - GLM, GLMNET, NLS, SUR, MLE.


Web Crawling 

1. Python - request, BeautifulSoup, lxml, selenium.
2. selenium.
3. Auto recognition captcha code by CNN model.

Message Queue

1. Rabbitmq & Celery & Flower.
2. Build works on 6 Linodes ( Cloud ) Distributed queue system for Web Crawling.
3. deploy by docker.

Python Package 

1. FinMind 900 stars


WEB
2. nginx
3. django


Log Collect & Monitor

1. Distributed System Log collect by elk. 
2. Prometheus and Grafana
Monitor user usage, request latency, request count and alter by telegram bot. 

Docker 

1. dockerfile. 

2. docker-compose. 

3. docker swarm. 

4. Using docker swarm manage multiple services、node.


DataBase
1. MySQL. 
2. Redis



Education

National Dong Hwa University, Master of Science,  Sep. 2017.

Major : Mathematics and Statistics.

Tamkang University. Bachelor of Science, Sep. 2015.

Major : Mathematics

Languages


R, Python. Basic in English and proficient in Chinese.

Powered by CakeResumePowered by CakeResume