WangPengKang 王鵬綱

Currently, I am  a Senior Data Engineer at Far EasTone Telecom, focusing on data warehousing, data transformation, CICD , automation, web scraping, data science, and providing data platform services and insights.

  Taipei, Taiwan          

Skill

  • Data&ETL : DataBricks ,Trinity ,Airflow, Prefect, Spark(pyspark), RabitMQ
  • CICD: Docker, Jenkins, Git
  • Database: Teradata, MySQL, Oracle ,PostgreSQL
  • Programming languages: Python, SQL
  • Machine Learning: Classification, Clustering, Regression, Natural Language Processing, Fraud Detection, Recommender System, Social Network Mining
  • Visualization: Tableau, Metabase, Python(Matplotlib, Seaborn, plotly, DASH), Data Studio
  • Crawler: python(BeautifulSoup, Selenium, Scrapy, Mitmproxy)
  • Others  VSCode , Notion, Hackmd

Work Expertise and Experience

工作經歷

Data Engineer

FarEasTone Telecom   •  六月 2021 - Present

1. DataBricks & TeraData ETL Pipeline Dev and OP
2. Data Domain on App/Web Log, Leads, ChatBot
3. Python+ Jenkins + Tableau TeraData DB Usage Monitor Dashboard & Alert
4. Using LightGBM Predict Sales and won 2021 Farestone Tableau Hakthon First Place

Data Engineer

Anue - Data Application Department  •  六月 2019 - 六月 2021

  1. Web analytics platform: Matomo (piwik) data warehouse and ETL pipeline.

  2. Maintain CICD infrastructure: AWS-Bitbucket-Docker-PHP and migrate to GCP-Gitlab-Docker-Python.

  3. All web url tag and visitor tag for precision marketing and user profile analysis

  4. Keywords network and news pv analysis

  5. Web Crawl and automation for internal support

  6. Dashboard build with metabase and dash

Data Engineer

CrowdInsight   •  三月 2019 - 六月 2019

  1. Agricultural products Crawl

  2. Database schema design and build test data for martech system

Python Backend Engineer

JS tech  •  七月 2017 - 二月 2019

  1. High frequency web crawl and stable data source supply

  2. Build internal API application

  3. Tools: Flask, Request, Selenium, Beautifulsoup, Mitmproxy, Prorxypool, RabitMQ

Research Assistant

IES Academia Sinica  •  一月 2013 - 六月 2017

1. NOAA satellite data and Gamma Ray radiation analysis for possible earthquake precursor
2. Experiment instruments and data pipeline build.
3. Data analysis and data visualization.

學歷


National Chung Cheng University

Applied Geophysics Master's Degree

09/2007- 06/2011

學歷


National Chung Cheng University

Department of Earth and Environmental Sciences  Bachelor's Degree

06/2003- 06/2007