About

Connor Hsu

Curious about data and real world, build product to solve problem, make machine learning into product, writing is my interest.

Summary

5 years+ experience of real world AI product building.
Experienced in transferring real problem into requirements and solution planning.
Capable of building AI product from scratch both individually or co-work with various teams.
Extensive problem solving experience from data science to data engineering.
Analysis, Experiment, Debug, Improve AI system on large scale, millisecond-responsible environment.
Experienced in communication between scientists and engineering teams.

Experience

Software Engineer, SmartNews, July 2019 - present

Data platform team: build data infrastructure/platform to serve Ads/Data Science needs.

Improve Hive performance to 4x by utilizing partition after 1 month.

Lead team documentation culture, attend as representative in cross-team meeting after 2 months.

Senior Engineer II, Appier, Jan 2019 - July 2019

Data governance team / agile coach: Coordinating cross-team product features, burning down company level technical debts.

AI backend: Building extensive machine learning / analytical services through various approaches with F2E, Data Scientists, Data Infrastructure team.

Software Engineer, Appier, Jun 2014 - 2018

Build RTB bidding algorithm in a fast-growing, dynamic business environment.

Conduct experiments on real product to make daily improvements and achieve business goals.

Solve critical issues immediately and root cause analysis in uncontrolled, unreplayable, unbalanced data environments.

Enable product features with petabytes level data.

Experienced in data tools and operations (Spark, AWS RDS, Airflow)

Research Assistant, CSIE, NTU, Oct 2012 - May 2014

Build an online video retrieval application, including retrieval algorithms and backend systems.

Projects Detail

Ad Product Technical Debt Burn Down '18Q4

Co-work with scientists to migrate an legacy machine learning project from python2 to python3
Design, burn down and implement new log patch framework to secure safe patch behavior, also enable abstraction on log patch mechanism.

Data Governance '18Q2 ~

Form a Data Governance committee with tech leads to consistently improve data quality and availability. Focus involves: schema change, legacy deprecation, data platform migration planning, ..., etc.

Projects Detail

Automatical Refund System '17Q2 ~ '18Q2

Saved more than 10 millions TWD dollars for our business as well as tremendous human effort, milestones include:

Automate process to save support team and CM team's human effort (17'Q2)
Eliminate major data discrepancy (17'Q3)
Support various timezones, formats and make debug efficient. (17'Q4)
Different dimension breakdown and co-work with F2E to build a new UI (18'Q1)

Pipeline Reconstruction and Migration '17Q2 ~ '17Q3

Reconstruct ad-hoc pipeline and improve it by applying unit test, migrating DB, code-refactoring, and migrating to Jenkins.
Co-work with team members to migrate critical production pipelines to Airflow, till 2019Q1, more than 20 data pipelines are operated by Airflow.

Improve ML Model Performance '16Q2

Improve high quality inventory discovery by embedding inventory as vector: precision achieve 79% from 5.3%, volume increased to 12.8x
Extend CPA model to different ads vertical: CPA reduced to 68%, volume increased to 2.4x

Production Skills

Data Pipeline / Warehousing: Python, Spark, Scala, Airflow, Presto

AWS: EC2/ECS/ECR, EMR (Hive), RDS (MySQL), DynamoDB, ElastiCache (Redis), CloudWatch, ..., etc.

API Services: Flask, Django, Swagger (Flasgger)

CI/CD: Jenkins/Github Actions/Bitbucket Pipelines/TravisCI, Ansible

Machine Learning: TensorFlow Serving

Other Skills

Side Projects

Bots: Slack bot, Line bot, Twitch bot

Web & Visualization: d3.js, bokeh, jQuery

Writings

Education

National Taiwan University, Taiwan, Sep 2009 - Jun 2011

M.S., Department of Computer Science and Information Engineering

National Chiao Tung University, Taiwan, Sep 2005 - Jun 2009

B.S., Department of Computer Science