Svpzebntubzk3gt9d2f6

Connor Hsu

Curious about data and real world, build product to solve problem, make machine learning into product, writing is my interest.

Summary

  • 5 years experience of real world AI product building.
  • Experienced in transferring real problem into requirements.
  • Capable of building AI product from scratch both individually or co-work with various teams.
  • Extensive problem solving experience from data science to data engineering.
  • Analysis, Experiment, Debug, Improve AI system on large scale, millisecond-responsible environment.
  • Experienced in communication between scientists and engineering teams.

Experience

Senior Engineer II, Appier, Jan 2019 - Present

  • Data governance team / agile coach: Coordinating cross-team product features, burning down company level technical debts.
  • AI backend: Building extensive machine learning / analytical services through various approaches with F2E, Data Scientists, Data Infrastructure team.
  • Software Engineer, Appier, Jun 2014 - 2018

  • Build RTB bidding algorithm in a fast-growing, dynamic business environment.
  • Conduct experiments on real product to make daily improvements and achieve business goals.
  • Solve critical issues immediately and root cause analysis in uncontrolled, unreplayable, unbalanced data environments.
  • Enable product features with petabytes level data.
  • Experienced in data tools and operations (Spark, AWS RDS, Airflow)
  • Research Assistant, CSIE, NTU, Oct 2012 - May 2014

  • Build an online video retrieval application, including retrieval algorithms and backend systems.
  • Projects Detail


    Ad Product Technical Debt Burn Down '18Q4

    • Co-work with scientists to migrate an legacy machine learning project from python2 to python3
    • Design, burn down and implement new log patch framework to secure safe patch behavior, also enable abstraction on log patch mechanism.

    Data Governance '18Q2 ~

    Form a Data Governance committee with tech leads to consistently improve data quality and availability. Focus involves: schema change, legacy deprecation, data platform migration planning, ..., etc.

    Projects Detail


    Automatical Refund System '17Q2 ~ '18Q2

    Saved more than 10 millions TWD dollars for our business as well as tremendous human effort, milestones include:

    • Automate process to save support team and CM team's human effort (17'Q2)
    • Eliminate major data discrepancy (17'Q3)
    • Support various timezones, formats and make debug efficient. (17'Q4)
    • Different dimension breakdown and co-work with F2E to build a new UI (18'Q1)

    Pipeline Reconstruction and Migration '17Q2 ~ '17Q3

    • Reconstruct ad-hoc pipeline and improve it by applying unit test, migrating DB, code-refactoring, and migrating to Jenkins.
    • Co-work with team members to migrate critical production pipelines to Airflow, till 2019Q1, more than 20 data pipelines are operated by Airflow.

    Improve ML Model Performance '16Q2

    • Improve high quality inventory discovery by embedding inventory as vector: precision achieve 79% from 5.3%, volume increased to 12.8x
    • Extend CPA model to different ads vertical: CPA reduced to 68%, volume increased to 2.4x

    Production Skills


    Data Pipeline / ETL Tools: Python, Spark, Scala, MySQL (AWS RDS), AWS S3, Jenkins, Airflow, Grafana

    API Services: Flask, Swagger (Flasgger)

    CI/CD: Pyflakes, Pylint, Vulture, Coverage, Travis CI, Coveralls

    Machine Learning: TensorFlow Serving

    Other Skills


    Side Projects

    Bots: Slack bot, Line bot, Twitch bot

    Web & Visualization: d3.js, bokeh, jQuery

    Education

    National Taiwan University, Taiwan, Sep 2009 - Jun 2011

    M.S., Department of Computer Science and Information Engineering

    National Chiao Tung University, Taiwan, Sep 2005 - Jun 2009

    B.S., Department of Computer Science