Chun Wei Lu

A Data / Software engineer who is interested in solving data-related problems.

EXPERIENCE

Data Engineer  •  Carousell

Jan 2020 - Jun 2021

- Implemented the new distributed computing algorithm on existed recommendation spark jobs to have better flexibility to adapt different types of data embedding with custom weight, also increase the stability of the pipeline from failing once a week to every two months.

  • 20M products across 6 countries
  • Finished in 2 hours with limited computing resources
  • Spark / Scala / Cassandra

- Worked with data scientists to apply different data models to the recommendation-related system, including pre-computed spark jobs and real-time similar product recommendation backend system powered by Spotify annoy.

  • 2 ~ 3 different data models
  • Backend service with 500up QPS
  • Spark / Python / Golang / Spotify Annoy

Build, maintain and scale efficient data infrastructure, all daily jobs from different teams in ETL( Apache Airflow), and reporting pipelines.

  • Python / Airflow

- Develop and maintain internal Google BigQuery authority management system and experimentation tools such as A/B Test Platform.

  • Python / Golang / Protobuf

Develop and maintain APIs in data-related backend microservice including recommendation, keyword suggestion, and product collection.

  • Golang / Protobuf / Cassandra

Backend Engineer in Convenience Team  •  Carousell

Sep 2018 - Sep 2019

- Using Golang to build a service from scratch for synchronizing payment transactions data to the Finance accounting system to help the Finance team do the auditing more efficiently.

- Using Golang to develop backend API micro-service for both web and mobile, including payment system, logistics system, and business logic across 5+ marketplace in the SEA ( SG, TW, HK, MAS...).

- Implementing unit testing and integration testing to ensure the stability of system development.

  • Golang / Ginko / Protobuf

Backend Engineer in Core-Tech Team  •  Carousell

Sep 2019 - Dec 2019

- Developing the 2nd generation of the fieldset also known as a server-side rendering system which helps us to control the data and format on the client from the backend side.

  • Golang

EDUCATION

Interdisciplinary Program Of Nuclear Science, Computer Science And Engineering Science, Bachelor Degree

National Tsing Hua University: Taiwan, Hsinchu


SKILL

  • Languages: Golang, Python, and Scala
  • Database: Cassandra, Postgres, and Redis
  • Container Technology: Docker, and k8s.
  • Cloud Service: Google Cloud Platform
Powered by CakeResumePowered by CakeResume