Avatar of Rui An Huang.
Rui An Huang
Software Engineer
ProfileResume
Posts
0Connections
Print
Avatar of the user.

Rui An Huang

Software Engineer
My Open Source Projects (https://github.com/rueian): * rueidis: A high-performance (14x throughput) Golang Redis client library (Moved to Redis Github Organization). * pgcapture: A unified Change Data Capture and Data Dumping solution for PostgreSQL via gRPC. * pgbroker: A Golang library for building PostgreSQL proxy. * aerial: A combination of Cilium Golang Envoy Filter and a TCP tunnel to bridge local servers into k8s. * zenvoy: A L4 TPROXY and XDS server supporting k8s endpoints to be scaled from&to zero. * kinko: A k8s operator and CLI tool for sealing secrets with GCP KMS. My Talks (https://speakerdeck.com/rueian): * 2020 Cilium & Cgroup eBPF (Trace Code) * 2020 Bridge to Kubernetes with Cilium's Envoy Go Filter * 2020 (Chinese) Popular Golang PostgreSQL Libraries Comparison with Wireshark * 2019 (Chinese) Scaling PostgreSQL for OLAP Usage on GCP My Publications (https://ruian.medium.com): * 2022 (Chinese) Saying Goodbye to Database Sorting and Pagination. * 2022 Working on High-Performance Golang Client Library — Remove the Bad Busy Loops with the sync.Cond * 2022 Working on High-Performance Golang Client Library — Reading Again From Channels? * 2022 Working on High-Performance Golang Client Library — Batching on Pipeline * 2022 Redis 6 Server-Assisted Client-Side Caching with Golang * 2020 (Chinese) Avoid Wasting PostgreSQL's Network Bandwidth with Extended Protocol * 2020 (Chinese) How does PostgreSQL estimate the row count of a LIKE statement? (Trace Code) * 2019 (Chinese) How does PostgreSQL estimate the row count of a HashAggregate step? (Trace Code) Experiences * Developing public APIs with CDN caching that serve 20000+ RPS. * Developing a transparent database disaster recovery API that restored 1000+ databases per day. * Scaling OLAP throughput transparently based on the previous disaster recovery API. * Having a strong understanding of PostgreSQL/MongoDB optimization for both OLTP/OLAP workloads. * Implementing a managed Ray (ray.io) service to spin and scale up ray clusters automatically on IBM Spectrum LSF. Tools I Currently Use * Golang, NodeJS, Python * PostgreSQL, MongoDB, Redis, ScyllaDB, Elasticsearch, GCP BigQuery * RabbitMQ, Apache Pulsar, GCP PubSub * Envoy, XDS, Nginx, Cilium, Wireshark, Ray (ray.io) * GCP Kubernetes, Pulumi, Hashicorp Packer, Kosko
Logo of the organization.
MediaTek 聯發科技
National Chiao Tung University
Taipei, Taiwan

Professional Background

  • Current status
    Employed
  • Profession
    Back-end Engineer
    Data Engineer
    Software Engineer
  • Fields
    SaaS / Cloud Services
    Software
    Information Services
  • Work experience
    6-10 years (6-10 years relevant)
  • Management
    I've had experience in managing 1-5 people
  • Skills
    Golang
  • Languages
    English
    Fluent
  • Highest level of education
    Master

Job search preferences

  • Desired job type
    Full-time
    Interested in working remotely
  • Desired positions
    Software Engineer
  • Desired work locations
    Taipei City, Taiwan
  • Freelance
    Non-freelancer

Work Experience

Logo of the organization.

Software Engineer

Jun 2022 - Present
Hsinchu City, Taiwan 300

Architect & Backend Engineer

Jul 2015 - May 2022
6 yrs 11 mos
2021 - 2022 * Using Linux TPROXY to scale up L4 K8S Pods on demand and save 50%+ costs of our dev K8S cluster. * Using my unified PostgreSQL CDC and dumping solution to handle data changes robustly, including indexing content into Elasticsearch, watching other micro services' events, and migrating the whole database with one event consumer implementation with nearly zero downtime. * Building CDN edge computing framework to make my colleagues utilize CDN cache at edges easily. * Improving 10x throughput on our usage of go-redis library transparently with auto pipeline technique. 2019 - 2021 * Building an internal IaC framework with Pulumi, Packer and GCP KMS. * Building an internal Envoy XDS control plane for better traffic management compared to K8S' service. * Building a scalable and transparent database snapshot as a service as well as a disaster recovery mechanism based on GCE snapshots and custom proxy of PostgreSQL and MongoDB. It has now become a core platform for ETL, OLAP, and even a testing environment for query optimization. It restored 1000+ database snapshots every day in 2020 with little GCP costs. Later, I enhanced the platform to real-time replication with my CDC solution in 2021. * Building a fast cardinality estimation service with HyperMinHash to give a quick business insight. * Building a CDN-friendly linked list service with Golang and PostgreSQL. It allows engineers to deliver content easily, including trying different content sorting algorithms. It has also become a core communication bridge between different engineer teams because of its simple API and high performance. It now inserts 10TB+ of data every day into just one n1-highmem-4 PostgreSQL instance thanks to optimization with ZSTD compression and batching on GRPC streams. 2015 - 2018 * Building a user event tracking and CDN-friendly A/B testing remote config service with Golang and Apache Pulsar. It has also become a core data source that drives other business insights and ML sources. It now collects 2TB+ of events every day. I also made it easy for our colleagues to consume these events in Apache Flink and Spark seamlessly by writing our own connectors in Java. * Building NodeJS APIs and internal Golang GRPC/HTTP/AMQP framework with our code generation tool.

Software Engineering Intern

Jan 2017 - Dec 2018
2 yrs 0 mos
Developing and operating web systems of the Department of Computer Science, National Chiao Tung University with PHP.

Software Engineering Intern

Jan 2013 - Dec 2016
4 yrs 0 mos
Building web UI with AngularJS.

Software Engineering Intern

Sep 2013 - Sep 2016
3 yrs 1 mo
Developing and operating web systems of the National Central University with PHP.

Software Engineering Intern

Jan 2013 - Dec 2014
2 yrs 0 mos
Building both backend API and web UI with Ruby On Rails.

Education

Master of Science (MS)
Master of Computer Science
2016 - 2018
Bachelor of Science (BS)
Bachelor of Computer Science
2012 - 2016