CakeResume 找人才
進階搜尋
On
4 到 6 年
6 到 10 年
10 到 15 年
15 年以上
Data Engineer @Tesla2023 ~ 2023
・
Data engineer / Data anyayst
兩個月內
data that included geo-location data from BigQuery and deployed it on the GCP environment. The API saved 80% of the time on fetching data (Cloud Run, IAM, BigQuery) 十月七月 2021 Data engineer • 富盈數據 Maintained distributed system and database • Constructed and managed the Hadoop ecosystem with Ambari. Built ETL pipeline to query multi-source database which processing more than three terabytes (TB) provided 90% of the analysis needs (Hive, HBase, Python, ELK, MySQL) • Established data collection and analysis workflow, saving Data scientists’ 30% of the time to analyze and build machine learning
就職中
・
正在積極求職中
全職 / 對遠端工作有興趣
4 到 6 年
University of Texas at Dallas
・
Information Technology and Management
Principal Engineer, Data Engineering @KKCompany2023 ~ 現在
・
Backend Engineer, Data Engineer, MLOps Engineer
一個月內
Chin-Hung (Wilson) Liu I am a lead architect responsible for designing and implementing a large-scale data pipeline for Lomotif, Paktor x 17LIVE, utilizing GCP/AWS/Python/Scala, in collaboration with data science and machine learning teams in Singapore and TW HQ, as well as with the Hadoop ecosystem (HDFS/HBase/Kafka) at JSpectrum in Hong Kong and Sydney. With over 15 years of experience in designing and developing Java/Scala/Python-based applications for daily operations, I bring: ● At least 8 years of experience in data analysis, pipeline design
就職中
・
目前會考慮了解新的機會
全職 / 對遠端工作有興趣
10 到 15 年
National Taiwan University
・
EMBA Programs, Business Administration, Accounting, Finance and International Business.
Staff Cloud Architect Enginner @域動行銷股份有限公司2023 ~ 現在
・
雲端工程師，雲端架構師，數據架構師
一個月內
及進步的團隊。 [email protected], Taiwan Education 輔仁大學圖書資訊學系,Skills Python 程式寫作、實作爬蟲及資料清理作業。 Database SQL : PostgreSQL, MySQL, MSSQL NoSQL: MongoDB, Redis, DynamoDB, Hadoop Docker 容器化技術 Data Lakehouse Databricks Azure 持有Microsoft Certified: Azure Solutions Architect Expert 及 Data enginner 證照 AWS 持有 AWS Certified Solutions Architect – Professional 證照 工作經歷 Staff Cloud Architect Enginner , 域動行銷股份有限
就職中
・
目前沒有興趣尋找新的機會
全職 / 對遠端工作有興趣
4 到 6 年
輔仁大學
・
圖書資訊
就職中
全職 / 對遠端工作有興趣
6 到 10 年
Senior Data Engineer @美光科技2021 ~ 現在
・
Software Engineer / Backend Engineer / DevOps Engineer
半年內
CD pipelines from scratch which follow GitOps flow and deploying service to GKE cluster using Helm . Familiar with GCP service , IAM, GCS, Big Query, Cloud Function, Pub/Sub, Cloud Scheduler Data Engineer Micron OctOct 2021 Taichung, Taiwan Developed and maintained ETL processes using Python to transfer data into Hadoop Ecosystem, including HBase and Hive, for efficient data storage and retrieval. Proficient in SQL for data manipulation and query optimization. Collaborated with cross-functional teams to design and implement data pipelines, ensuring data integrity and accuracy. Streamlined data processing workflows, resulting in significant time and resource
就職中
全職 / 我只想遠端工作
4 到 6 年
National Chiao Tung University
・
資訊管理學系
Senior Data Engineer @Garena2021 ~ 現在
・
Data engineer
一年內
Aiden Wu Data Engineer / Machine Learning Engineer Taipei, Taiwan • Enthusiastic software developer: focus on distributed systems, especially Hadoop ecosystem • Experience in data engineering: develop batch and real-time data pipelines with an average of TBs per month via Spark and Airflow • Experience in machine learning: develop machine learning (ML) and deep learning (DL) models while providing services on RESTful API https://www.slideshare.net/ssuserf88631/presentations 工作經歷 Senior Data Engineer • Garena 八月Present • Build and manage self-distributed systems (e.g., Hadoop, Spark, and Kafka Cluster) • Design
就職中
全職 / 對遠端工作有興趣
4 到 6 年
National Cheng Kung University
・
Department of Electrical Engineering
Senior Data Engineer @Microsoft2021 ~ 現在
・
資料科學家、資料工程師、資料分析師
一個月內
Ching-Chuan Chen 陳慶全 資料科學家、資料工程師、資料分析師 • City, TW • [email protected] Data engineer and data scientist with over four half years of experience. Proven success in processing big volume of data (6TB per day) in Spark in Scala and MPI in R and Python, developing a machine learning model with Spark in Scala on 30 billions of records for IoT device recognition and developing algorithms to classify unlabeled network behaviors of customers to protect their devices from compromising. Skilled in programming
就職中
全職 / 對遠端工作有興趣
4 到 6 年
National Cheng Kung University,
・
Statistics就職中
全職 / 對遠端工作有興趣
10 到 15 年
Jr. Programmer @德義資訊股份有限公司2013 ~ 2015
・
Developer Team Leader, Architect, FullStack Developer
超過一年
就職中
全職 / 對遠端工作有興趣
6 到 10 年
National Taiwan University
・
Bachelor of Bio-Industrial Mechatronics Engineering
Big data developer
超過一年
ZOOKEEPER. Summary Excellent understanding /knowledge on HADOOP(Gen-1 and Gen-2) and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, Resource Manager (YARN), Node Manager and Aplication Master. Expert in understanding the data and designing/implementing the enterprise platforms like Hadoop data lake and huge Data warehouses. Have over 2 years of experience as Hadoop Architect with very good exposure on Hadoop Technologies like HDFS, YARN, MapReduce, Sqoop, Flume, HBase, Hive, Presto, Oozie and Spark. Good understanding of NoSQL databases and hands on working experience in writing applications
全職 / 對遠端工作有興趣
6 到 10 年
JNTUH
・
Computer science最輕量、快速的招募方案，數百家企業的選擇
搜尋履歷，主動聯繫求職者，提升招募效率。
- 瀏覽所有搜尋結果
- 每日可無限次數開啟陌生對話
- 搜尋僅開放付費企業檢視的履歷
- 檢視使用者信箱 & 電話
搜尋技巧
1
嘗試搜尋最精準的關鍵字組合
資深 後端 php laravel
2
將須完全符合的字詞放在雙引號中
"社群行銷"
3
在不想搜尋到的字詞前面加上減號，如果想濾掉中文字，需搭配雙引號使用 (-"人資")
UI designer -UX
職場能力評價定義
專業技能
該領域中具備哪些專業能力（例如熟悉 SEO 操作，且會使用相關工具）。
問題解決能力
能洞察、分析問題，並擬定方案有效解決問題。
變通能力
遇到突發事件能冷靜應對，並隨時調整專案、客戶、技術的相對優先序。
溝通能力
有效傳達個人想法，且願意傾聽他人意見並給予反饋。
時間管理能力
了解工作項目的優先順序，有效運用時間，準時完成工作內容。
團隊合作能力
具有向心力與團隊責任感，願意傾聽他人意見並主動溝通協調。
領導力
專注於團隊發展，有效引領團隊採取行動，達成共同目標。
兩個月內
Sr. Data Engineer / Team Lead
新加坡商競舞電競娛樂有限公司 Garena Online Private Ltd
・
2021 ~ 現在
專業背景
目前狀態
就職中
求職階段
專業
其他
產業
工作年資
6 到 10 年
管理經歷
我有管理 1～5 人的經驗
技能
語言能力
Chinese
・
母語或雙語
English
・
進階
求職偏好
希望獲得的職位
資料工程師
預期工作模式
全職
期望的工作地點
Taipei, 台灣
遠端工作意願
對遠端工作有興趣
接案服務
學歷
學校
主修科系
Yi-Lun Wu (Velen)
Summary
- 7+ years of experience in big data fields both Cloud(GCP, AWS) and on-premised (Cloudera CDH).
- Develop data catalog with hybrid cloud environment(on-perm+AWS) for global commerce at Yahoo.
- Lead a machine learning team to build up offline/real-time platforms for recommendation systems from scratch at Garena.
- Lead and architect large-scale data pipelines/warehouses from Innova Solutions(AWS) and 17LIVE (GCP). And Cooperate with data science/machine learning team/TW HQ data team.
- Expert of Hadoop ecosystems such as HDFS/Hive/Hbase/Spark in Athemaster and Xuenn.
Sr. Big Data Engineer / Team Lead
Taoyuan，TW
[email protected]
Skills
Languages
- Python
- SQL
- Linux Shell Script
- Scala
Big Data Solutions
- Cloudera CDH
- Hadoop echosystems
- AWS EMR
- GCP Dataporc
Data Warehouse
- Hive
- Google BigQuery
- MySQL
- HBase
- Clickhouse
- AWS RDS
- AWS Athena
ETL Skills in Big Data
- Spark / Spark Streaming
- Hive (for ELT)
- Impala
- Kafka
- Cloud DataFlow (GCP)
Workflow Skills
- Airflow
- Digdag
- NiFi
- AWS CloudFormation
Other Skills
- Great communication
- Leaderships
- Scrum
- JIRA
- Linux
- Git
Experience
Senior Engineer at Yahoo．Jan, 2023 - Present
Working at Yahoo. The greatest challenge lies in developing in response to rapid market changes, where the data catalog needs to integrate with various complex systems. Simultaneously, maintaining the highest stability and ensuring high-quality data is essential.
More responsibilities/details as below.
- Fetch providers data with various way via Java. Such as fetching data from client's API, GraphQL, FTP, S3 or GCP... etc.
- standardizing data from above to feed into data warehouse in Hadoop/Hive/HBase using Spark.
- Implement a checking system to guarantee high quality data via Java.
- Migrate part of services from on-perm(Hadoop) to cloud(AWS) to become a hybrid cloud environment.
Senior Data Engineer/ Team Lead, at BOOYAH! Live Garena．Oct, 2021 - Sep, 2022
Working at ML team as a first data engineer. The challenge include build up data warehouse/pipeline from scratch. And design the data flow to support both batch/real-time recommendation systems.
More responsibilities/details as below.
- Design data model from scratch and Manage Hadoop based data warehouse for the training system.
- Develop streaming ETL pipeline via Spark from Message queue(Kafka) into in-memory data structure store(Redis) and ClickHouse for real-time recommendation system.
- Design ELT job for offline report system in Hadoop/Hive using Spark.
- Build up monitoring dashboard on Grafana.
- Take leadership on TW side.
Senior Data Engineer at 17 Media．Jun, 2020 - Oct, 2021
The big challenge of 17 Media data teams is facing fast-growing data volume (processing 5-10x TB level daily), complex cooperation with stakeholders, the cost optimization of pipeline and refactor big latency systems .etc.
More responsibilities/details as below.
- Manage Google BigQuery based data warehouse/lake.
- Refactory architecture of data warehouse to enhance 2x performance.
- Develop batch/streaming ETL pipeline to process data from diverse data sources(e.g. MongoDB, MySQL, APIs) into GCP.
- Design workflow using Digdag.
- Implement CI/CD on BIgQuery.
- Build up visualization tool(Superset) via Kubernetes.
- Well leadership and guiding junior members.
Senior Software Engineer at Innova Solutions Ltd．Oct, 2018 - May, 2020
Development Intelligent Healthcare Data Platform(IHDP) for empowering compony solutions using AWS service.
More responsibilities/details as below.
- Build APIs to the processing of patient records and providing access for downstream usages.
- Build Infrastructure on AWS and compliance for HIPPA and GDPR standard.
IT Consultant at Xuenn Pte Ltd．May, 2018 - Sep, 2018
The biggest challenge in Xuenn is facing performance issues in the original data warehouse. And I lead a project to build up a Hadoop cluster to reduce original EDW loading and improve various data pipelines.
More responsibilities/details as below.
- Perform adopting new technologies to implant into, to fuse into or to replace with existing systems for gaining leaps in performance, benefits and capabilities of the users.
- Building-up multiple systems and integrating with existing system, implemented Hadoop, data mining or data warehouse systems.
- Design an architecture which processing real-time data end-to-end using various Hadoop solutions without coding.
Software Engineer at Athemaster Co., Ltd．Jan, 2016 - Apr, 2018
Athemaster is a technology company offering solutions and expertise in implementing Enterprise Data Hub and automating Data integration with Open Source technologies such as Apache Hadoop and Spark.
More responsibilities/details as below.
- Focus on Enterprise Big Data solution such as Hadoop and Spark (Cloudera CDH).
- Maintain and improved other companies' Hadoop cluster.
- Help other companies integrate with Hadoop and resolved the technical issues.
- Build data pipelines via python ETL data to Hadoop.
Certification and License
CCA-175: CCA Spark and Hadoop Developer
Education
Undergraduate studies at Tamkang University, with a concentration in Department of Management Sciences.
2008 - 2012
Yi-Lun Wu (Velen)
Summary
- 7+ years of experience in big data fields both Cloud(GCP, AWS) and on-premised (Cloudera CDH).
- Develop data catalog with hybrid cloud environment(on-perm+AWS) for global commerce at Yahoo.
- Lead a machine learning team to build up offline/real-time platforms for recommendation systems from scratch at Garena.
- Lead and architect large-scale data pipelines/warehouses from Innova Solutions(AWS) and 17LIVE (GCP). And Cooperate with data science/machine learning team/TW HQ data team.
- Expert of Hadoop ecosystems such as HDFS/Hive/Hbase/Spark in Athemaster and Xuenn.
Sr. Big Data Engineer / Team Lead
Taoyuan，TW
[email protected]
Skills
Languages
- Python
- SQL
- Linux Shell Script
- Scala
Big Data Solutions
- Cloudera CDH
- Hadoop echosystems
- AWS EMR
- GCP Dataporc
Data Warehouse
- Hive
- Google BigQuery
- MySQL
- HBase
- Clickhouse
- AWS RDS
- AWS Athena
ETL Skills in Big Data
- Spark / Spark Streaming
- Hive (for ELT)
- Impala
- Kafka
- Cloud DataFlow (GCP)
Workflow Skills
- Airflow
- Digdag
- NiFi
- AWS CloudFormation
Other Skills
- Great communication
- Leaderships
- Scrum
- JIRA
- Linux
- Git
Experience
Senior Engineer at Yahoo．Jan, 2023 - Present
Working at Yahoo. The greatest challenge lies in developing in response to rapid market changes, where the data catalog needs to integrate with various complex systems. Simultaneously, maintaining the highest stability and ensuring high-quality data is essential.
More responsibilities/details as below.
- Fetch providers data with various way via Java. Such as fetching data from client's API, GraphQL, FTP, S3 or GCP... etc.
- standardizing data from above to feed into data warehouse in Hadoop/Hive/HBase using Spark.
- Implement a checking system to guarantee high quality data via Java.
- Migrate part of services from on-perm(Hadoop) to cloud(AWS) to become a hybrid cloud environment.
Senior Data Engineer/ Team Lead, at BOOYAH! Live Garena．Oct, 2021 - Sep, 2022
Working at ML team as a first data engineer. The challenge include build up data warehouse/pipeline from scratch. And design the data flow to support both batch/real-time recommendation systems.
More responsibilities/details as below.
- Design data model from scratch and Manage Hadoop based data warehouse for the training system.
- Develop streaming ETL pipeline via Spark from Message queue(Kafka) into in-memory data structure store(Redis) and ClickHouse for real-time recommendation system.
- Design ELT job for offline report system in Hadoop/Hive using Spark.
- Build up monitoring dashboard on Grafana.
- Take leadership on TW side.
Senior Data Engineer at 17 Media．Jun, 2020 - Oct, 2021
The big challenge of 17 Media data teams is facing fast-growing data volume (processing 5-10x TB level daily), complex cooperation with stakeholders, the cost optimization of pipeline and refactor big latency systems .etc.
More responsibilities/details as below.
- Manage Google BigQuery based data warehouse/lake.
- Refactory architecture of data warehouse to enhance 2x performance.
- Develop batch/streaming ETL pipeline to process data from diverse data sources(e.g. MongoDB, MySQL, APIs) into GCP.
- Design workflow using Digdag.
- Implement CI/CD on BIgQuery.
- Build up visualization tool(Superset) via Kubernetes.
- Well leadership and guiding junior members.
Senior Software Engineer at Innova Solutions Ltd．Oct, 2018 - May, 2020
Development Intelligent Healthcare Data Platform(IHDP) for empowering compony solutions using AWS service.
More responsibilities/details as below.
- Build APIs to the processing of patient records and providing access for downstream usages.
- Build Infrastructure on AWS and compliance for HIPPA and GDPR standard.
IT Consultant at Xuenn Pte Ltd．May, 2018 - Sep, 2018
The biggest challenge in Xuenn is facing performance issues in the original data warehouse. And I lead a project to build up a Hadoop cluster to reduce original EDW loading and improve various data pipelines.
More responsibilities/details as below.
- Perform adopting new technologies to implant into, to fuse into or to replace with existing systems for gaining leaps in performance, benefits and capabilities of the users.
- Building-up multiple systems and integrating with existing system, implemented Hadoop, data mining or data warehouse systems.
- Design an architecture which processing real-time data end-to-end using various Hadoop solutions without coding.
Software Engineer at Athemaster Co., Ltd．Jan, 2016 - Apr, 2018
Athemaster is a technology company offering solutions and expertise in implementing Enterprise Data Hub and automating Data integration with Open Source technologies such as Apache Hadoop and Spark.
More responsibilities/details as below.
- Focus on Enterprise Big Data solution such as Hadoop and Spark (Cloudera CDH).
- Maintain and improved other companies' Hadoop cluster.
- Help other companies integrate with Hadoop and resolved the technical issues.
- Build data pipelines via python ETL data to Hadoop.
Certification and License
CCA-175: CCA Spark and Hadoop Developer
Education
Undergraduate studies at Tamkang University, with a concentration in Department of Management Sciences.
2008 - 2012