Research Interests
- Artificial Intelligence:
- Natural Language Understanding and Natural Language Generating
- Computer Vision
- Data Mining:
- Social media mining
- Visualization
Work Experience
Microsoft AI Research and Development Center
Research Software Development Engineer Taipei, Taiwan Jan. 2018 - Now
- Natural language and computer vision techniques survey and investigation
- Enhance document understanding accuracy by deep learning approach
- Enhance Chinese IME quality by unsupervised word segmentation
Microsoft Research Asia
Research Intern (Mentor: Ruihua Song) Beijing, China Feb. 2016 - Aug. 2016
- Advanced research on natural language understanding and generation
- Research and production on image inspired Chinese poetry generation
- Research and implementation on personalized chatting robot
National Institute of Informatics
Research Intern (Mentor: Shin’ichi Satoh) Tokyo, Japan Aug. 2015 - Feb. 2016
- Social media mining by cross-modal understanding
- Deep learning on social media images and metadata
- Location-based applications by combining social media data and traffic data
Academia Sinica
Summer Intern (Mentor: Tsan-sheng Hsu) Taipei, Taiwan Jul. 2013 - Aug. 2013
- Research on large graph mining
Awards
- Best Demo Award Microsoft Research Asia Academic Day 2017
- Multimodal Award Grand Challenge, ACM Multimedia 2014
Projects
- Document understanding
Deep Learning Computer Vision Nature Language Processing
Designed and implement deep learning approaches to improve the quality of document understanding, hugely surpassing all competitors on English receipts.
- Unsupervised word segmentation
Learning Nature Language Processing
Designed an unsupervised word segmentation algorithm and implement a word segmentation tool for all languages. This approach reduces the end-to-end relative error rate by 12% for Chinese IME (input method editor).
- Xiaoice writes poetry inspired by images
Deep Learning Nature Language Generating
Main contributor of this work, including the implementation of an application prototype. By leveraging multi-modal understanding with deep learning, this application takes an image from users and extracts information from the image to construct fluent poems. Within the first twenty-four hours of release, this application received near 300 thousand requests from users and has generated over 2 million poems since launch. A collection of the generated poem was published in 2017.
- Landmark analysis and recommendation for local, domestic and foreign travelers
Social Media Mining Deep Learning Computer Vision Nature Language Processing
Collected a social media dataset of Sapporo (0.64M image posts on Instagram and tweets on Twitter). By leveraging multi-modal deep learning approaches, this work combines both location types prediction and user hometown predictions to discover different travel behaviors in Sapporo for 15,563 travelers from different countries.
- Location classification on social media by multi-modality engagement
Social Media Mining Deep Learning Computer Vision Nature Language Processing
Collected a large scale Instagram dataset including more than 14 million photos from half a million locations in New York. By leveraging a multi-modal classification model, the model achieves 73.18% accuracy on locations type prediction.
Education
National Taiwan University Sep. 2013 - Jan. 2017
M.S. in Graduate Institute of Networking and Multimedia
Communications and Multimedia Laboratory (CMLAB), MiRA Group (Advisor: Winston H. Hsu)
Thesis: Location classification on social media by multi-modality engagement
National Taiwan University Sep. 2009 - Jan. 2013
B.S. in Computer Science & Information Engineering
Publications
- Accepted:
- Wen-Yu Lee, Yin-Hsi Kuo, Peng-Ju Hsieh, Wen-Feng Cheng, Ting-Hsuan Chao, Hui-Lan Hsieh, Chieh-En Tsai, Hsiao-Ching Chang, Jia-Shin Lan, Winston Hsu, “Unsupervised Latent Sub-events Discovery based on Multi-content and Human Activities Analysis for Diverse Event Summarization,” ACM Multimedia 2015. (Grand Challenge)
- Ching-Hsuan Liu, Yen-Liang Lin, Wen-Feng Cheng, Winston H, Hsu, “Exploiting word and visual word co-occurrence for sketch-based clipart image retrieval,” ACM Multimedia 2015. (Short paper)
- Pei-Yun Hsu, Wen-Feng Cheng, Peng-Ju Hsieh, Yen-Liang Lin, Winston H. Hsu, “Real-Time Instant Event Detection in Egocentric Videos by Leveraging Sensor-Based Motion Context,” ACM Multimedia 2015. (Short paper)
- Yi-Chih Tsai, Liang-Chi Hsieh, Wen-Feng Cheng, Yin-Hsi Kuo, Winston H. Hsu, Wen-Chin Chen “Trending pool: Visual analytics for trending event compositions for time-series categorical log data,” IEEE VIS (vast), 2015. (Short paper)
- Yin-Hsi Kuo, Yan-Ying Chen, Bor-Chun Chen, Wen-Yu Lee, Chun-Che Wu, Chia-Hung Lin, Yu-Lin Hou, Wen-Feng Cheng, Yi-Chih Tsai, Chung-Yen Hung, Liang-Chi Hsieh, Winston H. Hsu, “Discovering the city by mining diverse and multimodal data streams,” ACM Multimedia 2014. (Grand Challenge, all authors are equally contributed)
- Arxiv:
- Cheng, Wen-Feng, et al. "Image inspired poetry generation in xiaoice." arXiv preprint arXiv:1808.03090 (2018).
- Other:
- “阳光失了玻璃窗”, the first AI-authored collection of poems published in the name of AI (Xiaoice).
