Collect and analyze Ptt discussions about beauty products to assist marketing staff to understand user reviews of the products.
Core Features & Technologies:
Custom search keywords:Create various combinations of keywords by users. For example, users can set brand and multiple product keywords and connect them by different conditions.
Automated web crawler: Use cheerio to crawl websites and crontab to active web crawler on schedule.
Data pipeline:Established an automatic process including web crawler, unifying data format, deleting repeated data, and storing data on schedule.
Search for articles: Use MySQL(in AWS RDS) to store Ptt discussions and search related data in MySQL by using custom search keywords.
Ratio of positive and negative reviews: Use Google Natural Language API to identify emotions of articles.
Monitoring negative reviews and alert: Store negative reviews in RDS and alert users when they log in.
Word cloud: Use nodejieba to acquire the most popular words. Use wordcloud2 for data visualization.
Spent over 10 hours every day on coding, a total of 20 weeks.
Built an E-commerce website and personal project with https servers by using Node.js, AWS and MySQL.
Developed and launched a personal project "Product Analysis Website" independently in 5 weeks, using the web crawler to monitor Ptt discussions over specific brands, products, and keywords.
Co-worked with Front-End and Android trainees to add new features to the E-commerce website. Created APIs for the product review system. Combined LINE Bot with MySQL to let users search for products and check their orders by LINE.