Product Analysis Website

Avatar of Yi-hua Fang.
Avatar of Yi-hua Fang.

Product Analysis Website

Back-End Developer
Taiwan

Product Analysis Website

Collect and analyze Ptt discussions about beauty products to assist marketing staff to understand user reviews of the products.

Architecture

Data Pipeline

Core Features & Technologies: 

◆ Custom search keywords: Create various combinations of keywords by users. For example, users can set brand and multiple product keywords and connect them by different conditions.

◆ Automated web crawler: Use cheerio to crawl websites and crontab to active web crawler on schedule.

◆ Data pipeline: Established an automatic process including web crawler, unifying data format, deleting repeated data, and storing data on schedule.

◆ Search for articles: Use MySQL(in AWS RDS) to store Ptt discussions and search related data in MySQL by using custom search keywords.

◆ Ratio of positive and negative reviews: Use Google Natural Language API to identify emotions of articles.

◆ Monitoring negative reviews and alert: Store negative reviews in RDS and alert users when they log in.

◆ Word cloud: Use nodejieba to acquire the most popular words. Use wordcloud2 for data visualization.

Testing: Use Mocha to implement unit test.

Collect and analyze Ptt discussions about beauty products to assist marketing staff to understand user reviews of the products.
Avatar of the user.
Please login to comment.

Published: juil. 25ème 2021
122
6
0

Tools

trello
Trello
nodejs
Node.js
nginx
nginx
mysql
MySQL
mocha
Mocha
linux
Linux
javascript
JavaScript
html5
HTML5
github
GitHub
express
Express
css3
CSS3
bootstrap
Bootstrap
amazonwebservices
Amazon Web Services

Share