• Developed and maintained using Django + Django DRF + Celery + Redis
• Added RESTful API
• Increased scalability
• Developed each subsystem in detail
• Migrated the system from MySQL to Greenplum
• Optimized SQL queries, reducing the time from one hour to 10 minutes
• Improved performance by using temporary tables, write compression, multiple write executions, indexes, composite indexes, and query analysis
• Reduced the deployment time from 30 minutes to 2 minutes using scripts
• Implemented Celery for parallel tasks and Redis for caching frequently used data
• Used different queues and workers for different Celery tasks, as well as scheduled and delayed tasks
• Optimized the performance analysis subsystem, reducing the query time from 30 minutes to 5 minutes, a 500% improvement
• Improved performance by using temporary tables, write compression, multiple write executions, and query analysis
• Implemented a comparison subsystem for large-scale comparisons, using multithreading and algorithms to accelerate the process
• Implemented Git hooks
• Updated Dockerfile to reduce the number of stack layers
• Updated Celery broker from Redis to RabbitMQ for increased performance
• Updated celery_config to improve performance and reduce the probability of failure
• Added compression function to effectively reduce the amount of data transmitted, which can be reduced to one-tenth in the best case

System Environment

• Operating system: Ubuntu and CentOS
• Containerization: Docker
• Deployment tool: Shell script
• Django web server: Gunicorn
• Database: MySQL and Greenplum

Web Crawler System Development and Maintenance

Developed and maintained a web crawler system using Celery + Redis for concurrency
Used Fabric for internal host management and maintenance
Internal Management System Development

Assisted in the development and maintenance of systems related to the MediaWiki project

Primarily used Python
Developed backend APIs (RESTful) using Gunicorn + Flask
Developed crawlers and systems for the creation, deletion, and modification of MediaWiki content
Used PHP to develop WordPress plugins
Used Scrapy to crawl Taiwanese news websites (Apple Daily, Public Television Service, Central News Agency, etc.)
Used Celery + Redis or RabbitMQ for task queues
Have experience using Facebook APIs (extracting data from fan pages)
Developed WordPress plugins that can synchronize posts with Facebook or vice versa
Have experience using Flask
Have experience using the following databases:
MySQL
PostgreSQL
MongoDB
Used SQLAlchemy for SQL object-relational mapping
Have experience using Bitbucket and Git for version control
Here are some additional details about the translation:

I used the term "MediaWiki project" to translate "字媒體計畫". This is the project that develops