【About me】 I'm Wei-Jou Lee, with 8 years of Python programming experience and a Master's in Information Management from National Central University. During my tenure at Lumentum, I specialized in designing, maintaining, and optimizing data infrastructure to facilitate collection, management, transformation, and access. Additionally, I managed the data pipeline for a globally renowned tech company, ensuring stability, reliability, and alignment with customer needs. 【Skills】 Development: REST APIs, Web Development (Flask), Swagger API, Python, Linux, SQL, Bash Data Management: Apache NiFi, Apache Kafka, NoSQL, MongoDB, Cassandra, Redis Cloud Technologies: AWS Lambda, AWS Glue, Amazon S3, Amazon Athena Automation: Jenkins, Kubernetes, Docker, Fluentd, Prometheus.io, CI/CD Testing: Unit Testing, Stress Testing, A/B Testing 【Work Experience】 In my role, I'm responsible for building and maintaining data infrastructure to ensure optimal extraction, transformation, and loading of large datasets from various channels such as API payloads, AWS S3, and Cassandra. My focus lies in enhancing data system reliability, speed, and overall performance. Moreover, I specialize in developing data tools, including performing SPC calculations, automating report email generation, and creating APIs tailored for our production line needs to ensure smooth operations. Achievements: Refactored data flow, reducing memory usage from 32 Gib to 1.6 Gib while increasing transfer rate by 3x. This initiative resolved data missing issues, augmented yield rates, and enhanced overall performance and stability. Redesigned data processes and database structures to support datasets five times larger, facilitating high throughput and accommodating large data sizes seamlessly. Engineered a microservices solution enabling real-time data uploads from Box to S3. Leveraging Box's webhook, real-time event triggers synchronize any user's actions to S3, reducing operational costs and significantly enhancing user experience.