abr 2021 - Presente
- Develop web crawlers for social media using Python, Scrapy, Playwright, etc.
- Build and maintain RESTful API server with FastAPI in Python.
- Use MongoDB/MySQL as data store and a part of data pipeline.
- Manage ETL system and web crawler with Airflow.
- Store and apply cached data with Redis.
- Pack and deploy modules and environment with Docker.
- Deploy service on GCP/AWS.
#Python #FastAPI #Airflow #Redis #Docker #MongoDB