In this article, we will explore the process of extracting data from an AWS RDS database, and then publishing it to S3 with AWS Glue. We will cover the following details:
- The ability to support data stored in Amazon Aurora and all other Amazon RDS engines, Amazon Redshift, and Amazon S3 (such as XML, CSV format)
- Big data size
- Different data schemas
- A solution to easily switch environments from Development → Test → User Acceptance Testing (UAT) → Staging → Production
- Autoscale hardware related to data size