If you’re working with AWS Infrastructure, you may know that currently there are some tools/frameworks support to implement your AWS infrastructure such as CloudFormation, Terraform, AWS CDK. What is the best tool that your team can rely on and use? What is the best tool can help you increase productivity and quality? Do you have the answer? If not, read this post, I will give you our answer and reasons.
In this article, we will explore the process of extracting data from an AWS RDS database, and then publishing it to S3 with AWS Glue. We will cover the following details:
- The ability to support data stored in Amazon Aurora and all other Amazon RDS engines, Amazon Redshift, and Amazon S3 (such as XML, CSV format)
- Big data size
- Different data schemas
- A solution to easily switch environments from Development → Test → User Acceptance Testing (UAT) → Staging → Production
- Autoscale hardware related to data size