Running ETL Jobs using Amazon Athena

Amazon Athena supports the ‘CREATE TABLE AS’ and ‘INSERT INTO’ statements, which can be used to transform data between a source and a destination.

In this post I’m using them to optimise the storage of data that is received into S3 as files of JSON objects. Specifically, I’m using CREATE TABLE AS (CTAS) to […] “Running ETL Jobs using Amazon Athena”

Integrating AWS Glue Jobs with S3 Notifications and EventBridge

In my last post I created a standalone AWS Glue job for loading and transforming data from S3 into Redshift.

Here, I’m expanding that scenario by integrating the job with S3 notifications, Lambda functions, EventBridge rules and SNS messaging. Specifically this will […] “Integrating AWS Glue Jobs with S3 Notifications and EventBridge”

Loading data into Amazon Redshift using AWS Glue

AWS Glue is a fully managed, cloud-native, AWS service for performing extract, transform and load operations across a wide range of data sources and destinations.

It supports connectivity to Amazon Redshift, RDS and S3, as well as to a variety of third-party database engines running on EC2 instances. It’s a serverless […] “Loading data into Amazon Redshift using AWS Glue”