Data Engineering · AWS Data Engineering
Build Data Pipelines on Amazon Web Services
Learn how to build and operate data pipelines on AWS, from storing data in Amazon S3 and transforming it with AWS Glue to querying with Athena and warehousing in Redshift.
- Amazon S3 data lakes
- AWS Glue for data integration
- Amazon Athena and Amazon Redshift
- Orchestration and monitoring
Ask about AWS Data Engineering on WhatsApp (opens in a new tab)
Course overview
About This Course
AWS Data Engineering covers the Amazon Web Services used to collect, store, transform and analyse data, and how to connect them into dependable pipelines.
AWS is one of the most widely used cloud platforms for data. Data engineers who can design S3 data lakes, build Glue jobs and serve data through Athena and Redshift can support analytics and reporting at almost any scale.
You will learn how the AWS data services fit together, how to organise and catalogue data in S3, how to transform it with AWS Glue, how to query and warehouse it, and how to orchestrate and monitor the whole flow.
This course follows SQL and Python for Data Engineering in our Data Engineering learning paths. Azure Data Engineering covers the equivalent skills on Microsoft Azure.
Who it is for
Who Should Take This Course?
-
Learners with SQL and Python
You have the foundations and want to apply them on a cloud data platform.
-
ETL and BI developers
You build data integration or reporting solutions and want to work on AWS.
-
AWS professionals moving into data
You work with AWS and want to specialise in its data services.
-
Aspiring data engineers
You want practical experience with a widely used cloud data platform.
Learning outcomes
What You Will Learn
By working through the course, you will build towards being able to:
Describe the main AWS data services and how they combine into a data platform.
Design an Amazon S3 data lake with sensible zones, formats and partitioning.
Catalogue datasets with the AWS Glue Data Catalog and crawlers.
Write AWS Glue jobs that clean and transform data.
Query data in S3 with Amazon Athena and load analytical data into Amazon Redshift.
Orchestrate, schedule and monitor pipelines and respond to failures.
Apply security, performance and cost practices to AWS data workloads.
Curriculum
Course Curriculum
The course is organised into modules. Session order and depth may vary by training arrangement.
-
Module 1: AWS data platform fundamentals
The AWS services used for data and how they work together.
Topics covered include
Practical activity: Sketch an AWS data architecture for a sample reporting requirement.
-
Module 2: Amazon S3 data lakes
Store data durably and organise it for processing.
Topics covered include
Practical activity: Design an S3 layout and partitioning scheme for sample event data.
-
Module 3: AWS Glue
Catalogue and transform data with a managed integration service.
Topics covered include
Practical activity: Catalogue raw sample data and write a Glue job that cleans it.
-
Module 4: Amazon Athena
Query data where it sits in S3 using SQL.
Topics covered include
Practical activity: Query the cleaned data with Athena and compare partitioned and unpartitioned queries.
-
Module 5: Amazon Redshift
Load and serve data from a cloud data warehouse.
Topics covered include
Practical activity: Load curated sample data into Redshift tables designed for reporting.
-
Module 6: Orchestration and monitoring
Run AWS data pipelines on schedule and keep them healthy.
Topics covered include
Practical activity: Schedule and monitor a sample pipeline from S3 ingestion to Redshift.
Practical learning
Hands-On Learning
Practical work follows data from raw files to an analytics-ready warehouse. Examples of activities include:
Hands-on work on AWS needs access to an AWS environment. Whether an environment is provided, or you use your own account, will be explained when you enquire.
- Designing an S3 data lake layout
- Writing AWS Glue transformation jobs
- Querying data with Amazon Athena
- Loading and modelling data in Amazon Redshift
Before you start
Prerequisites and Learning Requirements
-
Recommended background
Working knowledge of SQL and basic Python. Our SQL and Python for Data Engineering courses cover these.
-
Technical prerequisites
Familiarity with basic cloud concepts is helpful but not required.
-
Software and tools
A computer with a reliable internet connection for live online sessions. Access to an AWS environment is needed for hands-on work; the options will be confirmed when you enquire.
-
Prior experience
Not intended as a first course for complete beginners in data.
Learning Format
IT Champion Hub offers live, instructor-led online training and workshops. Formats, duration and schedules vary by course and are not the same for every program.
- Live online sessions led by an instructor
- Practical activities alongside explanations
- Upcoming batches and schedules shared on enquiry
Enquire to find out the formats and upcoming batches available for this course. Enquire about this course
FAQ
Frequently Asked Questions
Do I need an AWS account?
Hands-on work needs access to an AWS environment. We'll explain the options, and any costs involved, when you enquire.
What should I know before starting?
You should be comfortable with SQL and basic Python. Our SQL and Python for Data Engineering courses are the recommended preparation.
Is this an AWS certification course?
No. The course focuses on practical AWS data engineering skills. It is not an official AWS certification course.
Should I choose Azure or AWS Data Engineering?
Both teach the same data engineering ideas on different platforms. Choose the platform used where you work or want to work, or enquire and we'll help you decide.
How can I enquire about upcoming training?
Select Enquire About This Course. The enquiry form will note that you are asking about AWS Data Engineering, and we'll get back to you with upcoming batches and available schedules.
Ready to Explore This Learning Path?
Have questions about the curriculum, prerequisites, or training arrangements? Get in touch with IT Champion Hub to discuss your learning goals.
Ask about AWS Data Engineering on WhatsApp (opens in a new tab)