Accelerating Data Engineering Pipelines
Contact IT Dojo for current pricing, available dates, and a custom quote tailored to your team or organization.
Course Duration
1 Day
Audience
Employees of federal, state and local governments; and businesses working with the government.
Prerequisites
Familiarity with data engineering concepts and basic Python experience recommended.
Course Description
Today's teams need to move fast to deliver value for their organizations and customers, and slow data pipelines are a common bottleneck. In this workshop, participants explore how GPUs can improve data pipelines, and how advanced data engineering tools and techniques can result in significant performance acceleration. Faster pipelines produce faster insights and free up engineering time for higher-value work.
Learning Objectives
- Identify common performance bottlenecks in data engineering pipelines
- Explain where and how GPU acceleration improves pipeline performance
- Apply parallelization and batching strategies to a data pipeline
- Benchmark and evaluate the performance impact of pipeline acceleration techniques
Course Outline
1. Where Data Pipelines Bottleneck
- Common performance bottlenecks in traditional data pipelines
- Where GPU acceleration provides the most benefit
- Profiling a pipeline to find the real bottleneck
2. GPU-Accelerated Data Processing
- Overview of GPU-accelerated data processing libraries
- Porting common pipeline operations to run on GPU
- Measuring the performance gains realistically
3. Pipeline Architecture and Techniques
- Parallelization and batching strategies
- Memory management for large-scale data processing
- Choosing where acceleration is and is not worth the added complexity
4. Hands-On Lab and Wrap-Up
- Accelerating a sample data pipeline end to end
- Benchmarking before-and-after performance
- Final questions and key takeaways
Frequently Asked Questions
What does this course cover?
This workshop covers how GPUs and advanced data engineering techniques can significantly accelerate data pipeline performance.
How long is this course?
Accelerating Data Engineering Pipelines is a one-day workshop, available as live remote online instruction or on-site at your facility.
Who should attend?
Data engineers and data scientists who want to speed up data pipeline performance.
Does IT Dojo offer this training on-site at government or DoD facilities?
Yes. IT Dojo delivers this course on-site at government agencies, DoD commands, and contractor facilities, in addition to live online delivery.
How do I register for this course?
IT Dojo training is employer sponsored. Contact IT Dojo via the Request Training form or call 757-216-3656 to schedule this course for your team.