Real-Time Data Processing with Kafka Training Course
Course Overview
Developers who code in Java, Python or Scala spend three days building event pipelines on Apache Kafka that react the moment data arrives. Day one compares streaming with batch work, explains event-driven design and the broker, topic, partition and offset model, and writes a producer and consumer app. Day two processes streams with the Kafka Streams API, links Kafka to databases and analytics tools through Kafka Connect, and secures and monitors a cluster. Day three scales for high volume with partitions and replication, studies log aggregation and event sourcing cases, and ends with teams presenting a Kafka design for a business use case.
Format of Training
- Whiteboard sessions on brokers, topics and partitions before any code
- Coding labs that write producers, consumers and Kafka Streams jobs
- Demo of an end-to-end pipeline wired up with Kafka Connect
- Cluster security and scaling exercises, then a team design pitch
Course Objectives
- Explain how real-time processing and event-driven architectures work.
- Describe Apache Kafka’s design and its core parts.
- Set up and manage Kafka clusters.
- Produce, consume and process live data streams.
- Connect Kafka to outside systems such as databases and stream processing tools.
- Apply Kafka to practical cases such as log aggregation, data pipelines and monitoring.
- Deploy and scale Kafka in production.
Prerequisites
- A basic grasp of data and how it moves through systems
- Some coding in a language such as Java, Python or Scala
- No Kafka experience needed
- A wish to build real-time data processing systems
Course Outline
Day 1: Introduction to Real-Time Data Processing and Kafka
Session 1: Fundamentals of Real-Time Data Processing
- Real-time processing compared with batch processing
- What event-driven architectures are
Session 2: Apache Kafka Overview
- How Kafka is built: brokers, topics, partitions and offsets
- Preparing a Kafka environment
Session 3: Producing and Consuming Data Streams
- Coding producers and consumers in Kafka
- Writing a Kafka producer and consumer application
Day 2: Advanced Kafka Features and Integration
Session 1: Kafka Streams and Processing
- Processing streams with the Kafka Streams API
- Running live data transformations in Kafka Streams
Session 2: Integrating Kafka with Other Systems
- Linking Kafka to databases, monitoring tools and analytics platforms
- Demo: an end-to-end pipeline built with Kafka Connect
Session 3: Securing and Monitoring Kafka Clusters
- Role-based access control, encryption and monitoring
- Locking down and watching a Kafka cluster
Day 3: Scaling and Real-World Applications
Session 1: Scaling Kafka for High-Volume Data
- Setting partitions, replication and fault tolerance
- Tuning Kafka so it scales
Session 2: Real-World Kafka Applications
- Cases on log aggregation, event sourcing and real-time analytics
- Group task: designing a Kafka solution for a business case and presenting it
Session 3: Best Practices and Future Trends
- Tuning Kafka for production use
- Talk: new ideas and directions in real-time data processing
Book This Course
Customise This Course for Your Team
Teams can bring sample events from their own apps, sensors or logs so the producer and stream labs work on familiar data. If your Kafka cluster is already running, Day 3 can focus on scaling and securing that setup instead of a fresh one. Let us know your event sources and we will plan the labs around them.
Need help with the right course to choose?
Explore more opportunities
- Duration: 3 Days
- 4.5 Ratings
Building Scalable Data Pipelines with Apache Airflow Training Course
- Duration: 3 Days
- 4.5 Ratings
Data Warehousing with Snowflake and Redshift Training Course
- Duration: 3 Days
- 4.5 Ratings
Real-Time Data Processing with Kafka Training Course