Cloudera DataFlow: Flow Management with Apache NiFi (C-DATAFLOW-FMAN)

This three-day hands-on training course provides the fundamental concepts and experience necessary to automate the ingest, flow, transformation, and egress of data using Apache NiFi. 


Along with gaining a grasp of the key features, concepts, and benefits of NiFi, participants will create and run NiFi dataflows for a variety of scenarios. Students will gain expertise using processors, connections, and process groups, and will use NiFi Expression Language to control the flow of data from various sources to multiple destinations. Participants will monitor dataflows, examine progress of data through a dataflow, and connect dataflows to external systems such as Kafka and HDFS. After taking this course, participants will have key knowledge and expertise for configuring and managing data ingestion, movement, and transformation scenarios for the enterprise.


Audience & Prerequisites 

This course is designed for Developers, Data Engineers, Data Scientists, and Data Stewards. It provides a no-code, graphical approach to configuring real-time data streaming, ingestion, and management solutions for a variety of use cases. Though programming experience is not required, basic experience with Linux is presumed. Exposure to big data concepts and applications is helpful.


What You Will Learn 

Students who successfully complete this course will be able to: 

  • Understand the role of Apache NiFi and MiNiFi in the Cloudera DataFlow platform 
  • Describe NiFi’s architecture, including standalone and clustered configurations 
  • Use key features, including FlowFiles, processors, process groups, controllers, and connections, to define a NiFi dataflow 
  • Navigate, configure dataflows, and use dataflow information with the NiFi User Interface 
  • Trace the life of data, its origin, transformation, and destination, using data provenance 
  • Organize and simplify dataflows 
  • Manage dataflow versions using the NiFi Registry 
  • Use the NiFi Expression Language to control dataflows 
  • Implement dataflow optimization methods and available monitoring and reporting features 
  • Connect dataflows with other systems, such as Kafka and HDFS 
  • Describe aspects of NiFi security 


This training is provided in collaboration with PUE, Cloudera Authorized Training Center.

Mostra dettagli


Course Contents

Introduction to Cloudera Flow Management

  • Overview of Cloudera Flow Management and NiFi
  • The NiFi User Interface


Processors

  • Overview of Processors
  • Processor Surface Panel
  • Processor Configuration


Connections

  • Overview of Connections
  • Connection Configuration
  • Connector Context Menu


Dataflows

  • Command and Control of a Dataflow
  • Processor Relationships
  • Back Pressure
  • Prioritizers
  • Labels


Process Groups

  • Anatomy of a Process Group
  • Input and Output Ports


FlowFile Provenance

  • Data Provenance Events
  • FlowFile Lineage
  • Replaying a FlowFile


Dataflow Templates

  • Templates Overview
  • Managing Templates


Apache NiFi Registry

  • Apache NiFi Registry Overview
  • Using the Registry


FlowFile Attributes

  • FlowFile Attributes
  • Routing on Attributes


NiFi Expression Language

  • NiFi Expression Language Overview
  • Syntax
  • Expression Language Editor
  • Setting Conditional Values


Dataflow Optimization

  • Dataflow Optimization
  • Control Rate
  • Managing Compute


NiFi Architecture

  • NiFi Architecture Overview
  • Cluster Architecture
  • Heartbeats
  • Managing Clusters


Site-to-Site Dataflows

  • Site-to-Site Theory
  • Site-to-Site Architecture
  • Anatomy of a Remote Process Group
  • Adding and Configuring Remote Process Groups


Cloudera Edge Management and MiNiFi

  • Overview of MiNiFi
  • Example Walk-through


Monitoring and Reporting

  • Monitoring from NiFi
  • Overview of Reporting
  • Examples of Common Reporting Tasks


Controller Services

  • Controller Services Overview
  • Common Controller Services


Integrating NiFi with the Cloudera Ecosystem

  • NiFi Integration Architecture
  • NiFi Ecosystem Processors
  • A Closer Look at NiFi and Apache Hive
  • A Closer Look at NiFi and Apache Kafka


NiFi Security

  • NiFi Security Overview
  • Securing Access to the NiFi UI
  • Authentication
  • The Importance of Kerberos
  • NiFi Registry Security
  • NiFi Security Summary