Hadoop for System Administrators Training

Duration:

3 Days

Audience:

Employees of federal, state and local governments; and businesses working with the government.

Qualified participants should be comfortable with the Linux commands and have some systems administration experience, but do not need previous Hadoop experience

Course Overview:

This course covers the essentials of deploying and managing an Apache™ Hadoop® cluster. The course is lab intensive with each participant creating their own Hadoop cluster using either the CDH (Cloudera’s Distribution, including Apache Hadoop) or Hortonworks Data Platform stacks. Core Hadoop services are explored in depth with emphasis on troubleshooting and recovering from common cluster failures. The fundamentals of related services such as Ambari, Zookeeper, Pig, Hive, HBase, Sqoop, Flume, and Oozie are also covered. The course is approximately 60% lecture and 40% labs.

Supported Distributions:

Red Hat Enterprise Linux 7

Course Outline:

HADOOP OVERVIEW

  • Data Analysis
  • Big Data
  • Origins of Hadoop
  • Hadoop Marketplace
  • Hadoop Core
  • Hadoop Ecosystem
  • Cluster Architecture
  • Hardware/Software Requirements

Labs:

  • Running Commands on Multiple Systems
  • Running Commands on Multiple Hosts
  • Preparing to Install Hadoop

HDFS

  • Design Goals
  • Design
  • Blocks
  • Block Replication
  • Namenode Daemon
  • Secondary Namenode Daemon
  • Datanode Daemon
  • Accessing HDFS
  • Permissions and Users
  • Adding and Removing Datanodes
  • Balancing

Labs:

  • Single Node HDFS
  • Multi-node HDFS
  • Files and HDFS
  • Managing and Maintaining HDFS

YARN

  • YARN Design Goals
  • YARN Architecture
  • Resource Manager
  • Node Manager
  • Containers
  • YARN: Other Important Features
  • Slider

Labs:

  • YARN

MAPREDUCE

  • MapReduce
  • Terminology and Data Flow

Labs:

  • Mapreduce

INSTALLING HADOOP WITH AMBARI LAB TASKS

  • CDH Uninstall
  • Installing Hadoop with Ambari
  • Tez

DATA INGESTION

  • Sqoop
  • Flume
  • Kafka

Labs:

  • Sqoop

DATA LINEAGE AND GOVERNANCE

  • Falcon
  • Atlas
  • Oozie

DATA PROCESSING FRAMEWORKS

  • The Bane of MapReduce
  • Tez overview
  • Pig
  • Hive
  • Spark
  • Storm
  • Solr

Labs:

  • Pig

NOSQL IMPLEMENTATIONS

  • HBase
  • Phoenix

CLUSTER MANAGEMENT

  • Ambari Metrics System (AMS)
  • Zookeeper