Become an expert in Hadoop by getting hands-on knowledge on MapReduce, Hadoop Architecture, Pig & Hive, Oozie, Flume and Apache workflow scheduler. Build familiarity with HBase, Zookeeper, and Sqoop concepts while working on industry-based use-cases and projects.
Why get Big Data Hadoop Developer Certification from Cognixia?
As job opportunities multiply for IT professionals in Big Data & Hadoop, career opportunities for industry-savvy professionals are everywhere. According to a recent study, by 2020, the Big Data & Hadoop market is estimated to grow at a compound annual growth rate (CAGR) of 58%, surpassing $16 billion.
Cognixia’s Big Data Hadoop Developer certification course highlights the key ideas and proficiency for managing Big Data with Apache’s open source platform – Hadoop. Not only does it impart in-depth knowledge on core ideas through the course, it also facilitates executing it through a variety of hands-on applications. Through this course, IT experts working in organizations of all sizes can learn to code within the MapReduce framework. The course also covers advanced modules like Yarn, Zookeeper, Oozie, Flume and Sqoop.
By the end of this course, participants will have the leadership toolkit to shape and steer GenAI portfolios across their organization.
- To date, approximately 20% of all data is in structured form. The limitation of RDBMS is that we can isolate and store only structured data. Hadoop, however, enables us to store or process all data – structured or unstructured.
- E-commerce websites have had a tremendous impact on local economies across the globe. As part of their operation, e-commerce websites maintain a detailed record of user-activity, storing it as clickstream. This activity is used to analyze the browsing patterns of a particular user, thus helping e-commerce technology to recommend products with high accuracy, during current and future visits. This also helps e-commerce marketers, as well as their technology platforms, to design personalized promotional emails for its users.
Strategic Application
Introduction/ Installation of Virtual Box and the Big Data VM, Introduction to Linux, Why Linux?, Windows and the Linux equivalents, Different flavors of Linux, Unity Shell (Ubuntu UI), Basic Linux Commands (enough to get started with Hadoop)
- 3V (Volume- Variety- Velocity) characteristics
- Structured and unstructured data
- Application and use cases of Big Data
- Limitations of traditional large scale systems
- How a distributed way of computing is superior (cost and scale)
- Opportunities and challenges with Big Data
- HDFS Overview and Architecture
- Deployment Architecture
- Name Node
- Data Node and Checkpoint Node (aka Secondary Name Node)
- Safe mode
- Configuration files
- HDFS Data Flows (Read/Write)
- CRC Checksum
- Data Replication
- Rack awareness and block placement policy
- Small file problems
- Command-Line Interface
- File Systems
- Administrative
- Web Interfaces
- Load Balancer
- Dist cp (Distributed Copy)
- HDFS Federation
- HDFS High Availability
- Hadoop Archives
- MapReduce overview
- Functional Programming paradigms
- How to think in a MapReduce way
- Legacy MR v/s Next Generation MapReduce (YARN/ MRv2)
- Slots v/s Containers
- Schedulers
- Shuffling, Sorting
- Hadoop Data Types
- Input and Output Formats
- Input Splits – Partitioning (Hash Partitioner v/s Customer Partitioner)
- Configuration files
- Distributed Cache
Word Count
- Adhoc Querying
- Graph Computing Engines
- Standalone mode (in Eclipse)
- Pseudo Distributed mode (as in the Big Data VM)
- Fully Distributed mode (as in Production)
- MR API
- Old and the New MR API
- Java Client API
- Hadoop data types
- Custom Writable
- Different input and output formats
- Saving Binary Data using Sequence Files and Avro Files
- Hadoop Streaming (developing and debugging non Java MR programs – Ruby and Python)
- Speculative execution
- Combiners
- JVM Reuse
- Compression
- Sorting
- Term Frequency
- Inverse Document Frequency
- Student Database
- Max Temperature
- Different ways of joining data
- Word Co-occurrence
- Click Stream Analysis using Pig and Hive
- Analyzing the Twitter data with Hive
- Further ideas for data analysis
- HBase Data Modeling
- Bulk loading data in HBase
- HBase Coprocessors – Endpoints (similar to Stored Procedures in RDBMS)
- HBase Coprocessors – Observers (similar to Triggers in RDBMS)
- PageRank
- Inverted Index
- Introduction and Architecture
- Different modes of executing Pig constructs
- Data Types
- Dynamic Invokers
- Pig streaming Macros
- Pig Latin language Constructs (LOAD, STORE, DUMP, SPLIT, etc)
- User-Defined Functions
- Use Cases
- NoSQL Databases – 1 (Theoretical Concepts)
- NoSQL Concepts
- Review of RDBMS
- Need for NoSQL
- Brewers CAP Theorem
- ACI D v/s BASE
- Schema on Read vs. Schema on Write
- Different levels of consistency
- Bloom filters
- Key Value
- Columnar, Document
- Graph
- HBase Architecture
- Master and the Region Server
- Catalog tables (ROOT and META)
- Major and Minor Compaction
- Configuration Files
- HBase v/s Cassandra
- Java API
- Client API
- Filters
- Scan Caching and Batching
- Command Line Interface
- REST API
- Use-case of Sqoop
- Sqoop Architecture
- Sqoop Demo
- Use-case of Flume
- Flume Architecture
- Flume Demo
- Use-case of Oozie
- Oozie Architecture
- Oozie Demo
- Usecase of YARN
- YARN Architecture
- YARN Demo
- Introduction to RDD
- Installation and Configuration of Spark
- Spark Architecture
- Different interfaces to Spark
- Sample Python programs in Spark
- Hadoop industry solutions
- Importing/exporting data across RDBMS and HDFS using Sqoop
- Getting real-time events into HDFS using Flume
- Creating workflows in Oozie
- Introduction to Graph processing
- Graph processing with Neo4J
- Using the Mongo Document Database
- Using the Cassandra Columnar Database
- Distributed Coordination with Zookeeper
- Stand alone mode (Theory)
- Distributed mode (Theory)
- Pseudo distributed
- Fully-distributed
- Cloudera Hadoop cluster on the Amazon Cloud (Practice)
- Using EMR (Elastic Map Reduce)
- Using EC2 (Elastic Compute Cloud)
hands-on learning model
Access to sanitized process maps, KPI definitions, candidate initiative lists, and basic cost baselines (time, cycle time, error or rework rates)
Speak with a Cognixia specialist about enrollment options, custom cohorts for your leadership team, or tailored delivery formats for your organization.
One of our specialists will contact you within one business day.
Asked Questions
Find details on duration, delivery formats, customization options, and post-program reinforcement.
Our instructors/trainers are Cloudera and Hortonworks-certified professionals. They have industry experience of more than 12 years and are Big Data SME’s.
To attend the live virtual training, at least 2 Mbps would be required.
Yes, Cognixia’s Virtual Machine can be installed on any local system. The training team of Collabera will assist you with this.
To install the Hadoop environment, you’ll need 8GB RAM, 64-bit OS, 50GB hard disk space free, and a Virtualization Technology-enabled processor.
The access to the Learning Management System (LMS) is lifelong. That includes class recordings, presentations, sample code, and projects.
If you miss a session, don’t worry! All our sessions are available in recorded form on the LMS. We also have a technical support team that’s ready to assist you with any questions you may have.
The online live training course runs for eight weekends (15-16 sessions)
of the future
Enroll your leadership cohort in Designing GenAI Use-Case Portfolios & Business Cases.
Custom cohorts available for enterprise teams.