This blog covers step by step instructions to setup single node lab for Hadoop and Spark eco system using Cloudera Distribution. Almost all the leading vendors provide virtual machine image to learn all the tools in Big Data eco system including but not limited to Hadoop and Spark. But virtual machine requires higher configuration such […]

Aggregations can be broadly categorized into totals and by key. Spark brings power of in memory as well as rich APIs to aggregate data sets. In this blog I will discussing about implementation of aggregations using different actions and transformations. Pre-requisites Spark Introduction Aggregations – totals Aggregations – by key Difference between groupByKey, reduceByKey, aggregateByKey […]

Introduction As part of this getting started or 101 course we will see how Java developers can start learning MongoDB. Here is the course link. Course topics Following are the topics covered: Setup MongoDB MongoDB data architecture Mongo Shell CRUD operations Mongo chef Data Model Setup development environment Develop simple java application Perform CRUD operations […]

There are several Hadoop and Spark Developer Certifications available today to test developer skills in Hadoop eco system and Spark. Before getting into details of certifications, first we need to understand different technologies available in Hadoop and Spark eco system. What is Hadoop and Spark? Hadoop has two primary components HDFS and Map Reduce. HDFS, […]

Apache Hadoop is an open-source software framework for distributed storage and distributed processing of very large data sets on computer clusters built from commodity hardware. All the modules in Hadoop are designed with a fundamental assumption that hardware failures are common and should be automatically handled by the framework. Typically Hadoop cluster can have few […]