IT Versity

making IT resourceful

  • Courses
  • Blog
  • Labs
  • Discuss

January 28, 2018 By Training Itversity Leave a Comment

Spark SQL – Standard Transformations

Topic Progress:
← Back to Lesson

Row level transformations

Joining data from multiple tables

Aggregations

Sorting data

Set Operations

Related

← Previous Topic

Filed Under: Apache Spark, Scala

Start the discussion at discuss.itversity.com

Socially Connected

  • Facebook
  • Google+
  • Linkedin
  • Twitter
  • YouTube
Getting Started
  • Setup Scala and IDE
  • Preview of itversity platforms
  • Ambari and MySQL
  • Overview of HDFS
  • Getting started with Spark
  • Review of Sqoop and Hive
Scala Fundamentals for Spark
  • Getting Started with Scala
  • Basic programming constructs
  • Object Oriented Programming
  • Collections and Map Reduce
  • I/O Operations and Tuples
  • Development Life Cycle - sbt and scala
  • Application Development using IntelliJ
Data Ingestion - Apache Sqoop
  • Validating MySQL and Environment
  • Querying using list and eval commands
  • Sqoop Import - Simple import and execution life cycle
  • Sqoop Import - Customizing split logic
  • Sqoop Import - File Formats and Compression
  • Sqoop Import - Customizing filtering of data
  • Sqoop Import - Delimiters and handling nulls
  • Sqoop Import - Incremental loads
  • Sqoop Import - Hive Import
  • Sqoop Import - Import all tables
  • Sqoop - Typical life cycle
  • Sqoop Export - Simple Export
  • Sqoop Export - Upsert/merge
Core Spark using Scala
  • Getting Started
  • Creating RDD from HDFS
  • Transformations Overview
  • Row level transformations
  • Filtering the data
  • Joining data sets
  • Performing aggregations
  • Global Sorting and Ranking
  • By Key Ranking
  • Set Operations
  • Save data to HDFS
Data Frames and Spark SQL
  • Create tables and loading data
  • Spark SQL - Functions
  • Spark SQL - Standard Transformations
  • Spark SQL - Analytics and Windowing Functions
  • Spark SQL - Processing data using Data Frames
Streaming Analytics - using Flume and Kafka
  • Flume - Getting Started
  • Flume - Web Server logs to HDFS
  • Kafka - Getting Started
  • Spark Streaming - Getting Started
  • Spark Streaming - Another Example
  • Flume and Spark Streaming Integration - Example
  • Flume and Kafka Integration - Example
  • Kafka and Spark Streaming Integration - Example
Tips and Evaluation
Return to CCA Spark and Hadoop Developer - Scala

Copyright © 2018 · Education Pro Theme On Genesis Framework · WordPress · Log in