Apache Spark is an open-source powerful distributed querying and processing engine. It provides flexibility and extensibility of MapReduce but at significantly higher speeds: Up to 100 times faster than Apache Hadoop when data is stored in memory and up to 10 times when accessing disk. Apache Spark allows the user to read, transform, and aggregate data, as well as train and deploy sophisticated statistical models with ease. The Spark APIs are accessible in Java, Scala, Python, R and SQL. Apache Spark can be used to build applications or package them up as libraries to be deployed on a cluster or perform quick analytics interactively through notebooks (like, for instance, Jupyter, Spark-Notebook, Databricks notebooks, and Apache Zeppelin).
Learn about Apache Spark and the Spark 2.0 and PySpark architecture
• Build and interact with PySpark DataFrames
• Read, transform, and understand data and use it to train machine learning models
• Build machine learning models with MLlib and ML
• Learn how to submit your applications programmatically using spark-submit
• ML Algorithms: common learning algorithms such as classification, regression, clustering, and collaborative filtering
• Features : feature extraction, transformation, dimensionality reduction, and selection
• Pipelines: tools for constructing, evaluating, and tuning ML Pipelines
• Persistence: saving and load algorithms, models, and Pipelines
• Utilities: linear algebra, statistics, data handling, etc.
Before proceeding with the various concepts given in this tutorial, it is being assumed that the readers are already aware about what a programming language and a framework is. In addition to this, it will be very helpful, if the readers have a sound knowledge of Apache Spark, Apache Hadoop, Scala Programming Language, Hadoop Distributed File System (HDFS) and Python.
Hardware : Intel Core 5 processor with 16GB Recommended RAM. OS : Ubuntu Server ( Latest Version ) or Cent OS or Mac OS or Windows 64 bit 7/8/10 ( Latest preferable version ) High Speed Internet Connection ( Open Port for Installations ) Software Prerequisites Java ( Latest Version ) , Scala ( Latest Version) Apache Spark [ Latest Version ] (Downloadable from http://spark.apache.org/downloads.html) A Python distribution containing IPython, Pandas and Scikit-learn Anaconda with Python3.6, PySpark Local Environment www.anaconda.com [ Local Machine ] Hadoop, PySpark PySpark on Hadoop Cloud Environment OR Cloudera Hadoop or Online Databriks Cloud
Course Outline
169
Kxc1oiFd9bcbVcsDe0CJgnTQEa8gMRW8ujivJExsKTe6AWDcthSGDNqPDcjdGhpp
{{ $index + 1 }}. {{ each.name }}
