subject

Big Data Integration and Processing

 with  Ilkay Altintas and Amarnath Gupta
At the end of the course, you will be able to:

*Retrieve data from example database and big data management systems
*Describe the connections between data management operations and the big data processing patterns needed to utilize them in large-scale analytical applications
*Identify when a big data problem needs data integration
*Execute simple big data integration and processing on Hadoop and Spark platforms

This course is for those new to data science. Completion of Intro to Big Data is recommended. No prior programming experience is needed, although the ability to install applications and utilize a virtual machine is necessary to complete the hands-on assignments. Refer to the specialization technical requirements for complete hardware and software specifications.

Hardware Requirements:
(A) Quad Core Processor (VT-x or AMD-V support recommended), 64-bit; (B) 8 GB RAM; (C) 20 GB disk free. How to find your hardware information: (Windows): Open System by clicking the Start button, right-clicking Computer, and then clicking Properties; (Mac): Open Overview by clicking on the Apple menu and clicking “About This Mac.” Most computers with 8 GB RAM purchased in the last 3 years will meet the minimum requirements.You will need a high speed internet connection because you will be downloading files up to 4 Gb in size.

Software Requirements:
This course relies on several open-source software tools, including Apache Hadoop. All required software can be downloaded and installed free of charge (except for data charges from your internet provider). Software requirements include: Windows 7+, Mac OS X 10.10+, Ubuntu 14.04+ or CentOS 6+ VirtualBox 5+.

Syllabus

Welcome to Big Data Integration and Processing
Welcome to the third course in the Big Data Specialization. This week you will be introduced to basic concepts in big data integration and processing. You will be guided through installing the Cloudera VM, downloading the data sets to be used for this course, and learning how to run the Jupyter server.

Retrieving Big Data (Part 1)
This module covers the various aspects of data retrieval and relational querying. You will also be introduced to the Postgres database.

Retrieving Big Data (Part 2)
This module covers the various aspects of data retrieval for NoSQL data, as well as data aggregation and working with data frames. You will be introduced to MongoDB and Aerospike, and you will learn how to use Pandas to retrieve data from them.

Big Data Integration
In this module you will be introduced to data integration tools including Splunk and Datameer, and you will gain some practical insight into how information integration processes are carried out.

Processing Big Data
This module introduces Learners to big data pipelines and workflows as well as processing and analysis of big data using Apache Spark.

Big Data Analytics using Spark
In this module, you will go deeper into big data processing by learning the inner workings of the Spark Core. You will be introduced to two key tools in the Spark toolkit: Spark MLlib and GraphX.

Learn By Doing: Putting MongoDB and Spark to Work
In this module you will get some practical hands-on experience applying what you learned about Spark and MongoDB to analyze Twitter data.

2 Student
reviews
Cost Free Online Course
Provider Coursera
Language English
Certificates Paid Certificate Available
Calendar 6 weeks long
+ Add to My Courses
Learn Data Analysis udacity.com

Learn to become a Data Analyst. Job offer guaranteed or get a full refund.

Advertisement
75+ Hour Free Coding Course flatironschool.com

Get started with Ruby & JS curriculum online with all-day instructor help.

Advertisement
FAQ View All
What are MOOCs?
MOOCs stand for Massive Open Online Courses. These are free online courses from universities around the world (eg. Stanford Harvard MIT) offered to anyone with an internet connection.
How do I register?
To register for a course, click on "Go to Class" button on the course page. This will take you to the providers website where you can register for the course.
How do these MOOCs or free online courses work?
MOOCs are designed for an online audience, teaching primarily through short (5-20 min.) pre recorded video lectures, that you watch on weekly schedule when convenient for you.  They also have student discussion forums, homework/assignments, and online quizzes or exams.

2 reviews

Write a review
1 out of 1 people found the following review useful
7 months ago
Ericdo1810 partially completed this course, spending 12 hours a week on it and found the course difficulty to be medium.
I have beta tested this course. As a person who audited the previous versions of this series, I can compare and contrast. My rating: this is probably the best course in both the new series and the old series. My background? As a data analyst (well, some calls me data scientist) working alongside a big data team, I am Read More
I have beta tested this course. As a person who audited the previous versions of this series, I can compare and contrast.

My rating: this is probably the best course in both the new series and the old series. My background? As a data analyst (well, some calls me data scientist) working alongside a big data team, I am familiar with most of the Big data technologies.

When I beta test this course, I was expecting to see only a few technologies covered: for instance, Spark or SQL. To my amazement, this course covers a lot more technologies. The technologies are those that I've heard of, and awesome, there are also several other technologies that are new to me.

Not only it covers the technologies, but it also covers their different uses. For instance, for Spark, the course covers its use in Database, Streaming, Machine learning and Graphing. Wow! Just wow!

Besides the very rigorous coverage of technologies and their underlying concepts, the course also provides hands-on programming assignments, with thorough instructions and gentle difficulty, so that learners can get hands-on with most of the technologies covered. I like the Splunk hands-on and the PySpark hands-on the most. The assignments of those two are classic dashboards and word-counts, applied in the big data context. The programming part is not hard, however, learning and getting accustomed to the interface is a very valuable experience (my big data department uses Splunk by the way, hence, it is very valuable that they teaches Splunk in great details). There are other programming assignments, but I haven't tried them yet as I need to finish this review before going to work. That said, the rigor and thoroughness of this course exceeds my expectation leaps and bounds.

Really, this is the best course in the old and new series. I highly recommend this. Forget about your bad experience with the old series. This course is the new face of this new series.

Was this review helpful to you? YES | NO
1 out of 1 people found the following review useful
6 months ago
Y. Nicodeme completed this course.
Was this review helpful to you? YES | NO

Write a review

How would you rate this course? *
How much of the course did you finish? *
Review
Create Review