admin has written 55 articles

Installing Hadoop

Here we set up and configure a single-node Hadoop installation so that you can quickly perform simple operations using Hadoop MapReduce and the Hadoop Distributed File System (HDFS). Installation is easy on Linux platforms than on Windows. Windows requires building Hadoop from source or using a pre-compiled binary for windows which is available here. Version…

Using CURL to test HTTP Requests

$ curl -i -X POST -H “Content-Type:application/json”-d ‘{ “firstName” : “Frodo”, “lastName” : “Baggins” }’ http://localhost:8080/people HTTP/1.1201CreatedServer:Apache-Coyote/1.1Location: http://localhost:8080/people/1Content-Length:0Date:Wed,26Feb201620:26:55 GMT -i ensures you can see the response message including the headers. The URI of the newly created Person is shown -X POST signals this a POST used to create a new entry -H “Content-Type:application/json” sets the…

Spark Machine Learning Example

Spark Machine Learning Application Machine Learning application using classification technique, specifically collaborative filtering method, to predict the movies to recommend to a user based on other users’ ratings on different movies. Our recommendation engine solution will use Alternating Least Squares (ALS) machine learning algorithm. Even though the data sets used in the code example in…

Spark Streaming Example

Spark Streaming Application This example illustrates a web server log analytics use case to show how Spark Streaming can help with running analytics on data streams that are generated in a continuous manner. These log messages are considered time series data, which is defined as a sequence of data points consisting of successive measurements captured…

Spark SQL Example

Spark SQL Application Once you have Spark Shell launched, you can run the data analytics queries using Spark SQL API. In the first example, we’ll load the customer data from a text file and create a DataFrame object from the dataset. Then we can run DataFrame functions as specific queries to select the data. Let’s…