Apache Spark - Could not locate executable null\bin\winutils.exe in the Hadoop binaries

I am passionate about BigData technologies. I am exploring the Apache Spark these days. I found IBM's BigDataUniversity the best place to learn the trending BigData technologies. I wrote my first Apache Spark application but when I ran it, I encountered the following error

Could not locate executable null\bin\winutils.exe

in the Hadoop binaries

I searched for it over the internet and found variously detailed solutions. In this post, I am listing down the steps in an easy way to explain how to fix this issue.

Step 1

Download the winutils.exe from the following link

winutils 32-bit (depends on your installed Windows version)

winutils 64-bit (depends on your installed Windows version)

Step 2
Create the following two directories in C or any other drive

c:\hadoop\bin
C:\tmp\hive

Step 3

Place the downloaded winutils.exe inside the c:\hadoop\bin folder

Step 4
Set the following environmental variable

Name : HADOOP_HOME

Value : C:\HADOOP

Step 5

Add the %HADOOP_HOME%\bin to the PATH variable

Step 6

Open the command prompt as administrator and run the following commands

>winutils.exe chmod -R 777 c:\tmp\hive

Cheers, you are done.

Comments

Eclipse - Server Tomcat v8.5 Server at localhost failed to start.

When I try to launch the tomcat from Eclipse, I encountered the following error Server Tomcat v8.5 Server at localhost failed to start. Solution Step 1 Delete the .snap file located at the following location eclipse workspace Path\ .metadata\.plugins\org.eclipse.core.resources Step 2 Delete the tmp0 folder from the following path eclipse workspace Path \.metadata\.plugins\org.eclipse.wst.server.core Step 3 Delete the server from servers list Step 4 Remove already added Tomcat Server i) Click on Define a new Server ii) Select Server Runtime Environments iii) Select the Tomcat Server and remove it as follows Remove Selected Server Step 5 Make sure that correct version of Server is configured in Project Properties Step 6 Restart the Eclipse IDE.

Intellij : How to add @author comment to every new class

Introduction In this tutorial, we will learn how to add @author comments to every new class that we create. We can achieve it using either of the following two solutions Solution 1: Automatically add @author comments to every new class using Files and Code Templates Open File -> Settings -> Editor -> File and Code Templates -> Includes Click on Includes . Under File Header , enter the following comments text /** * @author ${USER} * @Date ${DATE} */ Intellij - add @author comments Solution 2: Autocompletion of @author Open File -> Settings -> Editor -> Live Templates Select Java and then click on + button In Abbreviation, enter @a In template text , enter the following comments /** * @author ${USER} * @Date ${DATE} */ In option , Expands with select SPACE Intellij - Autocompletion @author You can simply add the @author comments by typing @a and then click SPACE

hibernate-release-5.4.4.Final - Required Jars

Introduction Hibernate (Object Relational Mapping framework) is an implementation of Java Persistence API (JPA) specification. Required Jars for Hibernate 5.4.4 Following Jars resided inside the required folder are the mandatory jars required for Hibernate 5.4.4 antlr-2.7.7.jar byte-buddy-1.9.11.jar classmate-1.3.4.jar dom4j-2.1.1.jar FastInfoset-1.2.15.jar hibernate-commons-annotations-5.1.0.Final.jar hibernate-core-5.4.4.Final.jar istack-commons-runtime-3.0.7.jar jandex-2.0.5.Final.jar javassist-3.24.0-GA.jar javax.activation-api-1.2.0.jar javax.persistence-api-2.2.jar jaxb-api-2.3.1.jar jaxb-runtime-2.3.1.jar jboss-logging-3.3.2.Final.jar jboss-transaction-api_1.2_spec-1.1.1.Final.jar stax-ex-1.8.jar txw2-2.3.1.jar Hibernate 5.4.4 release is compatible with Java 8 or 11 JPA 2.2 References https://hibernate.org/orm/releases/5.4/

Techie Shah

Search This Blog