Built with Tutify.io

    AIStudyLabs

    Spark | Hadoop - Exception in thread “main” java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z

    Spark | Hadoop - Exception in thread “main” java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z

    When you run the Spark program on windows OS, you often get the exception "Exception in thread “main” java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z"

    Full-stack trace of the error

    
Exception in thread “main” java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z

    Solution: In order to run the Spark program on windows, you would need Hadoop winutils.exe file as windows don't support HDFS and winutils provides a wrapper.

    If you don't have winutils.exe installed, please download the wintils.exe and hadoop.dll files from https://github.com/steveloughran/winutils (select the Hadoop version you are using as winutils are specific to Hadoop versions)

    Copy the downloaded files to a folder on your system, for example, let's say you have copied these files to c:/hadoop/bin, set the below environment variables.

    
set HADOOP_HOME=c:/hadoop
set PATH=%PATH%;%HADOOP_HOME%/bin;

    Close and reload the command line or terminal to initialize these variables. Sometimes you may also need to put hadoop.dll file into the C:/Windows/System32 folder.

    Now run your spark program and issue "Windows.access0(Ljava/lang/String;I)Z" should disappear.

    However, if it still doesn't work, try to restart your system as some of the above settings would get the effect with the restart.

    Happy Learning !!

    <a id="h-related-articles"></a>

    Related Articles