Skip to main content

spark-submit java.lang.NoClassDefFoundError: scala/runtime/java8/JFunction1$mcII$sp

Image
By Asif Shah

Exception 


Exception in thread "main" java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: scala/runtime/java8/JFunction1$mcII$sp
        at SparkPi$.main(SparkPi.scala:14)
        at SparkPi.main(SparkPi.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)


Problem:


It seems that you have compiled and generated your jar file with an upper version of the Scala compiler than the one spark is using.

Solution

Step 1

Run the following command from spark-installed-directory\bin

spark-shell.cmd (or.sh)

and note the Scala version, My Spark version was 2.4.3 and Scala version 2.11.12

Step 2

Change the scala version into your build.sbt to 2.11.12 (per your configuration).

My build.sbt is

name := "SparkPi Project"
version := "1.0"
scalaVersion := "2.11.12"
val sparkVersion = "2.4.5"

libraryDependencies += "org.apache.spark" %% "spark-core" % sparkVersion % "provided"

Now when you submit you jar to Spark, it will run without errors.

Hope you find this post helpful.

Labels:

Comments

  1. AnonymousOctober 18, 2026 at 10:23 PM

    Scala version compatibility is an important consideration when deploying Apache Spark applications. The NoClassDefFoundError involving scala/runtime/java8/JFunction1$mcII$sp can occur when an application is compiled against a different Scala version than the one used by the Spark runtime. Checking the Scala version reported by spark-shell and aligning the project's Scala compiler and dependencies with the installed Spark distribution is a good first troubleshooting step.

    For developers interested in learning distributed data processing with Python, PySpark Online Training in India is a relevant learning resource. Understanding Spark's runtime environment, dependencies, and execution model helps developers troubleshoot compatibility issues more effectively.

    ReplyDelete
  2. AnonymousOctober 18, 2026 at 10:23 PM

    Additionally, Data Engineering Online Training in India can help learners develop a broader understanding of data processing frameworks and pipeline architecture. Building a solid foundation in these areas is valuable when developing, deploying, and maintaining reliable big data applications.

    ReplyDelete

Post a Comment

Popular posts from this blog

Eclipse - Server Tomcat v8.5 Server at localhost failed to start.

By Asif Shah
When I try to launch the tomcat from Eclipse, I encountered the following error Server Tomcat v8.5 Server at localhost failed to start. Solution Step 1  Delete the .snap file located at the following location     eclipse workspace Path\ .metadata\.plugins\org.eclipse.core.resources Step 2 Delete the  tmp0  folder from the following path      eclipse workspace Path \.metadata\.plugins\org.eclipse.wst.server.core Step 3  Delete the server from servers list Step 4  Remove already added Tomcat Server      i)  Click on Define a new Server     ii)  Select Server Runtime Environments     iii) Select the Tomcat Server and remove it as follows Remove Selected Server Step 5 Make sure that correct version of Server is configured in Project Properties Step 6 Restart the Eclipse IDE.
223 comments
Read more

hibernate-release-5.4.4.Final - Required Jars

By Asif Shah
Introduction Hibernate (Object Relational Mapping framework) is an implementation of Java Persistence API (JPA) specification.   Required Jars for Hibernate 5.4.4 Following Jars resided inside the required folder are the mandatory jars required for Hibernate 5.4.4 antlr-2.7.7.jar byte-buddy-1.9.11.jar classmate-1.3.4.jar dom4j-2.1.1.jar FastInfoset-1.2.15.jar hibernate-commons-annotations-5.1.0.Final.jar hibernate-core-5.4.4.Final.jar istack-commons-runtime-3.0.7.jar jandex-2.0.5.Final.jar javassist-3.24.0-GA.jar javax.activation-api-1.2.0.jar javax.persistence-api-2.2.jar jaxb-api-2.3.1.jar jaxb-runtime-2.3.1.jar jboss-logging-3.3.2.Final.jar jboss-transaction-api_1.2_spec-1.1.1.Final.jar stax-ex-1.8.jar txw2-2.3.1.jar Hibernate 5.4.4 release is compatible with  Java 8 or 11  JPA 2.2 References https://hibernate.org/orm/releases/5.4/
Post a Comment
Read more

Intellij : How to add @author comment to every new class

By Asif Shah
 Introduction In this tutorial, we will learn how to add @author comments to every new class that we create. We can achieve it using either of the following two solutions Solution 1:  Automatically add @author comments to every new class using Files and Code Templates Open File -> Settings -> Editor -> File and Code Templates -> Includes Click on Includes . Under File Header , enter the following comments text /**  * @author ${USER}  * @Date ${DATE}   */ Intellij - add @author comments Solution 2: Autocompletion of @author Open File  ->  Settings  ->  Editor  -> Live Templates Select Java and then click on + button In Abbreviation, enter @a In template text , enter the following comments           /**             * @author ${USER}             * @Date ${DATE}            */ In o...
3 comments
Read more