Versions Compared

Old Version 27

changes.mady.by.user Brock Noland

Saved on

compared with

New Version 28

changes.mady.by.user Brock Noland

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

IssueCauseResolution

java.lang.NoSuchMethodError: com.google.common.hash.HashFunction.hashInt

(I)Lcom/google/common/hash/HashCode

Guava library version conflict between Spark and Hadoop.  See HIVE-7387 and SPARK-2420 for details.

Alternatives until this is fixed:

  1. Remove Guava 11 from HADOOP_HOME and replace it with Guava 14.
  2. Choose to build Spark assembly manually, apply HIVE-7387-spark.patch to Spark branch before building.

org.apache.spark.SparkException: Job aborted due to stage failure:

Task 5.0:0 had a not serializable result: java.io.NotSerializableException: org.apache.hadoop.io.BytesWritable

Spark serializer not set to Kryo.Set spark.serializer to be org.apache.spark.serializer.KryoSerializer as described above.

java.lang.NullPointerException

at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:257)

at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:224)

Hive is included in the Spark Assembly.Either build a version of Spark without the "hive" profile or unjar the Spark assembly and rm -rf org/apache/hive org/apache/hadoop/hive and then rejar. The fix is in SPARK-2741.

[ERROR] Terminal initialization failed; falling back to unsupported
java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected

Hive has upgraded to Jline2 but jline 0.94 exists in the Hadoop lib
  1. Delete jline from the hadoop lib directory (it's only pulled in transitively from zk)
  2. export HADOOP_USER_CLASSPATH_FIRST=true