基于BiSheng JDK 1.8.0.262执行Spark,偶现JVM coredump问题的解决方法
问题现象描述
配置LD_PRELOAD环境变量的情况下,基于BiSheng JDK 1.8.0.262执行Spark,偶现JVM coredump,报错堆栈如下。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
Stack: [0x0000ffd2ce0d0000, 0x0000ffd2ce2d00001, sp=0x0000ffd2ce2cb830, free space=2030k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) C [1ibzip.so+0x52a4] fi11_window+0x164 C [1ibzip.so+0x6460] deflate_slow+0x1b0 C [1ibzip.so+0x781c] deflate+0x1a4 C [1ibzip.so+0x3480] Java_java_util_zip_Deflater_deflateBytes+0x280 J 3376 java.util.zip.Deflater.deflateBytes (J[BIII)I (0 bytes) @ 0x0000fffe537ded84 [0x0000fffe537ded00+0x84] V ~Runtimestub::_register_finalizer Java J 12020 C2 org.apache.hadoop.io.WritableUtils.writeCompressedstringArray (Ljava/io/Dataoutput; [Ljava/lang/String;)V (43 bytes) @ 0x0000fffe55773a38 [0x0000fffe55773380+0x6b8] j org.apache.hadoop.conf.Configuration.write (Ljava/io/DataOutput;) V+104 j org.apache.spark.util.SerializableConfiguration.$anonfun$writeObject$1 (Lorg/apache/spark/uti1/SerializableConfiguration; Ljava/io/ObjectOutputStream; ) V+9 j org.apache.spark.util.SerializableConfiguration$$Lambda$2298.apply$mcv$sp()V+8 J 14862 C2 scala.runtime.java8.JFunction0$mcV$sp.apply()Ljava/lang/Object; (10 bytes) @ 0x0000fffe55e08b34 [0x0000fffe55e08b00+0x34] J 14673 C2 org.apache.spark.util.Utils$.tryOrIOException(Lscala/Function0;)Ljava/lang/Object; (100 bytes) @ 0x0000fffe55d5ddfc [0x0000fffe55d5ddc0+0x3c] j org.apache.spark.util.SerializableConfiguration.write0bject(Ljava/io/ObjectOutputStream;) V+10 V ~StubRoutines::ca1l_stub V [libjvm.so+0x6f20f4] JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCal1Arguments*, Thread*)+0xe54 V [libjvm.so+0xa82eac] Reflection::invoke(instanceKlassHandle, methodHandle, Handle, bool, objArrayHandle, BasicType, objArrayHandle, bool, Thread*) +0xaf4 V [1ibjvm.so+0xa8456c] Reflection::invoke_method(oopDesc*, Handle, objArrayHandle, Thread*)+0x144 V [1ibjvm.so+0x7b416c] JVM_InvokeMethod+0x11c J 5318 sun.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method; Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (0 bytes) @ 0x0000fffe5436d1a8 [0x0000fffe5436d100+0xa8] J 14063 C1 net.jpountz.lz4.LZ4BlockOutputStream.write([BII)V (106 bytes) @ 0x0000fffe55a967e0 [0x0000fffe55a96280+0x560] C 0x00000007822fef70 |
关键过程、根本原因分析
这是BiSheng JDK 1.8.0.262的一个Bug。如果JVM使用java.util.zip.ZipFile打开文件,并且该文件在打开时在磁盘上更改,JVM可能会崩溃。
结论、解决方案及效果
BiSheng JDK不具备向前兼容性,通过升级BiSheng JDK版本至推荐版本1.8.0.342可规避该问题。
父主题: OmniOperator算子加速