-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Description
Search before asking
- I searched in the issues and found nothing similar.
Paimon version
Manifest compression defaults to zstd, but the avro format still attempts to load snappy.
in the class `org.apache.avro.file.CodecFactory` we find :
static {
addCodec("null", nullCodec());
addCodec("deflate", deflateCodec(-1));
addCodec("bzip2", bzip2Codec());
addCodec("xz", xzCodec(6));
addCodec("zstandard", zstandardCodec(3, false));
addCodec("snappy", snappyCodec());
}
Could not load 'org-xerial-snappy.properties' from classpath: java.lang.NullPointerException
java.lang.NullPointerException
at org.xerial.snappy.SnappyLoader.getRootClassLoader(SnappyLoader.java:147)
at org.xerial.snappy.SnappyLoader.injectSnappyNativeLoader(SnappyLoader.java:256)
at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:227)
at org.xerial.snappy.Snappy.(Snappy.java:48)
at org.apache.paimon.shade.org.apache.avro.file.SnappyCodec$Option.(SnappyCodec.java:35)
at org.apache.paimon.shade.org.apache.avro.file.CodecFactory.snappyCodec(CodecFactory.java:69)
at org.apache.paimon.shade.org.apache.avro.file.CodecFactory.(CodecFactory.java:140)
at org.apache.paimon.shade.org.apache.avro.file.DataFileStream.resolveCodec(DataFileStream.java:158)
at org.apache.paimon.shade.org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:144)
at org.apache.paimon.shade.org.apache.avro.file.DataFileReader.(DataFileReader.java:143)
at org.apache.paimon.shade.org.apache.avro.file.DataFileReader.(DataFileReader.java:134)
at org.apache.paimon.shade.org.apache.avro.file.DataFileReader.openReader(DataFileReader.java:74)
at org.apache.paimon.format.avro.AvroBulkFormat$AvroReader.createReaderFromPath(AvroBulkFormat.java:84)
at org.apache.paimon.format.avro.AvroBulkFormat$AvroReader.(AvroBulkFormat.java:70)
at org.apache.paimon.format.avro.AvroBulkFormat$AvroReader.(AvroBulkFormat.java:57)
at org.apache.paimon.format.avro.AvroBulkFormat.createReader(AvroBulkFormat.java:54)
at org.apache.paimon.utils.FileUtils.createFormatReader(FileUtils.java:130)
at org.apache.paimon.utils.ObjectsFile.createIterator(ObjectsFile.java:180)
at org.apache.paimon.utils.ObjectsCache.readSegments(ObjectsCache.java:129)
at org.apache.paimon.utils.ObjectsCache.read(ObjectsCache.java:93)
at org.apache.paimon.utils.ObjectsFile.readWithIOException(ObjectsFile.java:149)
at org.apache.paimon.utils.ObjectsFile.read(ObjectsFile.java:134)
at org.apache.paimon.utils.ObjectsFile.read(ObjectsFile.java:105)
at org.apache.paimon.utils.ObjectsFile.read(ObjectsFile.java:101)
at org.apache.paimon.manifest.ManifestList.readDataManifests(ManifestList.java:86)
at org.apache.paimon.operation.ManifestsReader.readManifests(ManifestsReader.java:99)
at org.apache.paimon.operation.ManifestsReader.read(ManifestsReader.java:85)
at org.apache.paimon.operation.AbstractFileStoreScan.readManifests(AbstractFileStoreScan.java:393)
at org.apache.paimon.operation.AbstractFileStoreScan.plan(AbstractFileStoreScan.java:233)
at org.apache.paimon.table.source.snapshot.SnapshotReaderImpl.read(SnapshotReaderImpl.java:296)
Compute Engine
master
Minimal reproduce step
none
What doesn't meet your expectations?
none
Anything else?
No response
Are you willing to submit a PR?
- I'm willing to submit a PR!