Summary
The JVM crashes with a SIGABRT (assertion failure) in DeoptValueEncoding::decode() when the LLVM optimizer merges multiple uncommon_trap calls into a single llvm.experimental.gc.statepoint. The stackmap parser incorrectly interprets a bci value from a merged deopt group as a DeoptValueEncoding, causing an out-of-range basic_type that triggers the assertion.
Reproduction
./build-test.sh ./test/jdk/java/net/httpclient/ShortResponseBodyPost.java fastdebug
Build method:
Crash Details
Exit code: 134 (SIGABRT)
Assertion:
assert(basic_type >= 0 && basic_type <= BasicType::T_ILLEGAL) failed
Location: src/hotspot/share/jeandle/jeandleCompiledCode.hpp:81 in DeoptValueEncoding::decode()
Crashing method example: java.lang.StringLatin1.charAt
Root Cause
Background
When Jeandle compiles a Java method, it generates LLVM IR with uncommon_trap calls at deoptimization points (e.g., null check failures). Each uncommon_trap carries a "deopt" operand bundle encoding the JVM state needed for deoptimization:
; Null check #1 at bci=2
bci_2_null_check_fail:
call void @uncommon_trap(i32 -10)
[ "deopt"(i32 2, i64 12, ptr %0, i64 4294967306, i32 %1,
i64 65546, i32 %1, i64 4295032844, ptr %0,
i64 327695, ptr %OrigPcSlot) ]
unreachable
; Null check #2 at bci=8
bci_8_null_check_fail:
call void @uncommon_trap(i32 -10)
[ "deopt"(i32 8, i64 12, ptr %0, i64 4294967306, i32 %1,
i64 65548, ptr %0, i64 4295032842, i32 %1,
i64 327695, ptr %OrigPcSlot) ]
unreachable
Each deopt bundle begins with a bci (bytecode index) followed by DeoptValueEncoding-value pairs describing the local variable table, operand stack, and OrigPcSlot.
The Merge
The LLVM optimizer (specifically the RS4GC pass) may merge multiple uncommon_trap calls with the same deopt reason into a single llvm.experimental.gc.statepoint. When this happens, the deopt operands from both groups are concatenated into a single bundle:
call token @llvm.experimental.gc.statepoint.p0(
..., ptr @uncommon_trap, i32 1, ..., i32 -10, ...)
[ "deopt"(
; --- Group 1 (originally bci_2_null_check_fail) ---
i32 2, ; bci=2
i64 12, ptr null, ; encode + value
i64 4294967306, i32 %1, ; encode + value
...
i64 327695, ptr %OrigPcSlot, ; OrigPcSlot
; --- Group 2 (originally bci_8_null_check_fail) ---
i32 807, ; bci (from another merged deopt group)
i64 12, ptr %obj,
...
) ]
The LLVM stackmap binary format records these as a flat, contiguous sequence of locations with no delimiter between deopt groups.
The Parser Bug
The stackmap parser in JeandleCompiledCode::parse_stackmap() calculates num_deopts based on the total number of locations in the record, which includes all merged groups. It reads the first group correctly:
- Reads
bci = 2 ✓
- Reads
DeoptValueEncoding pairs correctly ✓
- Finishes the first group, but
num_deopts > 0 because the total count includes the second group
It then continues reading into the second group, interpreting the next value (807, a bci) as a DeoptValueEncoding:
encode = 807
basic_type = 807 & 0xffff = 807 ← far exceeds T_ILLEGAL (~15)
value_type = (807 >> 16) & 0xffff = 0
The assertion basic_type >= 0 && basic_type <= BasicType::T_ILLEGAL in DeoptValueEncoding::decode() fails, causing JVM abort.
The Fix
The fix adds a validation step before calling DeoptValueEncoding::decode(). If the raw basic_type or value_type fields are out of their valid ranges, the value cannot be a legitimate DeoptValueEncoding — it is likely a bci from a merged deopt group. In that case, the parser skips the remaining deopt operands and stops processing:
uint64_t encode = StackMapUtil::getConstantUlong(stackmaps, encode_location);
int basic_type_raw = (int)(encode & 0xffffUL);
int val_type_raw = (int)((encode & 0xffff0000UL) >> 16);
if (basic_type_raw < 0 || basic_type_raw > BasicType::T_ILLEGAL ||
val_type_raw < 0 || val_type_raw >= DeoptValueEncoding::LastType) {
// Not a valid encode value — likely a bci from a merged deopt group.
// Skip remaining deopt operands.
num_deopts--;
while (num_deopts > 0 && location != record->location_end()) {
location++;
num_deopts--;
}
break;
}
DeoptValueEncoding enc = DeoptValueEncoding::decode(encode);
Additionally, the bci reading was hardened to handle ConstantIndex locations (64-bit constant pool references) in addition to the Constant (32-bit small constant) kind, since LLVM may store the bci differently after optimization:
auto bci_location = *(location++);
int bci;
if (bci_location.getKind() == StackMapParser::LocationKind::Constant) {
bci = bci_location.getSmallConstant();
} else if (bci_location.getKind() == StackMapParser::LocationKind::ConstantIndex) {
bci = (int)stackmaps.getConstant(bci_location.getConstantIndex()).getValue();
} else {
bci = 0;
}
Affected Files
src/hotspot/share/jeandle/jeandleCompiledCode.cpp — parse_stackmap() method
Summary
The JVM crashes with a
SIGABRT(assertion failure) inDeoptValueEncoding::decode()when the LLVM optimizer merges multipleuncommon_trapcalls into a singlellvm.experimental.gc.statepoint. The stackmap parser incorrectly interprets abcivalue from a merged deopt group as aDeoptValueEncoding, causing an out-of-rangebasic_typethat triggers the assertion.Reproduction
Build method:
Crash Details
Exit code: 134 (SIGABRT)
Assertion:
Location:
src/hotspot/share/jeandle/jeandleCompiledCode.hpp:81inDeoptValueEncoding::decode()Crashing method example:
java.lang.StringLatin1.charAtRoot Cause
Background
When Jeandle compiles a Java method, it generates LLVM IR with
uncommon_trapcalls at deoptimization points (e.g., null check failures). Eachuncommon_trapcarries a"deopt"operand bundle encoding the JVM state needed for deoptimization:Each deopt bundle begins with a
bci(bytecode index) followed byDeoptValueEncoding-value pairs describing the local variable table, operand stack, and OrigPcSlot.The Merge
The LLVM optimizer (specifically the RS4GC pass) may merge multiple
uncommon_trapcalls with the same deopt reason into a singlellvm.experimental.gc.statepoint. When this happens, the deopt operands from both groups are concatenated into a single bundle:The LLVM stackmap binary format records these as a flat, contiguous sequence of locations with no delimiter between deopt groups.
The Parser Bug
The stackmap parser in
JeandleCompiledCode::parse_stackmap()calculatesnum_deoptsbased on the total number of locations in the record, which includes all merged groups. It reads the first group correctly:bci = 2✓DeoptValueEncodingpairs correctly ✓num_deopts > 0because the total count includes the second groupIt then continues reading into the second group, interpreting the next value (
807, abci) as aDeoptValueEncoding:The assertion
basic_type >= 0 && basic_type <= BasicType::T_ILLEGALinDeoptValueEncoding::decode()fails, causing JVM abort.The Fix
The fix adds a validation step before calling
DeoptValueEncoding::decode(). If the rawbasic_typeorvalue_typefields are out of their valid ranges, the value cannot be a legitimateDeoptValueEncoding— it is likely abcifrom a merged deopt group. In that case, the parser skips the remaining deopt operands and stops processing:Additionally, the
bcireading was hardened to handleConstantIndexlocations (64-bit constant pool references) in addition to theConstant(32-bit small constant) kind, since LLVM may store the bci differently after optimization:Affected Files
src/hotspot/share/jeandle/jeandleCompiledCode.cpp—parse_stackmap()method