Summary
A claim phrased "up to N%" (or "up to N ") is not recognized on the claim side: the
comparator parser falls through to exact. Evidence phrased "up to N%" is parsed as up_to, and
_evidence_entails_claim returns False for up_to evidence against an exact claim. The result:
a claim that is a verbatim substring of the abstract is scored PARTIAL.
There is an asymmetry — the evidence side understands "up to", the claim side does not.
Affected code
src/ref_verify/claim_check.py — _claim_percentage_comparator (no "up to" / "at most" /
"no more than" branch) and _evidence_entails_claim (up_to evidence + exact claim → False).
src/ref_verify/numeric_claim.py — _claim_comparator has the same gap for non-% units.
Repro (abstract contains the claim verbatim)
python3 -m ref_verify.cli check-claim 10.1126/science.287.5454.836 \
--claim "Actuated strains up to 117% were demonstrated with silicone elastomers." --source crossref --json
# Abstract (CrossRef): "...Actuated strains up to 117% were demonstrated with silicone elastomers..."
# Actual: PARTIAL / CLAIM_NOT_EXPLICIT
# Expected: ACCEPT
Suggested fix
Recognize "up to" / "at most" / "no more than" in the claim comparator (map to up_to/lte), and
make _evidence_entails_claim treat up_to evidence as supporting an up_to/lte claim when the
values match (and an exact claim of the same value, since "up to X" stated verbatim for the same X is
the same statement).
Tracking
Covered by the CLI regression corpus as row E2-pelrine-117 (gated_on up-to-comparator) in
PR #15. The gate will flip that row PENDING → PASS once this lands.
Summary
A claim phrased "up to N%" (or "up to N ") is not recognized on the claim side: the
comparator parser falls through to
exact. Evidence phrased "up to N%" is parsed asup_to, and_evidence_entails_claimreturns False forup_toevidence against anexactclaim. The result:a claim that is a verbatim substring of the abstract is scored PARTIAL.
There is an asymmetry — the evidence side understands "up to", the claim side does not.
Affected code
src/ref_verify/claim_check.py—_claim_percentage_comparator(no "up to" / "at most" /"no more than" branch) and
_evidence_entails_claim(up_toevidence +exactclaim → False).src/ref_verify/numeric_claim.py—_claim_comparatorhas the same gap for non-% units.Repro (abstract contains the claim verbatim)
Suggested fix
Recognize "up to" / "at most" / "no more than" in the claim comparator (map to
up_to/lte), andmake
_evidence_entails_claimtreatup_toevidence as supporting anup_to/lteclaim when thevalues match (and an exact claim of the same value, since "up to X" stated verbatim for the same X is
the same statement).
Tracking
Covered by the CLI regression corpus as row
E2-pelrine-117(gated_onup-to-comparator) inPR #15. The gate will flip that row PENDING → PASS once this lands.