Skip to content

[bug] Claim-side "up to N" comparator not recognized — verbatim "up to" claim scored PARTIAL #16

@moonweave

Description

@moonweave

Summary

A claim phrased "up to N%" (or "up to N ") is not recognized on the claim side: the
comparator parser falls through to exact. Evidence phrased "up to N%" is parsed as up_to, and
_evidence_entails_claim returns False for up_to evidence against an exact claim. The result:
a claim that is a verbatim substring of the abstract is scored PARTIAL.

There is an asymmetry — the evidence side understands "up to", the claim side does not.

Affected code

  • src/ref_verify/claim_check.py_claim_percentage_comparator (no "up to" / "at most" /
    "no more than" branch) and _evidence_entails_claim (up_to evidence + exact claim → False).
  • src/ref_verify/numeric_claim.py_claim_comparator has the same gap for non-% units.

Repro (abstract contains the claim verbatim)

python3 -m ref_verify.cli check-claim 10.1126/science.287.5454.836 \
  --claim "Actuated strains up to 117% were demonstrated with silicone elastomers." --source crossref --json
# Abstract (CrossRef): "...Actuated strains up to 117% were demonstrated with silicone elastomers..."
# Actual:   PARTIAL / CLAIM_NOT_EXPLICIT
# Expected: ACCEPT

Suggested fix

Recognize "up to" / "at most" / "no more than" in the claim comparator (map to up_to/lte), and
make _evidence_entails_claim treat up_to evidence as supporting an up_to/lte claim when the
values match (and an exact claim of the same value, since "up to X" stated verbatim for the same X is
the same statement).

Tracking

Covered by the CLI regression corpus as row E2-pelrine-117 (gated_on up-to-comparator) in
PR #15. The gate will flip that row PENDING → PASS once this lands.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions