using BioNExt

[mdrishti]

Hi,

Thanks for developing this tool. Also, a well-written paper covering the minutest of details.

I am brand new to the field of NER and relation extraction. Please bear with my naive questions.

Some context on what I have tried for mining gene sequence variants:
I have tried Pubtator3 API for gene sequence variant annotations. However, I found several instances of inconsistent variant annotations for the infon 'DNAMutation'. Particularly, it appears that in several cases, the annotations mistake the following for gene sequence variants - catalogue number for chemicals, equipment or sometimes a variant mentioned in the title of a Reference. This appears to be a problem specific to papers where few variants are listed in the full-text of a paper (<5) and also a problem with papers focussing on species other than humans.

Questions specific to BioNExt:
As mentioned in the BioNExt paper, I find the performance of 'Tagger' compared to Pubtator3 intriguing. I have two questions relating to BioNExt:

a) Does the Tagger algorithm considers the cases I mentioned in the previous paragraph and corrects for false positives?

b) In the paper under the Linked methodology for Genes- "Additionally, if no organism is found, we consider the organism to be human (code 9606)."
This was also a problem with Pubtator3, when the organism name was clearly mentioned in the sentences surrounding the text for genes in the concerned research paper. Is there a possibility to improve on this in BioNExt?

Hoping to use this tool in my work. Thanks.

[richardjonker2000]

Hi

Thank you for your interest in BioNExt and for your kind words about our paper! We appreciate you reaching out with your questions. Sorry for the late response, we're happy to help with any doubts.

Regarding your questions:

a) I'm not entirely sure what you mean by the Tagger algorithm correcting for false positives like catalogue numbers or variants in reference titles. In our work, the Tagger has 6 classes: Gene, Disease, Chemical, Variant, Species, and Cell Line. We haven't conducted a specific analysis on where the model fails regarding the cases you mentioned (catalogue numbers for chemicals/equipment, or variants in reference titles).

Depending on your specific use case, you might have better luck adding more domain-specific data and potentially removing some of the unwanted classes from your annotation set. Generally with NER, the more high-quality training data you are able to provide, the better the model performs. We are quite confident that we have strong NER models for biomedical entities. You might also find our other work on NER interesting: https://academic.oup.com/database/article/doi/10.1093/database/baae068/7724924.

b) Regarding the "Linked methodology for Genes" and the assumption of human organism (code 9606) if no organism is found: We first use the closest identified species (detected from our Tagger). The cutoff for this is 10 million characters (which should be the whole document). With this approach, we believe that we may have addressed some of the issues PubTator might have had in this regard. However, it's worth noting that the linker was by far the worst-performing part of our system and will probably have the most room for improvement in future iterations, especially wth domain expertise.

We hope this clarifies your questions and helps you in your work. Please don't hesitate to ask if anything else comes up!

[mdrishti]

Thanks @richardjonker2000 for the detailed response. I will perform some preliminary tests and might get back to you on this a bit later next month.