LLMs can be, and are, often wrong

[punkish]

@punkish thanks for this - yes I was looking for it.

I just checked to get a description of Formica rufa http://askzai.net/?heyzai=can%20you%20describe%20Formica%20rufa? which claims that the wood ant is a beetle... see eg https://tb.plazi.org/GgServer/srsStats/stats?outputFields=doc.uuid+bib.year+bib.source+tax.genusEpithet+tax.speciesEpithet+tax.authName&groupingFields=doc.uuid+bib.year+bib.source+tax.genusEpithet+tax.speciesEpithet+tax.authName&FP-tax.genusEpithet=Formica&FP-tax.speciesEpithet=rufa&format=HTML

Originally posted by @myrmoteras in #1

[punkish]

Yea, the "answers" have to be taken with due caution as they can and are often wrong or off-point. Here is what I have learned during my explorations with LLMs --

1. You would expect almost all computer programs to produce the same answer when asked a question with the same inputs. But, LLMs can and do produce different answers. Not just different LLMs, but the same LLM, given a certain input, can produce a different answer. In other words, 2 + 2 may not always be 4 under certain conditions. I have tried three different LLMs, and they all have their weaknesses and strengths.

Possible solution: If the cost of producing the answer is low enough, we can provide the user a choice of answers.

2. Some of the variation (extent to which the answer may be wrong) is likely the fault of my approach. When the user asks a question, Zai breaks the question into tokens, removes the tokens that it considers unimportant ("stopwords"), then conducts a search with the rest of the tokens. It then takes the highest ranked search result and uses that as the context with which the LLM has to answer the original question. If the highest ranked search result is "wrong", then the answer will be wrong.

Possible solution: Give the user the ability to pick the search result to use as the context.

3. A species binomen is usually associated with more than one treatment. Depending on which treatment is used as the context, the answer can be different.

Possible solution: Same as above, that is, give the user the ability to pick the search result to use as the context.

4. LLMs only understand english. If the treatment is in a different language, or doesn't have enough text to allow the LLM to extract the answer, then the answer may be wrong.

Possible solution: Give the user the ability to modify the search and use different contexts.

In summary, are the LLMs correct. No, certainly not, and certainly not all the time. But, are the LLMs useful. I think "yes", as long as the user knows the limitations.

Also, this is just the first attempt at making a user-facing application. There are many other approaches that might yield different or even better results. We need to try those. I certainly intend to try a few now that I know a lot more about how the LLMs work.

Originally posted by @punkish in #1

[punkish]

@myrmoteras I asked Zai "Describe Formica rufa" (slightly different wording), and I got the following answer

Formica rufa is a species of ant that belongs to the genus Formica. It is native to Europe and Asia, with a range that includes the UK, France, Germany, and Japan. The species is characterized by its reddish-brown coloration and distinctive red or orange markings on its elytra (wing covers).

http://askzai.net/?heyzai=Describe%20Formica%20rufa

[punkish]

a follow-up:

Internally, Zai handles two kinds of queries:

1. Queries that begin with the word "Describe" and are followed by a two-word binomen. For example, "Describe Formica rufa" (can be lowercase or uppercase -- casing does not matter). If you just type "describe xxx" (the word "describe" followed by a space and any three letters shown here as xxx), Zai fetches all species binomens starting with "xxx" and shows a dropdown list from which you can choose. Choosing a binomen and hitting the button returns a short description of that species. This query usually does NOT end with a ?. Zai has already created summary descriptions for almost ¾ million treatments, so the answer comes back really fast.

Note: In your case above, you didn't type "describe formica rufa". Instead, you typed (a semantically equivalent) "can you describe formica rufa", and that threw Zai off causing it to conduct a different kind of query… see next

2. Queries that do NOT begin with the word "describe". In this case, Zai tags certain words in the query as "stopwords" (they show up in a grey font) and other words as the valid search terms (in green font). In the case of "can you describe formica rufa", the first three words got greyed out. Zai then conducts a fulltext search using the words in the green font, the search terms. Depending on the words used to search, there might be 10s, even hundreds of answers. Internally, Zai ranks them using a statistical method (bm25()), finds the top-ranked treatment, and uses the fulltext of that paper to ask the LLM the original question: "can you describe formica rufa?" These queries usually end in a ?. Since the LLM figures out the answer on the fly, the result comes back quite slow (our server is very low-powered, lot less than my laptop).

I am trying to figure out the best way to ensure that queries such as "can you describe foo bar" are treated the same as "describe foo bar", that is, use the semantic cache that I have developed to answer semantically equivalent queries.

cc @myrmoteras