Diversity of inference

[Fpooyan]

Thanks for providing the code and well-organized repo!

I have fine-tuned Gemma-3 on my recommendation dataset, now I need to get some diverse outputs instead of the top one.

I used the beam search and tricks to diversify the result like temperature and top-k, but the result mostly sticks to one product(type).

Any guidance on how to achieve more genuinely diverse recommendations would be greatly appreciated.

[krishnacharya]

Thanks for the kind words!

Regarding diversity in recommendations, the current pipeline generates k texts and then performs a search (semantic or lexical) to pick the top-1 matching item from the catalog.

Here are a couple of ways to improve item diversity:

• Generation strategy tweaks – As you mentioned (temperature, top-k), this can help, but it might hurt core retrieval metrics and is more expensive since multiple generations are needed per request.

• Expand candidate selection – Instead of picking the top-1 item per generated text, consider taking the e.g. top-10 items. This gives you k*10 potential items, which can then be filtered using a post-processing pipeline, e.g., pydiversity

I’m currently working on the latter approach.