SentenceTransformer model reloaded on every search request

[adarshkumar23]

Problem

The milvus_search() function in both server/app.py (line 77) and server-https/app.py (line 123) initializes a new SentenceTransformer instance on every call:

encoder = SentenceTransformer(EMBEDDING_MODEL)

Loading sentence-transformers/all-mpnet-base-v2 takes ~2-5 seconds and ~400MB of memory each time. In agentic RAG workflows with multiple tool calls per user turn, this adds 10-25 seconds of unnecessary latency per query.

Proposed Fix

Initialize the model once at module level and reuse it across all requests.

[Ayush-kathil]

Resolved via PR #186.

The root cause was repeated initialization of the SentenceTransformer model inside milvus_search(), which introduced significant latency and memory overhead during query execution.

The fix moves model loading to a shared instance, ensuring it is initialized once per process and reused across requests. This removes redundant loading and substantially improves response time, especially in multi-step RAG workflows.