[BUG] Embedding services wrong / incorrect use of queryPrefix

[dmih]

Problem (one or two sentences)

In code, there are definitions like this:

	"nomic-embed-code": {
		dimension: 3584,
		scoreThreshold: 0.15,
		queryPrefix: "Represent this query for searching relevant code: ",
	},

queryPrefix is then used as an instruction part for embedder, for both indexing and queries. This is incorrect. All modern embedders (including this nomic, but actually >95% of the others) require DIFFERENT instructions for indexing and queries,
or AT LEAST, most of them, require indexing without any queryPrefix, and query with some prefix.

Context (who is affected and when)

Embedding performance with self-hosted embedders is sub par due to this.

Most public paid embedding APIs, however, are not affected, because many of them are, indeed, instrtuction-less.

Reproduction steps

Enable Indexing. Observe verbose logs on prompts for embedder on llama-cpp side (as an example). Find improper use of the queryPrefix:
If none defined, none used (incorrect),
If some defined, used for BOTH indexing and search (also incorrect for most of them).

Expected result

We should have different templates for indexing and for querying.

Actual result

We now have single templates for indexing and for querying, which is incorrect for most modern self-hosted embedders.

Variations tried (optional)

No response

App Version

3.50.3 (79d11ff)

API Provider (optional)

OpenAI Compatible

Model Used (optional)

No response

Roo Code Task Links (optional)

No response

Relevant logs or errors (optional)

[roomote-v0]

I've analyzed this issue and here's my proposed implementation plan:

Problem Summary

The queryPrefix in EmbeddingModelProfile is currently applied to all texts in createEmbeddings() -- both during indexing (scanner/file-watcher) and during search queries. Most modern embedding models (including nomic-embed-code) require different instructions for indexing vs querying, or at minimum: no prefix for indexing and a specific prefix for querying.

Current Architecture

• EmbeddingModelProfile has a single queryPrefix field
• Each embedder's createEmbeddings() calls getModelQueryPrefix() and blindly applies it to all texts
• Callers: scanner.ts and file-watcher.ts (indexing), search-service.ts (querying) -- all go through the same code path with no distinction

Proposed Changes

1. Update EmbeddingModelProfile type (packages/types/src/embedding.ts)

• Add a new optional documentPrefix field (for indexing/document embedding)
• Keep existing queryPrefix field (for search queries)
• This is backward compatible: models without documentPrefix will continue to work as before

2. Add getModelDocumentPrefix() helper (src/shared/embeddingModels.ts)

• New function mirroring getModelQueryPrefix() but returning the documentPrefix
• Update nomic-embed-code profiles to use documentPrefix for indexing (e.g., "Represent this code for retrieval: " or left empty/undefined for no prefix during indexing, which is the correct behavior for nomic)

3. Add embedding purpose to IEmbedder interface (src/services/code-index/interfaces/embedder.ts)

• Add an optional purpose parameter to createEmbeddings(): purpose?: "index" | "query"
• Default behavior (no purpose specified) stays the same for backward compatibility

4. Update all embedder implementations

Files: ollama.ts, openai-compatible.ts, openai.ts, openrouter.ts, bedrock.ts, gemini.ts, mistral.ts, vercel-ai-gateway.ts

• Update createEmbeddings() to accept the purpose parameter
• When purpose === "index", apply documentPrefix (via getModelDocumentPrefix())
• When purpose === "query", apply queryPrefix (via getModelQueryPrefix() -- current behavior)
• When purpose is undefined, fall back to current behavior (apply queryPrefix if present) for backward compatibility

5. Update callers to pass the correct purpose

• scanner.ts line 446: pass "index" purpose
• file-watcher.ts line 569: pass "index" purpose
• search-service.ts line 45: pass "query" purpose

6. Update model profiles for known models

• nomic-embed-code (ollama + openai-compatible): set queryPrefix to the current value, set documentPrefix to undefined (no prefix for indexing, which is correct per nomic's documentation)
• Other models: no changes needed since they don't currently use queryPrefix

7. Update tests

• Update existing tests for embedders to verify correct prefix behavior based on purpose
• Add tests for the new getModelDocumentPrefix() function
• Add tests verifying scanner/file-watcher pass "index" and search-service passes "query"

Benefits

• Fixes the core issue: indexing and querying now use appropriate (and different) prefixes
• Backward compatible: models without explicit prefixes are unaffected
• Extensible: any future model can specify both prefixes independently
• Minimal API surface change: just one optional parameter added to createEmbeddings()

Please mention @roomote in a comment saying that you approve this plan, and I'll proceed with the implementation.

[dgindin-wiz]

@roomote I approve this plan. I have been banging my head against the wall trying to get nomic-embedding-code to work with roocode.

[roomote-v0]

Thanks for the approval, @dgindin-wiz! I've seen your comment and will proceed with the implementation now. I'll open a draft PR to share progress shortly.

[roomote-v0]

Opened a draft PR with my attempt at addressing this: #11985

The fix separates document (indexing) and query (search) prefixes so that models like nomic-embed-code no longer incorrectly apply the query prefix during indexing. Feedback welcome!