-
Notifications
You must be signed in to change notification settings - Fork 2.9k
Description
Problem (one or two sentences)
There are two things I noticed.
The first is that when editing a file fails, the spinner at the top, meaning the loading icon, remains active. It is hard to tell whether anything is still happening here.
I strongly assume this is just a UI bug: when the tool cannot write the file, caused by the LLM hallucinating during the freeform patch, or depending on how it calls the tool here, whether it is using freeform patch or working with lines, in any case, this is the standard issue that has always existed with edit tools. LLMs start hallucinating beyond a certain length and should really re-anchor beforehand.
The problem is that it simply keeps going, and the loading bar at the top remains active. It is difficult to say what exactly is happening here, because there is no error handling or anything similar in this regard. It just keeps loading.
However, I strongly assume that this is only a UI bug.
I was able to reproduce that, without my own system prompt that I set for my mode, this can happen up to four times in a row, to the point that it becomes unusable, until Roo Code eventually aborts after a certain minimum number of attempts because of the failure state it has built in.
That would be the second feature that would need to be integrated.
The system prompt that is currently set as the default for tool usage is simply phrased too loosely. This is the most important anchor point, because LLMs consistently hallucinate here.
Of course, it is fine to have a fallback so that it does not get stuck in an endless loop and stops after a certain number of failed attempts, and to have that option available. I do not want to argue against that. However, the core problem is the architectural system prompt.
From my own experience, I can say that this issue with these tools can be addressed architecturally through prompting by requiring a re-anchoring step before every edit to a file. In other words, a re-anchoring step must take place, and in addition there must be a multi-level evaluation principle for whatever is currently being patched.
That means:
- whether a full rewrite is being done,
- whether a sequential edit is being done,
- the size of what is currently being edited,
- and, for example, if only hypothetically 15% of the file is being edited, then a small sequential edit should be made,
- whereas if it is being edited more extensively, say 80%, then a full rewrite should be done.
At the same time, boundaries should also be set. For example, if the file is 1,200 characters long, then a full rewrite should not be the default, because proportionally, based on the weighting, the likelihood is simply higher that LLMs will hallucinate through the freeform patch.
On top of that, there are naturally also topics such as the context window here, how large it is, and if this is done properly at the architectural level, there is also access to the context size here, etc., so it would even be possible to handle this differently.
Either way, architecturally this can be solved in more than 80% of cases if a re-anchoring step is performed.
That means the option should be integrated that, although it is naturally more context-intensive in terms of token usage, a renewed re-anchoring step is always performed beforehand. There should be such an option in the settings.
If the option is enabled, the default system prompt should be extended so that a default re-anchoring step is always performed before any existing file is edited in any way, so that line-patch or freeform-patch errors are drastically reduced.
Context (who is affected and when)
see desc
Reproduction steps
see desc
Expected result
see desc
Actual result
see desc
Variations tried (optional)
No response
App Version
Version: 3.51.1 (7c9722b)
API Provider (optional)
None
Model Used (optional)
No response
Roo Code Task Links (optional)
No response