An important factor is to be very specific with what you want it to do. For that, many now use a “plan” mode, where you discuss about a change first, adding more and more context. In addition to that MCP and RAG will improve the quality tremendously, if you use them correctly. But even if you are careful and used all the tools possible, the LLM might randomly hallucinate. Maybe it affects only a misplaced semicolon in the code. But if it is happening to the orchestrator-agent, then a whole feature is affected. The worst part: you cannot test this, as in many cases, the code is syntactically correct. Just the implementation is not logical at all.
So even for the bad case, where you have a wrong feature with working code, you can mitigate this by introducing a critic-agent, which is by default critical towards everything unless proven otherwise.
That’s at least how I do it. And it works well. But it took me roughly 2 full months to get to this state. So you cannot compare this to classical “Oh I just open up CODEX and then I have working software”, because that will not work.
What do you man by tell it to write good code?
You mean specific prompts, which you mention afterwards, but as also hit and miss?
An important factor is to be very specific with what you want it to do. For that, many now use a “plan” mode, where you discuss about a change first, adding more and more context. In addition to that MCP and RAG will improve the quality tremendously, if you use them correctly. But even if you are careful and used all the tools possible, the LLM might randomly hallucinate. Maybe it affects only a misplaced semicolon in the code. But if it is happening to the orchestrator-agent, then a whole feature is affected. The worst part: you cannot test this, as in many cases, the code is syntactically correct. Just the implementation is not logical at all.
So even for the bad case, where you have a wrong feature with working code, you can mitigate this by introducing a critic-agent, which is by default critical towards everything unless proven otherwise.
That’s at least how I do it. And it works well. But it took me roughly 2 full months to get to this state. So you cannot compare this to classical “Oh I just open up CODEX and then I have working software”, because that will not work.