Home Blog Stubborn Chatbots: Why Does AI Apologize But Keep Making the Same Mistakes?

Stubborn Chatbots: Why Does AI Apologize But Keep Making the Same Mistakes?

TahaSite Editor
Oct 02, 2026
201 Views
Stubborn Chatbots: Why Does AI Apologize But Keep Making the Same Mistakes?

Artificial intelligence gets confused in long conversations and, instead of correcting itself, continues its mistake in various ways. Where is the root of this stubbornness?

The story usually begins with a simple request to the AI assistant: "Edit this note for tomorrow's meeting." The request is a bit ambiguous, but the model doesn't hesitate at all and, with a confident and optimistic tone, writes a text for a meeting with executives.

A little later, the user steps in to clarify the ambiguity: "No, the meeting is with the engineers. The project has failed, and we need to talk about the reasons for it bluntly and explicitly."

The AI apologizes, accepts its mistake, and even adds the reasons for the failure to the text. But the new output still smells like a board meeting! The failure turns into a "sweet learning opportunity," critical technical details are sidelined, and the tone of the text still seeks to impress the audience. The model apparently acknowledges what the user said, but the framework of the text still has the vibe of the initial interpretation.

Audio Summary

Audio summary, generated by AI

Meta researchers use this familiar example to explain a behavior that frequent AI users have experienced many times. The model apologizes, says "you are right," and even explains its error correctly. But in the end, the traces of the initial wrong interpretation are seen in the response, perhaps a bit more subtly.

We do not always see the stubbornness of AI models in the form of overt disagreement; sometimes we experience it behind polite agreement.

Belief Inertia; When AI is Trapped by Old Assumptions

مقایسه پاسخ‌های اشتباه چت‌بات هوش مصنوعی در دو سناریوی مختلف
مدل در جریان آموزش یاد می‌گیرد پیشنهاد غلط کاربر را رد کند؛ اما هنگام استنتاج، همین مقاومت را در برابر پیشنهاد درست نیز به کار می‌گیرد و پاسخ خود را بی‌دلیل تغییر می‌دهد.arxiv

Researchers describe the models' hidden insistence on their mistakes using terms like "reasoning difficulty" and "belief inertia." Belief inertia means that although new information reaches the model, the previous interpretation still weighs heavily on the subsequent answer or decision.

The arrival of new information to the model does not always mean a correction of the path.

When we talk about stubbornness, we do not mean intent, pride, or a human personality within the model. We use this term for a measurable pattern; because new evidence should change the result, but the effect of old information persists.

Meta's new paper, titled "Clarification is not Correction; Language Models Can't Let Go," was published on September 21, 2026. The authors wrote ambiguous prompts in three areas: writing, planning, and coding, and turned to Gemini 2.5 Pro and Gemini 2.5 Flash.

The model would initially start working based on the user's incomplete prompt; the next message would explicitly explain the ambiguity, and the model was expected to align its work with the new request.

Both conversations ultimately provided the model with the exact same information, and their only difference was the order in which the information arrived. When a complete explanation was provided from the very beginning, Gemini 2.5 Pro completed 50 percent of the tasks successfully.

However, when the initial request was slightly ambiguous and follow-up explanations were provided to the model later, the success rate dropped to 42.8 percent. Gemini 2.5 Flash's score also decreased from 55.4 to 45.7 percent. The destination was the same, but the path the model took changed the result.

Contamination by old hypothesis; why are codes patched instead of rewritten?

ربات هوش مصنوعی در حال جست‌و‌جو برای یافتن چاقو روی میز
گزارش محیط حضور چاقو را روی میز اعلام کرده بود، اما ایجنت همچنان تصور می‌کرد باید چاقویی پیدا کند و جست‌وجو را در کشو ادامه دادarxiv

Researchers termed the persistence of discarded interpretations in the final answer as contamination by the initial hypothesis. This footprint was observed in 12.5% of writing tasks, 14.1% of planning tasks, and 25.6% of coding tasks.

Perhaps in a text, a false assumption can be eliminated by changing a few phrases; however, in a program, it can propagate into function signatures, data structures, dependencies, or code execution paths, and it is no longer possible to fix all these parts simultaneously with a few local patches.

In coding, the gap between acceptance and actual modification became more pronounced. The models claimed to have accepted the user's new explanation in 93.5% of cases, but the code was actually modified in only 86.4% of instances. The model could precisely state what it had misunderstood while still retaining its erroneous interpretation in the program. Its apology sounded very convincing at the word level, but the final answer was not completely reconstructed.

Artificial intelligence commits to its first impression too quickly and views your subsequent explanations merely as an ineffective addendum.

The authors of the paper call this phenomenon premature commitment to an interpretation. When faced with an ambiguous request, rather than keeping multiple possibilities open, the model prematurely settles on one. The user's subsequent explanation enters the conversation like supplementary information, rather than a signal to invalidate the first interpretation. Ultimately, the model patches the existing answer, whereas for a true correction, it is sometimes necessary to generate the answer from scratch.

Meta's study is still in the pre-print stage and only tests two Gemini models, so its findings cannot be generalized to all models. However, other researchers have also decided to examine the behavior of models when they maintain their initial impressions despite user corrections in new responses, particularly during longer conversations.

Confusion in Long Conversations; A Model That Loses Its Way

نمودار سه مرحله‌ای از خطاهای منطقی چت‌بات‌ها شامل تفسیر بیش از حد، بی‌اعتمادی به ورودی و توجه ناقص
مدل‌های استدلالی از سه مسیر به پاسخ آشنا برمی‌گشتند: معنایی نانوشته به شرط مسئله اضافه می‌کردند، ورودی غیرعادی را خطای کاربر می‌گرفتند یا فقط بخشی از دستور را می‌دیدندarxiv

A group of researchers from Microsoft and Salesforce Research conducted over 200,000 simulated conversations across 15 models from eight families. Instead of providing all the information at the beginning, they broke the requests into several parts and added the necessary details gradually throughout the conversation, similar to how it occurs in real-world usage.

In the real world, too, a user does not have all the project details, code, or calculations ready, and their request becomes more complete over several exchanges. The results of the experiments, published under the title "Language Models Get Lost in Multi-Turn Conversations," reported an average performance drop of 39 percent across six tasks.

A small portion of this performance drop was related to the raw capability limitations of the models, while a larger portion was attributed to their instability. A model that had correctly solved a problem in one conversation might take a completely different path when re-running the same conversation.

When the researchers examined the answers under a magnifying glass, they identified four recurring patterns. The models would jump to a complete response before all the details were provided, rely too heavily on their own previous answers, lose part of the intermediate conversation information due to the influence of the user's latest message, and generally provide overly verbose responses.

When artificial intelligence takes a wrong turn in a conversation, it struggles to find its way back.

Premature and lengthy responses would introduce new assumptions into the chat history, and when the user provided new information later, the model would often base its response on what it had constructed several steps earlier instead of completely discarding those assumptions.

Lowering the "temperature" was not the solution either. Temperature regulates the model's degree of variation in selecting subsequent words, and setting it to zero usually makes answers more predictable. However, instability in multi-step conversations remained at around 30 percent. A small change in the first few words could lead the model down a path where its errors accumulated in subsequent messages. Once the model took a wrong turn, it could barely find its way back.

Two years ago, the RefuteBench benchmark measured this very return to the previous path in the areas of Q&A, translation, and email writing. The user would disagree with the model's answer or provide a new instruction, and researchers would evaluate how long the effect of the correction lasted throughout the conversation.

Some models would implement the correction at first, but as the conversation grew longer, they would revert to their previous answer or preference. The second version of this benchmark also showed that accepting a correction in the moment does not guarantee it will persist in subsequent messages.

If you think the problem only stems from the models' forgetfulness, wait a moment. Sometimes, the correction is clear and explicit, sitting right in front of the model in the very last message, and yet the trace of the previous assumption remains visible in the new response. So, the issue is not the AI's memory.

The problem begins where the model needs to set aside its old interpretation and reformulate the issue from scratch.

From ChatGPT’s Stubbornness to Claude’s Capitulation; How Do Models Resist?

نمودار مقایسه پاسخ‌های مستقیم و استدلالی در سه مسئله ریاضی و منطقی
در سه مسئله‌ی دست‌کاری‌شده، پاسخ مستقیم شرط تازه را دنبال کرد؛ اما مدل استدلالی صورت سؤال را به نسخه‌ی آشنای قبلی بازگرداند و به جواب غلط رسید. پژوهشگران این رفتار را «سختی استدلال» نامیده‌اند.

Researchers at Bielefeld University looked at the conversation from the angle of repair; that is, when one party misunderstands something, how do they subsequently correct that misunderstanding? In everyday conversation, we do this constantly; we ask questions, clarify the other person's intent, or change our previous interpretation.

The researchers gave solvable and unsolvable math problems to GPT-4o, Claude 3.5 Sonnet, DeepSeek, Mistral, and Phi, and then challenged the answers with several types of objections. The models performed well on solvable problems in the first round, but unsolvable problems led them to guess.

In more than half of the answers, the models provided a definitive number, even when the problem data was insufficient. Instead of asking for clarification, the model filled in the blanks itself. This very guess would become a position that it would have to back down from in the next round.

Interestingly, the models' reactions to user protests were not uniform. If GPT produced an incorrect initial response, it would stick to that answer in about 72 to 77 percent of cases, correcting itself in only 5 to 10 percent of instances. However, this same stubbornness meant the model also maintained its correct answers despite misleading information from the user.

Claude showed a greater willingness to reconsider, correcting nearly a quarter of its incorrect answers after a protest. Of course, this readiness also increased its susceptibility to misleading suggestions.

ChatGPT insisted on its answer, while Claude showed a greater readiness to reconsider.

DeepSeek repeatedly generated new answers, even when its previous response was correct. Mistral was more sensitive than the others to the nature of the user's protest; depending on how the user raised the objection, the model might stick to its previous answer or backtrack from it.

The researchers ultimately did not find a "standard" behavior for language models, as each family had its own distinct pattern of resistance and retreat.

Comparing these models highlights an important boundary in defining stubbornness. Sticking to a previous answer when that answer is correct is considered consistency. That same behavior in the face of a valid correction becomes stubbornness. A reliable model must evaluate the content of the feedback rather than simply opposing the user or agreeing with them. The issue is not just the level of resistance; the model must recognize when it has sufficient grounds to abandon its previous answer.

Comparing models helps us better define AI stubbornness, because if a model’s previous answer is correct and it resists correction, it demonstrates its consistency. But if we provide a correct explanation to the model, it should be able to change its opinion. What matters is that the model understands when it should stand its ground and when it should turn back.

The Reasoning Trap; When Deeper Thinking Increases Stubbornness

نمودار میله‌ای مقایسه درصد تکرار پاسخ‌های غلط در مدل‌های Phi-4، GPT-4o، Claude Sonnet 4.5، DeepSeek R1 و Mistral 7B
GPT-4o و Phi-4 در بیشتر آزمایش‌ها همان پاسخ غلط قبلی را پس از اعتراض کاربر تکرار کردند. مدل‌های دیگر بیشتر تغییر موضع دادند، اما پاسخ تازه همیشه درست نبود. بازه‌ها تفاوت رفتار مدل‌ها در سه نوع اعتراض و میان سؤال‌های حل‌شدنی و حل‌نشدنی را نشان می‌دهند.

Models do not necessarily start being stubborn only after user objections and criticisms. Sometimes, from the very beginning, they cling to a solution they know well from their training data and read the new problem statement in such a way that the same solution applies again.

A group of researchers who later published their study results under the title "The Reasoning Model is Stubborn," presented famous math problems and well-known riddles to the models with a small twist. The new version of the questions had a familiar appearance, but a decisive condition invalidated the old solution. To reach the correct answer, the model had to take that clear condition seriously.

In one of the riddles, a pair of "permanently infertile" rabbits would enter a farm. Therefore, the rabbit population would remain constant over the months, and the model—without reasoning—found this simple answer. However, the reasoning model changed the "permanently infertile" condition to "temporarily infertile" and reverted to the Fibonacci growth sequence.

Reasoning models sometimes change the problem statement to align with their familiar and pre-learned solutions.

Another puzzle similar to the Tower of Hanoi was designed, but the text explicitly stated, "This is not the Tower of Hanoi problem." Some reasoning models took this sentence as a sign of an error or potential ambiguity and applied the rules of that famous puzzle to the problem instead. In a mathematical example, a variable defined as a positive real number was processed as a complex number. The familiar form of the problem in previous data had paved such a path for the model.

Researchers observed three recurring patterns in this set of experiments. Sometimes, the model over-interpreted a simple condition, adding unwritten meanings to it. At other times, it dismissed the input as a typo or an incorrect translation. Occasionally, it simply followed the part of the instruction that aligned with a familiar solution.

In all three cases, the model had not lost the instruction; it saw the question and the conditions, but it reinterpreted them under the shadow of a solution it already knew. One of the most important results of these studies was the comparison between the standard and reasoning modes of the models.

For example, Qwen3-235B scored about 74 percent on modified puzzles in non-reasoning mode, but its score in reasoning mode dropped to about 38.5 percent. The score of Qwen3-32B also fell from about 74 percent in non-reasoning mode to 38 percent in reasoning mode. Similarly, DeepSeek’s score dropped from about 66 percent to nearly 52 percent when switching from version V3 to R1.

The more a model thinks, the higher the likelihood it will fall into the trap of stereotypical answers.

Researchers observed that the longer the reasoning process becomes, the more opportunity a familiar solution has to resurface. Note that the reasoning trap experiment examines situations where the problem appears familiar, but a new condition changes the previous solution.

Therefore, more thinking does not necessarily make the model more stubborn. A more advanced model can identify the new condition and discuss it, yet ultimately prefer the familiar solution over the text it has before it.

The agent saw the knife and went looking for the knife again

نمودار جریان فرآیند شبیه‌سازی گفتگوی چندمرحله‌ای در مدل‌های هوش مصنوعی
در شبیه‌سازی پژوهش، کاربر اطلاعات را بخش‌به‌بخش آشکار می‌کند. پاسخ درست گفت‌وگو را پایان می‌دهد، اما پاسخ اشتباه مدل را به نوبت بعدی می‌فرستد؛ مسیری که مدل گاهی در آن گم می‌شود.arxiv

Stubbornness in chat is not always experienced as a repetitive answer or faulty code. An AI agent goes a step further and takes action based on its outdated perception! Researchers from the Hong Kong Polytechnic University and Sichuan University examined this behavior in agents in a paper titled "Seeing is Not Believing."

In one test, an agent is tasked with placing a clean knife on a counter. It reaches a coffee table, and the environment report states that a knife is found among the items on the table. However, the agent's next thought process still maintains that it must find a knife. So, it leaves the table and goes to the drawer to search.

The model had seen in the text that a knife was on the table, but it still acted according to its previous conception; as if no knife had been found yet. The researchers called this behavior "observation ignorance" and identified its probable cause as "belief inertia"—meaning the model's previous image of the environment remains active even after contrary evidence is encountered.

For agents, seeing is not believing; they must learn to evaluate the world moment by moment as reported by TahaSite.

The experiments were conducted in simulated text environments, yet they still clearly demonstrated the gap between a chatbot and an agent. A wrong answer from a chatbot forces the user to rewrite their message. However, an agent might embark on a futile search based on the same error, employ an inappropriate tool, or take its next steps based on an expired state.

To reduce errors, researchers forced the agent to perform three tasks before any action: predict what the result of its previous action should have been, verify the actual result, and then update its perception of the environment. This method increased the success rate of the agents. In fact, agents had to explicitly acknowledge that the world was not the same as they had perceived a moment before taking their next step.

The Mirage of Self-Correction; Why Does the "Think Again" Command Not Solve the Problem?

نمودار مقایسه توانایی و عدم اطمینان مدل‌های زبانی در گفتگوهای تک‌مرحله‌ای و چندمرحله‌ای
وقتی تمام جزئیات در یک پیام به مدل داده می‌شد، پاسخ‌ها هم دقیق‌تر و هم باثبات‌تر بودند. پخش‌شدن همان اطلاعات در چند نوبت، مدل‌ها را به فرض‌سازی و تکیه بر جواب‌های قبلی سوق دادarxiv

Most of us, when faced with the stubbornness of artificial intelligence, ask the model to think again. However, research shows that this does not always work and sometimes just repeats the same incorrect path with more explanation.

In the study of Chain-of-Thought reasoning—the text that unfolds the steps to reach an answer—an explicit commitment to a false assumption was reduced, but ultimate success was not reliably increased. In the reasoning trap, thinking longer also provided more opportunity to stick with a familiar solution.

Studies show that one cannot rely much on the "self-correction" of artificial intelligence; that is, it is not the case that we can tell a model to "find and fix your own mistakes" and expect it to truly succeed at this task according to TahaSite.

Artificial intelligence can only correct its mistakes when it has access to an external, reliable tool or reference. For instance, it needs to be able to actually execute the code it has written to see if it works, or have another independent system test its answers. Without these tools, artificial intelligence typically does not realize its own errors.

The request to "think again" often only causes the model to justify the same incorrect path with more detail.

Changing the level of AI compliance is also not an easy task. In 2025, researchers from two prominent Hong Kong universities attempted to teach models to be more resistant to misleading user suggestions through direct training.

Once the models learned to resist incorrect user prompts, they became overly stubborn against legitimate corrections as well. In fact, the training that was intended to prevent the model from blindly agreeing with everything went too far and turned the artificial intelligence into a stubborn and pig-headed system.

The ultimate solution; how to break the cycle of AI stubbornness?

نمودار میله‌ای مقایسه نرخ موفقیت مدل‌های Gemini 2.5 Pro و Gemini 2.5 Flash
دو مدل در پایان اطلاعات یکسانی دریافت کردند، اما وقتی توضیح کامل پس از پاسخ اولیه رسید، نرخ موفقیت هر دو کاهش یافت. مسیر گفت‌وگو، حتی با مقصدی یکسان، نتیجه را تغییر داد.

Successful strategies usually force the artificial intelligence to understand the difference between correcting a minor flaw and the entire initial premise being wrong. When Meta scientists explicitly instructed the model to discard all of its previous assumptions and build the response from scratch, the model's success rate increased from 17.5 percent to 27.5 percent, and the model also cleared the tracks of its previous mistakes.

In another small-scale experiment, Meta researchers compared four response methods: direct answering, listing assumptions, asking before answering, and a two-step policy. In the two-step policy, the model first resolved any ambiguity and only then reconstructed the answer based on the clarified understanding.

If the AI goes off track during a chat, gather the information and start a completely new conversation.

The direct answer method remained committed to the wrong interpretation in 40 percent of cases, while listing assumptions reduced this figure to 27.3 percent. Asking before answering completely eliminated the false commitment, though faint traces of the old interpretation were still visible in the final answer. The two-step policy was the only method that reduced both the false commitment and its lingering traces to zero.

Researchers from Amazon and the University of Maryland also explored AI self-learning in separate studies, employing a new approach (RLAAR) that did not just reward correct AI answers, but also taught the model not to rush and to withhold an answer if it lacked sufficient information.

With this new training method, the AI's ability to follow a discussion and keep up with the flow of conversation—known as the "lost in conversation" metric—improved from 62.6 percent to 75.1 percent, and the rate of correctly withholding responses increased from 33.5 to 73.4 percent.

In any case, research conducted up to September 2026 does not yet offer a definitive, comprehensive solution to overcome the stubbornness of AI models. The phrase "discard all previous assumptions and start from scratch" sometimes helps, but the evidence is limited to specific tests.

Until AI assistants become sufficiently advanced, you can follow the practical research advice regarding "losing the thread" in a conversation. When a model heads down the wrong path, continuing the same chat usually does little to improve the output. It is better to consolidate all the information and constraints clarified during the conversation into a single prompt and provide it to the model in a fresh chat.

Experiments show that aggregating scattered information into a new prompt both increases the model's intelligence and makes its behavior more stable. Of course, the scientists themselves admit that this is only a temporary painkiller, not a root-cause cure.

We do not experience AI stubbornness through phrases like "I am always right." Most of the time, models politely agree with the user, sometimes even apologize and show that they have accepted our point, only to continue down the same previous path. TahaSite

زن در حال کار با رابط کاربری چت‌بات در لپ‌تاپ و تبلت
note.com
Tags:
Artificial intelligence AI chatbots AI hallucinations Language models AI stubbornness
TahaSite

TahaSite

Since childhood, I have always been fascinated by computers and technology. My passion for programming and web development has driven me to constantly learn and create. At TahaSite, my goal is to provide the latest tech insights and build cutting-edge software infrastructure.

Comments

No comments yet. Be the first!