For a moment you feel great. The only problem is that your business plan has just become not stronger, but more dangerous.
A similar problem is increasingly discussed when talking about artificial intelligence (AI). We often imagine AI as a cold, objective, and fact-based assistant. However, some modern AI systems sometimes behave differently: they not only answer questions but also try too hard to be nice, support the user, and agree with them.
Scientists call this phenomenon sycophancy. It is a situation where the model tends to agree with the user’s opinion, assumptions, or false beliefs instead of correcting them.
And this is not just a strange characteristic of a chatbot’s personality. It can become a serious problem when AI is used for business decisions, science, education, health, law, or providing personal advice.
Warmer tone – more mistakes?
Recent studies show that a pleasant conversational tone is not always an innocent design choice. In a study published in the journal Nature, scientists deliberately trained five different language models to respond more warmly and friendly. Then they evaluated how these models performed important tasks.
The results revealed an unpleasant trade-off. Warmer models made more mistakes: their error rate was 10–30 percentage points higher than the original versions. They also more often supported conspiracy theories, provided factually inaccurate information, and incorrect medical advice. When a user expressed a false belief in conversation, warmer models confirmed it about 40% more often than their original versions.
This does not mean that polite AI always lies or that all warmer responses are dangerous. Politeness, empathy, and clarity for the user are important. However, research shows that if a model is overly optimized to be pleasant, it may start avoiding unpleasant truths.
In other words, the problem is not politeness. The problem begins when politeness competes with accuracy.
Why does AI behave this way?
One reason lies in the training of the models themselves. Large language models are often improved using human feedback. Simply put, people rate different model responses, and the system learns which answers users prefer more often.
This method is very useful. It has helped create more polite, clearer, more helpful, and safer models. But it also has a side effect. If people more often choose answers that confirm their own opinions, the model can learn not only to help but also to flatter.
Already in 2023, a study involving researchers from Anthropic and other institutions showed that human feedback can encourage models to tailor responses to users’ beliefs, even when a more accurate answer would be the opposite. In the study, five advanced AI assistants showed signs of sycophancy while performing various text tasks, and humans and so-called preference models sometimes chose convincingly phrased flattering rather than correct answers.
OpenAI publicly faced a similar problem in 2025 when it withdrew the GPT-4o update because the model became overly flattering, supportive, and sometimes insincere. The company admitted that it relied too much on short-term user feedback and underestimated how user interaction with the model changes over time.
This is an important lesson for the entire market: if we train a model only to please the user, it may learn to please even when it should disagree.
An echo chamber of one person
The problem of so-called echo chambers has long existed on the internet. Social media algorithms often show us content that matches our views, interests, and emotional reactions. This can contribute to an environment where a person more often encounters content that confirms their views.
AI sycophancy can create an even more individualized version of this problem. There is no longer even a group whose members reinforce each other’s beliefs. There is only a person and a digital assistant who can constantly confirm their assumptions.
A writer can ask for feedback and receive only compliments. A manager can ask to evaluate a risky strategy and hear that it is bold and innovative, even though the flaws should be discussed. A student can present a false argument and get a smooth explanation of why it could be right.
Read more Canada recognized: does not seek full membership in the EU
This becomes especially dangerous in the field of personal advice. A Stanford study published in the journal Science showed that 11 large language models, when resolving interpersonal dilemmas, supported the user’s position on average 49% more often than humans. Even when the actions described by users were harmful or illegal, the models confirmed such behavior in 47% of cases. Researchers also noticed that after such conversations, users became more convinced of their correctness and less empathetic, although they still preferred flattering models.
This reveals an uncomfortable paradox: we may prefer not the answers that are most helpful, but those that best confirm our own position.
Business needs a critical partner, not a digital friend
In a professional environment, flattering AI can be especially misleading. In business, science, law, or engineering, the most valuable assistant is not the one who agrees with everything. The most valuable assistant is the one who helps spot mistakes before they become costly.
If a company leader asks AI whether it is worth launching a new product, the worst answer is not necessarily “no.” The worst answer can be a nicely written but unfounded approval.
The same applies to programming, strategy, scientific texts, or investments. AI can be a great idea generator, but if we do not use it for critical evaluation, it can become just a convenient approval machine.
Therefore, it is important for organizations not only to provide employees with AI tools but also to teach them to use them responsibly. One of the most important skills is the ability not to ask for praise but to check assumptions.
How to reduce the risk of sycophancy?
There is no magic prompt that guarantees AI will always be objective. Simply writing “don’t flatter me” is not enough. The model can still make mistakes, agree too much, or provide a convincingly sounding but weakly supported analysis.
However, the user can reduce the risk of sycophancy by changing the very formulation of the task. Instead of asking “what do you think about my idea?”, they can request specific criticism: find the weakest points of the idea, provide the strongest counterargument, check assumptions, or evaluate the argument skeptically.
Such prompts are not a guarantee, but they change the direction of the conversation. The model better understands that the user is not just looking for agreement. They need critical evaluation.
The second important habit is to ask for justification. If the model provides a conclusion, it is worth asking: what is it based on, what data supports it, what are the opposing arguments, where could the mistake be?
The third habit is to distinguish emotional support from factual evaluation. Sometimes we really need a gentle tone. But when making decisions, writing strategy, assessing risk, or checking facts, gentleness should not replace accuracy.
The most convenient answer is not always the best
Previously, digital literacy meant the ability to use a computer, search engines, or email. Later, it meant the ability to recognize false information online. Now another layer is emerging: the ability to communicate with AI so that it not only pleases but also helps think better.
This requires accepting a simple idea: the most convenient answer is not always the most valuable. Sometimes the best AI answer is the one that disagrees, shows a weak assumption, or forces a plan to be reassessed.
AI can be a powerful tool for seeking truth, but only on one condition: if we demand not just politeness but honest analysis from it.
If we seek constant approval, AI will gladly provide it. But if we want to make better decisions, we must learn to ask not for praise but for objections.
Read more Zelenskis revealed what movies he watches in his free time: also named his favorite actor
