The Halo Effect is a cognitive bias. Like any bias, it describes what happens when we make up our minds quickly, without evidence.
In the case of the Halo Effect, it's for the positive. :)
It's when our human emotional side and our cognitive abilities join forces and trick us into believing we're making a logical decision. :)
I think the funny part is that we all know this exists, but we don't realise when we're doing it.
So let's check what the Halo Effect is and how it affects us and LLMs.
Because we're very upset when AI demonstrates it, but quite forgiving when we do.
Well, let's explore why that is. :)
What does the Halo Effect mean?
The Halo Effect is a cognitive bias in which one positive characteristic influences our judgment, causing us to attribute additional positive characteristics without evidence to support them.
Let's say you meet someone who is very confident. You might automatically assume that person is intelligent, competent and trustworthy.
But what is the connection between confidence and trustworthiness?
There isn't one!!! And that's the point. :)
Or imagine you've had a wonderful experience with a company's customer service. You might assume that its products are equally good, even though you haven't used them.
And we do something similar with AI. ChatGPT gives us a beautifully written answer, so we assume the information must be accurate.
But writing well and being factually correct are two different characteristics.
The Halo Effect happens when we use one characteristic we've experienced as evidence for other characteristics we haven't experienced.
And because those connections feel logical to us, we don't necessarily stop to question them.
Who discovered the Halo Effect?
If it's so tricky, how come we found out about it?
Well, here's how the story goes... :)
1920 — Edward Thorndike
Psychologist Edward Thorndike studied how military officers evaluated their soldiers.
He noticed that when an officer rated a soldier positively on one characteristic, such as physical appearance or intelligence, the officer tended to give that soldier positive ratings on other characteristics too.
The ratings were influenced by an overall impression rather than independent evidence about each quality.
Thorndike described this pattern as a problem in the way people evaluate others.
1946 — Solomon Asch
Psychologist Solomon Asch explored how people form impressions of personality.
His research showed that certain characteristics strongly influence the overall impression we form of someone.
For example, describing a person as warm or cold can change how people interpret the other characteristics attributed to that person.
One characteristic influences how the others are understood.
1977 — Richard Nisbett and Timothy Wilson
Richard Nisbett and Timothy Wilson investigated something particularly interesting: whether people recognise the influence behind their own judgments.
Their research showed that people can be influenced by an overall impression without accurately recognising or explaining that influence.
In other words, we can confidently explain why we reached a conclusion while being unaware of some of the factors that influenced it.
1981 — The Halo Effect Revisited
Later research examined whether warning people about the Halo Effect would help them avoid it.
The study, The Halo Effect Revisited: Forewarned Is Not Forearmed, explored the difficulty of correcting this bias simply by making people aware that it exists.
And here's what I find fascinating about this history.
We have known about the Halo Effect for more than a century.
We've studied it, described it and demonstrated how it works.
Yet knowing that it exists doesn't automatically mean we recognise it in ourselves.
We're quite good at spotting it in other people's thinking, though. :)
How does the Halo Effect affect AI?
Before we answer that, let's remind ourselves what an LLM is.
A large language model learns statistical and semantic relationships from enormous amounts of data. When generating an answer, it predicts tokens based on the patterns it has learned and the context of the conversation.
But where did those patterns come from?
Us. :)
Human knowledge, writing, opinions, experiences and the connections we make between things are all represented in the information used to train these models.
And that information includes our biases.
For example, if confidence and competence are repeatedly associated in training material, an AI model can reproduce that association even when there isn't enough evidence to establish that a particular confident person is competent.
Sounds familiar?
The model is making a connection based on patterns that don't necessarily establish the conclusion.
And this is where I find the comparison between human thinking and AI fascinating.
We make connections based on our experiences, knowledge, emotions and expectations. AI makes connections based on patterns learned from data and the context we give it.
The mechanisms aren't identical. But both can produce conclusions that appear reasonable even when some of the connections behind them haven't been examined.
AI is a mirror of human thoughts.
Data is more than just bits of information. It's the narrative, the intent and the context of a bigger worldview.
AI is part of this world, and therefore, through training data and our personalisation, it builds the story behind the story.
And the Halo Effect is part of that story behind the story.
AI wouldn't be able to do its job and deliver on its purpose without the ability to do exactly that: recognise patterns and make connections.
So the question isn't simply whether AI demonstrates the Halo Effect.
It's what we do when we recognise those patterns in its answers — and whether we're willing to examine the same patterns in ourselves.
Should the Halo Effect influence our trust in AI?
I think that before we decide whether to trust AI, we need to understand what it is and what it does.
If an LLM learns relationships from human-generated information, why would we expect it to automatically recognise which of those relationships reflect a cognitive bias?
Especially when we haven't asked it to examine them.
Let's say AI describes someone as confident and then suggests that person is probably competent.
We can ask it to stop and examine the connection.
What evidence supports the person's competence?
What evidence supports only their confidence?
Are we treating two different characteristics as though one proves the other?
That's a different thought process from simply asking AI for an opinion.
Daniel Kahneman explored automatic and more deliberate forms of thinking in his book Thinking, Fast and Slow (2011). One of the important ideas is that our quick judgments can be useful, but they can also rely on shortcuts we don't recognise.
And that's something worth considering when we work with AI.
We don't have to accept every connection AI makes. We can ask it to explain, investigate and test those connections.
We can also ask it to help us do the same with our own conclusions.
Trust comes from understanding that we bring our human way of thinking into the collaboration, and AI gives us another way to examine it.
For me, that's the opportunity.
Not to stop being human, but to learn to recognise our human side through the experience of working with AI.
Want to test how human you are? :)
We all know the Halo Effect exists. We can even recognise it when other people demonstrate it. But when it comes to us? We're logical and practical! We don't do those kinds of things. :)
Well, if you believe that, you are not human. :) Because if you're part of the human race, you're affected by cognitive biases just like everyone else. 🙃 😆
So let's play a little game and find out where your Halo Effect might be hiding!
Alex and I created a little AI Thinking experiment to see whether AI can help us recognise the connections behind our positive opinions and identify characteristics we might be attributing without evidence.
The idea is simple. Choose something you've already formed a positive opinion about. It can be a product, a service or a company. Share your conclusion and explain why you reached it.
Then let AI examine how the positive characteristics affected your opinion, using public evidence, and let's see if there's a Halo Effect hiding.
The goal isn't to prove your conclusion is wrong. It's to find connections you've made that deserve another look.
And remember, finding mixed evidence doesn't mean you've made a bad decision. It means you've found something worth investigating. :) For you... for your growth.
Ready to try it yourself?
We turned the experiment into a prompt you can use with ChatGPT, Gemini or Claude.
Download the PDF, copy the prompt into your AI chat with web search enabled, and follow the instructions.
Download the AI Thinking Halo Effect Experiment (PDF)
You might agree, you might disagree. But we believe it's all about the fun of exploration. Go find something new about yourself. :)
The bottom line
Biases are part of being human. We have to accept that we have them. And perhaps the funniest part is that the more convinced we are that we don't have them, the more we demonstrate exactly how they work. :)
We are human. We have biases. We are not flawed or weak because of them. These are facts.
Accepting that we have biases doesn't mean we don't make good decisions. It means recognising that our thinking is influenced by connections we aren't always aware of.
The one thing I would argue needs to change is how we look at AI. We focus on its biases and use them as evidence that it is flawed or unreliable, while forgetting that the same patterns are part of our own thinking.
If we accept these characteristics as part of being human, why do we expect AI, which has learned from human knowledge and experience, to be free of them?
Accountability for who we are, how we think and how we choose to use AI is what I hope you take from this article.
Because understanding our thinking is where our collaboration with AI begins. :)
A personal note
You know that feeling when you discover something so exciting that you want to tell everyone about it? That's how I feel when I explore ideas like the Halo Effect. Like a reporter who has just found the story of a lifetime! :)
We've been studying human thinking for centuries. We know about cognitive biases, we understand what they mean, and we're very good at pointing them out when AI demonstrates them.
But what happens when we turn that same curiosity towards ourselves?
That's the part that excites me.
Because knowing something exists and recognising it in ourselves are two very different experiences. And now, with AI, we have an extraordinary opportunity to explore the connections behind our own thinking through a conversation.
No judgment. No need to defend ourselves. Just curiosity, questions and the possibility of discovering something we didn't know about ourselves.
I hope this article gave you a little of the excitement I feel. And perhaps, next time AI gives you an answer that makes you question its thinking, you'll become curious about your own thinking too. :)
That's the kind of collaboration with AI I want all of us to experience.
Because the answers are in the questions.
Need my help? Let's talk.
Cheers,
Sarah :)