Recently in a leadership meeting, we plunged into a conversation about how AI could help us communicate better, when something came up that I think deserves more attention.
If you haven’t seen it, here’s the short version: you ask it a medical question, or a complex strategic question, and instead of giving you one answer, it asks the same question to Claude, ChatGPT, Gemini, and Grok simultaneously. Then it shows you where they agree, where they diverge, and gives you a synthesis. This illuminated a really important idea (as it occurs to me).
Separately, AI’s will sometimes give you flatly contradictory answers about the same thing.
I said something in that meeting that I want to develop here, because I think it matters well beyond AI conversations.
Each AI source is a single occurrence.
And in our work, occurrences are things you test. Not trust.
Why Leaders Should Treat AI as an Occurrence, Not an Oracle
In Meta Performance, an occurrence is your perception of an event.
Your intuitive read on a person. It’s the story your brain constructs from incomplete data and fills in without telling you.
And occurrences feel like reality. They have the texture of truth. Your nervous system doesn’t label them “working hypothesis.” They just… run. Like a system based on “facts”.
But we know better. We’ve spent years teaching people that the occurrence you have of a situation is an invitation into investigation, not a conclusion.
You get curious about it.
You test it.
You ask…
Is what you’re seeing actually what’s there?
Or, are your history, assumptions, and fears building a story faster than the evidence warrants?
Thinking Fast and Slow is essentially a 500-page unpacking of this problem. I’ve read it twice and I have still only scratched the surface of what it reveals. Kahneman calls the phenomenon of trusting your occurrence WYSIATI: What You See Is All There Is.
System 1 grabs whatever’s in the room, builds the most coherent story it can from those inputs, and presents that story as the verdict. It doesn’t flag missing data. It doesn’t say “by the way, I only have about 30% of what I’d need to be confident here.”
It just tells you what’s “true”.
That’s our occurrence. Often wrong (or inadequately informed) and, the most dangerous part…
Confident.
The Problem With AI’s Confident Answers
When you ask Claude something, you get an answer. It’s well-written. Structured.
It’s confident.
And if you already lean toward trusting it, the halo effect (another frame from Thinking Fast and Slow) kicks in fast…
Well, it clearly remembers more than I do, so this must be right.
I said this in the meeting, and I want to say it again here: there’s a de-skilling risk here.
When you read an AI summary and accept it without stress-testing it, you’re doing the thing we warn coaches and clients against every week.
You’re treating an occurrence like a fact.
You’re letting a coherent-sounding story do the work of actual curiosity.
The difference is that with a human, you’d push back. You’d ask clarifying questions. You’d say,”Maybe”. You’d notice the tone of voice, the hesitation, the things not said. With AI, the delivery is clean every single time. There’s no hesitation. There’s no “I’m actually not sure about this part.”
Here’s the part that matters: AI will often appear to solve this problem. It cites sources. It flags confidence levels.
Sidekick (our proprietary AI Coaching Assistant), for instance, will tell you “high confidence” on a claim and be completely wrong. The confidence signal is itself an occurrence. It has the texture of verification without the substance of it. So the natural friction that would prompt your skepticism gets removed, and you go along with it.
That’s not a small thing. Over time, it atrophies a muscle that I, for one, do not want to weaken.
Why Disagreement Makes AI More Useful
What I love about the Council model isn’t the synthesis. It’s the conflict. The friction, the outliers.
When you see Claude say one thing and ChatGPT say the opposite, something important happens:
You realize these aren’t verdicts and you’re forced into the investigative posture.
You can’t just accept the answer, because there isn’t one answer. There’s a range of takes, and you get to do the work of deciding what’s actually true.
That’s a privilege, not a burden. It means your judgment still matters. It means the question isn’t settled until you settle it.That’s the right relationship (and, yes, I’m positioned on this) to have with AI.
And I’d argue it’s the right relationship to have with any source of input, including your own intuition.
In a meeting, when you bring 4 people with genuinely different lenses to a problem, you don’t get a clean answer.
You get friction, disagreement, and eventually something more thorough, nuanced and possibly helpful than any 1 of them would have reached alone.
The Council model mimics that to a degree. It re-distributes the authority of the conclusion back to you, rather than concentrating it in the AI.
One voice, no matter how capable, is one occurrence. Treating it as the final word is a mistake, whether that voice belongs to a luminary, a podcast, or a machine.
How AI Can Amplify the Halo Effect
There’s another dynamic at work here.
Daniel Kahneman wrote about the halo effect: our tendency to allow a positive impression of a person or source to influence how we evaluate everything else coming from that source.
We do this all the time.
We trust the podcast host, so we become a little less critical of the argument.
We respect the author, so we’re less likely to question the conclusion.
AI can turbocharge this tendency.
The tone is confident. The writing is polished. There’s no bad day, nervous energy, hesitation, or slip of the tongue that causes you to wonder:
Wait. How certain is this source, really?
Our natural skepticism has less to grab onto.
The antidote isn’t distrusting AI. The antidote is curiosity.
Ask:
What would have to be true for this answer to be wrong?
Ask:
What might be missing?
Ask:
What would someone who disagrees with this say?
And perhaps most importantly:
Does this match what I have actually seen?
Because you have something the machine doesn’t.
You were in the room.
How to Use AI Without Outsourcing Your Judgment
There’s an interesting idea in AI circles sometimes described as “loop maxing.” Instead of asking a question once and accepting the first answer, you build a loop.
Like this:
Ask → Get an answer → Interrogate the answer → Add context → Ask again → Look for contradiction. → Update → Repeat.
Interestingly, this is also what great coaching looks like.
You notice an occurrence. You test it. You update. You form another occurrence. You test that one.
The loop is the practice. The loop is the muscle.
AI can do this quickly and without a lot of attachment to its first answer, but humans have an additional problem.
We get attached to being right.
Once we’ve formed an opinion, changing it can feel like losing. So instead of remaining curious, we stop the loop early.
The best leaders and coaches I know learn to stay in that loop longer.
They become willing to revise and they keep asking questions even when those questions threaten their original conclusion.
Why Human Judgment Matters More in the Age of AI
Think about how you develop intuition in any domain. You get reps. You see a lot of situations, form a lot of reads, check yourself against your vision, against other factors and data, and you have a chance to recalibrate.
Over time, your occurrence gets sharper. You start seeing things that others don’t.
And here’s what AI cannot replicate: sharpness.
Your read of the room. The thing you notice in the second minute of a conversation that changes everything. The pattern you recognize because you’ve been in 400 rooms like this one.
Machines process information. Humans accumulate wisdom.
Those are not the same thing, and right now, in this particular moment in the rise of this technology, your accumulated wisdom is more valuable, not less.
AI can’t do that for you. And if you outsource too much of the reading-the-room work, the global listening, or the inquiry, you stop training. Your occurrences stay where they were, or (maybe worse) they drift toward whatever the machine is telling you to see.
So the practical commitment isn’t just “use multiple AI sources.” It’s also “keep building your curiosity.”
A Practical Framework for Using AI as a Leader
A few ideas to help:
1. Before you query, form your own hypothesis. (Even rough. Even uncertain.) Write a sentence. Then bring the AI in.
Now you have 2 occurrences to compare: yours and the machine’s.
If they diverge? Well then that divergence is the most interesting thing in the room.
When the AI says something surprising, get curious before you comply.
- Ask it to show its reasoning.
- Ask what it might be missing.
- Ask what the other side of that claim looks like.
2. When the stakes are high, ask more than one AI. The disagreements will tell you more than the consensus.
3. And maybe most important: treat your intuition as data worth integrating. You have context the AI doesn’t. You were in the room. You know the history. That’s not nothing.
AI is one of the most useful tools I’ve ever heard of. But a tool of such historic prowess is a tool you can stop thinking around.
And the day you stop thinking around your tools is the day the tools start thinking for you.
That’s not a future I want to build. For the Institute, for our firm, or for our clients.
So use AI.
Ask it hard questions.
Let it synthesize enormous amounts of information and challenge what you think you know.
But don’t hand it authority it can’t possibly earn.
ADDITIONAL RESOURCES FROM NOVUS GLOBAL
- The Real Risk of AI Dependency: It’s Not the Technology
- Work with a Novus Global Coach: Ready to explore what you’re actually capable of?Join us for a Free Discovery Call.
- Become a Coach: Learn more about the Meta Performance Institute for Coaching at www.mp.institute.
- Learn more about the author: Joseph King Barkley