My father had a line for this, and most people’s fathers had a version of it. If all your friends jumped off the Brooklyn Bridge, would you?
I thought of it this week while reading three sets of numbers that describe the same organizations from three different angles — and produce answers that cannot all be right.
I want to be careful here, because the easy version of this argument is wrong and I am not making it. The surveys are not fake. The firms producing them are not lazy, and they are not conspiring. The methodology is generally sound, the samples are large, and the people doing the work are competent.
The problem is structural, and it survives everyone doing their job properly.
What the numbers actually measure
Take four recent findings, all published within sixty days.
84 percent of organizations have not redesigned jobs or workflows around AI. That is executives reporting on their own organizations.
Around 1 percent consider themselves fully mature. Self-assessment.
74 percent believe they could pass an AI compliance audit today, while 27 percent describe their governance programs as fully mature. Both figures from the same respondents, about themselves.
Half of executives say they regularly verify the quality of AI outputs. Self-report again.
Every one of those is a genuine finding. And every one measures the same thing: what senior people believe about the organization they are running.
That is worth knowing. It is not the same as what the organization is doing, and the gap between the two is the entire subject of my working life.
The control group
Here is what makes the point concrete rather than theoretical.
Statistics Canada asked the workers.
Not executives. Not a vendor panel screened for people already familiar with AI tools. A national statistical agency, using supplementary questions to the Labour Force Survey, asking people what they actually did in their main job.
The answer: 35.9 percent of Canadian workers used generative AI in the twelve months to March 2026.
Over roughly the same period, 12.2 percent of Canadian businesses reported using AI at all, and 66.7 percent said they had no plans to adopt it.
Three workers using it for every business that has adopted it.
Statistics Canada’s own note, in an earlier release comparing the two series, is that this pattern may indicate that adoption is not solely firm-led.
That is a national statistical agency, with no product to sell, observing that the workforce is running ahead of the firms.
Same country. Same period. Same phenomenon. Ask the executive and you get one number. Ask the worker and you get three times that number.
Neither survey is wrong. They are measuring different people, and only one of those groups is doing the work.
Which is the whole of it. These surveys are accurately flawed. The measurement is sound and the picture is partial — and the flaw cannot be detected from inside the instrument, because the instrument only reports what somebody thought to ask.
Where the echo actually is
This is the part I want to state precisely, because “echo chamber” is usually thrown around as an accusation and I mean something narrower and more mechanical.
Research firms survey senior executives. The findings are published. Those findings become the frame within which the next round of questions gets written. And the executives answering the next round have read the previous round.
Nobody is coordinating anything. The loop closes on its own.
You can see it in the question wording. Have you redesigned jobs or workflows around AI? is a reasonable question. It also embeds a premise — that redesigning around AI is the objective — and every answer, yes or no, confirms the premise. The respondent is placed on a line whose direction has already been chosen.
A survey can tell you where an organization sits on a line. It cannot tell you whether the line is pointed the right way, because the line is the instrument.
What this costs
Here is the practical consequence, and it is not academic.
If your information about your own industry comes primarily from research assembled by asking people like you what they think, then your picture of the problem is an average of what your peer group already believes.
That is genuinely useful for some purposes. It tells you whether you are ahead or behind, which questions are being asked, where budget is moving. Benchmarking is real, and I have used it.
But it cannot surface a cause that nobody in the sample has thought to look for. And the causes that matter most are almost always in that category — because if they were visible from the executive suite, they would have been addressed already.
Which is my father’s bridge, more or less.
The four o’clock
In 1998 I worked on a defense maintenance operation delivering parts on time 51 percent of the time against a 90 percent requirement.
If you had surveyed that organization’s leadership, you would have received accurate answers. They understood the problem. Procurement was underperforming. Supplier reliability was inconsistent. The process documentation was current and everyone had read it.
Every one of those answers would have been given in good faith, and every one would have been useless.
The question that changed the outcome was not on any survey, and would never have been:
What time of day do orders come in?
Late afternoon. Almost all of them, around four o’clock — because service technicians were holding order releases and batching them at the end of the day. Releasing orders as they arose interrupted service calls, and technicians were measured on call volume.
A rational response to how they were being measured. One department away from the people being blamed. And in nobody’s answer to any question anyone would have thought to ask.
Delivery performance moved from 51 percent to 97.3 percent in three months. No new system. The technology was selected afterward, once the result was substantially in hand.
Nobody surveyed the four o’clock. You cannot. It is not something anyone knows to report — it is something somebody has to go and observe.
Genchi genbutsu
Toyota has a term for this and it is sixty years old. 現地現物 — go to the actual place, see the actual thing.
Taiichi Ohno’s version was to stand on the shop floor and watch until you saw what was actually happening, rather than what the process documentation said was happening. Not because the documentation was dishonest. Because a description of work and the work itself are different objects, and only one of them can be observed.
That is not a criticism of surveys. It is a statement about what class of instrument a survey is.
A survey collects reports. Observation collects behavior. When those agree, the survey is efficient and you should use it. When they diverge, the survey will not tell you they have diverged — because a survey has no channel for reporting the thing nobody was asked about.
What I would actually do
Read the research. It is well made, it is mostly free, and it will tell you things about your industry you cannot see from inside one company.
Then treat every figure in it as an answer to a question somebody chose to ask.
And before you act on any of it, spend a week finding out one thing about your own operation that nobody has reported to you: when do things actually happen, who actually decides, and what are people actually measured on?
Not from the process map, which returns the declared version, because that is what it was built to capture. By going and looking.
If what you find matches the documentation, you have lost a week and gained certainty, which is a good trade.
If you find a four o’clock — and most operations have one — you have found something no survey in your industry contains, about the only organization you can actually change.
The good that industry research does is real. What it cannot do is see the thing nobody thought to ask about — and in my experience, across nearly three decades of contemporaneous frontline engagement as a practitioner, provider, and analyst, that is nearly always where the answer is.
-30-
Truth Is Believing. Accuracy Is Knowing. Outcome Is Proof.™
Related
Nobody Surveyed the Four O’Clock, Or: Knowing That Everybody Jumped Is a Fact, Not a Reason
Posted on August 7, 2026
0
My father had a line for this, and most people’s fathers had a version of it. If all your friends jumped off the Brooklyn Bridge, would you?
I thought of it this week while reading three sets of numbers that describe the same organizations from three different angles — and produce answers that cannot all be right.
I want to be careful here, because the easy version of this argument is wrong and I am not making it. The surveys are not fake. The firms producing them are not lazy, and they are not conspiring. The methodology is generally sound, the samples are large, and the people doing the work are competent.
The problem is structural, and it survives everyone doing their job properly.
What the numbers actually measure
Take four recent findings, all published within sixty days.
84 percent of organizations have not redesigned jobs or workflows around AI. That is executives reporting on their own organizations.
Around 1 percent consider themselves fully mature. Self-assessment.
74 percent believe they could pass an AI compliance audit today, while 27 percent describe their governance programs as fully mature. Both figures from the same respondents, about themselves.
Half of executives say they regularly verify the quality of AI outputs. Self-report again.
Every one of those is a genuine finding. And every one measures the same thing: what senior people believe about the organization they are running.
That is worth knowing. It is not the same as what the organization is doing, and the gap between the two is the entire subject of my working life.
The control group
Here is what makes the point concrete rather than theoretical.
Statistics Canada asked the workers.
Not executives. Not a vendor panel screened for people already familiar with AI tools. A national statistical agency, using supplementary questions to the Labour Force Survey, asking people what they actually did in their main job.
The answer: 35.9 percent of Canadian workers used generative AI in the twelve months to March 2026.
Over roughly the same period, 12.2 percent of Canadian businesses reported using AI at all, and 66.7 percent said they had no plans to adopt it.
Three workers using it for every business that has adopted it.
Statistics Canada’s own note, in an earlier release comparing the two series, is that this pattern may indicate that adoption is not solely firm-led.
That is a national statistical agency, with no product to sell, observing that the workforce is running ahead of the firms.
Same country. Same period. Same phenomenon. Ask the executive and you get one number. Ask the worker and you get three times that number.
Neither survey is wrong. They are measuring different people, and only one of those groups is doing the work.
Which is the whole of it. These surveys are accurately flawed. The measurement is sound and the picture is partial — and the flaw cannot be detected from inside the instrument, because the instrument only reports what somebody thought to ask.
Where the echo actually is
This is the part I want to state precisely, because “echo chamber” is usually thrown around as an accusation and I mean something narrower and more mechanical.
Research firms survey senior executives. The findings are published. Those findings become the frame within which the next round of questions gets written. And the executives answering the next round have read the previous round.
Nobody is coordinating anything. The loop closes on its own.
You can see it in the question wording. Have you redesigned jobs or workflows around AI? is a reasonable question. It also embeds a premise — that redesigning around AI is the objective — and every answer, yes or no, confirms the premise. The respondent is placed on a line whose direction has already been chosen.
A survey can tell you where an organization sits on a line. It cannot tell you whether the line is pointed the right way, because the line is the instrument.
What this costs
Here is the practical consequence, and it is not academic.
If your information about your own industry comes primarily from research assembled by asking people like you what they think, then your picture of the problem is an average of what your peer group already believes.
That is genuinely useful for some purposes. It tells you whether you are ahead or behind, which questions are being asked, where budget is moving. Benchmarking is real, and I have used it.
But it cannot surface a cause that nobody in the sample has thought to look for. And the causes that matter most are almost always in that category — because if they were visible from the executive suite, they would have been addressed already.
Which is my father’s bridge, more or less.
The four o’clock
In 1998 I worked on a defense maintenance operation delivering parts on time 51 percent of the time against a 90 percent requirement.
If you had surveyed that organization’s leadership, you would have received accurate answers. They understood the problem. Procurement was underperforming. Supplier reliability was inconsistent. The process documentation was current and everyone had read it.
Every one of those answers would have been given in good faith, and every one would have been useless.
The question that changed the outcome was not on any survey, and would never have been:
Late afternoon. Almost all of them, around four o’clock — because service technicians were holding order releases and batching them at the end of the day. Releasing orders as they arose interrupted service calls, and technicians were measured on call volume.
A rational response to how they were being measured. One department away from the people being blamed. And in nobody’s answer to any question anyone would have thought to ask.
Delivery performance moved from 51 percent to 97.3 percent in three months. No new system. The technology was selected afterward, once the result was substantially in hand.
Nobody surveyed the four o’clock. You cannot. It is not something anyone knows to report — it is something somebody has to go and observe.
Genchi genbutsu
Toyota has a term for this and it is sixty years old. 現地現物 — go to the actual place, see the actual thing.
Taiichi Ohno’s version was to stand on the shop floor and watch until you saw what was actually happening, rather than what the process documentation said was happening. Not because the documentation was dishonest. Because a description of work and the work itself are different objects, and only one of them can be observed.
That is not a criticism of surveys. It is a statement about what class of instrument a survey is.
A survey collects reports. Observation collects behavior. When those agree, the survey is efficient and you should use it. When they diverge, the survey will not tell you they have diverged — because a survey has no channel for reporting the thing nobody was asked about.
What I would actually do
Read the research. It is well made, it is mostly free, and it will tell you things about your industry you cannot see from inside one company.
Then treat every figure in it as an answer to a question somebody chose to ask.
And before you act on any of it, spend a week finding out one thing about your own operation that nobody has reported to you: when do things actually happen, who actually decides, and what are people actually measured on?
Not from the process map, which returns the declared version, because that is what it was built to capture. By going and looking.
If what you find matches the documentation, you have lost a week and gained certainty, which is a good trade.
If you find a four o’clock — and most operations have one — you have found something no survey in your industry contains, about the only organization you can actually change.
The good that industry research does is real. What it cannot do is see the thing nobody thought to ask about — and in my experience, across nearly three decades of contemporaneous frontline engagement as a practitioner, provider, and analyst, that is nearly always where the answer is.
-30-
Truth Is Believing. Accuracy Is Knowing. Outcome Is Proof.™
Share this:
Like this:
Related