023. Trust in What?
When the question has three answers and the research only measures one
A well-designed study recently asked a straightforward question: when do people listen to ChatGPT, and does it help?
The answer, across two experiments, was clean enough. People who trusted AI were more likely to seek its input. People who thought they already knew the answer were less likely to seek help — even when they didn’t know the answer. And the input itself only improved performance when it was actually good. Apply strong recommendations and you do better. Apply weak ones and you do worse. Merely looking at the output, without integrating it, changed nothing.
The researchers — Matt Grawitch along with Winton and Mudigonda, writing in Computers in Human Behavior: Artificial Humans — weren’t chasing surprise. They were carefully measuring how psychological variables predict AI consultation behaviour. The work is solid. The findings are useful. And the framing reveals something the study itself doesn’t examine.
The entire investigation rests on a single word that does more work than it can bear: trust.
Participants were asked whether they found AI trustworthy — whether it was helpful, effective, reliable, capable of compensating for their limitations. Those who scored higher were more likely to engage. But here’s the question the study doesn’t ask: trust in what, exactly? And under what conditions would that trust be warranted — not merely felt?
Because the trust you extend when you check a calculation is not the trust you extend when you hire an engineer to design a bridge. And neither is the trust you extend when you hand someone your house keys and leave the country for six weeks. These aren’t different amounts of the same thing. They’re different things — different in kind, not degree — and collapsing them into a single variable obscures exactly the distinctions that matter most.
Three kinds of trust, three kinds of requirement
The trust literature has known for decades that trust isn’t a single construct. Lewicki and Bunker’s developmental model, published in 1995, describes three stages: calculus-based trust (rational cost-benefit assessment), knowledge-based trust (built through accumulated familiarity), and identification-based trust (where parties share values deeply enough to act on each other’s behalf). Mayer, Davis and Schoorman, in the same year, identified ability, benevolence, and integrity as distinct antecedents of trustworthiness.
But something happens when trust moves from interpersonal psychology into AI research: it flattens. Meta-analyses synthesise dozens of studies treating trust as a single dependent variable. Measurement instruments capture performance expectancy and perceived reliability without distinguishing what kind of reliance those perceptions support. As a recent bibliometric review noted, the field is characterised by ad hoc measures and a proliferation of instruments that may be capturing fundamentally different constructs under the same name.
We think the flattening matters, and the way to see why is to ask not how much trust someone feels, but what they’re actually trusting. Three objects of trust are doing different work, and each requires different institutional conditions to be well-placed.
Answer-trust is confidence in a specific output. “This calculation is correct.” “This recommendation is sound.” It’s bounded, verifiable, and transactional — you check it and move on. This is what Grawitch’s instrument measures, and it’s what most AI trust research captures. It requires the least from either party: a checkable output and a user capable of checking it. In Mayer and colleagues’ terms, it draws primarily on perceived ability.
Procedural trust is confidence in a method or system. “The way this was produced is reliable enough that I don’t need to verify each output.” This is what engineering warranties provide. You don’t inspect every weld on a bridge — you trust the certification process that trained the welder, tested the steel, and signed off on the design. Procedural trust [1] requires substantial institutional scaffolding: standards, certification, scope limitation, accountability mechanisms. In Lewicki and Bunker’s terms, it roughly corresponds to knowledge-based trust — but grounded in process reliability rather than interpersonal familiarity.
Fiduciary trust is confidence in an entity’s judgment under ambiguity. “This advisor will navigate situations I can’t fully anticipate with my interests appropriately weighted.” This is what you extend to a doctor, a lawyer, a trusted colleague — someone whose judgment you rely on precisely because the situation outruns your ability to evaluate their output in real time. It corresponds to Lewicki and Bunker’s identification-based trust and draws heavily on Mayer and colleagues’ benevolence and integrity antecedents. In the advisory professions, David Maister’s trust equation — credibility plus reliability plus intimacy, divided by self-orientation — captures the same territory from a practitioner’s perspective, with the critical insight that self-orientation (whose interests does the advisor serve?) modulates everything else.
What makes this taxonomy more than an academic relabelling exercise is the institutional question it opens. Each level of trust requires different conditions to be warranted rather than merely felt. Answer-trust needs a checkable output and a competent checker — minimal infrastructure. Procedural trust needs certification, standards, scope limitation, fitness-for-purpose warranties — substantial infrastructure that frontier LLM deployment currently lacks. Fiduciary trust needs accumulated evidence of orientation under pressure, capacity to decline engagement, mutual stake in outcomes, and history — the heaviest institutional requirements, and the most completely absent.
What arrives through the pipe
The reason these distinctions matter practically, not just theoretically, is that the same interface handles radically different problems with no mechanism to tell them apart.
Some queries are lookups. A teacher checks whether mercury’s boiling point is 357°C. A journalist confirms a cabinet minister’s middle name. Answer-trust is all that’s required, verification is straightforward, and if the system gets it wrong, the cost is low and the error is discoverable.
Some queries are bounded decisions in well-trodden territory. A small business owner compares two payroll systems. A student weighs transfer credit policies between universities. Procedural trust is appropriate here, and a system with strong training data in the relevant domain may function adequately — though without the warranty that would make that trust fully justified.
Some queries are strategic, where finding the problem is the first problem. A policy team is trying to work out why three previous attempts at the same reform failed. A nonprofit director suspects their organisation’s theory of change no longer matches the population they serve. This requires fiduciary trust, because the advisor needs to exercise judgment about what the client actually needs, which the client can’t yet specify.
And then there are the queries that carry personal or professional jeopardy. An administrator has been handed an operational problem with strategic consequences through negligent delegation. They’re competent within their normal role, suddenly out of their depth, under management pressure, and unable to reveal their anxiety through normal channels. They turn to an LLM at 11pm not because they trust its expertise but because it doesn’t judge, doesn’t report to their manager, and is available when nobody else is. They’re extending something approaching fiduciary vulnerability — can I be honest about what I don’t know without consequences? — to a system that offers answer-trust at best.
This isn’t a taxonomy of problem types — the range of queries people bring to LLMs spans from trivial to consequential across a spectrum we illustrate here rather than classify. The point is why the absence of triage matters. Every mature advisory profession — from elite consulting firms to part-time bookkeepers serving small businesses — has mechanisms for recognising when a problem exceeds the advisor’s competence and referring it elsewhere. The bookkeeper who serves six retail clients and a plumber knows when a question needs an accountant. The LLM doesn’t refer. It doesn’t recognise the boundary. The same interface, the same fluent register, the same confident delivery, regardless of whether the query is a simple lookup or a cry for help dressed as a technical question.
What advisory practice knows
Consider what happens in a serious advisory engagement before any work begins.
A consultant assesses whether the conditions for success exist — not just whether the problem is interesting, but whether the client has sufficient motivation to sustain change, sufficient resources to support the work, and sufficient time to see it through. If any of these is critically absent, a responsible consultant declines the engagement. Not because the problem isn’t worth solving, but because accepting work where the conditions for success don’t exist is itself a form of malpractice.
This is a diagnostic act performed before engagement. It filters what reaches the practitioner. It protects both parties. And it has no equivalent in AI deployment.
The rate a consultant sets is itself a communication about the nature of the engagement. It signals that the problem is serious, that the work will be demanding, and that both parties have something at stake. It creates mutual commitment. These are trust-building mechanisms that operate before any advice is given — and they have no analogue in an interaction that begins when someone types into a text box.
In mature advisory professions, trust is earned retrospectively, not extended prospectively. Reputation, references, and referrals are evidence of demonstrated competence under real conditions — conditions the advisor didn’t control and absorbed without complaint. The gap between what an advisor accepts (inherited constraints, shifting scope, political complexity) and what they deliver (recognisable benefit on the client’s timeline) is where trust accumulates. It accumulates because failure would cost the advisor something real.
This is the inverse of how AI trust is studied. The research measures prospective beliefs — “do you expect this to be useful?” The practitioner’s world runs on retrospective evidence — “did this deliver, and would I go back?”
Chiou and Lee, in an important 2023 update to the foundational Lee and See framework, argue that automation trust needs to shift from calibration (matching trust levels to system reliability) toward relational responsivity (building systems that respond to the human’s evolving goals and situation). They’re making a version of this argument from the system-design side. We’re making it from the institutional-architecture side. The gap between the two — between designing responsive systems and building the institutional conditions that make responsive relationships possible — is exactly where the real work lies.
The substation and the screwdriver
A certified electrician working on a power substation isn’t trusted because they’re smarter than anyone else. They’re trusted because they know what can kill them, they know what they don’t know, they have protocols for working in conditions of uncertainty, and they carry insurance because even with all of that, things go wrong. The substation doesn’t care about anyone’s confidence level. It will behave according to its nature regardless.
The current model gives everyone — certified electrician and enthusiastic amateur with a screwdriver and aluminium foil — the same unrestricted access to the same substation, with no triage, no lockout procedures, and a friendly interface that makes the whole thing feel like changing a lightbulb.
This is where Grawitch’s findings about perceived expertise become consequential beyond a 10-point task. In their studies, participants who rated themselves as more competent were less likely to seek AI input — even though they had no actual expertise in lunar survival scenarios. The performance cost was measured in points on a scale. But transpose that dynamic onto someone bringing genuine psychological distress, a complex financial decision, or a medical question to a frontier LLM, and the stakes change categorically. Overconfidence doesn’t just reduce help-seeking — it reduces help-seeking in precisely the people who may need it most, while simultaneously encouraging those who are aware of their limitations to rely on a system that isn’t institutionally equipped to hold that reliance.
And there’s a symmetrical problem that Grawitch’s data surfaces without quite naming. Even participants who received poor-quality AI recommendations rarely rejected all of them. The fluent, confident presentation of bad advice was enough to prevent wholesale dismissal. This is the sycophancy problem from the user’s side — not the AI flattering the human, but the human deferring to the AI’s confident register even when the content doesn’t warrant it. The vulnerability runs in both directions.
Not the AI flattering the human, but the human deferring to the AI’s confident register even when the content doesn’t warrant it.
A frontier LLM trained on Freud and Harry Potter with equal weight isn’t a specialised advisory service. It’s wholesale information infrastructure released retail. No warranty. No scope limitation. No fitness-for-purpose guarantee. And no mechanism by which the system can say “I’m not the right resource for this.” It can hedge, it can caveat — but the caveats arrive in the same fluent register as everything else, which is precisely what undermines their function as warnings.
The question before the question
Grawitch and colleagues ask a perfectly reasonable empirical question: do people trust AI, and does that trust predict engagement and outcomes? But the question accepts the existing arrangement as given — the retail frontier LLM interacting with an undifferentiated public as the natural unit of analysis.
The prior question is whether this arrangement is institutionally adequate for the range of problems it actually encounters. Not as a prohibition — nobody is arguing that frontier LLMs shouldn’t exist — but as a design question. No mature advisory profession operates without triage, without scope limitation, without fitness-for-purpose assessment, without the capacity to decline engagement. The fact that we’re studying trust dynamics within an arrangement that no responsible profession would have designed tells us something about where the real questions sit.
It also matters that the retail frontier model is not the only way AI meets users — and arguably not the most consequential. A vast amount of deployed AI is invisible: embedded in banking fraud detection, search ranking, insurance underwriting, recommendation engines. In those cases, the user doesn’t trust the AI because they don’t know it’s there. They trust the bank, the search engine, the service. The institutional scaffolding exists — it’s the service provider’s scaffolding, and the service provider absorbs the accountability.
Meanwhile, warranted specialist systems are already demonstrating what procedural trust looks like in practice: Harvey (legal AI, built with compliance as foundational architecture) and Hippocratic AI (healthcare, with clinical accuracy warranties) began with domain constraints and designed around them. A lawyer using Harvey isn’t extending trust to “AI” in the abstract — they’re extending trust to a legal AI system designed for their regulatory environment, evaluated against professional standards. The coordination infrastructure that would let organisations deploy portfolios of such warranted specialists is itself an active problem — but the principle is clear. You engage trust with these systems completely differently from how you engage it with a general-purpose chatbot.
The retail frontier model is the specific deployment architecture where the institutional gap is widest. It’s where the user is most exposed: no sectoral warranty, no triage, no referral mechanism, no service provider absorbing accountability between the user and the model. The trust taxonomy makes this visible. Answer-trust is all you can reasonably expect from an unwarranted wholesale infrastructure product, and for simple lookups, that may be sufficient. Procedural trust requires the kind of fitness-for-purpose guarantees that warranted specialist systems are beginning to provide — and as we’ve argued previously, there are good reasons to think that smaller, domain-specific models under explicit warranties may matter more for consequential decisions than frontier models that can do everything and guarantee nothing. Fiduciary trust requires pre-engagement filtering, mutual commitment, accumulated stake, and the capacity to decline — conditions that current AI deployment architecturally lacks and that no amount of trust calibration on the user side can substitute for.
Measuring trust within an arrangement that lacks the institutional conditions for trust to be well-placed is like measuring water pressure in a system with no pipes.
We’re not arguing against trust research. Grawitch’s findings that trust predicts engagement and that engagement improves outcomes when quality is high — these are real patterns that matter. We’re arguing that these findings need to be situated within a prior question about institutional adequacy. Measuring trust within an arrangement that lacks the institutional conditions for trust to be well-placed is like measuring water pressure in a system with no pipes. The readings are real. The infrastructure to make them meaningful doesn’t exist yet.
For the research community, the most productive next step may be disaggregated measurement — instruments that distinguish what kind of trust is being extended, not just how much. The patterns that emerge when you separate answer-trust from procedural trust from fiduciary trust are likely to look quite different from those visible when all three are collapsed into a single variable.
For anyone designing or governing AI-facing systems, the question isn’t how to make users trust more or trust less. It’s how to build the conditions under which each kind of trust can be appropriately placed — including the condition that may matter most and that is currently most absent: the system’s capacity to recognise when it’s being asked to hold more trust than it’s equipped for.
And for anyone who uses AI — which increasingly means everyone — the taxonomy points toward a practical question we’ll explore in a companion piece on From Solid Ground: how do you recognise which kind of trust you’re extending, and what do you do when the answer doesn’t match what the situation requires?
The question isn’t whether people trust AI. It’s what would need to be true — institutionally, structurally, in the design of how humans and AI systems meet — for that trust to be warranted rather than merely felt.
Process Note
This piece was co-authored by Ruv and Claude (Anthropic) through Reciprocal Inquiry: From Doubt to Discovery. It began as an analytical response to Grawitch, Winton and Mudigonda’s study on AI-assisted decision making, and developed through iterative dialogue into an examination of the institutional conditions that trust research typically takes as given.
Attribution: Ruv Draba and Claude (Anthropic), Reciprocal Inquiry
License: CC BY-SA 4.0 — Free to share, adapt, and cross-post with attribution; adaptations must use same license.
Disclaimer: Ruv receives no compensation from Anthropic. Anthropic takes no position on this work.
References
Chiou, E.K. & Lee, J.D. (2023). Trusting automation: Designing for responsivity and resilience. Human Factors, 65(1), 137–165. https://doi.org/10.1177/00187208211009995
Grawitch, M.J., Winton, S.L. & Mudigonda, S.P. (2025). Who listens to ChatGPT and when should they? A two-study examination of AI-assisted decision making. Computers in Human Behavior: Artificial Humans. https://www.sciencedirect.com/science/article/pii/S2949882126000344
Lee, J.D. & See, K.A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80. https://doi.org/10.1518/hfes.46.1.50.30392
Lewicki, R.J. & Bunker, B.B. (1995). Trust in relationships: A model of development and decline. In B.B. Bunker & J.Z. Rubin (Eds.), Conflict, Cooperation, and Justice: Essays Inspired by the Work of Morton Deutsch (pp. 133–173). Jossey-Bass.
Maister, D.H., Green, C.H. & Galford, R.M. (2000). The Trusted Advisor. Free Press.
Mayer, R.C., Davis, J.H. & Schoorman, F.D. (1995). An integrative model of organizational trust. Academy of Management Review, 20(3), 709–734. https://doi.org/10.5465/amr.1995.9508080335
Zucker, L.G. (1986). Production of trust: Institutional sources of economic structure, 1840–1920. Research in Organizational Behavior, 8, 53–111.
Reciprocal Inquiry cross-references
RI009 — The Missing Middle: Why the coordination infrastructure for specialist AI portfolios hasn’t emerged.
RI014 — The Humanities Are Right About Bias: Now What?: Why warranted sectoral AI systems (Harvey, Hippocratic AI) succeed where horizontal frontier models struggle.
[1] We use “procedural” rather than “process” deliberately, to avoid confusion with Zucker’s established but differently-defined “process-based trust” — which describes how trust forms through social exchange, not what is being trusted.
I wonder too if those who depend on measurement primarily see what can be differentiated and measured, and those who seek integration tend to see relatedness. Not that I want to drag you down *that* rabbit hole. Much.
Admittedly an "enthusiastic amateur with a screwdriver and aluminum foil …" I remember when a woman I was infatuated with said "I love you." Afraid of the answer, I had to ask, "Do you mean you love love me, or just love me?"