Product × Finance × AI: Financial Products in the Age of AIPart Two Users: what people want and what they can bear中文
Chapter 18

How users judge and trustTrust is a belief that updates with evidence

For A to trust B is for A to believe that B will act in A's interest even when A cannot monitor B. This is a belief, and it updates with evidence. Before there is any record, the belief comes from a prior: institutional safeguards, the brand and what others say. After that, every act of fulfilment is a piece of evidence. How far a piece of evidence moves the belief depends on its diagnostic power, that is, on how different the probabilities are of seeing it from a reliable trustee and from an unreliable one. The difficulty in finance is that the evidence people use most often, short-term returns, has almost no diagnostic power. What does have diagnostic power is the process record and serious failures, which also explains why trust is built slowly and lost quickly. Beyond being a belief, trust is also a decision: the larger the stakes, the stronger the belief required. Good trust is trust that matches how trustworthy the other party actually is.

1. Trust is a belief, and also a decision

Mayer, Davis and Schoorman (1995) separated two things. Trustworthiness is the principal's judgement of the trustee; trust is the principal's decision, on the basis of that judgement, to accept risk. The first can be written as a probability: how likely the trustee is to act in the principal's interest when no one is watching. The second is a choice made on top of that probability, and its threshold depends on the stakes. To ask someone to look up a figure, fifty per cent confidence may be enough; to hand over all of one's savings for discretionary management, ninety per cent may not be. The same belief can be sufficient for a small matter and insufficient for a large one, and how high the threshold sits depends in turn on how large a loss the principal can bear.

Whether the same belief is enough depends on the stakes

Drag the principal's belief about the trustee: the probability that the trustee acts in the principal's interest when no one is watching. The four tasks carry different stakes and so need different thresholds; switch the principal's capacity to bear loss.

Illustrative thresholds. The distinction between trustworthiness (a belief) and trust (the decision to accept risk) follows Mayer, Davis and Schoorman (1995).

2. What is being estimated: ability, benevolence and integrity

What the principal has to estimate is which kind of trustee this is. Mayer and colleagues argue that trustworthiness has three components: ability, whether the trustee has the skill to do the job; benevolence, whether the trustee cares about the principal's interest; and integrity, whether the trustee keeps to principles both sides accept and acts as it says. All three are needed. An adviser with great ability whose real aim is to sell, and a friend who sincerely wants the best for you but lacks expertise, are equally unfit to be entrusted with a retirement fund. Ability must also be estimated task by task: being good at research does not mean being good at executing trades.

The three components of trustworthiness

Pick a trustee, or set ability, benevolence and integrity yourself, and see whether a retirement fund could be entrusted to it.

How far it can be trusted

The three components follow Mayer, Davis and Schoorman (1995). "How far it can be trusted" is the product of the three, for illustration: if any one component is very low, the whole is very low.

3. Prior and update: how the record changes belief

With no record at all, the principal relies on a prior: whether the trustee is regulated, whether assets are held in independent custody, what its brand and reputation are. Each observed act of fulfilment then adjusts the belief once by Bayes' rule: posterior = prior × probability of the evidence if the trustee is reliable ÷ total probability of the evidence. This is how reputation theory describes the formation of trust (Diamond, 1989), and online platforms' reputation systems often use a Beta distribution to accumulate positive and negative records (Jøsang and Ismail, 2002). A trustee with a strong prior starts high and needs less evidence. A new trustee, such as a newly arrived AI agent, starts low, and every piece of evidence matters more.

Belief updates with evidence

The trustee is in fact reliable. Choose the prior and the type of evidence, observe the record of fulfilment step by step, and see how quickly the belief crosses the threshold needed for discretionary management; then try a serious failure and a model version update.

Illustrative model with two hypotheses, reliable and unreliable. Annual returns: a reliable trustee has a good year 55% of the time, an unreliable one 50%. Process records (fees, constraints, an account of each period): a reliable trustee keeps to the agreement in 97% of periods, an unreliable one 70%. Serious failure: 0.2% for a reliable trustee, 20% for an unreliable one. Old records are discounted at each step, keeping 90%; at a model version update, the belief falls halfway back to the prior. The approach follows Diamond (1989) and Jøsang and Ismail (2002).

4. The diagnostic power of evidence: which records are useful

How far a piece of evidence moves the belief depends on the likelihood ratio: the probability of seeing it from a reliable trustee divided by the probability of seeing it from an unreliable one. Evidence with a likelihood ratio close to 1 carries almost no information. The evidence most often used in finance is exactly of this kind: short-term returns are driven mainly by luck, and good and bad trustees alike can deliver a fine year. Sirri and Tufano (1998) found that money flows heavily into funds that ranked near the top the previous year; Jain and Wu (2000) found that funds that advertised attracted more money but did not perform better than their peers afterwards. Evidence with high diagnostic power is evidence on which reliable and unreliable trustees differ sharply: whether fees were kept to the agreement, whether constraints were respected, whether every period was accounted for, and whether ability-related failures were reported truthfully.

The diagnostic power of evidence

The probability of each piece of evidence from a reliable and from an unreliable trustee, and the ratio of the two (the likelihood ratio). The centre line is 1: the closer to it, the less useful the evidence. Tap a row for an explanation.

Tap a piece of evidence.

Illustrative probabilities, showing relative size only. Evidence that money chases past performance and advertising is from Sirri and Tufano (1998) and Jain and Wu (2000); Jain and Wu (2000) found that advertised funds did not perform better than their peers afterwards, so the likelihood ratios of past returns and advertising are close to 1.

5. Why trust is built slowly and lost quickly

Slovic (1993) observed that building trust takes a long record of good conduct, while a single negative event can destroy it quickly. Bayesian updating produces this asymmetry directly. A good act of fulfilment is common among reliable and unreliable trustees alike, so its likelihood ratio is close to 1 and the belief rises only a little. A serious failure is very rare among reliable trustees, so its likelihood ratio is large and the belief falls sharply. The response after a failure is also evidence, but the same response has different diagnostic power for different kinds of failure. Kim, Ferrin, Cooper and Dirks (2004) found that the effect of a response depends on which kind of problem the failure is seen as. When it is seen as a matter of ability, an apology repairs trust better than a denial. When it is seen as a matter of integrity, a denial works better than an apology, because to apologize is to admit a flaw of character. In terms of likelihood ratios: people generally believe ability can improve, whereas a single dishonest act is enough to reveal character. So the same apology is, for an ability failure, evidence of willingness to improve, which is more common among reliable trustees, and the belief recovers; for an integrity failure, it is evidence of genuine dishonesty, and the belief falls further. For products this means two things. Integrity failures, such as hiding fees, acting beyond authority or rewriting records after the fact, are almost impossible to repair afterwards and must be prevented at the source by rules and constraints. Ability failures, such as a calculation error or a wrong judgement, must be admitted candidly, with an account of what happened, whom it affected and how it will be improved.

Trust is built slowly and lost quickly

Forty interactions between a user and a service. At the 24th, an ability failure occurs, such as a calculation error. Switch the response to the failure and see how trust changes.

Illustrative curves: a good interaction has a likelihood ratio close to 1 and a serious failure a large one, so belief rises and falls asymmetrically. The asymmetry of trust is from Slovic (1993). Kim, Ferrin, Cooper and Dirks (2004) found that for ability failures an apology repairs trust better than a denial; for integrity failures the finding reverses, and a denial works better than an apology. The figure shows only the ability-failure case. The grey dashed line is the path with no failure.

6. Reputations can be exploited, and types can change

The updating rule has two weak points. First, the trustee knows the principal is updating. The reputation models of Kreps and Wilson (1982) and Milgrom and Roberts (1982) show that an unreliable party also has a motive to build a good record first; once a one-off gain is large enough, cashing in the reputation becomes profitable. A good record therefore cannot replace limits that match the stakes. Second, the trustee's type can change: an institution changes its management, an AI agent changes its model version, and the past record no longer fully applies. Old records must be discounted, the belief must move partly back towards the prior, and new evidence must accumulate again.

Build a reputation first, then cash it in

An unreliable trustee performs well for the first 15 periods. The principal's belief rises, and so does the amount entrusted to it; in period 16 a large one-off opportunity appears. Switch whether the principal has a limit that matches the stakes.

Principal's beliefAmount entrustedLoss

Illustrative. On reputations built and exploited strategically, see Kreps and Wilson (1982) and Milgrom and Roberts (1982).

7. Too little trust and misplaced trust

When belief departs from how trustworthy the trustee actually is, both directions have a cost. Guiso, Sapienza and Zingales (2008) found that people who trust others less hold fewer stocks, even at the same level of wealth and education, and so give up the long-run risk compensation. Gennaioli, Shleifer and Vishny (2015) showed that trust lowers the risk investors perceive and lets trustees charge higher fees; they call such trustees "money doctors". The first group loses returns the market could have provided; the second hands part of its returns to the trustee.

Too little trust and misplaced trust both have a cost

Drag the level of trust. With too little trust, the user puts only a small share of the money into the market; with trust beyond the trustee's actual trustworthiness, the user stops comparing fees and is willing to pay a higher management fee. See what 1 million yuan becomes after 30 years.

Share put into the market
Fees paid each year
After 30 years (million yuan)

Illustrative calculation: the part not invested grows at 1% a year, the invested part at 6% a year before fees; a trust level of 50 exactly matches the trustee's trustworthiness. On trust and participation, see Guiso, Sapienza and Zingales (2008); on trust and fees, Gennaioli, Shleifer and Vishny (2015).

8. Trust in proportion to trustworthiness

Good trust is trust whose posterior matches how reliable the trustee actually is. Lee and See (2004), studying how people trust automated systems, call this calibration. When trust exceeds reliability, people stop checking and errors slip through; when trust falls short of reliability, even a reliable system is left unused. Parasuraman and Riley (1997) call the first misuse and the second disuse. The way to bring trust into line with trustworthiness is to provide evidence with high diagnostic power: show the basis and the record, state the scope of application and the known limitations, and make it easy for users to check.

Trust in proportion to trustworthiness

The horizontal axis is how reliable a service actually is; the vertical axis is the user's trust. Near the diagonal, trust is calibrated. Turn on "Make trustworthiness visible" and see how the two imbalances return to the diagonal.

Illustrative. The concepts of misuse and disuse are from Parasuraman and Riley (1997); the calibration of trust is from Lee and See (2004).

9. Implications for product design

Trust cannot be won by promotion; it can only accumulate through evidence with high diagnostic power. The aim of design is trust in proportion, not more trust.

Chapter summary: diagnostic evidence in exchange for trust in proportion

Four design requirements, each corresponding to one mechanism in this chapter.

Trust is a belief plus a decisionSet thresholds by stakes and capacity to bear loss

Small matters can be handed over first; large ones wait until the evidence is sufficient. Thresholds adjust to the user's capacity to bear loss.

Diagnostic power drives updatingProvide process evidence with high diagnostic power

Make fees kept to the agreement, constraints respected, an account of every period and explanations after ability failures into records that can be checked. Do not seek trust with short-term returns and rankings.

Lost faster than builtAvoid integrity failures; admit ability failures candidly

Serious failures change belief sharply. Integrity failures are hard to repair afterwards and must be prevented by rules; ability failures must be explained truthfully and put right, which is evidence of willingness to improve.

Reputations can be exploited; types changeLimits stay as trust grows; a new version rebuilds the record

Exposure limits match the stakes. When the model or the team changes, say so, and build the record again.

References