Promises, the agreement, records, defences and compensation are all in place, yet the villager still hesitates. The scribe has only just opened for business; he has never seen it work. Why should he hand it the grain for his house and his retirement? Trust is not given all at once. He needs to believe three things: that it is capable, that it has his interests at heart, and that it does not say one thing and do another. Each has its own evidence. A scribe without a record can borrow signals that others cannot afford: the operator's capital, insurance and licence. Authority starts small and is widened by one level each time the agreement has been kept for a period, and the record of keeping the agreement gradually changes his judgement of the scribe. Trust is lost faster than it is built. After a mistake, trust should step back rather than drop to zero at once, and the amount of trust should always match how reliable the scribe actually is.
1. What he needs to believe
"Trusting the scribe" contains three different things. The first is believing that it is capable: it can keep accounts, understands the fleet, and does not make elementary mistakes. The second is believing that it has his interests at heart: its pay comes from him, not from the people who sell ship shares, and what it recommends is what he would choose himself after thinking it through. The third is believing that its words and actions match: it really keeps the boundaries written in the agreement, and the grounds written in the records really exist. Each has its own evidence: capability is judged by the past record, whose interests it serves by how it is paid and what its goals are, and consistency between words and actions by the agreement and its verification. If any one is missing, trust cannot stand.
There is one new constraint: the user has no prior about a new agent, and trust takes time.
Today's counterpart. The relationship summary that the U.S. Securities and Exchange Commission requires of brokers and investment advisers must describe the services offered, fees and costs, conflicts of interest, and whether there is a disciplinary history. These correspond roughly to ability, benevolence and integrity.
Three sources of trust, each with its own evidence
Select one to see what he needs to believe, what the product uses as proof, and what happens if it is missing.
The three-part division comes from Mayer, Davis and Schoorman (1995); the column "The product's evidence" is a design derived in this book.
2. Borrowed trust
A scribe that has just opened for business has no record, so there is no way to prove its ability. It can borrow trust from others: the operator that deploys it puts up capital as a deposit, buys insurance against its errors, obtains a licence, and has its accounts audited by a third party. What matters about all of these is that they are expensive. An operator that intends to cheat cannot afford such a cost, or would find it not worth paying. Precisely because they are costly, they can separate the reliable from the unreliable. Saying "we are trustworthy" costs nothing, and so it proves nothing.
Today's counterpart. Securities and investment advisory businesses are licensed, and firms are subject to requirements on capital, personnel and internal controls.
Only costly signals are credible
Two kinds of operator: reliable, and intending to cheat. Drag the cost of a signal and see whether each kind is willing to pay it, and whether the signal can tell them apart.
Illustrative: a reliable operator runs its business for the long term and, once trusted, gains 15 more portions over ten years. An operator intending to cheat wants only one big haul and then to leave; once trusted, it can cheat its way to 8 portions, but if things go wrong it loses everything it spent on the deposit and insurance. Each pays the cost of the signal only when what it gains exceeds what it pays (following Spence, 1973).
3. Starting small
Even with borrowed trust, there is no need to hand everything over at the start. Work it through to see the cheapest way to begin. First give the scribe the smallest task, such as investing on schedule each month, where one mistake can lose at most a small portion. After it has kept the agreement for a while, widen its authority by one level and let it adjust the proportions as the agreement provides. After another period of keeping the agreement, let it handle the rebalancing after a storm. At each level, the loss limit matches the confidence he has already built up. A scribe that intends to cheat must first do small tasks honestly for a long time before it gets a task large enough to be worth cheating on, and during that time its record has long been in front of him.
Today's counterpart. Many places have graduated licensing for new drivers: they first drive under restrictions, and the restrictions are lifted step by step as conditions are met. A Cochrane Collaboration systematic review concluded that such schemes are associated with fewer crashes among young drivers, and that the effect is more pronounced where restrictions are stronger.
Start small, widen level by level
Authority has four levels and is widened by one level each time the agreement has been kept for a period. Switch the type of scribe to see the largest loss it can cause at each level, and how long a scribe intending to cheat must keep the agreement before it gets authority worth cheating with.
Illustrative: at Level 1 the scribe can only make scheduled investments, moving at most 2% of household wealth at a time; at Level 2 it can adjust proportions as the agreement provides, up to 5%; at Level 3 it can handle rebalancing after a storm, up to 10%; at Level 4 it handles most matters on its own within the agreement, up to 25%. After a full year of keeping the agreement at one level, authority is widened by one level (starting small, as described by Watson, 1999).
4. The record gradually changes the judgement
His judgement of the scribe is not simply "trust" or "distrust" but a probability: how likely it is to be the reliable type. Each time it keeps the agreement, the probability moves up a little; each time it breaches the agreement, it drops a long way. The starting probability depends on the borrowed signals; changes after that depend on the record. With a reliable scribe, given a long enough record of keeping the agreement, his judgement will eventually approach the truth. An unreliable scribe that wants to keep its reputation must keep the agreement for a long time, and that in itself reduces the harm it can do.
Today's counterpart. Credit scores are updated monthly according to repayment history: on-time repayments raise the score slowly, while one serious delinquency lowers it sharply.
The record gradually changes the judgement
There are two types of scribe: a reliable one makes 0 to 1 small mistakes a year, an unreliable one makes one or two. His initial judgement depends on the borrowed signals. Switch the scribe's actual type and his initial judgement, and see how his estimate of the probability that "it is reliable" changes over ten years.
Illustrative: the number of mistakes per year follows a Poisson distribution with mean 0.5 for the reliable type and mean 1.5 for the unreliable type. Each year he updates his judgement by Bayes' rule according to the number of mistakes observed (judgements about type revised in light of behaviour, as in Kreps and Wilson, 1982). The number below each dot is the number of mistakes observed that year.
5. Lost faster than it is built
One instance of overstepping can wipe out years of keeping the agreement. People react far more strongly to bad news than to good news, and this is not wrong in itself: a single breach often reveals much more information than a single instance of keeping the agreement. But if every mistake sent trust back to zero, a single misjudgement or one small execution error by the scribe would put him back at the starting point. All the authority built up over years would be withdrawn, and he would return to his own, worse judgement.
Work it through to see what a reasonable response to mistakes looks like. First, look at what kind of mistake it is. Overstepping and concealment suggest the scribe may be the unreliable type, and its authority should be largely withdrawn. For a small execution error that has already been compensated, with its cause found and fixed, the authority steps back only one level. A misjudgement is a risk the agreement has accepted, so there is no need to withdraw authority, but the agreement should be reviewed to see whether it needs to change. Only by stepping back rather than dropping to zero can trust keep accumulating after a mistake.
Slovic (1993), discussing risk perception and trust, argued that negative events affect trust far more than positive ones, so that trust is easily lost and hard to build. Experiments by White and Eiser (2005) found that this asymmetry is less pronounced when the information is less specific and the risk is lower.
Today's counterpart. China's motor vehicle driving licences use a points system: each cycle allows 12 points, and each violation adds 1 to 12 points depending on its seriousness. A driver who reaches 12 points must attend a full-points study course and pass an examination, while serious violations such as drunk driving lead to revocation of the licence under the law.
After a mistake: back to zero, or one step back?
A reliable scribe makes an execution error in year 4, which is compensated and fixed. In year 7 it shifts the proportions back as the agreement says and then meets a storm, which is a misjudgement. Switch between two ways of treating mistakes, and see the authority level over ten years and how much it gets done for him.
Illustrative: authority has four levels (L1 to L4 on the vertical axis), rising one level for each year the agreement is kept. Under "Adjust by kind of mistake", an execution error that has been compensated and fixed steps back one level, a misjudgement does not step back, and only overstepping and concealment lead to authority being withdrawn. What it gets done for him in one year at each level counts as 1, 2, 4 and 7 respectively.
6. Ten years of trust: Civilization Simulator, version 24
The simulator unlocks the last item of Part Three: "the path to trust". Over ten years the villager deals with a new scribe. You can set whether the scribe is in fact reliable or unreliable, and choose how authority is granted: handing everything over at the start; widening level by level but dropping to zero after any mistake; or widening level by level and adjusting according to the kind of mistake. You can also turn the borrowed signals on or off. Then see how much the scribe got done for him over ten years, the losses he suffered from trusting too much, and whether he gave up entirely after a mistake.
Civilization Simulator · Chapter 24: ten years of trust
The villager deals with a new scribe for ten years. Choose the scribe's actual type and the way authority is granted, turn the borrowed signals on or off, and see the average result over 400 ten-year runs.
Product: what to promiseProduct: mandate and authorityProduct: process records and verifiable evidenceProduct: limits, confirmation and compensationProduct: the path to trust
Type
Authority
Illustrative model: authority has four levels; what the scribe gets done for him in one year at each level counts as 1, 2, 4 and 7, and a single action can move at most 2%, 5%, 10% and 25% of household wealth. Each year, both types of scribe have a one-in-four chance of one execution error, which is compensated. Once an unreliable scribe reaches Level 3 or above, it has a 60% chance each year of overstepping, losing an amount equal to that level's limit. When everything is handed over at the start, any single mistake has a 50% chance of making him give up entirely. Under "adjust by kind of mistake", an execution error steps back one level and overstepping ends the relationship. With signals, a reliable scribe starts at Level 2, while an unreliable operator cannot afford the signals and still starts at Level 1.
7. After this step
Five constraints, five parts: what to promise; mandate parameters and authority; process records and verifiable evidence; limits, confirmation and compensation; and the path to trust. Together they make up a financial product that can stand in a world where cognitive labour is almost free.
Five parts
The five parts that grew from five constraints. Select a part to see the question it answers.
Five partsTogether, one product
8. Implications for products
For a wealth-management product built around an agent, the last part to grow is "the path to trust". Three implications follow. First, provide separate evidence for ability, benevolence and integrity: ability rests on verifiable records, benevolence on charging the user and writing down the goals, and integrity on the agreement and independent verification. Second, a new agent first borrows expensive signals, namely the operator's capital, insurance, licence and audit, then starts small and has its authority widened level by level as its record of keeping the agreement builds up, with the loss limit at each level written down in advance. Third, adjust trust according to the kind of mistake: overstepping and concealment lead to a large withdrawal of authority, an execution error that has been compensated and fixed steps back only one level, and a misjudgement does not withdraw authority; the amount of trust always matches how reliable the agent actually is.
Chapter summary: trust takes time
Four conclusions, each tied to one mechanism in this chapter.
Three sources of trustworthinessAbility, benevolence and integrity, each with evidence
Records prove ability, fees and goals prove benevolence, and the agreement and its verification prove integrity.
SignallingOnly costly signals are credible
A deposit, insurance, a licence and an audit are what a cheating operator cannot afford.
Starting small and reputationWiden level by level; the record changes the judgement
The loss limit at each level is written down in advance, so a cheating scribe never gets authority worth cheating with.
AsymmetryAfter a mistake, step back rather than reset to zero
Overstepping withdraws authority, a compensated execution error steps back one level, and a misjudgement does not step back.
References
Colquitt, J. A., Scott, B. A., & LePine, J. A. (2007). Trust, trustworthiness, and trust propensity: A meta-analytic test of their unique relationships with risk taking and job performance. Journal of Applied Psychology, 92(4), 909–927.
Kreps, D. M., & Wilson, R. (1982). Reputation and imperfect information. Journal of Economic Theory, 27(2), 253–279.
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995). An integrative model of organizational trust. Academy of Management Review, 20(3), 709–734.
Milgrom, P., & Roberts, J. (1982). Predation, reputation, and entry deterrence. Journal of Economic Theory, 27(2), 280–312.
Ministry of Public Security of the People's Republic of China (2021). Measures for the Administration of Points for Road Traffic Safety Violations (Order of the Ministry of Public Security No. 163).
Nobel Prize Outreach. The Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel 2001. NobelPrize.org.
Road Traffic Safety Law of the People's Republic of China, Article 91.
Russell, K. F., Vandermeer, B., & Hartling, L. (2011). Graduated driver licensing for reducing motor vehicle crashes among young drivers. Cochrane Database of Systematic Reviews, CD003300.
Slovic, P. (1993). Perceived risk, trust, and democracy. Risk Analysis, 13(6), 675–682.
Spence, M. (1973). Job market signaling. Quarterly Journal of Economics, 87(3), 355–374.
Watson, J. (1999). Starting small and renegotiation. Journal of Economic Theory, 85(1), 52–90.
White, M. P., & Eiser, J. R. (2005). Information specificity and hazard risk potential as moderators of trust asymmetry. Risk Analysis, 25(5), 1187–1198.