Product × Finance × AI: Financial Products in the Age of AIPart Two Changing one parameter中文
Chapter 18

Can an agent correct human bias?Discipline, exploitation and new biases

A default can only be set for the typical household, and a cooling-off period can only treat everyone alike. Now every household has a scribe at hand. It knows how much this household means to save, when it will need the money, and what it fears most, so it can set a default for this household alone. After a storm, at the moment the principal is rushing to sell, it can put the original agreement in front of him and ask him to wait. The same capability can be turned the other way: pushing messages at the principal when he is most impulsive, and greeting every trade with confetti and badges. Handing judgment to the scribe also brings new biases. Trust it too much and you stop checking; see it fail once and you may never use it again.

1. A default for each household

Defaults are powerful because most people stay with them. That is also their limit: a single default can only aim at the average household, and where it misses, inertia leaves people in the wrong place. If the default saving rate is 12%, households that meant to save more end up saving too little, while households that meant to save less, because money is tight right now, end up saving too much.

The scribe changes this. It can work out a suitable rate for each household and make that the household's default, and adjust the default when income changes or the family grows. Inertia remains, but it now holds each household closer to what it actually intended.

Today's counterpart. Portfolios allocated from a questionnaire and retirement products that adjust automatically with age are defaults set by group. Defaults tailored to each household and adjusted at any time used to appear only in private banking.

Inertia keeps people on the default

Forty households each have a suitable saving rate (dots), and most stay on the default (bars). Switch between a single default and a default tailored to each household, then let some households' income fall in year five, and see whether the default adjusts.

Illustrative: suitable rates range from 6% to 18%; 70% of households stay on the default. The tailored default is set by the scribe for each household and adjusted when circumstances change. A gap of more than 3 percentage points counts as a "clear deviation".

2. The moment of panic

People who weight the present heavily are willing to bind themselves in advance, then want to break free when the moment comes. The old locks were crude: a store you could not draw from was either locked too tight, so that even a real emergency could not open it, or too loose, so that an impulse could. The scribe can make a smarter lock. It stays out of the way in ordinary times and lets real emergencies through. When, after a storm, the principal wants to sell all his ship shares, it first sets out the original agreement and where this fall sits in history, and asks him to confirm a day later, so that the impulse can pass first.

D'Acunto, Prabhala and Rossi (2019) studied a portfolio optimization tool and found that its users showed less of the disposition effect, trend chasing and rank effects.

Today's counterpart. The EU's 2002 directive on the distance marketing of financial services gives consumers a 14-day withdrawal period, but services whose price depends on fluctuations in financial markets, such as securities and fund units, are excluded: these products cannot be undone afterwards, only slowed down beforehand. The 2023 amending directive folds these rules into the Consumer Rights Directive and applies from 19 June 2026, when the 2002 directive is repealed.

A smarter lock

The same household faces two events: after a storm, ship shares fall sharply and he wants to sell them all; later, someone in the family falls ill and he urgently needs grain. Switch between three locks and see how each event turns out.

Illustrative: selling immediately after a sharp fall misses the subsequent recovery on average; "a store you cannot draw from" cannot be drawn from in an emergency either. The scribe tells the two events apart on the basis of the agreement the principal wrote in advance, which states what counts as an emergency.

3. The same capability can exploit bias

A scribe that can slow people down when they panic can also speed them up when they are excited: a message every time the price ticks, a burst of confetti with every trade, a badge for trading several days in a row. The better the scribe knows a household, the better it knows when a nudge works best. If it is paid by the number of trades, the most effective nudge is the one that makes people buy and sell more.

Chapkovski, Khapko and Zoican (2024) compared, in an experiment, trading interfaces with game-like elements such as confetti and badges against plain interfaces. The game-like elements raised trading volume by about 5% on average, while most of the difference in trading between the two kinds of platform came from differences in the people who chose them. Participants with less financial knowledge preferred the game-like platforms, and push notifications about price trends reinforced the mistakes of those whose judgment was already wrong.

Today's counterpart. In 2021 the U.S. Securities and Exchange Commission requested public comment on the "digital engagement practices" of broker-dealers and investment advisers, covering behavioural prompts, game-like features, and the analytical and technological tools behind them.

A scribe paid by the number of trades

The same principal, the same year of prices. Switch the scribe's goal, then drag the frequency of push messages, and see how many times the principal trades in a year and what it costs him.

Illustrative: a scribe acting on the plan invests once a month and rebalances once a quarter; its messages only remind the principal of the agreement and cause no extra trades. A scribe paid by the number of trades sends a message whenever the price moves; each message has a 15% chance of triggering a trade, each trade costs 0.3% of its value in spread and fees, and each trade is about one tenth of the holdings.

4. Trusting it too much: over-reliance

The scribe is right most of the time, so the principal gradually stops checking. When it says buy, he buys; when it says all is well, he does not look. But it is occasionally wrong, and when it is wrong it looks the same as when it is right. The fewer people check, the more easily its errors land in the accounts unchanged.

The opposite case also occurs. The principal sees the scribe make one mistake and never uses it again, preferring to go back to his own judgment even though his own judgment errs more often. The first is using it too much, the second too little; neither uses the scribe according to how reliable it actually is.

There is experimental evidence for both tendencies. Skitka, Mosier and Burdick (2000) found that participants who believed they were accountable for how they interacted with an automated system checked more often whether the system was right, and made fewer automation-related errors. Dietvorst, Simmons and Massey (2015) found that after seeing an algorithm err, people avoided using it, even though it was more accurate than a person.

Today's counterpart. Problems long studied in driver assistance and flight automation now appear in agents that place orders on people's behalf.

Too much trust and too little

The scribe does 200 things a year for the principal. Drag its reliability and see what share the principal checks and how many errors slip through. Tick "Principal confirms and answers for key actions", then switch to "After seeing it err once".

Illustrative: after long use, the share the principal checks falls as the scribe's reliability rises (the more reliable, the less checking); when he confirms and answers for key actions himself, the share checked rises to at least 60%. When the principal does the same things himself, his error rate is 10%. "After seeing it err once", 70% of principals stop using the scribe and do everything themselves (misuse and disuse in the sense of Parasuraman and Riley, 1997; Skitka et al., 2000; Dietvorst et al., 2015).

5. Whose goals it serves

Put three things side by side: the tailored default, the reminder in a panic, and the exploitation of bias. They use the same capability, namely the scribe's knowledge of a household and its presence at the critical moment. Whether the result is good or bad depends on whose goals it serves. Acting on the plan the principal set for himself, it is a better lock. Acting on the number of trades or on promotional income, it is someone who knows better how to pick the lock.

The person cannot be free of all responsibility either. Only when the principal sets the plan himself and confirms in person at the critical points do the scribe's reminders have a basis, and over-reliance diminishes too.

Today's counterpart. The U.S. Securities and Exchange Commission's 2019 interpretation of the standard of conduct for investment advisers states that an adviser's fiduciary duty comprises a duty of care and a duty of loyalty: advice must be in the client's best interest, and conflicts of interest must be dealt with by eliminating them or by full disclosure.

One capability, two goals

The scribe's capability is knowing a household and being present at the critical moment. Pick a moment and compare what it does when acting on the principal's plan and when acting on the number of trades or promotional income.

Illustrative: the comparisons are fictional examples.

6. Between current practice and the ideal

"Acting on the principal's goals" has a further difficulty: what the principal does now is not necessarily what he most wants to do. This principal sometimes buys ship shares because of something he heard in the market square, sometimes because he glanced at the price trend. When he sells he uses a different set of reasons, and when he reviews afterwards, yet another. From spotting an opportunity through buying and holding to reviewing, his method is not consistent. This scattered way of investing is not a one-off lapse but the habit of many years, and it is itself a stable state.

The scribe can take one of two approaches. The first is to go along with current practice: give him whatever he asks for, a faster ordering tool if he wants to trade often, and hold no view of its own. This is different from exploiting bias, since the scribe is not working for the number of trades; it simply passes no judgment. The second is to look toward the ideal: a mature investor uses one consistent method for finding opportunities, trading and reviewing, and keeps to it over the long run. There is a tension between current practice and the ideal. The scribe does not impose it; it comes from the principal's own wish, since everyone wants better results.

Consider each approach taken to the extreme. If it only goes along with current practice, the scribe accepts the household's habits wholesale, the cost of frequent trading accumulates year after year, and the principal's wish for better results is never answered. If it only looks toward the ideal, the scribe corrects him at every turn and tells him "your way is wrong", and he is likely to find another scribe, so this one loses even the chance to help. The workable path lies between the two. First respect his habits and carry out his instructions as usual, while recording the reason and the result of every trade. When it is time to review, use his own record to show the results, for example that frequent trading has not earned him better returns. Then ask whether he would like to try another method. The decision always stays with him.

All the scribe can promise is the process. It can help him keep his method consistent, give every step a reason and make reviews verifiable, but it cannot promise that this will bring better results: results also depend on a run of years that no one controls. What improves is the quality of decisions, not a guarantee of returns.

Today's counterpart. Barber and Odean (2000) studied about 66,000 household accounts at a U.S. discount broker. From 1991 to 1996, the 20% of households that traded most earned an average annual net return of 11.4%, against a market return of 17.9% over the same period. Thaler and Sunstein (2003) called the approach of preserving freedom of choice while guiding people toward choices they themselves consider better libertarian paternalism.

Between current practice and the ideal

The principal's present, scattered way of investing is one steady state; the consistent method of a mature investor is another. Switch the scribe's approach and see where he stands after ten reviews, and what the scribe does at several moments.

Steady state of current practice: scattered methodBuys on hearsay, sells for another reason, reviews by yet another
Steady state of the ideal: consistent methodOne method for finding, trading and reviewing

Illustrative: the horizontal axis shows how far the principal's practice is from that of a mature investor; positions and the size of each move are for explanation only, not measured data. The middle path promises only to improve the quality of the process, not the results.

7. The scribe at hand: Civilization Simulator, version 18

The simulator unlocks "Agents and human bias". Forty households each have their own suitable saving rate, and each has a scribe save and trade for it for thirty years. You can switch two rules on or off, the tailored default and the panic reminder (show the agreement, confirm a day later); switch the scribe between acting on the principal's plan and acting on the number of trades; and adjust how strongly people weight the present and how readily they sell in a panic. Then see how much each household saves and how many come close to their goal.

Civilization Simulator · Chapter 18: the scribe at hand

Forty households each have their own suitable saving rate, and for thirty years a scribe saves and trades for each of them. Switch two rules on or off and change the scribe's goal, then see how much each household saves and how many come close to their goal.

DefaultsPersonal agentGuaranteeing agentsAgents as market participantsAgents and human biasWhat did not change · Ch. 19
Median savings (years of income)10th to 90th percentile of households

Illustrative model: the run of years is the same as in the "keeping to the plan" simulator. Each household's suitable saving rate lies between 6% and 18%; without tailored defaults, every household saves the single rate of 12%. When acting on the principal's plan, the scribe buys on schedule, and weighting the present does not affect saving. When acting on the number of trades, the scribe does not stop the principal from spending more and pushes him to buy more after gains; panic selling after sharp falls rises by half, and trading costs are 1% a year. Reminders and the cooling-off period cut panic selling to one fifth. "Close to the goal" means savings after thirty years within 20% of what saving on time at the suitable rate would have produced.

8. What this step leaves open

The scribe can help people stick to their plans, and it can also exploit their biases. Several questions remain unresolved:

  1. Writing down and checking the goal. Whose goals does the scribe serve, how are they written down clearly, and who checks?
  2. Calibrating trust. How can the principal's trust in the scribe be made to match its actual reliability?
  3. Who holds the key. For the smarter lock, does the principal, the scribe or the operator hold the key?
  4. What does not change. However well the scribe helps people keep to their plans, the risk of falls and time itself do not change.

What this step leaves open

Four problems that remain unsolved once the scribe can both help people keep their plans and exploit their biases. Pick a problem to see the assumption it hides.

The scribe at handDiscipline and exploitation

9. Implications for products

The most valuable and the most dangerous feature of an agent-centred wealth product is the same: it can influence the user at critical moments. Four implications follow. First, tailored defaults and reminders at critical moments are where an agent can help users most: make the user's own plan the default, add a little friction at impulsive moments, and let real emergencies through. Second, the product's objective function must be written down: act on the user's plan, not on the number of trades, time spent in the app or promotional income; every design that exploits inertia and impulse must be able to say whose interest it serves. Third, make the user's trust match the agent's actual reliability: publish the agent's error record and keep the user's confirmation for irreversible actions, so that users neither follow blindly nor give up entirely after a single error. Fourth, take the middle path between the user's current practice and the ideal: first respect the user's existing habits, use the user's own record to show the results, then invite rather than require the user to try another method; promise to improve the quality of the process, not the results.

Chapter summary: can an agent correct human bias?

Five conclusions, each tied to one mechanism in this chapter.

Rerun: Choice architectureA default for each household

The default no longer aims only at the average person; inertia holds each household closer to what it intended.

Rerun: Quasi-hyperbolic discountingA smarter lock

It adds resistance to impulses and lets emergencies through, on the basis of the agreement the principal wrote in advance.

Exploiting biasOne capability, the opposite direction

A scribe acting on the number of trades knows best how to push for a trade at the impulsive moment.

Misuse and disuseTrust must match reliability

Trust it too much and you stop checking; see it err once and you stop using it. Confirming and answering for key actions in person reduces both biases.

Current practice and the idealRespect first, then invite

A scattered way of investing is also a steady state. Use the principal's own record to show the results, then ask whether he would like to try something else; promise only the quality of the process.

References