Product × Finance × AI: Financial Products in the Age of AIPart Four · Contract layer On what terms中文
Chapter 27

Liability and conflicts of interestWhom the agent works for, and who is responsible when it errs

A financial agent nominally works for the user, yet it is provided by the platform, paid for by the platform, and trained and evaluated against objectives the platform sets. If the platform's revenue comes from trading volume and time spent in the app, that interest will be written into the agent's objective: one more alert, one more trade, one more paid service. Disclosing the conflict does not remove it; research has even found that after disclosure, advice may become more biased, while those receiving it do not discount it accordingly. The more reliable remedy is structural: revenue that does not rise or fall with trading volume, an agent whose objective is measured by the user's net value, and independent audit. When the agent errs, liability should fall on the party that could have avoided the loss at the lowest cost; and to apportion liability, every step must first have a complete record.

1. Whom the agent works for

The classic principal–agent relationship has a single principal. A financial agent faces at least several parties at once: the user who uses it, the platform that provides and operates it, the issuers who sell products on the platform, and the company that supplies the underlying model. Bernheim and Whinston (1986) called the case of one agent serving several principals at once common agency: each party influences the agent's behaviour by paying it or setting its objectives, and the agent's eventual choice is a balance struck among these influences. The user is the nominal principal, yet is often not the party that pays most directly or has the most influence on the objective.

One agent, several principals

Switch views: where the money flows, who sets the agent's objectives, and whom the agent ought to serve. The thicker the line, the greater the influence (illustrative). "Issuer" means product issuers; "Model firm" means the model provider.

Illustrative structure. The case of one agent influenced by several principals at once is called common agency; see Bernheim and Whinston (1986).

2. The conflict is written into the objective

An AI agent's behaviour is determined by the objectives against which it is set up and evaluated. Holmström and Milgrom (1991) showed that when an agent's work has several aspects and only some are measured and rewarded, the agent shifts effort toward the part that is measured. If the platform evaluates the agent by trading volume, clicks and time spent, the agent will send alerts more often and recommend trades more actively; gains in these measures do not mean better outcomes for the user. In 2021 the U.S. Securities and Exchange Commission sought public comment on the digital engagement practices of brokers and investment advisers, and its concern was precisely the effect of gamified design, push notifications and prompts on investor behaviour.

How much weight the platform carries in the metrics

The agent is evaluated on a weighted mix of two parts: the user's net value, and the platform's trading volume and time spent. Drag the weight on the platform's metrics to see the agent's behaviour and the user's outcome (illustrative).

User's net annual returnPlatform's annual revenue from this user
Trade suggestions per month
User's net return
Platform revenue
Illustrative model: for each extra trade suggestion, about half of users follow it; each trade costs about 0.05% of assets, part of which goes to the platform; trading itself does not raise expected returns. On agents shifting effort toward the measured part when only part of the work is measured, see Holmström and Milgrom (1991).

3. Why disclosure is not enough

The most common remedy is to tell users about the conflict of interest. Yet experiments by Cain, Loewenstein and Moore (2005) found that disclosure can backfire: after disclosing, advisers felt they had met their duty to inform and gave even more biased advice; those receiving the advice knew of the conflict but did not discount the advice enough. As a result, after disclosure the advised party ended up worse off. Disclosure is necessary, but it hands the burden of recognizing the conflict to the people least able to recognize it.

Why outcomes can be worse after disclosure

There is a conflict of interest between the adviser and the person advised. Switch whether the conflict is disclosed, and see the bias in the advice, the discount the recipient applies, and the recipient's final error (illustrative).

How far the advice departs from the truth
Discount the recipient applies
Recipient's final error

Illustrative numbers; the directions follow the experiments of Cain, Loewenstein and Moore (2005): after disclosure, advisers gave more biased advice, and recipients' discounts were not enough to offset it.

4. Structural alignment

More reliable than disclosure is to make conflicts structurally hard to arise. First, the fee model: the agent's revenue does not vary with the number of trades or the products recommended, as with a fixed subscription. Second, objective constraints: the agent is evaluated on the user's net value and its adherence to the mandate, not on trading volume and time spent. Third, information separation: when making a recommendation, the agent cannot see how much revenue each product brings the platform. Fourth, independent audit: regular checks on whether the agent's recommendations lean systematically toward options that favour the platform. In 2023 the U.S. Securities and Exchange Commission proposed a rule requiring brokers and investment advisers to identify and eliminate conflicts of interest in predictive data analytics; the proposal was withdrawn in 2025, but the question it raised is exactly the one AI agents must answer.

Structural alignment, not just disclosure

Add four structural arrangements one by one and see how much room the agent has left to lean toward the platform's interest (illustrative).

Room to favour the platform
Illustrative. The U.S. Securities and Exchange Commission's 2023 proposed rule (Release No. 34-97990, withdrawn in 2025) would have required firms to identify and eliminate conflicts of interest in predictive data analytics.

5. Who is responsible when it goes wrong

Agents err for different reasons: the agent misread the mandate, a data source supplied a wrong price, the execution layer failed to block an instruction that crossed a limit, or the agent followed the user's explicit instruction and the result was poor. Calabresi (1970) proposed that liability for an accident should fall on the party able to avoid it at the lowest cost. By this principle, losses from execution beyond the limits should be borne by the platform, because it controls the execution layer's checks; data errors are borne by the party that supplied the data; if the agent misreads a clearly written mandate, the platform bears it; and if the agent follows the user's own explicit instruction and the result falls short of expectations, the user bears it. Liability follows control: whoever can avoid the loss is responsible for it.

Whoever can avoid it most cheaply is responsible

Six cases of an agent going wrong. Tap "Assign liability" to see who should bear the loss under the least-cost-avoider principle.

Illustrative judgments, not legal advice. For the least-cost-avoider principle, see Calabresi (1970); actual allocation of liability depends on local law and contract terms.

6. Liability must be traceable

Apportioning liability requires being able to reconstruct afterwards what happened: what data the agent saw at the time, which clause of the mandate it relied on, whose authorization it had, and what checks the execution layer performed. The securities industry has long had record-keeping requirements; for example, the U.S. Securities and Exchange Commission's Rule 17a-4 specifies the records brokers must keep and how they must keep them. For an AI agent, the record must cover every key action: the input data and its point in time, the clause relied on, the level of authority, and the result of execution. Only with a complete record can liability be apportioned, compensation be grounded, and the same error be avoided.

Records decide whether liability can be apportioned

An incident: the agent sold part of a holding beyond the agreed proportion. Afterwards, six questions need answers. Switch how complete the records are and see how many can be answered.

Illustrative. For the records brokers must keep and how, see the U.S. Securities and Exchange Commission's Rule 17a-4; for AI agents, records must also cover inputs, the basis for action and authorization.

7. Implications for product design

Put the agent's revenue and objective on the user's side, and handle conflicts through structure rather than disclosure; liability follows control and rests on a complete record.

Chapter summary: keeping the agent on the user's side

Four design requirements, each corresponding to one mechanism in this chapter.

The agent has several principalsMake clear that the agent serves the user

Write the mandate and the user's net value into the agent's objective, and tell users where the platform makes its money.

The conflict is written into the objectiveDo not evaluate the agent on trading volume or time spent

Base the metrics mainly on the user's net value and adherence to the mandate, and monitor how often trades are suggested.

Disclosure is not enoughHandle conflicts through structure

Revenue does not vary with trades or recommendations; product revenue is hidden when advising; bias in advice is independently audited at regular intervals.

Liability follows controlAssign liability to the least-cost avoider, and keep complete records

Execution and data errors are borne by the party in control; every key action records inputs, basis, authorization and checks, which ground compensation and improvement when things go wrong.

References