The PDF version of this article can be found here.
ALGORITHMIC
CONTRACTS: WHO IS TO BLAME?
Journe Fu*
I. Introduction
Traditionally, all the elements leading to
the formation of a valid contract require some form of human cognition. A party
will make an offer, while the other party will receive the offer, deliberate the
terms, and eventually decide to accept. Rapid advancements in artificial
intelligence technology has allowed algorithms to form contracts without human
cognition at the time of formation. This stretches traditional contract law
principles, which are premised on two human beings reaching an agreement, into
unchartered territory.
This
article closely examines how the Singapore Court of Appeal (‘Court of Appeal’)
in Quoine Pte Ltd v B2C2 Ltd[1] adapts the doctrine of unilateral mistake to
contracts formed using a deterministic algorithm and evaluates the approach
proposed by the Court of Appeal.
II. The Law on Unilateral
Mistake
The law on unilateral mistake was comprehensively laid out by the Court of Appeal in Chwee Kin Keong v Digilandmall.com Pte Ltd.[2] A unilateral mistake occurs when one party is mistaken as to a fundamental term of the contract and the other non-mistaken party is aware of this.[3] At common law, the non-mistaken party must have actual knowledge of the mistake, which would render the contract void.[4] The Court of Appeal also accepted that there exists an equitable doctrine of unilateral mistake that operates when the non-mistaken party, having constructive knowledge of the mistake, engages in some form of unconscionable conduct pertaining to the mistake. In equity, the contract will be voidable.[5]
III. THE Facts of Quoine
Both B2C2
and Quoine were market-makers who created liquidity on the trading platform (the
“Platform”) by actively buying and selling cryptocurrency. Both parties utilised
deterministic algorithms[6] to complete these trades, meaning
that the buy/sell contracts were concluded without any direct human involvement
apart from the initial programming.
Quoine’s
algorithm subsequently encountered software errors which set off a sequence of
events ultimately triggering margin calls against counterparties (the ‘Counterparties’),
who had borrowed Ethereum (‘ETH’) from Quoine to purchase Bitcoin (‘BTC’). Quoine’s
algorithm automatically sold ETH to the counterparties in exchange for BTC, in
order to repay the ETH loans (see diagram below, arrows 1 and 2).
These
orders were in part met by B2C2’s orders to sell ETH for BTC (see diagram
below, arrows 3 and 4). However, due to the software error, the ETH was sold at
approximately 250 times the going market rate, leaving B2C2 with a windfall.[7] In response, Quoine then
unilaterally cancelled and reversed these trades (see diagram below, arrows 1,
2, 3 and 4) leading B2C2 to claim against Quoine for breach of contract.
One of the defences raised by Quoine was that the underlying trades were void for unilateral mistake at common law or voidable in equity, therefore entitling them to reverse the trades. The main problem presented to the Court of Appeal in Quoine was ascertaining the type of knowledge possessed by the non-mistaken party, since the contract was formed solely using deterministic algorithms and neither party had any knowledge of its formation or the terms of the contract before the contract was concluded.
Quoine asserted that the Counterparties held two mistaken beliefs: first, that it was necessary to close out their positions in response to margin calls made by the Platform; second, that they were buying ETH for BTC at prices accurately representing the true market value of ETH relative to BTC.
(i) Unilateral mistake at common law
The first
alleged mistake was held to be a mistaken assumption and not a mistake as to a
term of the contract, and so in respect of which the doctrine of mistake would
not operate. As for mistake (2), the Court of Appeal held that it was wrongly
characterised and that since the prices had been arrived at by the parties’
respective algorithms as agreed upon, the actual premise for the mistake was
the belief that the platform would not fail. In other words, the Counterparties
mistakenly believed the Platform would either always operate as intended or
would otherwise provide adequate identification and protection to stop trading.
Even then, the Court of Appeal held this was merely a mistaken assumption as to
the circumstances under which the Trading Contracts were entered into and not a
mistake as to a contractual term.[8] The common law doctrine of
unilateral mistake therefore did not apply.
(ii) Unilateral mistake in
equity
The Court of Appeal left open the question of whether its equitable jurisdiction could be extended to mistaken assumptions,[9] but nevertheless explored the outcome on the assumption that it could. This led to a unique problem since, at both common law and equity, the requisite knowledge of the mistake must accrue prior to contract formation for the doctrine of mistake to operate. In the present case, the contracts were entered into without human cognition. The parties would therefore only have knowledge of the mistakes post-contract formation.
IV. Adapting the Knowledge
Rule for Algorithmic Contracts
The majority, led by Menon CJ, held that to ascertain ‘knowledge’, the state of mind of the programmer of the algorithm at the time of programming would be examined. This was supported by the view of amicus curiae Professor Goh Yihan, who recognised that the time of programming is when the programmer’s knowledge is the most concretised.[10] The doctrine of unilateral mistake would apply if the programmer had contemplated or ought to have contemplated that a future party would be mistaken and specifically designed the algorithm to exploit such a mistake.[11] Alternatively, if after programming, but before the time of contracting, the programmer or user of the algorithm becomes aware that the party might be mistaken but allows the algorithm to continue running in order to exploit this mistake, the doctrine will likewise apply.[12] The Court of Appeal held that B2C2 had no such contemplation and therefore neither possessed the requisite actual nor constructive knowledge for the doctrine of unilateral mistake to operate.
In his dissenting judgment, Mance IJ held that a contract could be set aside if it would be obvious to a reasonable person aware of the prevailing circumstances that some fundamental error had occurred, the timing of the mistake being irrelevant.[13] Mance IJ emphasized that algorithmic transactions with no human involvement did not fit with the principles of unilateral mistake at common law but what was certain was the fundamentality of the mistake. He held that the mistake in Quoine was just as fundamental as the mistake in Digiland and that any human trader viewing the transaction would have realized there was a fundamental error.
The majority approach therefore focuses on the reprehensible conduct of the non-mistaken party, stemming from the principle that where both parties had agreed to transact in a manner which prevents them from knowing if a contract would be formed or the contractual terms, the court ought to uphold this agreement and not intervene. Contrastingly, Mance IJ asserts that the court may intervene where there is substantive unfairness of the contract, choosing to focus instead on whether the contract is exceedingly onerous to one party. This requires a comparison of the transaction in question with other similar transactions in the market.
V. Evaluation OF THE TWO
APPROACHES
The Majority’s Approach
(i) Lack of protection for
end users of trading platforms
The
majority approach upholds contracts where the algorithm-induced mistake was
neither contemplated nor ought to have been contemplated by the programmer. For
protection against commercially unsound contracts formed by way of algorithmic
errors, it is crucial that parties contract for such unwanted events by
allowing for the reversal of transactions when such unforeseen circumstances
arise. The problem here is that trading agreements are usually entered into
based on standard forms put forth by the platform operator, with the end users
having little say as to the terms of the agreement.
Conveniently,
in Quoine, the mistake was caused by the Platform operator, who dictates
the terms of the standard form agreements and can easily contract for such
reversals. However, should the mistake arise due to the fault of the end user’s
algorithm (ie, B2C2 or the Counterparties’ algorithms, in the case of Quoine),
then there is little that the end user can do to contractually provide for a
right to reverse transactions. The majority approach therefore overlooks
whether parties are able to bargain for risk allocation when entering trading
agreements, leaving the end users of trading platforms unduly vulnerable.
(ii) Distinction between
contemplation and possibility: black swan events
In Quoine,
B2C2 knew of the possibility that the mistake would occur but, because
such an event was unlikely, never contemplated its actual occurrence.[14] Based on this finding, the Court of
Appeal affirmed the Singapore International Commercial Court’s decision[15] that B2C2 did not have actual or
constructive knowledge of the mistake since the B2C2 programmer had not turned
his mind in any detail to the circumstances in which the mistake would happen.
By drawing
a distinction between the possibility of a mistake occurring and the
contemplation of that possibility, end users become more vulnerable to black
swan events.[16] Such events are rare, and therefore
may never be in the contemplation of the non-mistaken party, yet there is a
real risk that they will occur and lead to severe consequences. The majority
approach precludes a finding of unilateral mistake under black swan events
since the non-mistaken party would not have considered the real possibility
of it happening. This further exacerbates the vulnerability of end users.
(iii) Impracticality and
unfairness
Low and Mik
posit that the majority’s approach unduly focuses on the deterministic nature
of the algorithm, using it as the grounds on which to assess the programmer’s
knowledge at the time of programming by working backwards.[17] This overlooks how market conditions
(and therefore prices) are likely to change between programming and
contracting, and the parties’ state of mind at the time of contracting has to
be evaluated in light of the prevailing market conditions.[18] ‘Working backwards’, merely because
the algorithm is deterministic, would require the programmer to have
near-prophetic knowledge which would undermine the doctrine’s practical
operation tremendously.[19] The authors therefore argue
that the focus should be on market indeterminacy and not algorithmic
determinism. Moreover, the authors identified that the Court of Appeal’s
approach may be largely impractical in certain situations. It is convenient
that Mr. Boonen, who programmed B2C2’s algorithm, was also the director of B2C2
who made important business decisions.
In reality,
this is the exception rather than the norm. Complex algorithms are generally
written by multiple programmers over long periods of time, and these
programmers typically follow the instructions of their bosses or clients. It is
therefore appropriate for a distinction to be drawn between writing the program
(a purely technical activity) and setting the transactional parameters
(involving commercial decisions), as aptly demonstrated by Software Solutions Partners Ltd, R (on the application of) v HM
Customs & Excise.[20] It would be useful for the Court of Appeal to
clarify whose knowledge under such circumstances should be assessed, as well as
the time at which to do so.
Increasingly,
parties are also utilising ‘off the shelf’ algorithms, which are programmed by an
external party and made available for purchase in a digital library. Surely, it
would seem absurd to trace knowledge of a mistake back to the programmers of
such software. Applying the majority approach in Quoine to such situations
would find the mistaken party having to prove a prophetic and dishonest intent
on the party using the algorithm, which seems plainly unfair. Further, the
commercial reality is that the party employing the algorithm is also likely to
be in a stronger bargaining position. Surely the doctrine of unilateral mistake
ought not to exacerbate inequality.[21]
The Minority’s Approach
(i) Third-party interests
While Mance
IJ’s approach appears to resolve the impracticality of the majority approach, problems
arise with Mance IJ’s insistence on a purely equitable doctrine of unilateral
mistake for automated contracts—supposedly because these contracts do not fit within
the principles governing unilateral mistake at common law. This is especially since
the operation of algorithms might lead to third-party interests arising before
any human discovery of the mistake.[22]
Assuming Mance
IJ was referring to algorithms which are capable of both contract formation and
contract performance, if third-party interests are the sole reason for voiding
contracts at common law, perhaps the consequences of unilateral mistake ought
to be revised generally, as opined by Mik and Low.[23]
(ii) Reliance interests
In
Singapore and England, benefits transferred under a contract that has been
rescinded are reversed in the law of unjust enrichment. The law, however, will not
allocate losses incurred in reliance of the transaction in the absence of any
transferred benefits. Perhaps it would be apt for the court to protect the
reliance interest of users since foreign jurisdictions have done so. In Germany,
non-mistaken parties will be entitled to compensation if their reliance on the
rescinded contract was justified. However, they might lose the right to
compensation if they knew of the grounds for rescission vitiating the contract
or ought to have known it (and were therefore negligent).[24] Provisions where reliance is given
similar protection can also be found in the Principles of International Commercial
Contracts[25] and the
Second Restatement of Contracts in the US[26].
VI. Conclusion
The
majority approach in Quoine remains the law on algorithmic contracts in
Singapore. Nevertheless, it remains to be seen how the Singapore courts will
assess knowledge in the context of non-deterministic algorithms (ie,
machine learning) whereby the same input may lead to differing outputs. It is
also clear that the approach in Quoine will benefit from further
refinement over time. Until then, it seems prudent for parties to contract for the
right to reverse a transaction should algorithmic errors occur.
* LLB (Candidate) (NUS), Class of 2023. All errors and views expressed in this article remain my own.
[1] [2020] SGCA(I) 2 [Quoine].
[2] [2005] 1 SLR(R) 502 [Digiland].
[3] Ibid at paras 34 and 80.
[4] Ibid at para 37.
[5] Ibid at para 80.
[6] Quoine, supra note 1 at para 82 defines deterministic algorithms as those which ‘will always produce precisely the same output given the same input’.
[7] At the material time, the market rate was approximately 0.04 BTC to 1 ETH. The impugned trades in Quoine
were made at the rate of 9.99999 BTC to 1 ETH.
[8] Quione,
supra note 1 at
para 115.
[9] Ibid at para 92.
[10] Quione, supra note 1 at para
99.
[11] Ibid at para 104.
[12] Ibid at para 99.
[13] Ibid at paras 181-183.
[14] Quione, supra note 1 at para 121.
[15] B2C2 Ltd v Quoine Pte Ltd [2019] SGHC(I) 3.
[16] A black swan event is an unpredictable event that is beyond what is
normally expected of a situation and has potentially severe consequences.
[17] Kelvin F.K. Low & Dr Eliza Mik, “Lost in Transmission: Unilateral Mistakes in Automated Contracts” Law Quarterly Review (forthcoming, Jun 2020) [Low and Mik] at p 3.
[18] Ibid.
[19] Ibid.
[20] [2007] EWHC 971.
[21] Low and
Mik, supra note 17 at p 4.
[22] Quoine, supra note 1 at para 182.
[23] Low and Mik, supra note 17 at p 5.
[24] § 122 BGB (Germany).
[25] Article 3.2.2(1)(a), Principle of International Commercial Contracts.
[26] §
153 Restatement (Second) of Contracts, The American Law Institute.