Module 2 - Rational Decisions Under Risk
Quick Access to Introduction | Risky Decision | Rational Decision as Expected Utility Maximization | Utility and Probability
An Argument for Expected Utility Maximization | Arguments Against Expected Utility Maximiaztion | An Alternative to MaxEU: Risk Aversion | Further Reading | Self Test
Instructed by Prof. Peter Hawke & Prof. Darrell Rowbottom
Introduction
What is the rational way to make a decision when the available options are all risky? The aim of this module is to introduce a classic but contentious proposal: to choose rationally is to maximize expected utility. We outline some famous arguments for and against this proposal, and consider one alternative.
Risky Decisions
To face a decision problem is to choose between different options (e.g., different actions) that lead to different possible outcomes. Typically, which outcome occurs depends partly on the choice of the decision-maker and partly on factors beyond her control. Typically, some outcomes are more preferable than others.
There is often a reasonable way to represent whether outcomes are “better”, “worse”, or “equal” to each other by attaching a real number to each outcome (situation). We call this the utility of the outcome and write u(O) to represent the utility of outcome O. If u(O1) > u(O2), this represents that O1 is strictly better (preferable) to outcome O2, while O2 is strictly worse than O1. If u(O1) = u(O2), this represents that O1 and O2 are equal —we are indifferent between them. Finally, if u(O1) ≥ u(O2), then O1 is better than O2 or equal to it.
Utility cannot always be equated with a mere monetary payoff. We will have more to say about this, and the nature of utility, in Section “Utility”. However, for many illustrative decision problems (like those we survey) one can equate, for simplicity, the utility an outcome produces with the amount of money the decision-maker gains or loses in that outcome.
A probability is a real number between 0 and 1 that measures how likely an outcome is: if an outcome has a probability of 0, it is minimally likely; if an outcome has a probability of 1, it is maximally likely. A decision problem is risky when a probability can be assigned to each possible outcome. In this case, we call each available option a gamble. We write p(O) for the probability of outcome O.
Simple decision problems can be represented as a table. A decision between two gambles, with two possible outcomes for each, looks like this:
State 1 | State 2 | |
Gamble 1 | Outcome 1 | Outcome 2 |
Gamble 2 | Outcome 3 | Outcome 4 |
A state (of the world) is a way for things to be that (i) is independent of the decision-maker’s choice but (ii) determines, along with the choice, which outcome occurs. In the above table, the rows correspond to gambles; the columns correspond to states; and each entry corresponds to an outcome, recording the utility associated with that outcome.
Rational Decision as Expected Utility Maximization
Is there a principle that precisely determines which gamble to choose in a risky decision problem? Here is a highly influential proposal:
The principle of expected utility maximization (MaxEU). To choose rationally in a risky decision problem is to choose a gamble with the highest expected utility.
The expected utility of a gamble is the sum of the utility of the possible outcomes of that gamble, each weighted by that outcome’s probability. More precisely, for gamble G:
eu(G) = p1u1 + p2u2 + ... + pnun
where O1,...,On are the possible outcomes of G and pi = p(Oi|G) and ui = u(Oi) for every i between 1 and n, where p(O|G) is the probability of O given that G is chosen. MaxEU says that it is rational to pick G exactly when eu(G) is as high as it gets, out of the available gambles.
Simple examples illustrate the appeal of MaxEU.
Utility and Probability
There are two key parts of expected utility. The first is the utility of outcomes, given the choice of a gamble. The second is the weights of how likely these outcomes are given that gamble. For instance, in Example 2, the utilities are listed in the rows. To apply MaxEU to that problem, each of these is weighted by the probability and then summed for that action. However, utility and probability are themselves ambiguous ideas. Decision theorists disagree about exactly what we should mean by these terms.
1. Utility
The amount of utility one receives in a certain outcome cannot always be equated with the monetary payoff (if any!) that one receives in that outcome. For example, you would be more worried about losing your last dollar than losing one of many dollars you own, because your last dollar would have more utility for you. Similarly, marginal utility is the extra utility you receive from something. Your marginal utility from winning 1 million in a lottery would be greater than your marginal utility from somehow winning another 1 million in a lottery the next week. The money is the same, but the utility is not.
Whose utility is important? That depends on what we are doing. If we are making decisions for someone else’s benefit, then their utility is the relevant quantity. If we are making deci- sions for just our own benefit, then our utility is the relevant quantity. These utilities can be related: for example, if you want someone to get what they want, then your utility depends on their utility.
However, it is often hard to work out what others want. It can even be hard to work out what we want! Stories are full of people struggling to determine what they want. It is very hard to identify and quantify our preferences when picking a career, making a policy decision, or choosing whether to marry someone. Since utilities are essential elements in expected utilities, this means that it is hard or even impossible to calculate our expected utilities.
One way to quantify a person’s expected utilities is to look at their actual decisions and assume that they were expected utility maximising for that person. You could also apply this approach to your own utilities, by looking at your own behaviour. So, if someone chose to make a particular gamble, then that was because they thought it would maximise their utility. If we can also learn the probabilities that they are using in their decision-making, then we can also calculate their utilities. This approach to utility is called revealed preference theory, because it involves assuming that a particular person’s actions will reveal their expected utilities.
Revealed preference theory works better in some contexts than others. For example, people’s decisions are not always rational in the MaxEU sense. When that happens, their choices can be misleading about their true preferences. Another problem is that people’s utilities can change. You might care a lot about money and luxury one day, then read a book about life that convinces you that money and luxury are not so important. Your actions that occurred before you read the book will be a misleading guide to your expected utilities after you read it. Decision theorists disagree about the extent to which we can use revealed preferences.
Andrey Kolmogorov
2. Probability
What about probability in decision theory? A function takes things in its domain (such as the set of all possible events) and gives an output (such as a number). A probability function p is a function that maps events to real numbers, with the following features:
Pr1. p assigns a number between 0 and 1 to every event (or, equivalently, between 0% and 100%), including negative events (e.g., the coin does not land tails), disjunctive events (e.g., either Ann chooses ice-cream or Ann chooses cake for dessert) and conjunctive events (e.g., Ann chooses ice-cream and Ann chooses cake for dessert);
Pr2. If e1 and e2 are events, the probability that p gives to the disjunctive event e1 or e2 is:
p(e1 or e2) := p(e1 and not-e2) + p(e2 and not-e1) + p(e1 and e2);
Pr3. p assigns 1 to events that are guaranteed to occur (e.g., p(e or not-e) = 1) and p assigns 0 to events that are guaranteed not to occur (e.g., p(e and not-e) = 0).
When these are spelled out with mathematical precision, they are called the Kolmogorov axioms, after a mathematician that pioneered the mathematics of probability.
The interpretation (meaning) of the numbers involved in probability calculations is a controversial subject. There are many different interpretations of probability. One popular idea during the early years of modern decision theory (in the mid-20th century) was to focus on our estimates of what would happen if some event was repeated over and over again. For example, imagine a fair coin flip. If the coin was flipped repeatedly and it never wore out, then sooner or later a pattern would emerge, with the frequency of the coin landing heads becoming closer and closer to 50%. According to a Frequentist view of probability, it is this fact that would make 0.5 the probability of a coin flip landing on heads.
A Frequentist analysis can make sense of many traditional decision problems, where we are making a decision about a repeatable event like flipping a coin, cars crossing a bridge that we have engineered, or taking a medicine. However, what about a unique event? For instance, in managing a nuclear power plant, we might be interested in whether a particular set of particles decay in a particular time period. Even though we are interested in a single non- repeatable event, it seems like we can still talk about the probability that the particles will decay. Even in a gambling problem, imagine we are throwing away objects for recycling. We might know that a deck of cards will never be used again. Yet, if we are about to randomly take a card from the deck, might still know the probability of randomly selecting the Ace of Spaces from the deck. The general problem is that when using a Frequentist interpretation (meaning) of probability we can only ever talk about the probability of a type of event, like coin tosses in general. So it is unclear how to use this way of understanding probability when making decisions about unique events.
For this reason, philosophers like Karl Popper developed the Propensity view of probability. Let’s use the word “experiment” in a very general way, to include any particular situation with clearly specified features, like a coin toss or a group of particles being in a nuclear reaction. On the Propensity view, each experiment has physical properties of the objects involved that determine what would happen if the experiment was repeated, and it is these that determine the probability. For example, although each particle’s radioactive decay is unique, a particle has physical properties that determine how frequently it would decay if it could somehow be recreated repeatedly. In general, with a Propensity view, we can apply MaxEU to unique or repeatable events, provided we know enough about the physical properties of the experiment. The Propensity interpretation differs from the Frequentist interpretation since propensities are the causes of relative frequencies, rather than the relative frequencies themselves.
On either a Frequentist or Propensity view, we can only sometimes know the probability and apply standard MaxEU when making our decisions. Often, we will only be able to approximately estimate the probability: using statistical methods, we can estimate that a coin toss’s probability of landing heads is roughly 0.5, but not exactly. However, many decision theorists have a different view of probability. According to them, the probability that an event occurs is how confident we are about that event occurring. This view of probability is called the Subjective Bayesian interpretation of probability. Our confidence that an event will occur is called a credence.
Karl Popper
Edwin Jaynes
In the Subjective Bayesian interpretation of probability, if our credences can be calculated by a probability function p, then they are our probabilities. So, if you are very confident that a coin toss will land heads, then your probability that the toss will land heads is a value close to 1. If you are very confident that a coin toss will not land heads, then your probability is close to 0. Insofar as you are uncertain (less sure about) what will happen, your probability is closer to 0.5.
This interpretation of probability is a very popular option among decision theorists using MaxEU. It seems to often give us the probabilities that we need to calculate expected utilities. For example, most important economists who have contributed to decision theory, like Leonard J. Savage, Milton Friedman, Frank Ramsey, and John von Neumann, all adopted this interpretation of probability for decision theory.
However, decision theorists like Edwin Jaynes agree with Subjective Bayesians that probability should be about beliefs, but criticise the idea that the relevant credences are those that we happen to believe. Instead, the credences that we should use are those that we would have if we were rational. So, according to this Objective Bayesian view, probability is rational credence. What’s rational? Objective Bayesians have many ideas about what makes a credence rational and they don’t all agree. However, Objective Bayesians like Jaynes think that there is usually a unique rational credence, whereas Subjective Bayesians think that there is usually an infinite number of possible credences that are just as “rational” as any others.
Their debates are part of a wider debate among Bayesians, who are thinkers that use either Subjective Bayesianism or Objective Bayesianism. In practice, even Subjective Bayesians rarely think that the right credence is purely a matter of personal choice or psychological peculiarity. For example, imagine someone choosing whether to buy fire insurance. Many Subjective Bayesians would say that the probabilities and hence the rational choices according to MaxEU are not totally unconstrained. For example, they would say that you should not be absolutely certain that a fire will not occur, because you know that your house could burn down.
Similarly, many Objective Bayesians think that there are some types of decision problem where there is not a uniquely rational credence. So there is a spectrum in among Bayesians: at one extreme, there are always unique credences that we should have if we were perfectly rational; at another extreme, any credences that we can calculate using a probability function are perfectly acceptable for MaxEU.
There are other interpretations of probability, but these examples indicate how it’s a controversial subject. The meaning that you are giving to the probabilities when you make a decision is very important for how you should calculate the expected utilities.
An Argument for Expected Utility Maximization
Is MaxEU correct? We’ve seen that it matches intuition in some simple cases. A more general argument is that it follows from a short list of basic and intuitive claims (‘axioms’), purportedly describing how utility should rationally be assigned to gambles in a risky decision problem, given an assignment of utility to outcomes. A mathematical proof then establishes that if the axioms hold, the utility thereby attached to a gamble G is exactly eu(G). So, a commitment to MaxEU requires only that one accept the axioms and that rationality favors gambles with the highest utility.
Theorists have tabled various lists of axioms. Here are four (due to Martin Petersen) that together will do the job:
EU1. If all outcomes of a gamble have utility u, then the utility of the gamble is u.
EU2. If one gamble is certain to lead to better outcomes under all states than another, then the utility of the first gamble exceeds that of the latter; and if both acts lead to equal outcomes, they have the same utility.
EU3. Every decision problem can be transformed into a decision problem with equally probable states, in which the utility of all gambles is preserved.
EU4. If two outcomes are equally likely, and if the better outcome is made slightly worse, then this can be compensated for by adding some amount of utility to the other outcome, such that the overall utility of the gamble is preserved.
Arguments Against Expected Utility Maximization
An important objection to MaxEU is that there are decision problems where MaxEU gives arguably counter-intuitive judgments about the rational course of action. In some problems, it arguably allows irrational choices. In other problems, it does not allow apparently rational choices. As MaxEU has independent appeal, such cases are usually labeled as paradoxical. We note three prominent examples.
2. The St. Petersburg paradox
Our second paradox is due to Daniel Bernoulli. Suppose that one has the option to enter an unusual game called the St. Petersburg game: you make fair coin flips until one lands Heads, at which point you are paid $2n, where n is the number of times the coin was tossed. (Thus, there are an infinite number of different possible outcomes.) Alternatively, one can choose not to play and simply be given $1m.
Heads after 1 toss | Heads after 2 tosses | Heads after 3 tosses | … | |
Play | $2 | $4 | $8 | … |
Don't Play | $1m | $1m | $1m | … |
Should one play? This seems irrational: it is highly unlikely that the coin will land Tails
sufficiently many times for the pay-off to exceed $1m. Yet MaxEU says that playing the game is the rational choice, on the grounds that it has infinite expected utility. To see this, note
that the probability that the coin will land Heads on the n-th toss is 0.5 × 0.5 × ... × 0.5 (n times) = 1/2n
eu(Play)
= ½ ×2+ ¼ ×4 × ...
= 1+1+1+...
= ∞
3. The Ellsberg Paradox
This paradox is named after the economist Daniel Ellsberg. Imagine that you are playing a game where you bet on balls drawn from a box. You know that the balls will be randomly drawn and that they can be red, yellow, or black. You also know that 1/3 of the balls in the box are red. The other 2/3 are some unknown mix of yellow and black: they could be all yellow, all black, or any proportion in between.
First, you are offered a choice between two gambles, Gamble A and Gamble B. If you choose Gamble A, then you win $100 if the randomly drawn ball is red, and win nothing otherwise. If you choose Gamble B, then you win $100 if the randomly drawn ball is yellow, and nothing otherwise.
Red | Yellow | Black | |
Gamble A | $100 | $0 | $0 |
Gamble B | $0 | $100 | $0 |
Second, you are offered a choice between Gamble C and Gamble D. If you choose Gamble C, then you win $100 if the randomly drawn ball is red or black, and win nothing otherwise. If you choose Gamble D, then you win $100 if the randomly drawn ball is yellow or black, and nothing otherwise.
Red | Yellow | Black | |
Gamble C | $100 | $0 | $100 |
Gamble D | $0 | $100 | $100 |
Daniel Ellsberg
In this type of problem, most people prefer Gamble A to Gamble B and Gamble D to Gamble C. One possible explanation is that most people want to avoid making “ambiguous” decisions, which are situations in which the probabilities in people’s choice-making are not based on prior knowledge of relative frequencies or propensities. In the choice between Gamble A and Gamble B, you know that 1/3 of the balls are red, and that you will win with Gamble A if one of these balls is selected. Therefore, you seem to know more about the relative frequencies and the “experiment’s” propensities with respect to selecting red balls. In contrast, you do not know the relative frequency or propensity of selecting yellow balls. Similarly, in the choice between Gamble C and Gamble D, you know that 2/3 of balls are yellow or black (the winning condition for Gamble D) but you do not know the relative frequency or propensity of selecting red or black balls (the winning condition for Gamble C).
However, while there are ways in MaxEU of assigning probabilities that justify preferring Gamble A to Gamble B or Gamble D to Gamble C, the combination is forbidden. If you think that the proportion of black balls is greater than 1/3, then you can justify preferring Gamble A to Gamble B, because you think that you are more likely to win the $100 prize. Yet if the proportion of black balls is greater than 1/3, then the randomly drawn ball is more likely to be red or black than yellow or black, so you should prefer Gamble C to Gamble D.
Imagine you guess that 3/6 of balls are black. Since you know that 1/3 = 2/6 are red, it follows that just 1/6 are yellow. Multiplying by the prize moneys, we find:
eu(Gamble A) = (2/6) × (100)
eu(Gamble B) = (1/6) × (100)
Therefore, eu(Gamble A) > eu(Gamble B).
However, with these assumptions, the probability of winning with Gamble D is 3/6 + 1/6, while the probability of winning with Gamble C is 2/6 + 3/6. Therefore:
eu(Gamble C) = (5/6) × (100)
eu(Gamble D) = (4/6) × (100)
Therefore, eu(Gamble C) > eu(Gamble D).
On the other hand, if you think that the proportion of black balls is less than 1/3, then you can justify preferring Gamble D to Gamble C. Yet if the proportion of black balls is less than 1/3, then the randomly drawn ball is more likely to be red than yellow, so you should prefer Gamble A to Gamble B.
Imagine you guess that just 1/6 of balls are black. Since 2/6 are red, it follows that 3/6 are yellow. Then, 3/6 balls are red or black, whereas 4/6 are yellow or black, so:
eu(Gamble C) = (3/6) × (100)
eu(Gamble D) = (4/6) × (100)
Therefore, eu(Gamble D) > eu(Gamble C).
This justifies a preference for Gamble D over Gamble C, but not a preference for Gamble A over Gamble B:
eu(Gamble A) = (2/6) × (100)
eu(Gamble B) = (3/6) × (100)
Therefore, eu(Gamble B) > eu(Gamble A).
Finally, if you think that there are equal proportions of 1/3 for each type of ball, then you should be indifferent between Gamble A and Gamble B, because both have expected utilities of (1/3)(100). You should also be indifferent between Gamble C and Gamble D, because both have expected utilities of (5/6)(100). Hence, there is no way of justifying most people’s preferences in the Ellsberg Paradox using MaxEU.
So, many people seem to have preferences that are forbidden by MaxEU. Unlike some other non-MaxEU choices, we cannot show that their decisions will lead them to lose money or make them more likely to lose money. Thus, the Ellsberg Paradox is puzzling: there is an apparently rational pair of preferences that a decision-maker might have, which cannot be rationalised using MaxEU.
An Alternative to MaxEU: Risk Aversion
Some take the moral of the Allais paradox to be that MaxEU overlooks the possibility of rational risk aversion. We can sharpen this observation.
Suppose you can choose between two options. The first is that you receive $1m, no strings attached. The second is that you win $2m if a fair coin flip lands Heads, and $0 if it lands Tails.
Lands Heads | Lands Tails | |
Gamble 1 | $1m | $1m |
Gamble 2 | $2m | $0 |
MaxEU implies that the options are equally rational:
eu(Gamble2)= ½×2m + ½×0=1m=eu(Gamble1)
But, again, it is intuitive that a perfectly rational person might be risk averse, in which case Gamble 1 seems the obviously preferable option. If so, MaxEU fails to (sometimes) prescribe the most rational action.
Economists such as Kenneth Arrow appeal to the diminishing marginal utility of money to explain the irrationality of choosing Gamble 1: if the utility of receiving $2m is less than double that of receiving $1m, then a guaranteed $1m has higher expected utility than a toss- up between $2m and $0. The problem with this explanation, however, is our example can easily be reframed so that one chooses between a guaranteed x utility units and a toss-up between 2x utility units and nothing. Again, risk aversion here seems perfectly rational, but questions about the relative utility of money are now irrelevant.
Kenneth Arrow
Lara Buchak
As an alternative strategy, Lara Buchak proposes that we replace MaxEU with:
Risk-weighted MaxEU. To choose rationally in a risky decision problem is to choose a gamble with the highest risk-weighted expected utility, where the risk-weighted expected utility of gamble G is the sum of the utility of the possible outcomes O of G, but each weighted by r(p(O|G)), where p(O|G) is the probability of outcome O (given G) and r is a function that measures how much weight one gives an outcome in light of its probability, taking into account one’s level of risk aversion.
As a toy example, suppose that r(p) = p2. Intuitively, this captures systematic risk aversion. For example, if p(O)=1/4, then the weight given to O by r is 1/16. If p(O)= 9/10, then the weight given to O by r is 81/100. If p(O)=1,then the weight given to O by r is 1.
Given this r, Risk-weighted MaxEU implies that Gamble 2 isn’t a rational choice in the above example:
reu(Gamble2)= ¼ ×2m + ¼ × 0 = 500,000 < 1m = reu(Gamble1)
Note that this result depends on taking Gamble 1 to lead to exactly one possible outcome (i.e., receiving $1m with a probability of 1), while Gamble 2 leads to two possible outcomes (i.e., receiving $2m with a probability of 0.5 and receiving $0 with a probability of 0.5).
The case against MaxEU isn’t yet decisive, however. For example, it is no simple matter to weigh up an argument along the lines of that in the previous Section against the intuitions that inform the alleged counterexamples to MaxEU. Thus, the debates about MaxEU and proposed changes to it are likely to continue for many years to come.
Buchak, L. (2013), Risk and Rationality, Oxford University Press.
Jeffrey, R. (1983) The Logic of Decision, 2nd edn., University of Chicago Press.
Peterson, M. (2004) ‘From outcomes to acts: A non-standard axiomatization of the expected utility principle,’ Journal of Philosophical Logic 33: 361–78.
Peterson, M. (2017) An Introduction to Decision Theory (Cambridge Introductions to Philosophy), 2nd edn., Cambridge University Press.
Rowbottom, D. (2015) Probability (Key Concepts in Philosophy), Polity.
Savage, L.J. (1954) The Foundations of Statistics, John Wiley & Sons. (2nd edn. 1972, Dover.)
Q1) Suppose you enter the following game. A standard coin is to be flipped. The flip is biased: the probability that the coin will land Heads is 0.65 and the probability that it will lands Tails is 0.35. You win a prize of $2m if you choose Heads and the coin lands Heads; you win a prize of $5m if you choose Tails and it lands Tails; otherwise, you get nothing. Assuming that all you care about is the amount of money you receive (and the more the better), which option does MaxEU recommend that you choose?
Q2) Suppose that your level of risk aversion is captured by the risk weighting function r(x)=x2. Now recall the decision problem you faced in Question 1. What option does Risk-weighted MaxEU recommend that you choose?
Correct!
reu(Choose Heads) = r(0.65) × 2m + r(0.35) × 0 = 0.652 × 2m = 0.4225 × 2m = 845,000
reu(Choose Tails) = r(0.65) × 0 + r(0.35) × 5m = 0.352 × 3m = 0.1225 × 5m = 612,500
As reu(Choose Heads) is strictly greater than reu(Choose Tails), Risk-weighted MaxEU recommends that you choose Heads.
(Incorrect) The correct answer is Heads.
Q3) You are to flip a standard coin in a fair way. We write H for the event that the coin lands with Heads facing up; we write T for the event that it lands with Tails facing up. As the flip will be fair, we can calculate the probability of different possible outcomes of the flip with a probability function p that says: p(H) = p(T). Further, as the coin will land one and only one way, our basic information about the set-up also includes:
- the events ‘H’ and ‘both H and not-T’ are identical
- the events ‘T’ and ‘both not-H and T’ are identical
- the event ‘both H and T’ is impossible
- the event ‘either H or T’ is guaranteed to happen
Given just this basic information, what do the axioms PR1-PR3 imply that p(H) equals?
Correct!
As the event ‘H or T’ is guaranteed to happen, it follows from axiom PR3 that:
(1) p(H or T)=1
On the other hand, it follows from axiom PR2 that:
(2) p(H or T)= p(H and not-T) + p(not-H and T) + p(H and T)
The event ‘H and not-T’ is identical to H, and ‘not-H and T’ is identical to T. Further, as ‘H and T’ is impossible, it follows from PR3 that p(H and T)=0. Thus, (1) and (2) imply:
(3) p(H) + p(T) = 1
Finally, recall that p(H) = p(T), since the coin flip is fair. Thus, (3) implies:
(4) p(H) + p(H) = 1
So, p(H) = ½.
(Incorrect) The correct answer is ½.
(Incorrect) The correct answer is ½.
Return to the Module List