Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit

STAT 132:  Quiz 1

Spring 2022

This quiz is based on the important and pioneering article

Tversky A, Kahneman D (1974). Judgment under Uncertainty: Heuristics and Biases. Science, 185, 1124– 1131,

which is available in the Pages tab of the course Canvas page. This article has been cited almost 45,000 times (an enormous citation count) over the past 49 years; it was written by two cognitive psychologists from the Hebrew University in Jerusalem (Israel), Amos Tversky (AT, now dead, unfortunately) and Daniel Kahneman (DK, still alive, and winner of the 2002 Nobel Prize in Economics for his work with Tversky and extensions of that work).  They were interested in documenting the extent to which capable but untrained people reason in a logically-internally-consistent manner in the presence of uncertainty (it turns out that by and large such people don’t); AT and DK performed a number of highly original experiments to gather data illustrating the types of innate biases that capable but untrained people exhibit in settings featuring uncertainty. Since, according to the definition we’re using in this class, statistics is the study of uncertainty, this article is deeply relevant to STAT 132.

In the introduction AT and DK say that they will describe three Heuristics that people use to assess probabilities and make predictions; this serves as an organizing device for the paper. Within each Heuristic they identify three or more Mistakes that people make when utilizing that particular heuristic; the structure of the article is thus as in Table 1. Please read the article carefully, several times if necessary and useful, to familiarize Yourself with all three Heuristics and 13 Mistakes,  before working on  Your answers to the questions below.  The article is so rich with ideas that are central to uncertainty quantification that I may well offer You more than one Quiz based on it in this course.

(1)  [30 total points for this problem] In the discussion of Mistake (M1b) the paper includes a striking and really unfortunate typo:   “The  similarity  of a  sample statistic  to  a population parameter does  not  depend on the  size  of the sample . ”  Consider independent identically distributed (IID) sampling of subjects from a population, in which You record, for each of the n subjects in the sample (in which n is a finite positive integer), the value of a quantitative outcome variable Y of interest to You with finite mean θ and finite and non-zero standard deviation 0 < SD(Y) ≜ σ < ∞ . Denote by Y = (Y1 , . . . ,Yn ) the random variables representing the possible samples and by y = (y1 , . . . ,yn ) the actual observed sample; let n  =   P Yi  be the random variable representing possible sample means and let n  =   P yi  be the actual observed sample mean.  In the language of the typo quote above, the population parameter is θ and the sample statistic is n . Let us agree to

Table 1:  The structure of the AT/DK article.

Heuristics:

(H1) Representativeness: Judgments of Similarity.  Mistakes:

(M1a) Insensitivity to prior probability of outcomes.

(M1b) Insensitivity to sample size.

(M1c) Misconceptions of chance.

(M1d) Insensitivity to predictability.

(M1e)  The illusion of validity.

(M1f) Misconceptions of regression.

(H2) Availability.  Mistakes:

(M2a) Biases due to the retrievability of instances.

(M2b) Biases due to the effectiveness of a search set.

(M2c) Biases of imaginability.

(M2d) Illusory correlation.

(H3) Adjustment and Anchoring.  Mistakes:

(M3a) Insufficient adjustment.

(M3b) Biases in the evaluation of conjunctive and disjunctive events.       (M3c) Anchoring in the assessment of subjective probability distributions.

measure“the similarity of a sample statistic to a  population parameter”by computing

the root mean squared error RMSE(n ) pMSE(n ) pE(θ)2 , in which

n  and θ and the RMSE is approximately the amount (on the scale of the data) by

which we expect n  to differ from θ .  Show by means of a simple STAT 131-style

calculation that

MSE(n ) =    and   RMSE(n ) =                            (1)

[20 points] .  Briefly explain in what sense this shows that the typo quote is quite wrong, and in fact that the quote runs counter to the message that AT/DK were trying to convey in this section of the paper [10 points] .


Table 2: Results from Monte Carlo generation, in the freeware statistical computing envi- ronment R, of 15 independent sets of n = 10 IID samples from the Bernoulli(0.5) probability mass function.

set .seed(  42  )

n  <-  10

#  results

sample( #  [1]  1 sample( #  [1]  0 sample( #  [1]  1 sample( #  [1]  1 sample( #  [1]  0 sample( #  [1]  0 sample( #  [1]  1 sample( #  [1]  0 sample( #  [1]  1 sample( #  [1]  1 sample( #  [1]  1 sample( #  [1]  1 sample( #  [1]  0 sample( #  [1]  1 sample( #  [1]  0

c( 1  c( 1  c( 0  c( 1  c( 0  c( 0  c( 1  c( 0  c( 0  c( 0  c( 0  c( 1  c( 0  c( 1  c( 0

0,  1  ), n, 0  1  1  1  1  0 0,  1  ), n, 1  0  0  1  1  0 0,  1  ), n, 1  1  0  1  0  1 0,  1  ), n, 0  1  0  1  0  0 0,  1  ), n, 0  1  0  1  1  1 0,  1  ), n, 0  1  0  1  1  0 0,  1  ), n, 1  1  1  0  0  1 0,  1  ), n, 0  0  0  1  0  0 0,  1  ), n, 0  1  1  1  0  0 0,  1  ), n, 0  1  1  1  0  1

0,  1  ),  10, 0  0  1  1  1  1

0,  1  ),  10, 1  0  1  1  0  0

0,  1  ),  10, 1  1  1  0  1  1

0,  1  ),  10, 1  1  1  1  1  0

0,  1  ),  10, 0  1  1  1  0  1

replace  =  T  )

1  1

replace  =  T  )

0  1

replace  =  T  )

0  1

replace  =  T  )

1  1

replace  =  T  )

1  1

replace  =  T  )

0  1

replace  =  T  )

1  0

replace  =  T  )

1  0

replace  =  T  )

0  0

replace  =  T  )

1  1

replace  =  T  ) 1  0

replace  =  T  ) 1  1

replace  =  T  ) 0  1

replace  =  T  ) 1  1

replace  =  T  ) 0  0

apparently unusual  features

4  1s  in  a  row

1  1  0  0  1  1  0  0

0  1  0  1  0  1

1  0  1  0  1  0

5  1s  in  a  row

3  0s  in  a  row

5  1s  in  a  row

5  0s  in  a  row

4  0s  in  a  row,  3  1s  in  a  row 2  sets  of  (  3  1s  in  a  row  )  5  1s  in  a  row,  3  0s  in  a  row 3  1s  in  a  row

3  1s  in  a  row,  1  0  1  1  0  1

7  1s  in  a  row

0  0  0  1  1  1

(2)  [30 total points for this problem] R is a freeware statistical data science package that’s been in general public use since 1995; over the past 27 years it’s been the basis of Monte Carlo simulation experimental results published in thousands of good articles across the entire spectrum of quantitative scientific disciplines, and (along with Python) it’s one of the two most widely used statistical data science environments today.   Table 2 presents the results of a tiny Monte Carlo experiment I recently performed, in which I asked R to generate 15 independent sets of n = 10 IID samples from the Bernoulli( ) probability mass function. You will note that, from the point of view of pattern recognition, every one of the 15 results in Table 2 exhibits features

that are apparently unusual if R were indeed correctly generating random outcomes: 7 1s in a row, 3 instances of 5 1s in a row, 1 instance of 5 0s in a row, obvious patterns such as (i) 1  1  0  0  1  1  0  0, (ii) 0  0  0  1  1  1, (iii) 1  0  1  1  0  1, and so on.

(a) Here are two competing theories that attempt to explain the apparently unusual results in Table 2:

(I) There’s something wrong with the random number generation process in R: long runs of consecutive 0s and consecutive 1s and other obvious patterns should not occur if the random number generators are working properly.

(II) There’s nothing wrong with the random number generation process in R: long runs of consecutive 0s and consecutive 1s and other obvious patterns occur all the time if the random number generators are working properly.

Into which section (Heuristic and Mistake) of the AT/DK paper does this ex- ample fall? Explain briefly. Hint: You may find the paper at

https://www.pnas.org/doi/pdf/10.1073/pnas.1422036112 to be relevant. [10 points]

(b) Based on Your reading of the AT/DK paper and the contextual information about R given at the beginning of this problem, which theory ((I) or (II) above) seems more plausible to You? Explain briefly. [10 points]

(c) Someone says “Humans are really good at finding patterns, but this means that we’re also really good at finding patterns that aren’t really there, in the sense that if we go looking for them again they may well have vanished.”After reading the AT/DK paper and considering this problem, do You agree with this statement? Explain briefly. [10 points]

(3)  [20  total points for  this  problem]  Consider this quote from a section of the AT/DK paper.

For an illustration of judgment by representativeness, consider an individ- ual who has been described by a former neighbor as follows:  “Steve is very shy and withdrawn, invariably helpful, but with little interest in people, or in the world of reality. A meek and tidy soul, he has a need for order and struc- ture, and a passion for detail. ”  How do people assess the probability that Steve is engaged in a particular occupation from a list of possibilities (for

example, farmer, salesman, airline pilot, librarian, or physician)? . . . In the case of Steve, for example, the fact that there are many more farmers than librarians in the population should enter into any reasonable estimate of the probability that Steve is a librarian rather than a farmer. . . . Subjects were shown brief personality descriptions of several individuals,  allegedly sam- pled at random from a group of 100 professionals — engineers and lawyers. The subjects were asked to assess, for each description, the probability that it belonged to an engineer rather than to a lawyer.  In [experimental con- dition  (1)], subjects were told that the group from which the descriptions had been drawn  consisted of 70 engineers  and 30 lawyers.   In [condition (2)],  subjects  were  told that the group  consisted  of 30  engineers  and  70 lawyers. . . . Specifically, it can be shown by applying [Bayes’s Theorem] that the ratio of [the odds that any particular description belongs to an engineer rather than to a lawyer, in a comparison of the form

posterior odds in favor of engineer in condition (1)  posterior odds in favor of engineer in condition (2)

should be (0.7/0.3)2 , or  ] 5.44 [to 1]

(text inside [ ... ] edited from the original quote for clarity). To make the translation from natural language to math language, let’s use the following definitions:

E   =   (the person chosen at random is an Engineer)

L   =   (the person chosen at random is a Lawyer)

D   =   (description of the chosen person)

Pj ( · | ·)   =   (probability assessments made by a subject in condition (j)) .   (2)

By explicitly writing out Bayes’s Theorem in odds ratio form for conditions (1) and (2) and dividing the resulting expressions in the order (1)/(2), show that AT/DK are correct in their calculation that the resulting ratio should be   5.44, which is far from 1 (recall that 1 was the approximate answer given by the experimental subjects, in violation of basic probability theory). [20 points]

(4)  [30 total points for this problem] Consider the role of luck as a causal expla- nation for outcomes of human processes.

(a) Someone says “I’m very unlucky: whenever I’m driving in a hurry to make an appointment I always hit all the red lights.”Which Heuristic/Mistake combina- tion in the AT/DK paper identifies an alternative explanation that would make the luck theory unnecessary? Explain briefly. [10 points]

(b) In the NCAA men’s basketball playoff game on 2 Apr 2022 between Duke and North Carolina, a Duke player Mark Williams, who had made 71.4% of his free throws1  during the 2021–2022 regular season, prior to the playoff game — missed two consecutive free throws late in the game in a high-pressure situation in which Duke was losing; video replays showed that his first attempt went too far in the air and his second try did not go far enough. Here are two competing cause-and-effect theories that attempt to explain this outcome.

(III) Williams’s misses were caused by bad luck; after all, he typically makes almost  of his attempts.

(IV) Williams missed the free throws because he mistakenly caused the trajec- tories of his shots to be too long (attempt 1) and too short (attempt 2).

Nothing is  ever caused by bad luck  (chance);  causally attribut- ing an observable (macro- or quantum-level) outcome to chance (good or bad luck) is just an informal way for people to say that they’re uncertain about the underlying cause of the outcome .  It does  make  sense  to  say,  e .g.,  that  “I was  (lucky)(unlucky)  in how  long  it  took  me  to  get  to  Your house,”  but  this  is  not  a causal statement; it merely means that my actual transit time T was (shorter)(longer) than expected (in STAT 131 language, this is a comparison of the oberved T with its expected value E(T)) .

Having read the AT/DK paper, which of these theories do You support? Explain briefly; include a brief discussion of Your views about the strong statement in bold italic font in theory (IV). (There are no “right”and “wrong”discussions here: I’d like You to think carefully about this issue; all reasonable arguments, pro or con, will get full credit.) [20 points]