Table 1: Results for the taxicabs and human cases
Dr. Lívia Markóczy
L.Markoczy@Cranfield.ac.uk
This note is a feedback report to those who participated in the judgment under uncertainty study. First of all let me thank you all again for helping.
Each of you were given one of the three (six) problems below. For each one listed below there as a ``15 percent'' variant and an ``85 percent'' variant.
A witness reported seeing the suspect flee in a green taxicab at dusk; you know that witnesses report the color correctly 70 percent of the time for similar conditions, and incorrectly 30 percent of the time.
The candidate passed a test which may predict success in the particular job; you know that the test predicts correctly in 70\ percent of the cases, and incorrectly in 30 percent of cases.
The candidate passed a test which may predict honesty in the particular job; you know that the test predicts correctly in 70\ percent of the cases, and incorrectly in 30 percent of cases.
You were then asked to estimate the probality that the taxi was green, that the women would succeed in the job, or that the convict would act honestly in the job.
Problems like the taxicab ones have been been used in experiments like this since the 1970s. Those studies systematically show that people don't make full use of the ``base rate'' information when estimating probabilities. In what you did the base rate information is the 15 or 85 percent figure given (depending on which question you got) about the population of green taxicabs, successful women, and honest convicts.
My aim was to show that people are more likely to use the base rate information when it is given about types of people than when the base rate information is given about things like taxicabs. This is because the human mind - some researchers argue - is specifically evolved to make and use generalizations about people. In this particular case it improves the accuracy of probability judgments, but in other cases base rate information about types of people might be over used.
Your responses completely confirmed my prediction! As expected, you behaved much like others who had been given the taxicab problem and under utilised the base rate information, relaying instead mostly on the 70 percent ``individuating'' information. For those given the taxicab problem, most of you guessed answers near 70 percent irrespective of whether you were given the high base rate of 85 or the low base rate of 15. Where the base rate information was given about humans there was a much larger difference between the high base rate cases and the low base rate cases. These numbers and other statistics are shown in table 1. Statistical analysis confirms that this result is not due to chance.
Table 1: Results for the taxicabs and human cases
First of all, no one was expected to come up with the correct answer.
You were asked to give your estimation and not calculate
answers.
So your answers were in line with how
people all over the world with all levels of education judge these.
Your answers were ``right'' with respect to how the human mind works,
but ``wrong'' mathematically. Also no part of your evaluation
depends on your responses (which were completely anonymous anyway),
and you are not expected to be able to make such calculations and
will not be examined for any of these.
If you use all and only the information in the stated problems, the correct way to judge is to use ``Bayesian'' analysis. Let's work though the taxicab case for the high base rate of 85 percent green ones. Just to make things simpler, imagine that there are exactly 1000 cabs in the city, 850 are green and 150 are blue. One of these 1000 cabs was involved in a crime. We have a witness report that it was green.
First imagine the case where the the cab really was green. If the cab really is green and the witness report is accurate 70 percent of the time, then a witness would report green for 595 of these 850 green cabs.
Now consider that the cab is actually blue. There are 150 of those cabs. If a witness incorrectly reports 30 percent of the time then they may have reported green if the cab were really blue for 45 of these cases (30% of 150).
So altoghether there are 640 ways that a witness could have reported a green cab (45 ways for when the cab is blue plus 595 ways for when it is really green). Since we know that the witness did report green, the question is what proportion of these 640 ways of reported a green cab actually has a green cab behind it. So the number of correct ways to report a green cab (595) divided by the total number of ways to report a green cab (640) is the probability that any particular report of a green cab is correct. That is is 595/640 = .9296875 (approx 93%).
So the ``correct'' answer for the case with the high base rate of 85 percent is approximately 93 percent. Only two of you (out of 37 who had the relevant question) answered above 80 for the cab case, while 26 of you (of the 49 who had the relevant questions) came that close for the cases about people and a high base rate.
If you wish to test your ability to now calculate these, you might wish to try to calculate the ``correct'' answer for where the base rate is 15 percent instead of 85. The answer that that will be listed on a link from http://www.cranfield.ac.uk/~mn795a/baserate/. When more information is available (e.g., data analysis, drafts of the paper, graphs, data set, etc) it will also be appearing at that address.
Report to participants in ``judgment under uncertainty study''
This document was generated using the LaTeX2HTML translator Version 96.1 (Feb 5, 1996) Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
The command line arguments were:
latex2html -split 0 report.tex.
The translation was initiated on Fri Oct 31 18:37:15 GMT 1997