The American Statistician ISSN: 0003-1305 (Print) 1537-2731 (Online) Journal homepage: http://www.tandfonline.com/loi/utas20 Reply: Hypotheticals and Hypotheses Joseph L. Gastwirth , Abba M. Krieger & Paul R. Rosenbaum To cite this article: Joseph L. Gastwirth , Abba M. Krieger & Paul R. Rosenbaum (1997) Reply: Hypotheticals and Hypotheses, The American Statistician, 51:2, 120-121 To link to this article: http://dx.doi.org/10.1080/00031305.1997.10473943 Published online: 17 Feb 2012. Submit your article to this journal Article views: 39 View related articles Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=utas20 Download by: [University of Massachusetts, Amherst] Date: 22 March 2016, At: 15:40 Reply: Hypotheticals and Hypotheses Downloaded by [University of Massachusetts, Amherst] at 15:40 22 March 2016 Joseph L. GASTWIRTH, Abba M. KRIEGER, and Paul R. ROSENBAUM Mintz and Dixon offer a plausible hypothetical, but a fairly implausible hypothesis. By a hypothetical we mean a claim that a particular conclusion would follow from a particular premise, with no claim that either the premise or the conclusion is true. A hypothetical is what Nelson Goodman (1965), for instance, calls a “counterfactual conditional” or a statement about a “possible world.” Clark Glymour (1986) offers a typical example: “. . . if I had been a girl, I would have been named Olga.” A hypothetical may be correct as a hypothetical even if it is wrong front and back, that is, even if both the premise and the conclusion are wrong as descriptions of the actual world. Clark is not a girl and was not named Olga, but the hypothetical may nonetheless be true as a hypothetical. Some hypotheticals are important and true as hypotheticals, for instance: “if q,1c2, . . . is a Cauchy sequence of real numbers, then the sequence converges to a limit.” By a hypothesis we mean a claim, perhaps tentative or ultimately erroneous, that something is actually true, and moreover, a claim that is at risk of being refuted on the basis of empirical observations. In this sense a hypothesis is what Sir Karl Popper (1965) calls a “conjecture.” Popper discusses carefully and at length what is meant by saying that a conjecture is at risk of refutation, but suffice it to say here that a conjecture must be sufficiently specific about something that can be observed that observations can lead to strong evidence against the conjecture. Hypotheticals and hypotheses may be true or false, plausible or implausible, believed correctly or incorrectly, but they are, nonetheless, very different claims. Consider the sentence: If there were a single quantity called “cognitive ability,” if this one quantity were a major determinant of successful performance in clerical work at the Revenue Service, if test scores were reliable and valid measures of this quantity, and if men had vastly more “cognitive ability” than women at the Revenue Service, then this large unobserved disparity in cognitive ability would be a business necessity, justifying promotion of men at twice the rate of women. That sentence is a hypothetical. Consider an alternative sentence: There is a single quantity called “cognitive ability,” and it determines successful clerical work at the National Revenue Service, and the test is a reliable and valid measure of this quantity, and moreover, men employed in clerical jobs at the Revenue Service have vastly more “cognitive ability” than similarly employed women, so this large unobserved disparity in cognitive ability between men and women is a “business necessity,” and it justifies promoting men at twice the rate of women. The alternative sentence is a hypothesis. The hypothetical is not implausible as a hypothetical, but as with poor, counterfactual Olga, the hypothetical may be false front and back. The hypothesis does not strike us as particularly plausible. Mintz and Dixon even say that they do not regard the hypothesis as plausible. They present calculations that support the hypothetical, but no evidence in support of the hypothesis. So Mintz and Dixon have offered a plausible hypothetical, but neither they nor we regard it as a particularly plausible hypothesis. Consider, now, the legal context into which their hypothetical is offered. The courts have established guidelines for the shifting burdens of production of evidence in discrimination cases. Although the data arose in a disparate impact case, the framework the Supreme Court laid out in the disparate treatment case, Texas Dep’t. of Comm. Affairs v. Burdine, 450 U.S. 248 (1981) describes the burden of production placed on a defendant who needs to respond to a legally meaningful and statistically significant disparity. The opinion states that the defendant needs to “produce evidence that the plaintiff was rejected or someone else was preferred for a legitimate non-discriminatory reason” and “[tlhe explanation provided must be legally sufficient to justify a judgment for the defendant.” Then the factual inquiry proceeds to a new level of specificity as the plaintiff now has the opportunity of showing that the defendant’s reason was not the true reason, and by persuading the Court that a discriminatory reason more likely motivated the employer. In discussing the defendant’s burden of production, the Court noted that the defendant’s reason should “frame the factual issue with sufficient clarity so that the plaintiff will have a full and fair opportunity to demonstrate pretext. The sufficiency of the defendant’s evidence should be evaluated by the extent to which it fulfills these functions.” Thus the Court is requiring the defendant to provide substantial, tangible evidence in support of what we have called a hypothesis, not merely to assert a hypothetical. The hypothesis above concerns, not just the claim that men employed in clerical positions at the Revenue Service have vastly more “cognitive ability” than women, but also that this leads to job-related characteristics that are necessary to the business. As the Supreme Court stated in Griggs v. Duke Power Co., 91 S.Ct. 849 (19711, “The Act proscribes not only overt discrimination but also practices that are fair in form but discriminatory in operation. The touchstone is business necessity. If an employment practice which operates to exclude Negroes cannot be shown to be related to job performance, the practice is prohibited.” The opinion goes on to state that although the Civil Rights Act of 1964 allows employers to use tests or other measures, “Congress has forbidden giving these devices and mechanisms controlling force unless they are a demonstrably reasonable measure of job performance.” z zyxwvutsrq zyxw zyxwvuts zyxwvu z zyxw zyxwvutsr Joseph L. Gastwirth is Professor, Department of Statistics, George Washington University, Washington, DC 20052. Abba M. Krieger is Professor and Paul R. Rosenbaum is Professor, Department of Statistics, University of Pennsylvania, Philadelphia, PA 19104-6302. 120 The American Statistician, May 1997, Vol. 51, No. 2 @ 1997 American Statistical Association Downloaded by [University of Massachusetts, Amherst] at 15:40 22 March 2016 zyxwvutsrqpo zyxwvuts zyxw In Albemarle Paper Co. v Moody, 422 U.S. 405,422 (1975), a case concerning the required proof of business necessity, which is discussed in Gastwirth (1988, pp. 381383), the Court noted that after the complaining party has shown that the tests or requirement in question have a disparate impact on a protected group, the employer has the burden of showing that the requirement has a manifest relationship to the job. The opinion notes that if the employer demonstrates job relatedness, the complainant has the opportunity to show that other selection procedures could serve the employer’s legitimate interest in an efficient workforce without such an undesirable effect on minority applicants. Mintz and Dixon specifically say that, in the general population, they do not believe men have vastly greater “cognitive ability” than women, so they would be hard pressed to argue that women with acceptable “cognitive ability” are unavailable for promotion. Moreover, in the Canadian case all test takers were already employed as lower level clerical workers, and presumably were performing their current jobs satisfactorily. This suggests that a huge difference in “cognitive ability” is unlikely to exist. If there were an enormous difference in the job-related “cognitive ability” of men and women at the Revenue Service, then one would expect a pattern of superior performance evaluations among men in their present jobs, but Mintz and Dixon offer no such evidence. In short, even their hypothetical, viewed strictly as a hypothetical and not a hypothesis, is inconsistent with their own expressed beliefs. Mintz and Dixon use the Pearson correlation to describe the relationship between two binary variables, and this might mislead the unsophisticated reader because, as Gastwirth and Nayak (1996) recall, moderate Pearson correlations between binary variables may correspond to very large odds ratios. Specifically, Mintz and Dixon note that the minimum partial correlation between passing and gender occurs when the Pearson correlation between passing and some college education is 364. If, as in Tables 1 and 2, 136/366 = 37.16% of test takers pass and also 123/366 = 33.61% have some college education, then a Pearson correlation of 3 6 4 corresponds to the following bivariate distribution for test performance and college attendance among clerical workers. workers with some college have an odds of passing an IQ test that is 286 times more than clerical workers with no college. In Kendall’s Advanced Theory of Statistics, Stuart and Ord (1991, p. 995) suggest appropriate measures of association for binary data. Our paper reached two conclusions. The first conclusion concerned a particular argument that had been accepted by the court in Maloley v. National Revenue Sewice of Canada. Specifically, we concluded: “There may or may not be discrimination against women, but whether there is or is not, the explanation the Appeals Board accepted is simply wrong. The difference in passing rates [for men and women] is far larger than can be explained based on the difference in proportions attending college.” Mintz and Dixon agree with this conclusion. The second conclusion was a general statement about the role of a statistician when confronted with claims about unobserved variables. “Some arguments involving unobserved variables are just wrong. Others are possible in principle, but not plausible. Still others are entirely plausible. Faced with an argument involving an unobserved variable, the statistician’s responsibility is to clarify what that argument objectively entails and implies so that listeners may appraise whether the argument is plausible.” Plausible is much less than “true,” but much more than “possible.” To say that a claim is plausible is to say that a reasonable person might assert that the claim is true in the face of available evidence. We believe that a critic who seeks to explain an association in terms of an unobserved variable must show that that explanation is a plausible hypothesis, a plausible description of what is so, not merely a hypothetical. In this we agree with Bross (1960) and disagree with Mintz and Dixon. zyxwvutsrq zyxwv Pass Did not pass Total Some college No college Total .3221 .0140 .3361 .0495 .6144 .6639 .3716 .6284 1.oooo ~ This table has a Pearson correlation of 3 6 4 and an odds ratio of 285.57. It is difficult for us to imagine that clerical [Received June 1996.1 REFERENCES Bross, I. (1960), “Statistical Criticism,” Cancer, 13, 394-400. Reprinted in The Quantitative Analysis of Social Problems, ed. E. Tufte, Reading, MA: Addison-Wesley, 97-108. Gastwirth, J. L. (1988), Statistical Reasoning in Law and Public Policy, New York: Academic Press. Gastwirth, J. L., and Nayak, T. K. (1996), “Statistical Aspects of Cases Concerning Racial Discrimination in Drug Sentencing: Stephens v. State and US.v. Armstrong,” Technical Report, George Washington University, Dept. of Statistics. Glymour, C. (1986), “Comment: Statistics and Metaphysics,” Journal of the American Statistical Association, 8 1, 964966. Goodman, N. (1963, Fact, Fiction and Forecast, Cambridge, MA: Harvard University Press. Popper, K. (1965), Conjectures and Refutations, New York Harper & Row. Stuart, A,, and Ord, K. (1991), Kendall’s Advanced Theory of Statistics. zyxwvu zyx The American Statistician, May 1997, Vol. 51, No. 2 121