291_l22 [compatibility mode]

• Chapter 11 Testing Hypothesis
Concepts of Hypothesis Testing
• Bonus Homework, due in the lab April 20-22:
Essay “How would you test the ‘hot hand’ theory in basketball games?” (~400-600 words / approximately one typed page) • Be as specific as you can: what data to collect? how many cases to collect? What hypothesis you are testing? • A significance test checks whether data agrees characteristic of a population parameter or parameters • If the data is very unreasonable under the hypothesis, then we will reject the hypothesis • Usually, we try to find evidence against the
1. State a (null) hypothesis that you would 2. Get data and calculate a statistic (for the sampling distribution of our statistic 4. If the calculated value in 2. is very unreasonable given 3 (i.e. almost impossible), then we conclude that the hypothesis was wrong • Somebody makes the claim that “Nicotine Patch and Zyban has same effect on quitting smoke” • You don’t believe it. So you conduct the experiment and collect data: Patch: 244 subjects; 52 quit. Zyban: 244 subjects; 85 quit.
• How (un)likely is this under the hypothesis of no • The sampling distribution helps us quantify the (un)likeliness in terms of a probability (p-value) • Mr. Basketball was an 82% free throw shooter last season. This season so far in 59 free throws he only hit 40.
• (null) Hypothesis: He is still an 82% • alternative hypothesis: his percentage has • How unlikely are we going to see 52/244 verses 85/244 if indeed Patch and Zyban are equally effective? (Probability = ?) • How unlikely for an 82% shooter to hit only • A small probability imply very unlikely or impossible. (No clear cut, but Prob less than 0.01 is certainly small) • A larger probability imply this is likely and no surprise. (again, no clear boundary, but prob. > 0.1 is certainly not small) • For the Basketball data, we actually got • For the Patch vs. Zyban data, we actually • Suppose we pick alpha = 0.05, then Any probability below 0.05 is deemed “impossible” so this is evidence against the null hypothesis – we say that “we reject the null hypothesis” • Otherwise, we say “we cannot reject the null hypothesis” imply there is not enough • Notice “not enough evidence against null • “validated the null hypothesis”, “accept • It could mean there is simply not enough • If the basketball data were 14 hits out of 20 shoots (14/20 = 0.7), the P-value would be 0.16247.
• Usually we cut off ( that’s the alpha level) • A significance test is a way of statistically
testing a hypothesis by comparing the data to values predicted by the hypothesis • Data that fall far from the predicted values provide evidence against the hypothesis
• Conclusion (reject, or not reject, that – Qualitative or quantitative? – Different types of data require different test procedures– If we are comparing 2 population means, then how the SD • What is the population distribution? – Is it normal? Or is it binomial?– Some tests require normal population distributions (t-test) – We usually assume Simple Random Sampling – Some methods require a minimum sample size Either “quit smoke” or “not quit smoke” • What is the population distribution? – It is Bernoulli type. It is definitely not normal since it • The null hypothesis (H ) is the
0
hypothesis that we test (and try to find evidence against) • The name null hypothesis refers to the fact that it often (not always) is a hypothesis of “no effect” (no effect of a medical treatment, no difference in characteristics of populations, etc.) • The alternative hypothesis (H ) is a
1
hypothesis that contradicts the null hypothesis • When we reject the null hypothesis, we are in favor of the alternative hypothesis.
• Often, the alternative hypothesis is the actual research hypothesis that we would like to “prove” by finding evidence against the null hypothesis (proof by contradiction) • Null hypothesis (H0):
The percentage of quitting smoke with Patch H0: Prop(patch) = Prop(zyban)
• Alternative hypothesis (H1):
• Null hypothesis (H0):
The percentage of free throw for Mr. Basketball H0: Prop = 0.82
• Alternative hypothesis (H1):
• The test statistic is a statistic that is
• Formula will be given for test statistic, but • Test statistic:
• How unusual is the observed test statistic when the null hypothesis is assumed true? • The p-value is the probability, assuming
that H is true, that the test statistic takes values at least as contradictory to H as • The smaller the p-value, the more strongly • Sometimes, in addition to reporting the p- value, a formal decision is made about rejecting or not rejecting the null hypothesis • Most studies require small p-values like p<.05 or p<.01 as significant evidence against the null hypothesis • “The results are significant at the 5% level” Highly Significant / “Overwhelming Evidence” – The alpha-level (significance level) is a number such that one rejects the null hypothesis if the p-value is less than or equal to it. The most common alpha-levels are .05 and .01 – The choice of the alpha-level reflects how – The significance level needs to be chosen before analyzing the data
– The rejection region is a range of values such that if the test statistic falls into that range, we decide to reject the null hypothesis in favor of the alternative hypothesis • Type I Error: The null hypothesis is • Type II Error: The null hypothesis is not Type I
Correct
error
Type II
Correct
error
– Alpha = Probability of a Type I error
– Beta = Probability of a Type II error
– Power = 1 – Probability of a Type II error
• The smaller the probability of Type I error, the larger the probability of Type II error and the smaller the power • If you ask for very strong evidence to reject the null hypothesis, it is more likely that you fail to detect a real difference • In practice, alpha is specified, and the probability of Type II error could be calculated, but the calculations are usually difficult • How to choose alpha?
• If the consequences of a Type I error are very serious, • For example, you want to find evidence that someone is • In exploratory research, often a larger probability of • If the sample size increases, both error probabilities can – What is “alpha-level” (in hypothesis

Source: http://www.ms.uky.edu/~mai/sta291/291_L22.pdf