Investigation 5.11: Running Out of Time (cont.)

Section 27.2 Investigation 5.11: Running Out of Time (cont.)

Exercises 27.2.1 The Study

In the previous investigation, we analyzed the statistical significance of our sample data by replicating how often we might get a random sample of 247 runners (after removing an extreme outlier) with a slope this extreme from a larger population of runners that did not have an association between finishing time and age. In other words, we assumed there was no association between finishing time and age and the pairing of these values for our runners was just arbitrary. This suggests another way of assessing the statistical significance of our results.

Paste the Talley5K2013.txt data file into the Analyzing Two Quantitative Variables applet.
Press Use Data and make sure age is used as the explanatory variable (if not press the (response, explanatory) button to toggle their roles).
Remove the outlier and check the Show Regression Line box.

1. Regression Equation.

Report the regression equation.

Solution.

Example output:

Applet output with Show Regression Line checked: predicted Time = 26.96 + 0.1403 times Age.

Another way to investigate whether the association observed in these sample data could have arisen by chance is a randomization test that shuffles the outcomes for one of the variables and reexamines the correlation coefficient and/or sample slope for the shuffled or re-randomized data.

Check the Show Shuffle Options box. Press the Shuffle Y-values box. Select the Plot radio button.

2. Single Shuffle.

Describe the blue line that appears. Is the association as strong as the one that we found in the actual data? How are you deciding?

Solution.

The slope will most likely be smaller than observed (as well the correlation coefficient, indicating a weaker association).

Press the Shuffle Y-values button 5-10 more times.

3. Repeated Shuffles.

Do you ever find a negative association? Are any of the associations as strong as the one observed by the student group? What pattern is beginning to emerge in the blue regression lines on the scatterplot?

Solution.

You should get a negative association about half the time, but probably never one as extreme as the observed slope (red line).

Change Number of Shuffles to some large number (like 1000 or the difference between how many you have done so far and 1000) and press the Shuffle Y-values button.

4. Null Distribution of Shuffled Slopes.

Describe the shape, center, and variability of the resulting distribution of re-randomized slopes. How does this distribution compare to the one you found in Investigation 5.10, Question 11?

Solution.

Example results

Histogram of 1000 shuffled slopes, roughly symmetric and centered near zero, with mean = -0.002 and SD = 0.032.

The shape is rather symmetric, centered around zero, with a standard deviation of about 0.03. This is similar to what we saw in Investigation 5.10.

Use the applet to estimate the p-value by using the Count Samples box: choosing to count “beyond” (for the two-sided p-value) and specifying the observed sample slope.
Press Count.

5. Empirical p-value.

Report your estimated p-value.

Solution.

Example results

Histogram of 1000 shuffled slopes with vertical lines at the observed slope in each tail; counting shuffles Beyond .1403 gives Count = 0/1000 (0.0000).

None of the simulated slopes was as extreme as the observed sample slope.

You should see many similarities between the statistic distributions from random sampling (Inv 5.10) and random shuffling). So let’s again consider applying the t-probability model to the standardized statistics.

6. Standardize Observed Slope.

Use the standard deviation for the slopes found in Question 4 and standardize the value of the observed sample slope.

Solution.

If we standardize, we again get something like \((0.14-0)/.03 \approx 4.67\text{.}\)

Using an observed t statistic of 4.49, we are again on the right edge of the distribution.

Histogram of 1000 shuffled t-statistics with mean = -0.058 and SD = 0.993; counting shuffles Beyond 4.49 gives Count = 0/1000 (0.0000).

Now use the pull-down menu to select t-statistic.

7. t-statistic p-value.

Use the applet to estimate the p-value corresponding to the observed t-statistic. How does it compare to the p-value you found in Investigation 5.10? (Note: The applet may complain due to rounding, estimation of the standard error.)

Solution.

The results are very similar to the previous investigation.

Histogram of 1000 shuffled t-statistics with an overlaid t-distribution curve; the simulated count beyond 4.49 is 0/1000 and the theory-based p-value is less than 0.0001.

Check the box to Overlay t-distribution.

8. Overlay t-distribution.

Does it appear to be a reasonable model of the shuffled t-statistics distribution?

Solution.

The t-distribution appears to be a good model for this randomization distribution.

Check the Regression Table box.

9. Regression Table Comparison.

Compare your calculations (e.g., what value does the table use for \(SE(\hat{\beta})\text{?}\)

Solution.

Applet regression table: Intercept coefficient 26.961 with SE 1.314, t-stat 20.52, p-value less than 0.0001; Age coefficient 0.140 with SE 0.031, t-stat 4.49, p-value less than 0.0001.

Note: the two simulation models agree because we are getting a similar standard deviation of the sample/shuffled slopes. If the association had been stronger, this would not necessarily be the case. Also keep in mind the distinction between strength of evidence (which is strong here because the sample size is rather larger) and strength of association. This regression model explains less than 8% of the variability in finishing times!

Discussion.

It is interesting to compare these two analysis approaches. In Investigation 5.10, we modeled repeated random sampling from a population but this required us to make certain assumptions about the data in the population (which we will spell out in more detail in the next Investigation). In Investigation 5.11, we didn’t make any assumptions about a larger population; instead we modeled what the sample data could look like if the observed response values had been randomly paired, over and over again, with the observed explanatory variable values. This approach is sometimes referred to as “conditioning on the observed data.” It is not surprising that the results are similar for both approaches, but some consider the second approach to be more “flexible” and “less restrictive” in the assumptions it makes. However, the first approach allows for more elegant mathematical derivations predicting how the sampling distribution of sample slopes will behave.

If we had had a stronger association in our sample (and \(s\) and \(s_y\) were less similar in value), you would have seen more differences between these two approaches. In particular, the random shuffling will have a larger standard deviation of the sample slopes than the random sampling. We saw with random sampling that the variability in the slopes is controlled by \(\sigma\text{,}\) not by SD(Y). However, with random shuffling, any \(x\)-value can be paired with any \(y\)-value, so the slopes are able to “swing around” a little more freely especially with smaller sample sizes. When you view the Regression Table, the SD(\(b_1\)) reported there will correspond to random sampling.

Subsection 27.2.2 Practice Problem 5.11

Checkpoint 27.2.1. Confirm with Height and Foot Length Data.

Confirm the preceding paragraph using the height and footlength data of Investigation 5.8.

You have attempted of activities on this page.

Prev Top Next