Skip to main content

ISCAM: Investigating Statistical Concepts, Applications, and Methods

Beth L. Chance, Allan J. Rossman

Section 21.5 Investigation 4.11: Smoke Alarms

Exercises 21.5.1 McNemar’s Test

A study published in the journal Pediatrics (Smith et al., 2006) addressed the important issue of how to awaken children during a house fire so they can escape safely. Researchers worked with a volunteer sample of 24 healthy children aged 6-12 by training them to perform a simulated self-rescue escape procedure when they heard an alarm. Researchers then compared the children’s reactions to two kinds of alarms: a conventional smoke alarm and a personalized recording of the mother’s voice saying the child’s name and urging him or her to wake up. All 24 children were exposed to both kinds of alarms, with the order determined randomly.
Smoke alarm study illustration
Children’s Responses to Alarms
Child # Response to mother voice Response to conventional Child # Response to mother voice Response to conventional
1 wake wake 13 wake wake
2 wake wake 14 wake wake
3 wake wake 15 wake not
4 wake wake 16 wake not
5 wake wake 17 wake not
6 wake wake 18 wake not
7 wake wake 19 wake not
8 wake wake 20 wake not
9 wake wake 21 wake not
10 wake wake 22 wake not
11 wake wake 23 wake not
12 wake wake 24 not not

1. Identify Study Components.

Identify the observational units, explanatory variable, and response variable. Are the variables quantitative or categorical? Is this an observational study or an experiment? If this is an experiment, how was randomization used/completely randomized or paired?
Observational units:
Explanatory variable:
Type:
Response variable:
Type:
Type of Study:
Randomness:
Solution.

2. Initial Analysis.

What proportion of children woke up to the conventional alarm and what proportion woke up to the mother’s voice? Does this provide preliminary evidence that the mother’s voice alarm is more effective? State the hypotheses of interest. Could you make a two-way table for these data? Could we carry out Fisher’s Exact Test or a two-sample z-test to assess statistical significance?
Solution.
The research question is whether a child was more likely to wake up to the mom’s voice than to the conventional alarm. If we let \(\pi_1\) and \(\pi_2\) represent these two probabilities, we can write this as \(H_0\!: \pi_1 = \pi_2\) vs \(H_a\!: \pi_1 > \pi_2\) but we need to keep in mind that these are not independent samples. In the data, \(23/24 \approx 0.958\) woke up to the mom’s voice; \(14/24 \approx 0.583\) woke up to the conventional alarm. This provides strong preliminary evidence that the probability of waking is larger with the mom’s voice, but we can’t apply Fisher’s Exact Test because this was not a randomized experiment with two separate treatments - we do not have two independent random samples.
The initial analysis reveals the proportion waking to the mother’s voice was larger than the proportion that woke to the conventional alarm, but we can’t consider these two independent samples – it’s the same children! This is again a matched pairs design, but with a categorical response.

3. Simulation Consideration.

Consider the data table. Suppose we conducted a simulation analysis by shuffling the response variable outcomes back to the explanatory variable outcomes. What’s going to happen in many of the cases?
Solution.
It’s not clear which column (mom’s voice or conventional alarm) would be considered the response variable, but if we mix up the entries in one column, most of the rows will not see a change because most of the rows are (awake, awake) anyway.
As before, with a matched pairs design we will focus on the differences in responses. First, we need an appropriate way to organize the data. We can do this with a two-way table looking at the two variables measured on each children – whether or not woke up to mom’s voice and whether or not woke up to the conventional alarm.

4. Create Two-Way Table.

Complete the following table.
Hint.
What should your table total be?
Mom \ Conventional Awakened to conventional alarm Did not awaken to conventional alarm Row total
Awakened to mother’s voice
Did not awaken to mother’s voice
Column total
Solution.
Mom \ Conventional Awakened to conventional Did not awaken to conventional Row total
Awakened to mother’s voice 14 9 23
Did not awaken to mother’s voice 0 1 1
Column total 14 10 24

5. Non-Ties.

How many children responded differently to the two alarms? (We will treat these "non-ties" as our sample size.)
Number of children:
Solution.
\(9 + 0 = 9\) children responded differently to the two alarms.

6. Mother’s Voice Only.

Of the children you identified in Question 5, how many woke to the mom’s voice alarm only? What proportion of the children you identified in Question 5 is this?
Solution.
None of the children in Question 5 awoke to only the conventional alarm. This is a proportion of 0.

7. Expected Success Rate.

Suppose there was no difference in the effectiveness of the two alarms in the long-run. If a child does react differently to the two alarms (wakes to one but not the other), how often would you expect the "success alarm" to be the mother’s voice in the long run?
Probability:
Solution.
Half the time

8. State Hypotheses.

Let \(\pi\) represent the probability of waking to the mother’s voice alarm when a child wakes to only one of the two alarms. State appropriate null and alternative hypotheses in terms of \(\pi\) for deciding whether the mother’s voice is the more effective alarm.
\(H_0\!: \pi = 0.50\)
\(H_a\!: \pi > 0.50\)
Solution.
\(H_0\!: \pi = 0.50\)
\(H_a\!: \pi > 0.50\)
where \(\pi\) = probability of waking to the mother’s voice alarm when a child wakes to only one of the two alarms (equally likely to wake up to only mom and only conventional alarm under null; more likely to wake up to mom only than to conventional only under alternative)

9. Statistical Test.

Use a method discussed previously in this text to assess whether there is statistically significant evidence that the mother’s voice is more effective than the conventional alarm. Be sure it is clear how you are finding the p-value (e.g., simulation, exact, theory-based?)
Solution.
With a binomial distribution with \(\pi = 0.5\) and \(n = 9\text{,}\) \(P(X \geq 9) = 0.00195\) (exact)

McNemar’s Test.

This test, using the binomial distribution to look at the distribution number of successes in the off-diagonal cells, is called McNemar’s Test.

10. Study Conclusion.

Summarize your conclusion for this smoke alarm study, and explain the reasoning process by which it follows.
Solution.
We have strong evidence (p-value = 0.00195) to reject \(H_0\!: \pi = 0.5\) (equally likely to wake up to mom/conventional) when only wake up to one of the two alarms. This is convincing evidence that mom’s voice increases (causation because this was a randomized matched-pairs experiment) the likelihood of waking up, at least for children like those in this study (healthy children aged 6-12).

Study Conclusions.

In this study, nine children reacted differently to the two alarms. If there were no effect from the type of alarm, we would expect half the children to respond to the mother’s voice but not the conventional alarm and the other half of the children to respond to the conventional alarm but not the mother’s voice. Instead, all 9 of the one-alarm children responded to the mother’s voice but not the conventional alarm. This provides evidence that the mother’s voice is more effective than the conventional alarm. To see whether this difference is statistically significant, we can calculate the binomial probability (assuming \(n = 9\) and \(\pi = 0.5\)) to determine the exact p-value \(P(X \geq 9) = 0.00195\text{.}\) (Or we could use simulation for one proportion. The sample size \(n = 9\) is too small to use a theory-based approach.) This provides strong evidence (p-value \(< 0.05\)) that the observed result did not arise from the randomness in the process alone (which alarm was tested first). Because this was a randomized matched-pairs experiment, we will attribute this difference to the type of alarm. Keep in mind that this was a sample of volunteers and may not represent a larger population of children.

Subsection 21.5.2 Practice Problem 4.11

Another aspect of this same study considered not just whether the child woke up but whether he/she successfully escaped from the house within 5 minutes of the alarm sounding. The article reports that 20 children escaped when they heard the mother’s voice, and only 9 escaped when they heard the conventional tone.

Checkpoint 21.5.1. Additional Information Needed.

Describe what additional information you need before you can analyze the data as you did above.

Checkpoint 21.5.2. Complete the Table.

Two children did not escape to either kind of alarm. Use this information to complete the following table:
Mom \ Conv Escaped to conventional alarm Did not escape to conventional alarm Total
Escaped to mother’s voice 20
Did not escape to mother’s voice
Total 9 24

Checkpoint 21.5.3. Simulation Analysis.

Checkpoint 21.5.4. Exact p-value.

Use the binomial distribution to calculate the exact p-value for this test.
p-value:

Checkpoint 21.5.5. Conclusion for Escaping.

Summarize your conclusion from this "escaping within 5 minutes" aspect of the study, and explain the reasoning process behind your conclusion.
You have attempted of activities on this page.
PrevTopNext