Treffer: Evaluating the Use of Random Distribution Theory to Introduce Statistical Inference Concepts to Business Students

Bootstrapping methods and random distribution methods are increasingly recommended as better approaches for teaching students about statistical inference in introductory-level statistics courses. The authors examined the effect of teaching undergraduate business statistics students using random distribution and bootstrapping simulations. It is the first such empirical demonstration employing an experimental research design. Results indicate that students in the experimental group--where random distribution and bootstrapping simulations were used to reinforce learning--demonstrated significantly greater gains in learning as indicated by both gain scores on the Assessment of Statistical Inference and Reasoning Ability and final course grade point averages, relative to students in the control group. (Contains 3 tables and 1 figure.)

As Provided

AN0054533309;jeb01jan.11;2019Mar06.14:49;v2.2.500

Evaluating the Use of Random Distribution Theory to Introduce Statistical Inference Concepts to Business Students. 

Bootstrapping methods and random distribution methods are increasingly recommended as better approaches for teaching students about statistical inference in introductory-level statistics courses. The authors examined the effect of teaching undergraduate business statistics students using random distribution and bootstrapping simulations. It is the first such empirical demonstration employing an experimental research design. Results indicate that students in the experimental group—where random distribution and bootstrapping simulations were used to reinforce learning—demonstrated significantly greater gains in learning as indicated by both gain scores on the Assessment of Statistical Inference and Reasoning Ability and final course grade point averages, relative to students in the control group.

Keywords: bootstrapping; random distribution; statistical inference

INTRODUCTION

Discovering how students learn most effectively is one of the major goals of research in education. Over the last 30 years, many researchers and educators have called for reform in the area of statistics education in an effort to more successfully reach the growing population of students across an expansive variety of disciplines, who are required to complete coursework in statistics (e.g., Garfield, Hogg, Schau, & Whittinghill, 2000; [13]). Many of these students have very little interest in learning mathematics, and even less interest in learning statistics ([2]; [11]; Hollis, 1995). In light of this, reform efforts have proposed that statistics education should abandon the "information transfer model in favor of a constructivist approach to learning" ([18], p. 124) in an effort to help students develop an understanding of statistical concepts, beyond the use of mathematical formulas.

The constructivist approach to learning is based on the premise that learning is the result of mental constructions in which students are able to incorporate new information by building on knowledge they already have acquired ([3]). [18] maintained that statistics is not a subcategory of mathematics, but rather it is a science, much like physics is a science. Statistics, like physics, can depend heavily on mathematical computations; however, unlike mathematics, statistics is the science of inference with different modes of thinking and concepts distinct from mathematics. Moore posited that successful statistics education for the nonmath majors should include a balance of content, pedagogy, and technology.

Statistics is an important core course in most undergraduate business programs in the United States ([26]). Although statistics is a core course in the sequence of business classes, many business students do not understand its relevance to their education or future job prospects ([32]). Many business students consider required statistics coursework to be "irrelevant to their discipline, difficult, unattractive, and boring" ([25], p. 1352). As a result, statistics professors are challenged to present the required course content in such a way that students are motivated to engage material they initially believe is uninteresting and irrelevant ([6]). In light of the significance of statistics and statistical reasoning in the field of business, pedagogical approaches that engage and entice this otherwise not-so-interested audience of future business graduates must be examined and researched.

Statistics is not a spectator sport, and learning about statistics should not be either. Over the last 10 years, research in statistics education has revealed that cooperative, activity-based, and technology-assisted learning are the primary pedagogical approaches that have the potential to enhance students' understanding, and, in effect, enhance student's ability to effectively apply statistical concepts ([8]; [9]; [10]; Ware & Chastain, 1991; [31]). Research indicates that a majority (over 66%) of the postsecondary level statistics professors surveyed in 2002 reported using constructivist approaches (Garfield et al.). However, although some studies examining the use of constructivist- or activity-based approaches to the teaching of statistics have shown some promise in improving student perceptions (e.g., Fernandez & Liu; Ware & Chastin; Yesilcay), more recent research (e.g., Onwuegbuzie, 2004) indicates that approximately 80% of students surveyed continue to experience negative feelings and bad attitudes about their statistics education.

Recently, Rossman, Chance, Cobb, and Holcomb (2008) proposed that statistics education should move from the Ptolemaic curriculum, based on estimation procedures supported by normal distribution theory, to the now technologically enabled random distribution theory approach. Traditionally delivered statistics education is built around the concept of a normal distribution as approximating a sample distribution. The results that are computed are, at best, approximations. But, with the availability and power of computers today, [5] maintained that statistics education should now evolve to the presentation of random distribution theory concepts, based on permutation tests, as the central paradigm for statistical inference (p. 12). He maintained that this approach is "simple and easier to grasp" (Cobb, p. 12) for the novice, and can encourage students to more easily embrace the logic of inference. Cobb, like Moore (1997, 2001), suggested that understanding the science of inference is fundamental to an authentic understanding of statistics and statistical applications.

Specifically, the random distribution theory approach allows for the rerandomizing of all possible combinations of outcomes to see what is typical and what is not. According to [20], resampling procedures represent one of the single most important developments in statistics education in a generation, without changing the fundamental reasoning of statistical inference. Resampling provides the teacher and learner with a visual presentation of random samples from the population that is not hindered by the need for normally distributed or large samples. Resampling helps to develop students' understanding by providing a medium through with students can carry out repetitions, while controlling for the number of repetitions as well as the sample size (del Mas, Garfield, & Chance, 1999). Once these repetitions are completed, students can describe and explain the behavior that they have observed with their data. Moore and McCabe maintained that these procedures are intuitively more appealing because they "appeal directly to the basis of all inference: the sampling distribution demonstrates for the student what would happen if we took many repeated samples under the same conditions" (p. 2). Researchers have found that with simulations abstract concepts such as sampling distributions, confidence intervals, and conclusions regarding statistical significance become more conceptually clear to students (Rossman & Chance, 2006).

[24] developed a number of applet-type simulations that can facilitate the use of the random distribution theory approach in the classroom. Each of the learning modules developed by Rossman et al. guide students through the ideas of randomization, repeating the random selection process through resampling, and making decisions as to whether or not the null hypothesis is plausible or should be rejected. This approach to introducing statistical inference provides students with a better chance of developing an understanding of how to interpret the results of a study based on null hypothesis statistical testing; more specifically, to help them understand what p values indicate. Students use these applet-type simulations to construct for themselves an understanding of the connection between a randomly designed experiment, and the conclusions that result from the statistical analysis.

An example of one of the applets used in this investigation is presented in Figure 1 (http://www.rossmanchance.com/applets/Dolphins/Dolphins.html). In this simulation activity, the students are asked to consider a study conducted by [1]. The study examines the effectiveness of dolphin-mediated water therapy, relative to traditional group therapy, in the treatment of mild to moderate depression. Students are presented with the results of the study and asked to explore whether it is possible that these findings indicate that dolphin-mediated water therapy resulted in significantly greater number of patients who showed substantial improvement, or whether the findings were simply the result of chance variation.

Graph: FIGURE 1 Dolphin Applet example.

As can be seen in Figure 1, the observed results of the study are presented to the student. The simulation provides the student with the ability to repeatedly resample the 30 individuals into the experimental and control group conditions, in an effort to see if resampling produces results as extreme as 10 improved patients in the experimental group, as was the case in the original study. The graph on the right half of Figure 1 depicts the results of one such resampling (n = 1000) and illustrates that the result of 10 patients showing improvement is clearly a rare outcome.

In the present study we sought to examine the impact of random distribution theory-based applications proposed by [5] and developed by [24] relative to the impact of assignments based on normal distribution theory, on students' understanding of statistical reasoning and statistical inference. Specifically, half of the students in an activity-based, cooperative learning facilitated undergraduate business statistics class were assigned to complete eight two-part sections of Rossman et al.'s learning modules. Their peers were asked to complete an assignment of equal difficulty level and time-commitment that was based on more traditional approaches. The goal of this investigation was to see if the introduction of the random distribution simulations did in fact impact student understanding of statistics and statistical inference, specifically in a statistics section that was constructed as an activity-based learning community. At present, there are no published studies in the available literature in which the researchers employed an experimental design examining whether random distribution theory can facilitate students' understanding of statistics and statistical inference. The present study is the first such attempt.

METHOD

Participants

Participants included second- and third-year undergraduate business students that ranged from 18 to 52 years of age (M = 20.82, SD = 5.08), including 33 men (59.6%) and 21 women (40.4%). These university students were enrolled in a Monday-Wednesday-Friday morning business statistics course section. Students in this course were expected to develop the statistical tools used in business decision making, including but not limited to determination and interpretation of measures of central tendency, variance, probability, regression and correlation analysis, hypothesis testing, frequency and probability distributions, and sampling issues. Students were also introduced to graphical, tabular, and mathematical depictions of statistical information.

Instrumentation

The Assessment of Statistical Inference and Reasoning Ability (ASIRA) was used to assess the students' inferential and reasoning skills. Specifically, the ASIRA was constructed of 20 questions intended to measure both statistical reasoning skills and understanding of statistical inference. The first 13 questions were adapted from the Statistical Reasoning Assessment (SRA) developed by Joan Garfield (2003) and further adapted for computer-administrated assessment. The last seven questions were adapted from an assessment developed by [24], and were intended to measure a student's understanding of statistical inference. This 20-question assessment took students approximately 25 min to complete. The same assessment was used for both the pre- and posttest measures. The Cronbach's alpha, a test of internal consistency, for the pre and posttest measures was 0.79 and 0.84, respectively. A copy of the ASIRA inventory is found in Appendix A.

The student's final course grade (based on 400 possible points) also was used to assess learning. The final course grade included 4 exam scores, 10 quiz scores, 4 application assignments, and 4 research-related assignments.

Procedure

At the beginning of the first class meeting, all students were asked to complete the ASIRA. Student names were randomly assigned to the treatment group (Random Distribution Applications) or the control group (Normal Distribution Applications). The original sample of students included 58 students; however, four students dropped out of the class within the first four weeks of the semester, leaving a final sample of 54 students for the analyses. Students in the control and experimental groups were asked to complete their specific assignments independently, as an out-of-class homework assignment. Other than the experimental group's exposure to the learning modules, and the different assignments, students in both groups received the same instruction, and completed the same activities in and out of class, and completed the same quizzes each week of the semester.

The treatment group participants were given four assignments that were based on peer-reviewed research projects. These peer-reviewed research projects were a part of learning modules developed by [24]. Each one of the four assignments included two sections of Rossman et al.'s learning modules. These lessons asked students to review a brief discussion of a research project, and guided students through an application intended to develop and reinforce student's understanding of statistical inference.

The first assignment consisted of two mini-lessons that developed the ideas of random distribution theory using a study by [1]; the second assignment also consisted of two mini-lessons that were used to further reinforce the ideas of random distribution theory, and expanded to consideration of binomial distributions (using data from Hamlin, Wynn, & Bloom, 2007). The third assignment also consisted of two mini-lessons and focused on random distribution with binomial models (using data from Todorov, Mandisodza, Goren, & Hall, 2005); and the fourth assignment incorporated bootstrapping procedures to further develop the idea of statistical inference (using data from [27]). For each of these assignments, students read a brief description of a study, and then completed resampling procedures using playing cards or poker chips. Students responded to a few questions regarding their findings from their initial resampling with the poker chips and playing cards. Students were then asked to use online applets that would repeat the resampling procedures upwards of 1000 times, and answer questions regarding their findings, based on the larger samples. On each exam, students in the treatment group were asked to respond to four questions about each of the research studies and applets, which they completed.

The control group participants were also given four assignments over the course of the semester in which they were to review and critique peer-reviewed published research projects. These assignments were estimated to require the same amount of time and the same amount of writing as the assignments given to the treatment group. Unlike the shortened research project summaries read by the treatment group as part of the learning modules, the control group participants' assignments required that they read and respond to an entire research article. These papers incorporated the four primary topic areas being discussed during that particular section of the course when the project was assigned. The four assignments included a paper by [22]; reporting only descriptive statistics, a study by [17] incorporating correlation analyses; a study including regression analyses by Troisi, Christopher, and Marek (2006); and a paper by Luskin, Aberman, and DeLorenzo (2006) including both a t-test analysis and analysis of variance (ANOVA). In a two-page response, students were asked to summarize the papers, discuss the sample and sampling procedures, the use of the data, and identify variables and whether the variables were used appropriately by the researchers. Students were to defend their response with information from the class text and class notes and discussion. On each exam, students in the control group condition were asked to respond to four questions about each of the research studies that they reviewed.

The ASIRA was administered to the students during the first class meeting, and again during the last class meeting, in an effort to assess students' inferential and reasoning skills. This assessment was taken at the beginning of the first and last class sessions, and was administered as a computer-adapted assessment.

RESULTS

The data indicated that the control group performed slightly better on the pretest (M = 42.04, SD = 13.53) than did the treatment group (M = 37.59, SD = 11.17). The results from the pretest were examined in an effort to assess whether there were any pre-existing differences in the two study groups. An independent sample t test indicated that there were no significant differences in the pretest ASIRA scores of the students (n = 54) from the two groups, t(52) = 1.32, p =.193. This data is presented in Table 1.

TABLE 1 Pretest Means and Standard Deviations, by Group

<table><thead valign="bottom"><tr><td /><td>Control</td><td>Experimental</td><td /></tr><tr><td>Measure</td><td><italic>M</italic></td><td><italic>SD</italic></td><td><italic>M</italic></td><td><italic>SD</italic></td><td><italic>t</italic></td></tr></thead><tbody><tr><td>Pretest</td><td>42.04</td><td>13.53</td><td>37.59</td><td>11.17</td><td>1.32 (52)</td></tr></tbody></table>

At the end of the semester, the ASIRA was readministered to the same students (n = 54). A repeated measures ANOVA was conducted and results indicated significant differences in students' scores from pre- to posttest, F(1, 52) = 45.94, p <.001, partial η<sups>2</sups> =.469, with the pretest scores being significantly lower (M = 39.81, SD = 12.49) than the posttest scores (M = 53.15, SD = 13.81). Additionally, significant differences were revealed between groups from pre- to posttest, F(1, 52) = 17.56, p <.001, partial η<sups>2</sups> =.248. The data are presented in Table 2.

TABLE 2 Within- and Between-Groups Results

<table><thead valign="bottom"><tr><td>Source</td><td><italic>df</italic></td><td><italic>MS</italic></td><td><italic>F</italic></td></tr></thead><tbody><tr><td>Within groups</td><td>&#160;1</td><td>4800.00</td><td>45.94^***</td></tr><tr><td>Between groups</td><td>&#160;1</td><td>1792.59</td><td>17.56^***</td></tr><tr><td>Total</td><td>52</td><td>104.469</td><td>&#8212;</td></tr><tr><td>***<italic>p</italic> &#60;.001.</td></tr></tbody></table>

Specifically, students in the control group showed significantly lower posttest scores on the ASIRA (M = 47.22, SD = 13.25) relative to students in the treatment condition (M = 59.07, SD = 11.85). An independent sample t test examining final grade point average by group also revealed significant differences, t(52) = 4.35, p <.001; the control group ended the semester with significantly fewer points (M = 277.94, SD = 70.04) relative to the treatment group (M = 342.27, SD = 31.79) based on a maximum of 400 total points. This data is presented in Table 3.

TABLE 3 Final Grade Point Average for Students in the Control Group Relative to Experimental Group

<table><thead valign="bottom"><tr><td /><td>Control</td><td>Experimental</td><td /></tr><tr><td /><td><italic>M</italic></td><td><italic>SD</italic></td><td><italic>M</italic></td><td><italic>SD</italic></td><td><italic>t</italic></td></tr></thead><tbody><tr><td>Course points (out of 400 possible points)</td><td>277.94</td><td>70.04</td><td>342.27</td><td>31.79</td><td>4.35 (52)^***</td></tr><tr><td>***<italic>p</italic> &#60;.001.</td></tr></tbody></table>

DISCUSSION

The present study demonstrated the effectiveness of pedagogical tools designed to introduce random distribution theory concepts to undergraduate business statistics students. Relative to students in the control group who were exposed to teaching exercises based on traditional statistics instructional techniques employing normal distribution theory concepts, students in the experimental group exposed to teaching techniques based on random distribution theory concepts demonstrated greater mastery of course material as measured by final course point totals, and demonstrated superior gains on pre- and posttest measures of statistical inference and reasoning skills. These findings are consistent with a growing consensus among statistical researchers and educators that statistics instruction would prosper from a shift in focus from normal distribution theory to random distribution theory. The present study represents the first empirical demonstration of the veracity of these assertions.

One possible limitation to the current investigation is the difference between the assignments for the experimental and control groups. The learning modules were originally written with three sections in each module. The decision was made to assign only two sections of the learning modules for each assignment, in an effort to make the time necessary to complete the assignment equivalent to the control group assignments. Students from prior semesters reported that the completion time for one section of the simulation assignment was approximately 30–45 min; students reported spending approximately 1 hr on their article assignments. Another possible criticism of the experimental conditions of the present study might concern the fun factor of the simulation assignments relative to the article assignments. It is arguable that the greater fun and enjoyment associated with the experimental group exercises was responsible for the gains in the experimental group relative to the control group, rather than the shift in focus to the random distribution theory-based techniques. However, the control group articles were chosen because of their relevance to the students' lives and the potential tie-in with the business students' other coursework. Although reading and responding to an article may not be as entertaining for the student as working through activities with a computerized simulator, these articles do offer the student an opportunity to apply the knowledge covered in required course matter.

At the 2005 U.S. Conference on Teaching Statistics, Cobb (2005) insisted that statistics educators should stop focusing on classical methods of approximation, such as t tests and F tests, and introduce the concepts of statistical inference with simulations and randomization. Cobb posited that with the present availability of computer technology, the time has come to leave behind the teaching of outdated approximation procedures based on assumptions of normal distribution theory. He maintained that the logic of inference should be introduced through randomization and bootstrapping techniques through which students can observe randomization procedures and how statistical conclusions come about through these randomization simulations.

In the present study we sought to incorporate the cooperative learning approaches, heavily supported in the literature, with the random distribution applications suggested by [5]. Although 30 years of research suggests that constructivist approaches to teaching statistics should improve student attitudes and learning, research continues to suggest that these approaches are not in fact alleviating the negative feelings shared by an overwhelming majority of students surveyed regarding their statistics education (e.g., Onwuegbuzie, 2004). The results of the present investigation suggest that students can benefit from an introduction to the ideas of statistical inference based on randomization and bootstrapping techniques. In addition to the much supported constructivist pedagogy, random distribution simulations, such as those developed by [24], can help to improve student perceptions regarding the difficulty and relevance of statistics to their future business careers.

According to [15], the aim of business statistics coursework is to develop statistical thinking skills that help students to understand and interpret data. They maintained that in order to achieve this objective students must be engaged not only in the course information, but actively work at discovering the meaning of data, the importance and relevance of statistical concepts, and they must be actively involved in constructing an understanding of the influence of data where different distributions, different sample sizes, and differing degrees of variability are at play.

The random distribution theory approach to teaching and learning statistics provides business professors with a tool that can engage students to construct an understanding of statistics at this level. Teaching traditional estimation approaches may result in students who understand the subtle nuances of abstract statistical theory, but who also continue to struggle to actually use statistical methods to analyze data effectively. One student involved in the present study, as a participant in the experimental group, who had previously completed an AP statistics course in high school and an introductory level statistics class in his first year in college, indicated in an email: "although I have had statistics in the past, and did very well in those classes, I did not realize until now what I did not understand. The idea of 'significantly different than what would be expected to occur by chance' had no real meaning until I completed these simulation assignments. So, thank you. I thought I might be bored in this class, but now I know that I really do understand the concepts that are foundational to statistics."

Today's business student is interested in more than the traditional talk and chalk lecture, coupled with well-meaning assignments, followed by midterm and final exams. Activities using random distribution theory, such as those presented here, can engage and intrigue students, and enhance their understanding and their performance in their present coursework and enhance their ability to think statistically in their future professional and academic careers.

APPENDIX A

Assessment of Statistical Inference and Reasoning Ability

By KarenH. Larwin and DavidA. Larwin

Reported by Author; Author