Print View
Download Citation
Erratum Alert
Research Toolkit
Blog This Article
Add to MyScitation
Tools
Email Abstract
Connotea
CiteULike
del.icio.us
BibSonomy
Tweet this Article
Add to Facebook
Share
INTRODUCTION
We frequently argue that college-level, general education, introductory astronomy classes (hereafter Astro 101) play a critical role in the science education of our society (e.g., Prather, Rudolph, and Brissenden 200929; Prather et al. 200931; Wallace, Prather, and Duncan 201238). After all, Astro 101 is the terminal science course in life for many of our nation's future politicians, business leaders, journalists, historians, artists, school teachers, parents, taxpayers, policy makers, and voters (Deming and Hufnagel 200111; Rudolph et al. 201035). Consequently, much of astronomy education research to date has focused on (1) uncovering and understanding the common conceptual and reasoning difficulties of students, and on (2) developing research-validated activities and curricula (e.g., Crouch and Mazur 20019; Hudgins et al. 200618; LoPresto 201023; Prather et al. 200432; Wallace, Prather, and Duncan 201238) that can raise the level of Astro 101 students' conceptual understandings and reasoning abilities. But should we be satisfied if improved conceptual understandings are all that students get out of Astro 101? We emphatically answer “no!”
In a seminal article, Cobern (1996)4 argues that much of the conceptual change research suffers because it fails to attend to the fundamental role played by students' worldviews. A person's worldview constitutes a set of (often implicit and often non-rational) beliefs, presuppositions, and assumptions about reality (Cobern 19964; Irzik and Nola 200919; Kearney 198420; Koltko-Rivera 200422). These beliefs, presuppositions, and assumptions affect our emotions, thoughts, and behaviors (Cobern 19964); influence our symbolic creations (Kearney 198420); and determine what constitutes valid and important knowledge about the world (Cobern 19964; Irzik and Nola 200919; Koltko-Rivera 200422).
Cobern (1996)4 asserts that much of the conceptual change research implicitly assumes that if students better understand the content of science then they will be more likely to adopt scientific ways of understanding the world and develop positive beliefs about the role of science in society. Cobern's concern is that science, its body of knowledge, its methods, and its culture are often not concordant with many of our students' worldviews. A student may master the content of a science course (and may even be able to carry out a scientific investigation) and yet maintain a worldview in which the enterprise and culture of science is marginalized because the course content, or the context in which it is presented, simply is not credible to the student. To give two examples, Hansson and Lindahl (2010)15 found that some students reject the scientific enterprise because they believe the universe is fundamentally incomprehensible and do not believe that humans can understand a whole by studying its component parts individually. Dagher and BouJaoude (1997)10 found that students may reject scientific ideas, such as biological evolution, not because these students are irrational, but because they have different notions than scientists about what constitutes sufficient evidence for accepting or rejecting an idea. The students in these two examples reject science because of their worldview beliefs about how knowledge is obtained and evaluated, and not because they are ignorant of science.
The way science is traditionally taught may be a significant part of the problem. Many agree that science promotes certain worldview beliefs over others (Cobern 19964; Irzik and Nola 200919). However, the promotion of these beliefs is often implicit (Ausubel 19663; Kilbourn 198021; Proper, Wideen, and Ivany 198833; Säther 200336), fragmented (Helve 199117) and may present a distorted view of science (Smolicz and Nunan 197537).
What is a science teacher to do? Cobern (1996)4 suggests that a teacher should “create a classroom atmosphere that invites students to express and discuss important personal viewpoints as they pertain to the science curriculum” (p. 604). In this way, scientific concepts may develop scope (“relevance for an individual over a wide range of contexts”; p. 581) and force (“central in an individual's thinking rather than marginal”; p. 580), since students can begin to see how science answers fundamental questions of existence and addresses important issues they and their society face.
One way to promote scientifically compatible worldviews and to give scientific ideas scope and force is to help students understand the role played by science in contemporary society. This has a direct bearing on improving the scientific literacies of people living in our society, as articulated by Mooney and Kirshenbaum (2009)25:
“[W]e don't need average citizens to become robotic memorizers of scientific facts or regular readers of the technical scientific literature. Rather, we need a nation in which science has far more prominence in politics and the media, far more relevance to the life of every American, far more intersections with other walks of life, and ultimately, far more influence where it truly matters—namely, in setting the agenda for the future as far out as we can possibly glimpse it. That would be a scientific America, and its citizens would be as scientifically literate as anyone could reasonably hope for. We will never have a nation that is fully composed of Ph.D.s.” (p. 18, italics in original).
This view of what it means to be scientifically literate is shared by Cobern (1996)4. Note that efforts to raise students' awareness of the role of science in society are different from, but not antithetical to, more traditional methods of addressing scientific literacy by improving students' content knowledge and/or their understandings of the process of science.
As discussed by Cobern (1996)4, many instructors implicitly (and perhaps incorrectly) assume that simply learning science content is sufficient to change students' worldviews. This paper presents data that tests the validity of this assumption. We report on our first step toward uncovering Astro 101 students' worldviews generally and their understandings of the role and impact of science in society specifically. This research was conducted in an Astro 101 course shown to be effective at increasing students' conceptual knowledge and reasoning abilities through the use of active engagement instructional strategies (Hudgins et al. 200618; Prather, Rudolph, and Brissenden 200929; Prather, Rudolph, and Brissenden 201130; Prather et al. 200432; Rudolph et al. 201035; Wallace, Prather, and Duncan 201238). We look at the effect that this class and its instructional strategies had on students' worldviews and their understandings of the role science plays in society. To address our research questions, and to provide a baseline for future studies, we administered the Thinking about Science Survey Instrument (TSSI; see Cobern 20015 and Cobern and Loving 20026) to an Astro 101 class of over 700 students. The TSSI is designed to measure people's agreement with a view of science and its relationship to society that is commonly portrayed to the general public by scientists, educators, and journalists (Cobern 20015; Cobern and Loving 20026).
This paper is organized as follows. In Section 2, we provide additional details about our administration of the TSSI, the population of students who took the TSSI, and the class in which they were enrolled. Section 3 describes the TSSI in more detail. Our results are presented in Section 4. Section 5 summarizes our conclusions and discusses the lessons we can learn from our results.
COURSE INFORMATION AND DEMOGRAPHICS
We administered the TSSI pre- and post-instruction to students enrolled in a 719 person Astro 101 “mega-course" (Prather, Rudolph, and Brissenden 201130) taught in the spring of 2011. This course made extensive use of research-validated interactive engagement learning strategies designed to improve students' conceptual understandings of astronomy, including interactive lecture demonstrations (animations and simulations), Think-Pair-Share, Ranking Tasks, and Lecture-Tutorials (Crouch and Mazur 20019; Hudgins et al. 200618; Prather, Rudolph, and Brissenden 200929; Prather, Rudolph, and Brissenden 201130; Prather et al. 200432; Rudolph et al. 201035; Wallace, Prather, and Duncan 201238). Additionally, this Astro 101 mega-course utilized a cadre of undergraduate peer instructors (a.k.a. “Ambassadors”), who had earned an “A” in a previous semester in the same Astro 101 course, taught by the same instructor. Similar to students participating in Supplemental Instruction (SI) Programs, developed at the University of Missouri-Kansas City (Arendale 19972), and Learning Assistant (LA) Programs, developed at the University of Colorado at Boulder (Otero et al. 200626; Otero, Pollock, and Finkelstein 201027), these Ambassadors helped implement the interactive teaching strategies during the lecture portion of class, and they ran supplemental help sessions (or office hours) outside of class. This Astro 101 course had no recitation or lab sections. All instruction was provided through 75 min “lecture” sessions offered twice a week. Note that the conceptual learning gains of students in this mega-class were among the highest in the nation, as measured by two different concept inventories (Prather, Rudolph, and Brissenden 201130).
The content of this Astro 101 mega-course was traditional: Students received lessons on common Astro 101 topics such as the phases of the Moon, light and spectroscopy, stellar evolution, and cosmology. Because this 719-student class only met in a “lecture” style auditorium classroom, there was no instruction or use of activities that required students to conduct experiments or perform authentic scientific investigations using real data. Note that the instructor included frequent comments within the lecture portion of the course on the history of astronomy and physics, and the role it plays in advancing our understanding of the natural world, the advancement of our society, and its importance with regard to our quality of life and prosperity. However, we neither developed nor implemented any new curricular activities specifically designed to influence students' worldviews. Therefore, all the instruction provided on worldviews was delivered through lecture. By comparing students' pre- and post-instruction responses to the TSSI, we gain an understanding of which of their ideas about the role of science in society may or may not change as the result of taking an Astro 101 class that explicitly focuses on interactive learning strategies designed to increase students' conceptual and reasoning abilities with commonly taught topics in astronomy. Although we required students to take the TSSI by assigning a nominal amount of points simply for participating, students were not penalized based on their responses or on whether or not they granted us permission to use those responses in this study. Our dataset includes the responses of 442 students who took the TSSI pre-instruction and consented to have their responses used for research purposes, and the 294 consenting students who took the TSSI post-instruction; 277 students gave responses to the TSSI both pre- and post-instruction.
THE TSSI
The TSSI is a sixty-item survey. Each item contains a statement expressing a particular view about science. Respondents indicate their level of agreement with the statement on a 1–5 scale, where 1 = “strongly disagree,” 2 = “disagree,” 3 = “uncertain,” 4 = “agree,” and 5 = “strongly agree.”
Each item on the TSSI is designed to fall into one of nine categories. Each category represents one aspect of what Cobern (2001)5 calls “a common image of science” (p. 8). The categories are described in Table 1 (Descriptions are taken verbatim from Cobern 20015 and Cobern and Loving 20026.) Note that this “common image of science” is not meant to be authoritative (Cobern 20015; Cobern and Loving 20026); a given scientist may disagree with one or more of the statements in Table 1. Instead, it is meant to capture a certain perspective of science that is frequently portrayed in the media, science education, and popular science books (Cobern 20015; Cobern and Loving 20026). Respondents who agree with this perspective will score high on the TSSI.
When scoring the TSSI, we reversed the scores (5s became 1s, 4s became 2s, and vice versa) on Items 1–3, 7, 9, 12–15, 18, 21–23, 26–28, 32, 37, 38, 41, 44, 45, 50, 53, 54 57, and 58 (Cobern 20015). These items have statements that contradict the “common image of science” in Table 1. Students who agree with this image should answer 1 or 2 to these items, while students who disagree should answer 4 or 5. We reversed the scores on these items in order to (1) follow the analysis procedure of Cobern (2001)5, and (2) to facilitate the comparison of responses to items across the entire TSSI. In other words, in our results in Section 4 below, a score of 5 always corresponds to a strong agreement with Cobern's “common image of science” and a score of 1 always corresponds to a strong disagreement.
In order to check the reliability of the TSSI, we calculated Cronbach's α for both the pre- and post-instruction data. Pre-instruction, we found α = 0.87, while post-instruction we found that α = 0.92. These values are well above the conventionally accepted lower bound for Cronbach's α of α = 0.70 (George and Mallery 200912) and are therefore consistent with the hypothesis that the TSSI is a reliable instrument for our population (i.e., students' “true scores” are a significant fraction of their “observed scores”; see Lord and Novick 196824 for more details on this classical test theory perspective).
RESULTS
Table 2 summarizes our results. Table 2 shows, for each item, the average and standard deviation of students' responses, both pre- and post-instruction. While the averages and the standard deviations summarize the central tendencies and spreads in the responses, respectively, the reader may be interested in details of the pre- and post-instruction response distributions that are not necessarily immediately obvious from the numbers in Table 2. Consequently, in the Appendix, we include histograms showing the distribution of pre- and post-instruction responses for each item.
Table 2 also shows the difference in the means (Δ Avg. = post-instruction average – pre-instruction average). To determine whether or not the differences in the distribution of pre- and post-instruction responses are statistically significant, we applied the non-directional Mann-Whitney test to the pre- and post-instruction data for each item. Mann-Whitney is the appropriate test, as opposed to the t-test, because our data falls on an ordinal, not interval scale, and because we cannot assume students' responses come from a Gaussian distribution, as evidenced by the histograms contained in the Appendix. Our null hypothesis was that, for a given item, there is no difference in the distribution of pre- and post-instruction responses. Table 2 reports the p-values from the Mann-Whitney test. We bolded and italicized all p-values that indicate a statistically significant (p < 0.001) difference in the pre- and post-instruction distributions of responses.
We also report in Table 2 the effect size for each item. Specifically, we use Cohen's d as a measure of the effect size. Cohen's d is defined as
where μf is the average of the post-instruction responses, μo is the average of the pre-instruction responses, and σ is the “pooled sample standard deviation” (Cohen 19888; Hartung, Knapp, and Sinha 200816). This “pooled sample standard deviation” is given by
where nf is the number of students in the post-instruction data (294), σf is the standard deviation of the post-instruction responses for a given item, no is the number of students in the pre-instruction data (442), and σo is the standard deviation of the pre-instruction responses for that item (Hartung, Knapp, and Sinha 200816). The effect size of each item in Table 2 expresses the difference in means (Δ Avg.) as a fraction of the “pooled sample standard deviation.” Cohen (1988)8 defines effect sizes of d = 0.20 to be small, d = 0.50 to be medium, and d = 0.80 to be large. If we elaborate on these definitions so that d ≤ 0.20 is considered small, 0.20 < d ≤ 0.50 is considered medium, and d > 0.50 is considered large, then we see that none of the effect sizes in Table 2 fall into the “large” category. Most items have small effect sizes, although nineteen items (3, 5, 8, 10, 18, 20, 23, 29, 33, 34, 36, 38, 42, 43, 49, 50, 53, 59, and 60) have medium effect sizes. We bolded and italicized all medium effect sizes in Table 2.
Note that there are some items, whose effect sizes are medium, yet do not have a statistically significant p-value. How can this be? Note that all of these items would be statistically significant if we adopted a less stringent criteria for statistical significance, such as p < 0.05 or p < 0.01. Furthermore, many of the TSSI's items have response distributions, both pre- and post-instruction, which exhibit various levels of departure from Gaussianity. (See the Appendix.) The effect size has a well-defined relationship to the amount of overlap between two distributions when both distributions are Gaussian. However, two non-Gaussian distributions may have a much different amount of overlap than two Gaussian distributions, even if their effect sizes are equal (Coe 20027). This departure from Gaussianity is why some items have medium effect sizes, but not statistically significant (according to our stringent criteria) differences between their pre- and post-instruction response distributions.
While Table 2, in combination with the histograms in the Appendix, technically contains all the information about students' responses to the TSSI's items, the inferences one can draw from this are not immediately obvious. To provide a context to better understand the significance of the results contained in Table 2, we now group the items according to the categories in Table 1 to which they were assigned by Cobern (2001)5. In the following analysis, we only look at post-instruction responses. We do this for two reasons. First, the majority of items do not have significant differences between their pre- and post-instruction averages; this is not surprising, given that all our instruction on worldviews was provided only through lecture, and that research shows that instructors often struggle to positively change students' attitudes and beliefs (e.g., Adams et al. 20051; Perkins et al. 200528; Redish, Saul, and Steinberg 199834). Second, this analysis is primarily concerned with identifying those items for which students' post-instruction responses fall well outside of the neutral range of scores.
The first category in Table 1, “Epistemology,” is composed of nine items, which are shown in Table 3. Figure 1 plots the average post-instruction response for each of these nine items.
Fig 1.
The average post-instruction responses of the nine items in the “Epistemology” category of the TSSI. White diamonds indicate items for which there was no statistically significant difference in the pre- and post-instruction response distributions. Black dots indicate items for which there was a statistically significant difference in the pre- and post-instruction response distributions. Response values below the grey band indicate disagreement with the “common image of science,” response values above the grey band indicate agreement with the “common image of science,” and response values within the grey band indicate neutrality with respect to the “common image of science”
Figure 1 also contains two other representations to help the reader visualize some of the information in Table 2 and the Appendix. First, we used white diamonds to represent items for which there was no statistically significant (p < 0.001) difference in the pre- and post-instruction response distributions. For items that have a statistically significant difference, we used black circles. Additionally, Figure 1 contains a grey band that extends from 2.5 to 3.5 on the y-axis. According to Cobern (2001)5, a response r is aligned with the “common image of science” if 3.5 < r ≤ 5, is neutral with respect to this image if 2.5 < r ≤ 3.5, and is anti-aligned if 1 ≤ r ≤ 2.5. The grey band in Figure 1 thus marks the region into which neutral responses fall. An item with an average response that falls below the grey band indicates an item for which students tend to disagree with the response one would give if one accepted every facet of the “common image of science.” The more items that fall above the grey band, the more students are aligned, on average, with the claims of the “common image of science.” Figures 2,3,4,5,6,7,8,9 also use these representations.
Fig 2.
The average post-instruction responses of the ten items in the “Science and the Economy” category of the TSSI
Fig 3.
The average post-instruction responses of the four items in the “Science and the Environment” category of the TSSI
Fig 4.
The average post-instruction responses of the ten items in the “Public Policy and Science” category of the TSSI
Fig 5.
The average post-instruction responses of the four items in the “Science and Public Health” category of the TSSI
Fig 6.
The average post-instruction responses of the seven items in the “Science, Religion, and Morality” category of the TSSI
Fig 7.
The average post-instruction responses of the four items in the “Science, Emotion, and Aesthetics” category of the TSSI
Fig 8.
The average post-instruction responses of the four items in the “Science, Race, and Gender” category of the TSSI
Fig 9.
The average post-instruction responses of the eight items in the “Science for All” category of the TSSI
As Figure 1 shows, none of the average responses to the nine “Epistemology” items exhibit an anti-alignment with the “common image of science.” Five items (Items 2, 27, 29, 44, and 60) fall in the neutral region, while four items (Items 17, 33, 34, and 46) fall in the aligned region. The average post-instruction response across all nine “Epistemology” items is 3.29, which falls in the neutral region of Figure 1. While five items (Items 17, 29, 33, 34, and 60) have medium effect sizes, only Items 34 and 60 have statistically significant differences between the pre- and post-instruction response distributions.
The ten “Science and the Economy” items are shown in Table 4. Figure 2 plots the average post-instruction response for each of these ten items. Figure 2 implies that there is a uniform strong agreement among students about the importance of science for the economy, since all but one item fall in the aligned region (Item 22 falls in the neutral region). Consequently, the average post-instruction response across all ten of the “Science and the Economy” items is 3.80, which is in the aligned region. All three items that have medium effect sizes (Items 20, 42, and 49) also exhibit statistically significant differences in the pre- and post-instruction distributions of responses.
The four “Science and the Environment” items are shown in Table 5. Figure 3 plots the average post-instruction responses of these items. Three out of the four items have average post-instruction responses that place them in the aligned region of Figure 3 (Item 3 falls in the neutral region). The average post-instruction response across all of the “Science and the Environment” items is 3.90. This suggests that there is strong and uniform positive agreement among many students that science is “necessary for the discovery, development, and conservation and protection of natural resources and the environment in general” (Cobern 20015; Cobern and Loving 20026). All four items in this category have medium effect sizes, although only Items 38 and 43 have statistically significant differences between the pre- and post-instruction response distributions.
Table 6 contains the ten items from the “Public Policy and Science” category. Figure 4 plots the average post-instruction responses to these items. Four of the ten items (Items 6, 10, 18, and 57) fall in the aligned region of Figure 4, five items fall in the neutral region (Items 5, 26, 28, 45, and 50), and Item 19 falls in the anti-aligned region (with an average post-instruction response of 2.48). The average post-instruction response across all ten items is 3.44, which falls in the neutral region. Items 5, 10, 18, and 50 have medium effect sizes and all of these items except Item 50 also exhibit statistically significant differences between their pre- and post-instruction response distributions.
The “Science and Public Health” category has four items, which are shown in Table 7. Figure 5 plots the average post-instruction responses of these items. Items 8, 48, and 58 have average post-instruction responses that fall in the aligned region of Figure 5, while Item 9 falls in the neutral region. The average post-instruction response across all four items is 4.00, which falls into the aligned region. While Item 8 has a medium effect size, none of the four items exhibit statistically significant differences between the pre- and post-instruction distributions of responses.
Table 8 shows the seven items that comprise the “Science, Religion, and Morality” category. The average post-instruction responses to these items are plotted in Figure 6. The average post-instruction response to Item 7 places it in the anti-aligned region, while Item 32 falls in the aligned region. The remaining items all lie within the neutral region and, consequently, the average post-instruction response across all seven items is 3.03, also within the neutral region. All of these items have small effect sizes and none exhibit any statistically significant difference in the pre- and post-instruction response distributions.
Table 9 contains the four items in the “Science, Emotions, and Aesthetics” category. Figure 7 plots the average post-instruction responses for these items. Items 12 and 36 have average post-instruction responses that place them in the aligned region, while Item 1 falls in the neutral region and Item 21 falls in the anti-aligned region. The average post-instruction response across all four items is 3.35, which is in the neutral region. Item 36 has a medium effect size and has a statistically significant difference in the distributions of pre- and post-instruction responses.
The four items in the “Science, Race, and Gender” category are shown in Table 10 and their average post-instruction responses are plotted in Figure 8. All four items fall in the aligned region, and the average post-instruction response across these items is 3.94, implying that most students agree that “[r]ace, gender, and other personal factors are irrelevant in science” (Cobern 20015; Cobern and Loving 20026). Items 23 and 53 both have medium effect sizes and statistically significant differences between their pre- and post-instruction reponse distributions.
The final TSSI category, “Science for All,” has eight items. These items are shown in Table 11. Their average post-instruction responses are plotted in Figure 9. The average post-instruction response to Item 37 places it in the neutral region. The other seven items all fall in the aligned region, as does the average post-instruction response across all eight items (3.90). This suggests that many students agree with the idea that everyone should know and learn at least some science. All the items in this category have small effect sizes, and none show statistically significant differences between their pre- and post-instruction distributions of responses.
As a final way to analyze the data, we grouped all items in a given category together and calculated the average and standard deviation (pre- and post-instruction) of the responses to all of these items. We also calculated the differences in these pre- and post-instruction averages and the effect sizes of these differences. Additionally, we used the non-directional Mann-Whitney test to determine whether or not the distribution of students' total scores within a category were different pre- to post-instruction at a statistically significant level (p < 0.001). We also performed these calculations for the TSSI as a whole. Table 12 contains our results.
Table 12 shows that the average post-instruction response to items in the categories “Science and the Economy,” “Science and the Environment,” “Science and Public Health,” “Science, Race, and Gender,” and “Science for All” are all aligned with the “common image of science.” The remaining categories have post-instruction averages that place them in the neutral region. “Epistemology” and “Science and the Environment” are the only categories that show statistically significant differences in the distributions of students' pre- and post-instruction total scores. “Science and the Environment” is the only category with a medium effect size; the rest are small. If we look at the average scores for the TSSI as a whole, we see that, pre-instruction, the average falls in the neutral region, while post-instruction, the average falls in the aligned with “the common image of science” region. The shift in the distribution of students' total scores on the TSSI pre- to post-instruction is statistically significant; however, the effect size is small.
CONCLUSIONS AND DISCUSSION
What conclusions can we take away from this study? We observed a statistically significant improvement in students' overall responses to the TSSI's items (although the effect size is small). Additionally, from the data in Section 4 we conclude that students generally have positive (or at least not negative) views about the relationship between science and many aspects of society. These views may improve (or at least not decrease) after taking an Astro 101 course, even when the instruction on worldviews was only presented through lecture.
At this point, we must comment on the wording of some of the TSSI's items. Recall that the TSSI was written in order to measure agreement with the “common image of science” described in Table 1 (Cobern 20015; Cobern and Loving 20026). While we are interested, for our larger research project on students' worldviews, in exactly the contexts of science's interactions with society that appear on the TSSI, we take issue with how some of the TSSI's items are written; this is due to the fact that our perspective of science—and (what we hypothesize is) students' perspective of science—differs from Cobern's “common image of science” in multiple areas. One category of items with which we take issue includes Items 2, 3, 17, 21, 22, 27, 44, 45, and 60. These items, many of which come from the “Epistemology” category, present science in absolute and superlative terms. Someone who believes in all aspects of the “common image of science” must agree, for example, that science provides absolute truth (Item 2), that scientific knowledge is more important for a person than an appreciation of the arts (Item 21), and that science is conducted free of economic and political considerations (Item 45). Furthermore, statements such as Item 17 (“Science is the most objective form of knowledge”) require students to compare science to all other forms of knowledge (a daunting task) and agree that it is the most objective form (above, for example, mathematics). In many of the questions noted above, we would be more comfortable with the wording of the items if “the most” were replaced by “a very.” Such absolute and superlative framings may cause general education students to mark lower levels of agreement with these statements, even if they possess positive views of science.
A second category of items whose wordings we take issue with includes Items 7, 11, 32, 39, 40, and 47. All of these items appear in the “Science, Religion, and Morality” category, and all position science and religion in opposition to one another. We argue that there are many people, including many of our students, who are religious and simultaneously possess positive ideas about science's role in society. These people may think about the relationship between science and religion in terms similar to Gould's (1997)13 characterization of science and religion as “nonoverlapping magisteria.” People who do not view science and religion as necessarily in conflict with one another are likely to give answers to many of these items that are not aligned with “the common image of science” put forth by Cobern.
We have identified a subset of items on the TSSI for which our view of science's role in society differs from Cobern's “common image of science.” This disagreement is actually an important part of our interpretations of the results. If we had observed responses (either pre- or post-instruction) to these items from students that illustrated a strong agreement with Cobern's view, this would have indicated that students have worldviews regarding science and society that are different from ours. However, that is not what we observed. This suggests that our concerns about the TSSI's superlative framing of science are justified and that students might be more inclined to agree with certain items if they were written in less absolute and hierarchical terms.
If we ignore our issues with the wordings of some of the items, we must ask a more fundamental question: Should we take students' responses at face value? One of the limitations of this study is that it uses a Likert scale survey. While such surveys can provide useful information, students often know what responses their instructors would like to see (e.g., Gray et al. 200814). Even if students are trying to give their honest opinions, they may readily agree with a statement such as “Developing new scientific knowledge is very important for keeping our country economically competative in today's world” (Item 49) without being able to provide substantial evidence or robust lines of reasoning to defend their opinions. If we truly wish to better understand, measure, and affect students' worldviews and their understandings of the role science plays in society, then we must collect more than just Likert responses.
As a next step along this research pathway, we administered a series of provocative free-response writing prompts during class time to students enrolled in the spring 2012 version of the Astro 101 mega-course. Students were not told the content of these writing prompts before class, nor were they allowed to use outside resources while in class to help with their writings. While a detailed analysis of their responses is beyond the scope of this paper (and will be the subject of future publications), we note that their responses confirm our hypothesis that many students are unable to cite relevant examples and provide thoughtful, coherent, and logical arguments and correct representations to support their ideas about the relationships between science and society. We now show a selection of students' written responses in order to illustrate how much more information they capture about students, as compared to Likert scale items.
For example, we gave students the following writing prompt, which they had to prepare to respond to in class: “How do you think the taxpayers' return on investment in bailing out financial institutions ($2.5 trillion) will compare, or has compared, to the return on the money we have spent on NASA over the last 50 years (approximately $800 billion lifetime)? Provide evidence (more than just your opinion) to support your beliefs.” Many students wrote that the return on investment must be higher for the bank bailout since the amount of funds given to the banks was so much larger. One student claimed, “They both get a return amount, but clearly investments in bailing has a larger return, but it also has a larger starting point.” Another student elaborated on the idea that one always gets the greatest return on the program in which one invests the most money:
“I think the taxpayer's return on the investment in bailing out our financial institutions is higher than the amount of money we have spent on NASA over the last 50 years. The government spent about 9.0 trillion dollars on investments alone and in 2011, only about 17 million dollars was spent on NASA.”
A third student offered a different reason to support his belief that the bank bailouts will provide a larger return on investment:
“Over the past 50 years, the Government has spent a lot of money helping bankrupt companies. Many of those companies gave money back to the government whereas NASA has not given back. NASA only makes money when they make a discovery or off of patents. It is unfair for NASA to be getting so much money & never returning much of it. Although NASA has not gotten as much money as larger institutions such as mortgage companies, it makes sense because NASA does not return any of it.”
Since we are astronomers, one might think we would automatically favor responses promoting NASA over the bank bailout. However, we would have been happy to read a complete and coherent argument from a student in favor of supporting financial institutions over NASA. We would even have been impressed if a student questioned the premise of our prompt (for example, by arguing that NASA's primary mission does not include yielding a financial return). Yet we did not see those kinds of responses. Instead, we saw responses, such as those listed above, which indicate a troubling level of numerical and economic illiteracy among students, as well as ignorance about the role science plays in society.
Further evidence for students' economic and scientific illiteracies and innumeracies can be found in their responses to another free-response writing prompt. We gave students a pie chart representing the percentage of the federal budget spent on both mandatory programs (e.g., Medicare and Medicaid, Social Security, and interest on the national debt) and discretionary programs (e.g., veteran's benefits, transportation, defense, and the environment, energy, and science). We then asked students “If you could change how tax dollars are distributed to improve our society, what changes would you make? Be sure to describe which areas of tax spending you would decrease, increase, and why you believe these changes would have a benefit for society.” Here are a few randomly selected responses reproduced verbatim (including spelling and grammatical errors):
“I would decrease the amount of health care and medicare because people need health care and shouldn't have to be taxed on it. If you need health care you shouldn't be taxed and paying for it, it should be given to you for free to save a life. I would also decrease defense because people should have the right to get defense without being taxed on it.”
“I also think there should be a decrease on ‘Interest on Debt.’ The people are already trying to pay off their debt and with this interest, it makes it more difficult to obtain a debt-free life.”
“I don't pay tax. 1. maybe by spend more to raise the wages. 2 – you can improve the resuorse from where you get income. Like food resource, indestrail resources”
“Honestly, if someone gave me the opportunity to change all of these numbers around, however I wanted, I wouldn't. I don't even pay taxes and I dont feel as if I am in any position to decide anything like this. Truly, I don't know what should be changed and I feel like anyone who had any say in distributing the numbers as they are now, probably has a much better idea of how they should be distributed, than I do.”
Responses such as these illustrate some of the profound challenges facing instructors who strive to make science relevant within the larger context of issues facing our society. Many students appear to be ignorant of the programs supported by their national government, and that lack of knowledge may challenge instructors who wish to frame the role of science within the broader context of contemporary society. These responses also underscore the importance of helping students make deep and broad connections between what they learn in science and what they learn in other general education courses. Future studies will present the results of our analysis of these responses in more detail.
While this present study is a good first step toward helping us understand Astro 101 students' worldviews, the smorgasbord of written responses cited above demonstrates that much work remains to be done. Future studies must move beyond the straightforward statistical analyses we performed here in order to adequately capture students' worldviews. We are currently analyzing the rich dataset from our students' written responses to better understand what students think and how we can best measure and describe their worldviews. Only then will we be able to address the much more difficult issue of what Astro 101 instructors can do to influence their students' worldviews in a way that helps prepare students to appreciate and participate in a scientifically reliant society.
Acknowledgments
We owe a special thanks to David Consiglio for his help in collecting students' responses to the TSSI. Gina Brissenden provided valuable comments and feedback on an earlier version of this manuscript. We also thank the anonymous referee whose comments improved the overall quality of this paper. This material is based in part upon work supported by the National Science Foundation under Grant No. 0715517, a CCLI Phase III Grant for the Collaboration of Astronomy Teaching Scholars (CATS). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Appendix: Histograms of Students
Below are histograms for each item on the TSSI. Each histogram shows the percentage of students who answered using the responses 1–5. White bars represent pre-instruction percentages and black bars represent post-instruction percentages.
Tables
| Category | Description |
|---|---|
| Epistemology | Science is a superior, exemplary form of knowledge that produces highly reliable and objective knowledge about the real world. |
| Science and the economy | Modern industrial, commercial, and information-based economies depend on scientific developments for increasing production, wealth and general public welfare. |
| Science and the environment | Science is necessary for the discovery, development, and conservation and protection of natural resources and the environment in general. |
| Public policy and science | Science acts in the public interest. Science should thus be supported by public funds; however, the science community is more than capable of policing scientific activity. |
| Science and public health | The conquering of disease and physical affliction and the great advances in public health are made possible by science and will not continue without science. |
| Science, religion, and morality | People make moral choices about the use of scientific findings but science itself is morally neutral. Science is also neutral with regard to religion. The importance of science, however, is such that science must be protected from the intrusive activities of some religions. |
| Science, emotion, and aesthetics | Scientists are often passionate about their work but the work of science best proceeds on the basis of objective reason and empiricism. There is a beauty to science. Indeed, “elegance” is often required of scientific ideas. |
| Science, race, and gender | Science is an “equal opportunity employer.” Race, gender and other personal factors are irrelevant in science. |
| Science for all | The importance of science is such that it should be taught at all levels of schooling. Every citizen should have attained at least a minimal level of science literacy. |
| Pre-instruction | Post-instruction | Effect size | |||||
|---|---|---|---|---|---|---|---|
| Item | Avg. | Standard deviation | Avg. | Standard deviation | Δ Avg. | p-value | (Cohen's d) |
| 1 | 3.4 | 1.1 | 3.3 | 1.2 | −0.2 | 0.1118 | 0.14 |
| 2 | 2.9 | 1.3 | 3.1 | 1.3 | 0.2 | 0.0969 | 0.15 |
| 3 | 3.1 | 1.2 | 3.4 | 1.3 | 0.3 | 0.0026 | 0.25 |
| 4 | 4.2 | 1.0 | 4.2 | 1.1 | −0.1 | 0.4413 | 0.05 |
| 5 | 3.1 | 1.1 | 3.4 | 1.1 | 0.3 | 0.0004 | 0.28 |
| 6 | 4.5 | 0.8 | 4.5 | 0.8 | 0.0 | 0.4533 | 0.05 |
| 7 | 1.9 | 1.0 | 2.0 | 1.1 | 0.1 | 0.5961 | 0.05 |
| 8 | 3.9 | 1.0 | 4.2 | 0.9 | 0.2 | 0.0076 | 0.25 |
| 9 | 3.1 | 1.0 | 3.3 | 1.1 | 0.2 | 0.0076 | 0.15 |
| 10 | 3.9 | 0.9 | 4.2 | 0.9 | 0.3 | <0.0001 | 0.33 |
| 11 | 3.2 | 1.4 | 3.3 | 1.4 | 0.2 | 0.2585 | 0.12 |
| 12 | 3.7 | 1.0 | 3.9 | 1.1 | 0.1 | 0.0488 | 0.14 |
| 13 | 3.4 | 1.2 | 3.7 | 1.3 | 0.2 | 0.0060 | 0.19 |
| 14 | 3.5 | 0.9 | 3.6 | 1.1 | 0.2 | 0.0060 | 0.17 |
| 15 | 4.1 | 1.1 | 4.0 | 1.2 | −0.1 | 0.6892 | 0.08 |
| 16 | 4.2 | 0.9 | 4.3 | 0.9 | 0.0 | 0.4473 | 0.05 |
| 17 | 3.4 | 0.9 | 3.6 | 1.0 | 0.2 | 0.0220 | 0.21 |
| 18 | 3.3 | 1.0 | 3.6 | 0.9 | 0.3 | 0.0001 | 0.32 |
| 19 | 2.4 | 1.0 | 2.5 | 1.0 | 0.1 | 0.6527 | 0.08 |
| 20 | 3.8 | 1.0 | 4.0 | 0.9 | 0.3 | 0.0002 | 0.27 |
| 21 | 2.4 | 1.1 | 2.1 | 1.1 | −0.2 | 0.0178 | 0.19 |
| 22 | 2.4 | 1.0 | 2.6 | 1.0 | 0.2 | 0.0111 | 0.19 |
| 23 | 3.4 | 1.0 | 3.7 | 1.1 | 0.2 | 0.0008 | 0.24 |
| 24 | 4.3 | 0.8 | 4.3 | 0.8 | 0.0 | 0.3898 | 0.04 |
| 25 | 4.5 | 0.7 | 4.5 | 0.8 | 0.0 | 0.4065 | 0.01 |
| 26 | 3.6 | 1.1 | 3.5 | 1.2 | −0.1 | 0.3576 | 0.08 |
| 27 | 2.5 | 1.1 | 2.6 | 1.2 | 0.1 | 0.1868 | 0.11 |
| 28 | 2.8 | 1.0 | 2.8 | 1.1 | 0.0 | 0.6672 | 0.04 |
| 29 | 3.2 | 1.0 | 3.4 | 0.9 | 0.2 | 0.0019 | 0.24 |
| 30 | 4.1 | 1.0 | 4.1 | 1.1 | −0.1 | 0.3681 | 0.08 |
| 31 | 4.0 | 0.8 | 4.1 | 0.9 | 0.1 | 0.0688 | 0.10 |
| 32 | 3.4 | 1.2 | 3.6 | 1.2 | 0.2 | 0.0300 | 0.16 |
| 33 | 3.4 | 0.9 | 3.6 | 0.9 | 0.2 | 0.0015 | 0.24 |
| 34 | 3.3 | 1.0 | 3.6 | 1.0 | 0.3 | 0.0004 | 0.29 |
| 35 | 3.1 | 1.0 | 3.3 | 1.0 | 0.2 | 0.0034 | 0.20 |
| 36 | 3.9 | 0.9 | 4.1 | 0.8 | 0.2 | 0.0004 | 0.27 |
| 37 | 3.2 | 1.2 | 3.2 | 1.2 | 0.0 | 0.9442 | 0.01 |
| 38 | 3.7 | 1.1 | 3.9 | 1.1 | 0.2 | 0.0007 | 0.22 |
| 39 | 3.2 | 1.1 | 3.1 | 1.2 | −0.1 | 0.3030 | 0.09 |
| 40 | 3.1 | 1.1 | 3.1 | 1.2 | 0.0 | 0.6455 | 0.04 |
| 41 | 4.1 | 0.9 | 4.2 | 0.9 | 0.1 | 0.0524 | 0.11 |
| 42 | 3.3 | 1.0 | 3.6 | 1.0 | 0.4 | <0.0001 | 0.37 |
| 43 | 4.1 | 0.8 | 4.3 | 0.8 | 0.2 | 0.0002 | 0.27 |
| 44 | 2.5 | 0.9 | 2.6 | 1.0 | 0.1 | 0.6384 | 0.09 |
| 45 | 2.6 | 0.9 | 2.5 | 1.0 | 0.0 | 0.5961 | 0.05 |
| 46 | 3.4 | 0.8 | 3.6 | 0.9 | 0.1 | 0.0751 | 0.16 |
| 47 | 2.9 | 1.1 | 2.9 | 1.1 | 0.0 | 0.8259 | 0.02 |
| 48 | 4.4 | 0.8 | 4.5 | 0.7 | 0.1 | 0.2937 | 0.11 |
| 49 | 4.0 | 0.9 | 4.3 | 0.9 | 0.2 | 0.0007 | 0.24 |
| 50 | 3.2 | 0.9 | 3.5 | 0.9 | 0.2 | 0.0041 | 0.24 |
| 51 | 4.5 | 0.7 | 4.5 | 0.7 | 0.1 | 0.1236 | 0.13 |
| 52 | 4.1 | 0.9 | 4.2 | 0.9 | 0.1 | 0.0500 | 0.15 |
| 53 | 3.6 | 1.1 | 3.8 | 1.1 | 0.3 | 0.0005 | 0.27 |
| 54 | 3.6 | 1.1 | 3.7 | 1.1 | 0.1 | 0.0010 | 0.11 |
| 55 | 3.8 | 1.0 | 4.0 | 1.0 | 0.1 | 0.0455 | 0.15 |
| 56 | 3.8 | 1.0 | 4.0 | 1.0 | 0.2 | 0.0155 | 0.19 |
| 57 | 3.6 | 1.0 | 3.8 | 1.0 | 0.2 | 0.0131 | 0.17 |
| 58 | 4.0 | 1.1 | 4.1 | 1.2 | 0.1 | 0.1835 | 0.08 |
| 59 | 3.7 | 1.0 | 4.0 | 0.9 | 0.3 | 0.0013 | 0.26 |
| 60 | 3.1 | 1.0 | 3.4 | 1.0 | 0.3 | 0.0006 | 0.27 |
| Item 2 (reversed scoring) | No source of knowledge provides absolute truth—not even science. |
| Item 17 | Scientific knowledge is the most objective form of knowledge. |
| Item 27 (reversed scoring) | No form of knowledge can be completely certain—not even scientific knowledge. |
| Item 29 | We can be certain that scientific knowledge is reliable. |
| Item 33 | The methods of science are the most reliable source of true, factual knowledge. |
| Item 34 | Science is the best source of reliable knowledge. |
| Item 44 (reversed scoring) | No form of knowledge—including science—can ever be completely objective. |
| Item 46 | The methods of science are objective. |
| Item 60 | Scientific knowledge is the truest form of knowledge. |
| Item 14 (reversed scoring) | The strength of our national economy does not depend on scientific knowledge. |
| Item 16 | Science helps develop our natural resources such as coal, gas, oil, and solar energy. |
| Item 20 | Scientific knowledge is useful in keeping our national economy competative in today's world. |
| Item 22 (reversed scoring) | The development of our natural resources, such as coal, gas, oil, solar energy, requires much more than scientific knowledge. |
| Item 25 | There are many good things we can do today because of scientific knowledge. |
| Item 31 | The development of our natural resources, such as coal, gas, oil, solar energy, is dependent on having adequate scientific knowledge. |
| Item 41 (reversed scoring) | Scientific knowledge is useful for only a few people. |
| Item 42 | Science is our best source of useful knowledge. |
| Item 49 | Developing new scientific knowledge is very important for keeping our country economically competative in today's world. |
| Item 51 | Scientific knowledge is useful. |
| Item 3 (reversed scoring) | Scientific knowledge has often contributed to the destruction of our environment and natural resources. |
| Item 38 (reversed scoring) | Our natural environment would actually be helped by the absence of scientific knowledge. |
| Item 43 | Science can help us preserve our natural environment and natural resources. |
| Item 59 | Without science we would not be able to preserve our natural environment and natural resources. |
| Item 5 | Scientific research is rarely dangerous to the public. |
| Item 6 | Scientific research is generally very important. |
| Item 10 | Scientific research should be adequately funded by government. |
| Item 18 (reversed scoring) | Scientific research is often potentially dangerous to the public. |
| Item 19 | There is little need for the regulation of scientific research. |
| Item 26 (reversed scoring) | Scientists should not be allowed to research anything they wish. |
| Item 28 (reversed scoring) | Scientific research should be carefully regulated by law. |
| Item 45 (reversed scoring) | Scientific research is economically and politically determined. |
| Item 50 (reversed scoring) | Scientific knowledge influences government decision making too much. |
| Item 57 (reversed scoring) | The government should not be in the business of using tax dollars to fund scientific research. |
| Item 8 | Scientific knowledge is the single most important factor in the improvement of medicine and public health. |
| Item 9 (reversed scoring) | Common sense contributes more to good health than does scientific knowledge. |
| Item 48 | Scientific research makes important contributions to medicine and the improvement of public health. |
| Item 58 (reversed scoring) | Scientific knowledge contributes little to good health. |
| Item 7 (reversed scoring) | A person can be both religious and scientific. |
| Item 11 | Science is a more important source of knowledge than religion. |
| Item 32 (reversed scoring) | Religious knowledge contributes more to the well being of a person's life than does science. |
| Item 35 | Scientific research is morally neutral. |
| Item 39 | Religion and science are almost always at odds with each other. |
| Item 40 | Religion tends to impede scientific progress. |
| Item 47 | Scientific knowledge tends to erode spiritual values. |
| Item 1 (reversed scoring) | Human emotion plays no part in the creation of scientific knowledge. |
| Item 12 (reversed scoring) | Scientific explanations tend to spoil the beauty of nature. |
| Item 21 (reversed scoring) | It is equally important for a person to have scientific knowledge and an appreciation of arts. |
| Item 36 | Science can contribute to our appreciation and experience of beauty. |
| Item 4 | Women are welcome in science just as much as men are. |
| Item 23 (reversed scoring) | The scientific community is mostly dominated by men and is often unfriendly to women. |
| Item 30 | African Americans and other minority people are just as welcome in the scientific community as are white people. |
| Item 53 (reversed scoring) | The scientific community is mostly dominated by white men and is often unfriendly to minority people. |
| Item 13 (reversed scoring) | Students should not be forced to take science courses at the university. |
| Item 15 (reversed scoring) | Science should not be made an important subject for the elementary school grades. |
| Item 24 | Understanding science is a good thing for everyone. |
| Item 37 (reversed scoring) | Only a very few people really understand science. |
| Item 52 | All students should study science during the secondary school grade levels. |
| Item 54 (reversed scoring) | Most people really do not need to know very much science. |
| Item 55 | Even at the university level all students should study at least some science. |
| Item 56 | Science should be taught at all school grade levels. |
| Preinstruction | Post-instruction | Effect size | |||||
|---|---|---|---|---|---|---|---|
| Category | Avg. | Standard deviation. | Avg. | Standard deviation | Δ Avg. | p-value | (Cohen's d) |
| Epistemology | 3.10 | 1.06 | 3.29 | 1.10 | 0.19 | <0.0001 | 0.18 |
| Economy | 3.84 | 1.07 | 3.80 | 1.13 | −0.04 | 0.4473 | 0.04 |
| Environment | 3.64 | 1.08 | 3.90 | 1.09 | 0.26 | <0.0001 | 0.24 |
| Public policy | 3.31 | 1.14 | 3.44 | 1.18 | 0.13 | 0.0011 | 0.11 |
| Public health | 3.86 | 1.09 | 4.00 | 1.08 | 0.14 | 0.0045 | 0.13 |
| Religion | 2.95 | 1.21 | 3.03 | 1.27 | 0.08 | 0.1260 | 0.07 |
| Aesthetics | 3.34 | 1.21 | 3.35 | 1.30 | 0.01 | 0.7872 | 0.01 |
| Race and gender | 3.84 | 1.10 | 3.94 | 1.09 | 0.10 | 0.1052 | 0.09 |
| Science for all | 3.78 | 1.10 | 3.90 | 1.13 | 0.12 | 0.0193 | 0.11 |
| Whole TSSI | 3.48 | 1.17 | 3.62 | 1.19 | 0.14 | <0.0001 | 0.12 |
first citation in article
This Publication
Scitation
Google Scholar
PubMed