METHODS AND RESULTS
Research was conducted over two semesters at a large state university in the western U.S. In the first semester, digital-device use was investigated in three large introductory courses for nonscience majors (N = 318; Astronomy, Geology), and data gathering focused on students’ in-class use of laptop computers. The following semester, the study focused on in-class use of cellular phones in five additional courses. All eight courses used iClickersTM and Peer Instruction (sometimes called Think-Pair-Share) exercises to engage students. The instructors teaching the courses were asked to refrain from verbally discussing technology use and note-taking behavior with their students, and all aspects of the research were approved by the internal Human Subjects Review Board.
Utilizing a mixed-methods approach consisting of in-class observations, survey responses, and semi-structured interviews, we examined the effects of digital devices on student performance. Survey questions targeted demographics, student attitudes, and self-reported levels of technology use. The data were collected via a clicker polling system and then anonymously correlated with final course grades. A clicker system separate from that used by the instructor was brought into the classroom to collect our data, and students were assured of anonymity. Semi-structured interviews (N = 24) provided insight into student attitudes as related to behavior and the use of digital devices. In-class observations (N = 31 days) recorded the behavior of students and the instructors during lecture; this data permitted verification of students’ self-reports in interviews and survey responses.
In two of the three courses observed in semester 1, lecture notes were posted for student reference: one before class and in the other, after. Survey responses indicate 60% of the sample (N = 316; 91.6% response rate) took notes with pen and paper, 12% took notes on a laptop computer or some other electronic device, and 28% reported no note taking. In the third course, the instructor did not post notes. Students’ final grades were statistically indistinguishable across the three note-taking methods—we did not replicate the experience in the Engineering class that motivated this study. Our initial research therefore suggests that peer instruction and student engagement may play an important role in mitigating “digital distraction.”
It is interesting to note that when we presented the results of the initial Sieber study (personal communication) to faculty members, many interpreted the results as suggesting that writing on paper, drawing arrows or diagrams, and underlining topics engage the brain in a more complete way than typing. When we presented the results to students, the response was uniformly, “the laptop users are getting distracted by Facebook or other things they can do during class.” Our observations confirm that the student interpretation is more likely the correct one. However, we also learned that the laptop computer is no longer the device of choice for most students to bring to class.
Given recent national data confirming high rates of computer ownership (EDUCAUSE 20103
) among undergraduates, the number of students who took notes on laptop computers was much lower than we expected. At the same time, our observation and survey data indicate very high rates of in-class cell phone use: 75% of the sample (anonymously) reported regular phone use during lecture. These data motivated an extension of the study into the subsequent semester, where we sought to isolate the effects of cell phone use on learning outcomes. Students in five additional classes (N = 392, all introductory astronomy) reported their frequency of in-class cell phone use, and these data were then correlated with their final course grades.
All survey data were kept anonymous and participation in this study was voluntary. Response rates across the five courses ranged from 96% to 98% of those in attendance on the day the data were collected, and the number of responses ranged from 44 in the smallest class to 111 in the largest. Results from the five courses (combined) are presented in Table 1
and Figure 1
Self-reported frequency of cell phone use vs final grade.
The average grade difference between students who use their phones at all and those who do not is 0.36 ± 0.08. Cell phone use is significantly correlated with reduced learning outcomes: students who reported no cell phone use earned significantly higher grades than those who used their phones during class.
Interestingly, those who use their phones reported an average frequency of three times per class period, yet the observation data suggest this frequency may in fact be much higher. Observation notes suggest the average frequency of cell phone use is closer to seven times per class period (N = 32 observations, average = 6.85 incidents of use per user, per class period), indicating that undergraduates typically underreport their frequency of in-class cell phone use. These data are drawn from a relatively small sample; as we do not have more comprehensive data on rates of cell phone use across all five courses, we focus on the differences between phone users and nonusers in the ensuing discussion.
The difference in final grades between phone users and nonusers is highly statistically significant, but causality cannot be determined from this data. It could be that students who are already more disciplined are more likely to turn their phones off during class or that students who are busier outside of class are more likely to use their phones in class. Still, the observation and interview data suggest many undergraduates likely overestimate their ability to multitask. If students miss some instruction while engaging with digital devices, they may end up earning lower grades overall.
This conclusion is consistent with a recent finding that Stanford undergraduates consistently overestimate their ability to multitask (Ophir, Nass, and Wagner 20098
). In our interviews, students argued both for and against the ability to multitask effectively: “I could probably take notes on a computer, but I don’t [because] it’s really easy to be doing other things in class than taking notes.” “I feel phones can definitely be a distraction if you let them, but if you’re using your phone when you have a minute between taking notes or while the professor is going back and discussing something… I don’t find my phone distracting [then].”
This issue will become increasingly important for teachers as more and more students come to class with Internet-capable handheld devices (EDUCAUSE 20103
). On the bright side, our observation data suggest the use of clickers and peer instruction may mitigate distractions due to digital devices. We observed several occasions in which students stopped using their computers and cellular phones (for noncourse related purposes) when clicker questions were used. Yet the drive to multitask is strong, as confirmed by our interviewees: “I think I’ll still bring [my laptop] to class, but I’ll probably have to close all the applications except for [Microsoft] Word, that way I won’t be tempted… If they’re open, it’s just so easy to go over there and come back.” “It’s just staying connected. I know if I look at my phone, there’s a good chance someone’s gonna want to know if I want to [meet up] after class…there’s an email, I can check my horoscope. I could do something else right now.”