Classroom Technology and the West Point Study

[Edit, 1/2/17: With apologies to Kevin Gannon for the snark and straw men in this post. I’m leaving it as is, however, out of a historian’s perhaps misplaced sense of source integrity. For a great, and extensive, exchange with Kevin and Josh Eyler, see our Twitter convo.]

I completely missed this when it came out the other day, but the Social Sciences Department at West Point apparently ran a study on student use of technology in the classroom. [Correction: MIT ran the actual study.] The result? Students not using laptops or tablets did (marginally) better throughout the semester than those who did.Reaction GIF: eye roll, Stanley Hudson, The Office Naturally, this got some folks riled up, and not without some reason–media tend to go for the big headline, and people scanning headlines will simply take away “ban technology!” and not delve further.

In other words, discussion is locked down, rather than opened up.

I first learned of this when someone (I can’t remember who) retweeted Jonathan Becker’s article on the subject. From there I was led to Kevin Gannon’s column. Both authors make some good points, but both also fail somewhat in their broader analysis largely because neither seems to know anything about West Point, and each makes several errors because of that ignorance. So, I want to shed some light on these issues, because it impacts the overall credibility of these articles (we’ll put aside for a moment the typical paranoid reaction these studies seem to always elicit). I’m going to pick on Gannon more, both because of the tone of his column and because his claims come off (to me, anyway) as far less constructive than Becker’s. [This is actually the first of two columns taking issue with Gannon’s assertions. Sorry Dr. Gannon, it’s nothing personal, but some of your recent writing just happens to intersect with my end-of-semester reflections.]

First, Gannon’s key argument against the study is that

We learn that there were three sections of an economics course, consisting of nearly 800 students. I’m going to go out on a limb and say that they were taught in the large-lecture format. We know that lecture is one of the most ineffective ways to teach, and that student learning increases significantly when it is eschewed in favor of a pedagogy that embraces active learning (see the classic meta-study here). Yet we’re being asked to believe that hundreds of students packed into a lecture hall and subjected to demonstrably ineffective teaching methods aren’t learning because of their device use. That’s a design flaw characteristic of the entire genre of classroom-tech studies. They don’t even acknowledge, much less control for, pedagogy.

Nope, wrong on all counts. That’s not how West Point pedagogy works (I’ve already dealt with anti-lecture truthers, no need to repeat my version of that). Each class runs from 15 to 18 (occasionally 19, if enrollment is high) students per class. The study was grouping sections into three larger groups [you would know this, if you’d bothered to read the study: see page 8]. West Point is actually pretty cutting-edge when it comes to pedagogy, and has been since its foundation, because the prevailing philosophy has always been that small class sizes, with high levels of active student engagement, provide the best learning environment. It’s called the Thayer Method, after Sylvanus Thayer, superintendent from 1817-1833. A good description of the method is here, and more technical one, as applied to chemistry classes, here. I’m left wondering why, if you’re going to criticize something, you wouldn’t do a little research into the thing you’re criticizing…

A typical classroom at West Point, with COL Musteen leading the study of military history
A typical classroom at West Point, with COL Musteen leading the study of military history

But ultimately this is partly excusable, since, as Becker realizes, the study itself doesn’t really indicate what happens in a West Point classroom, and that leads to this:

are you able to tell me what happened in those classrooms? If not, how can we really know if the laptops are/were a distraction or not? What if the professors had the students doing lots of group projects that required them to do online research or collaborative work using Google Docs? If the course is designed for lecture and intense discussion, then maybe laptops are a “distraction.” But, then, that means the study would only generalize to classes where there is only lecture and intense discussion. Instead, we get articles like the Washington Post one that lead people to conclude that laptops should be banned from classrooms entirely, irrespective of instructional design.

And this leads me to my second point…

(Second) Becker’s point about instructional design is very well taken, because up to now West Point has actually strongly encouraged student use of technology in the classroom. The History Department in particular actually took the lead in promoting tablet use in class and tailoring its curriculum to a hybrid digital learning environment–the Thayer method with iPads (and now Android devices). The whole point of the iPads to begin with was to employ the power and potential of modern technology to make traditional subjects more dynamic. And the results seemed to bear it out. Granted, there was no study comparable to the current one (despite, cough, one or two people’s suggestion that one be implemented), but through extensive polling it was fairly well established that cadets found the new digital curriculum more engaging, practical, and rewarding than before. For example, with the ability for any participant to take over the projector, cadets could collaborate in real time in class, and then present (“brief” in military lingo) the class on their analysis. The interactive text illustrated concepts and ideas, and students had a lot more flexibility in studying to meet their individual learning styles than before. Not exactly Universal Design, but pretty darn close.

So, I’m not really sure what the internal politics of this new econ study actually were (and I agree with both Gannon and Becker, the performance deviations are not significant, especially when you remember that standard deviation includes outliers which impact the final number). When the iPad/tablet concept was introduced, there was a general concern that cadets would be more likely to goof off in class, and that this would be harder to correct with tablets (the basis of the “tablets flat on desks” lingo of the study), but that turned out to be marginal. Quizzes were still done via paper, exams were done via paper or laptop, at the instructor’s, and below that the student’s, preference. Certainly over-reliance on technology was not encouraged, not least because, well, this was the military, and everyone relies too much on technology (the Naval Academy has been sensing this too). Oh, and if you’re wondering how come I know so much about this, well, I’m the guy who piloted the History Department course, and then served as the assistant course director for 1100 students spread over about 60 individual classes.

There does remain, however, the issue of West Point’s “special” student body.

So, third and last is the issue of how “exceptional” West Point is in terms of the student experience at a four-year liberal arts college (well, engineering college, but a lot of people agree that that is one thing that has outlived its usefulness since Thayer’s day). West Point is, of course, different in some significant ways. The whole military side of the house, for example, is something regular college students don’t have to deal with, unless you’re at one of the other federal or state academies. West Point students tend to be very driven, very physically fit, and very resourceful. They’re also under a heck of a lot more pressure than their civilian counterparts, so it tends to balance out. They do also tend to be more respectful than your average student. If late for class, for example, cadets will stand at attention in the doorway until you give them permission to enter and sit down. Cell phones, which Gannon wonders about, are hardly to be seen, not least because the uniform pockets aren’t exactly made for cell phones, except maybe the back pocket, so if everyone’s cell phones are put away, you can’t very well sneak yours out in the small classroom environment.

But as for Becker’s guess that students at West Point are “focused on compliance”? The truth is, no more so than your average student. West Point students do what they need to do to survive and get through your class, and they’re like any other college student in that regard. Because they’re driven to succeed they tend to pick up your suggestions faster than many students, but by the standards of the civilian students I’ve taught over the last decade, West Point students are what we call “slightly right of center mass.”

It’s also worth noting the study’s authors are aware that West Point offers advantages to educators that many other schools don’t (page 26):

In a learning environment with lower incentives for performance, fewer disciplinary restrictions on distracting behavior, and larger class sizes, the effects of Internet-enabled technology on achievement may be larger due to professors’ decreased ability to monitor and correct irrelevant usage.

Now Gannon says what we should be asking is why that student felt it was more worth their time to surf the web than engage in class. To which I can only say, my friend, I know you teach surveys with unwilling conscripts. It ain’t always the professor’s fault that the student isn’t engaged (we can go down this road later…).

There is one point, however, where Gannon does hit the nail on the head, in that the study makes no provision for disability. And that is correct, because the brutal truth is that military schools are not geared to students with disabilities, nor can they be, in practical terms. In fact, in my last year there I found myself meditating on the curious fact that, while the support system includes counseling, resiliency training, and the arrival of therapy dogs during finals (very popular, those), the military schools are probably the last places where disability rights activism for students will take root, if it ever does, simply because the military is an outcomes-based organization. In that sense, the West Point study is informed by the “unique,” non-academic side of things, and will only tell you about a certain range of person. Granted, you have the ability, as the study authors noted, to control for a lot of extraneous factors that you normally wouldn’t in a regular college, which is, actually, a strength of the study. But it is only a range nonetheless, and even then, each cadet has his or her own struggles during the semester, so you can’t assume equal prep come the final exam.

Ok, so that’s two points. Yet, most of his premises being factually wrong, the rest of Gannon’s column is really more like a “get off my lawn” echo chamber than an actual attempt to engage with unwelcome findings. This is a typical reaction I’ve noticed from what I call “avant garde educators”: refuse to acknowledge or engage, heck, a refusal to even read, studies that contradict your beliefs, whether or not those studies are actually wrong (and I agree that this new study doesn’t tell us much). It only helps to bifurcate the profession into camps of the enlightened and the troglodytes. The complaint about exams, while not completely without merit, can be dismissed on much the same grounds as I pointed to for Twitter metrics: you can’t ditch metrics simply because you don’t like them on principle. Admin won’t find that a convincing reason, and Admin would be right. Persuade Admin to use different metrics because learning outcomes, but good luck on getting rid of finals.

I get that Gannon and Becker are concerned with people taking a study done at a military school and extrapolating to all of higher ed, but if you’re going to make that concern relevant you’ve got to get your facts right. It just underscores to me that most academics simply have no idea what goes on at places like West Point, and imagine some kind of authoritarian degree-mill dystopia where “compliance” is key. The truth of the matter is rather different, and because of that, if not out of principle, you should endeavor to do your homework before casting stones at straw men. The truth of the matter is that cadets are students–with a difference, to be sure, but still students in most of the ways that matter to higher ed in general. The truth of the matter is that, wherever your pedagogy is, West Point probably passed it five years ago.

Oh, and as for Becker’s parting query? Joke’s on him. Thayer Hall, where history and social science instruction takes place, is mostly windowless. Problem solved.

Thayer Hall, from the river courts

[Last edited at 7:48 p.m., 26 May]

2 Replies to “Classroom Technology and the West Point Study”

  1. As always, crisp and cogent, Dan!
    And an echo on class sizes: I taught for three years teaching at West Point, and the only time I saw any classes EVER in a large lecture format were the annual guest lecture by that year’s visiting distinguished professor, and the crisis days of the 2013 Sequester, when some class groups had to double up because a quarter of the faculty was in limbo status. Even then, the combined class groups did not exceed 40 students.

    1. Thanks Nick! I get that Gannon and Becker are concerned with people taking a study done at a military school and extrapolating to all of higher ed, but if you’re going to make that concern relevant you’ve got to get your facts right. It just underscores to me that most academics simply have no idea what goes on at places like West Point, and imagine some kind of authoritarian dystopia where “compliance” is key. Instead, I think you’d agree with me that they’re students with a difference, but still students in most of the ways that matter to higher ed in general.

Comments are closed.