Gaining political knowledge and raising political efficacy at school and at home
Millions of young Americans pay little attention to politics. They don’t follow the news, they lack even the most basic knowledge about political institutions, they don’t vote, and they don’t care. Identifying these behaviors as problems for the future of American democracy and recognizing that many of them are products of early-life socialization processes, numerous organizations are now pushing “civic education” efforts of various sorts, many of them targeted at elementary, middle school, high school, and college students.
In a new study, Timothy Vercellotti and Elizabeth Matto probe the impact of political participation in the school and at home on knowledge about politics and the sense of political efficacy. The design of the study is unusual – more innovative and ambitious than appears to be the norm in this research area. The participants were 361 high school students from four high schools who were assigned to either a treatment group that read newsmagazine articles weekly for eight weeks and discussed them in class, a treatment group that read the same articles and discussed them in class and with their parents, and a no-treatment control group. Each group was surveyed three times – first at the very start of the study, again after the eight-week treatment period, and once more six weeks later.
Political knowledge, as gauged by familiarity with various public figures, increased in all three groups, presumably because the study was conducted during the presidential caucus and primary period. Even so, the greatest increase occurred for the second treatment group – the one that discussed the articles both in class and at home. And knowledge remained at its second survey level six weeks later.
Internal efficacy, as measured by the National Election Study items that will be familiar to many “Monkey Cage” readers. Once again, the largest effects were for the second treatment group.
I have some methodological qualms. For one thing, intact classes, not individual students, were randomly assigned to the treatment groups, a feature that introduces uncertainty about what the treatment really is (the treatment itself, or something about the class, e.g., the quality of the instructor). Moreover, as Vercellotti and Matto recognize, using the same knowledge items in all three surveys could have produced a wave-to-wave learning effect. The timing of the survey, coming as it did in the middle of a high-visibility campaign, is also unfortunate. And I wish the design could have been expanded to include more groups: another control group that was surveyed only during the third wave; yet another control group that received a placebo of some sort like reading stories from, say, Sports Illustrated; and a third treatment group that discussed the articles at home but not at school.
Notwithstanding these qualms, there’s a lot to like about this study. Its subject matter is important; its application of a large-scale field experiment to address these issues is a definite step forward; and its findings, while hardly the last word on the subject, strike me as warranting greater confidence than those reported in many previous political socialization studies by political scientists and others.
Comments
I think this sounds like a really compelling study, granted there are the design flaws you indicate. A multi-level design that randomized schools, classrooms, and individuals might have clarified the treatment effect(s).
The more important, and unanswered question, is what about lasting effects? Sure, there’s measurement a few weeks after treatment, but that doesn’t necessarily mean altered behavior. For example, if you have an individual read Hamlet, they can probably answer questions on it (and maybe Shakespeare generally) with gradually-declining success. A test six weeks after reading it might show high levels of relevant knowledge, but it doesn’t show if the subject read any more Shakespeare or read more of anything.
The same principle applies here: if there’s a normative desire to increase civic participation, any effort to induce such behaviors in youths need to have some sort of meaningful effect on future action. Some research (from the Teaching/Education disciplines) suggests that engaging students in service-learning can have such lasting effects on future volunteering, voting, and self-efficacy.
Posted by: Thomas J. Leeper | June 25, 2009 01:38 PM
Realizing that many non-voters are youth (under 30), is there much data on non-voters over 30 that are “chronic” non-voters? Analyses tend to focus on snapshots, but is there much out there on non-voting over the lifespan (or chunks of it)?
Posted by: Doug Hess | June 25, 2009 10:51 PM
“…intact classes, not individual students, were randomly assigned to the treatment groups, a feature that introduces uncertainty about what the treatment really is…”
The potential for “classroom interference” could be analyzed through multi-level analysis: i.e., you put second-level predictors in the model (e.g., more experienced teachers may have greater effect, so you include a predictor for number of years each classroom teacher has taught and then cross it with the treatment dummy; or predictors describing the class racial makeup, or mean household income, etc.), but even if you don’t know what it is about the clustering in the data by classroom that might have an impact, you will be able to see if it clustering is an issue (i.e., the ICC can be calculated).
Andrew Gelman can correct me or explain it better.
Posted by: Doug Hess | June 25, 2009 11:00 PM
Doug, I think you raise a good question about long-term behavior. Does anyone know of any major longitudinal studies being conducted within political science?
Posted by: Thomas J. Leeper | June 26, 2009 10:20 AM
Doug and Thomas:
Some truly long-term, successive-wave studies would be great, for the major question is typically about enduring effects, not transitory ones. (Though sometimes about the transitory ones.)I don’t know of any such studies. (I’m too lazy to look up the Jennings-Niemi study to see how far separated the panel waves were, but it wasn’t very far.) The exception are a few extremely long-term panel studies by psychologists — 40 years or more. Unfortunately, I haven’t seen any of those that contain much, if anything, that would be of central interest to political scientists.
One other consideration: The longer the study goes on, the more uncontrollable sources of variance come into play. There’s a tradeoff between the length of the panel and the messiness or interpretability of the results. Nobody ever said this was going to be easy.
Posted by: Lee Sigelman | June 26, 2009 10:53 AM
Off the top of my head, the Jennings-Neimi study was ‘65-’73-’82-’97. I think that only the high school students from ‘65 were reinterviewed for ‘97. The other waves have both the students and their parents.
Posted by: Jason M | June 26, 2009 11:30 AM
Doug and Thomas: There’s your answer.
Jason: Thanks. I’m sure your memory is better than mine.
Posted by: Lee Sigelman | June 26, 2009 02:08 PM