Why does automation threaten the viability of qualitative research?

The question, from a backbencher: "How are quallys different from quantoids?"

The answer: Few topics are so perennially interesting for the individual political scientist and the discipline as the Question of Method. This is normally, quickly, and violently reduced to the Quant v. Qual debate, which is a battle of those who can't count against those who can't read. Even divisions that really affect other dimensions of methodological debate, such as those that separate formal theorists and interpretivists from case-study researchers and econometricians, are lumped into this artificial dichotomy. Formal guys know math, so they must be econometricians, or at least close enough; interpretivists use language, ergo they are case-study guys, or at least close enough. And so elective affinities are reified into camps, among which ambitious scholars must choose. (How else to understand EITM, which is the worst bastardization of all: the melding of deterministic theory with stochastic modeling?)

(Incidentally, let's not delude ourselves into thinking that mutli-method work is a via media. In everyday practice, quantoids proceed according to a one-drop rule: if a paper contains even the slightest taint of process-tracing or case studies, then it is irremediably quallish. In this, then, those of us who identify principally as multi-method guys stand in relation to the qual-quant divide rather as Third Way folks stand in relation to left-liberals and to all those right of center. That is, the qualitative folks reject us as insufficiently pure, while the quant camp thinks that we are all squishes.)

There is one point, however, in which it is incontestable that the quantoids have the upper hand on everyone else. And that is in the ease of transmission of the techne of data management, of data collection, and the analysis of data. I do not mean the high-concept stuff. All of that is usually about as difficult and as rarefied as the qualitative or formal high-concept readings, and about as equally useful to the completion of an actual research project--which is to say, not very, except insofar as it is beaten into everyday practice. (That R^2 continues to be reported as an independently meaningful statistic even 25 years after King (1986) is shocking, but the Kuhnian generational replacement has not yet so far really begun to weed out such ideological deviationists.)

No, when I talk about techne, I mean something closer to the quotidian translation of the replication movement, which when put into practice is rather like the business consultant notion of "best practices." There is a real craft to learning how to manage data, and how to write code, and how to present results, and so forth, and all of this is completely independent of the project on which a researcher is engaged. Indeed, it is perfectly plausible that I could take most of the thousands of lines of data-cleaning and analysis code that I've written in the past month for the GSS and the Jennings-Niemi Youth-Parent Socialization Survey, tweak four or five percent of the code to reflect a different DV, and essentially have a new project, ready to go. (Not that it would be a good project, mind you, but going from GRASS to HOMOSEX would not be a big jump.)

There is nothing quite like that for qualitative guys. Game theory folks come close, since you can tweak models indefinitely, but of course they then have to find data against which to test their theories (or not, as the case may be). Neither intepretivists nor case-study folks, however, can automate the production of knowledge to the same extent that quant guys can. And neither of those approaches appear to be as easily taught as quant approaches. The sage of Cambridge (that is to say, Gary Rex) is uncharacteristically almost snide in a passage from an unpublished paper:

A summary of these features of quantitative methods is available by looking at how this information is taught. Across fields and universities, training usually includes sequences of courses, logically taken in order, covering mathematics, mathematical statistics, statistical modeling, data analysis and graphics, measurement, and numerous methods tuned for diverse data problems and aimed at many different inferential targets. The specific sequence of courses differ across universities and fields depending on the mathematical background expected of incoming students, the types of substantive applications, and the depth of what will be taught, but the underlying mathematical, statistical, and inferential framework is remarkably systematic and uniformly accepted. In contrast, research in qualitative methods seems closer to a grab bag of ideas than a coherent disciplinary area. As a measure of this claim, in no political science department of which we are aware are qualitative methods courses taught in a sequence, with one building on, and required by, the next. In our own department, more than a third of the senior faculty have at one time or another taught a class on some aspect of qualitative methods, none with a qualitative course as a required prerequisite.

And yet there is something real here. If every quant scholar has gone from the probability theory --> OLS --> MLE --> {multilevel, hazard, Bayesian, ... } sequence, what is the corresponding path for a "qually"? What could such a path even look like?

Brad DeLong put the problem plainly in his obituary of J.K. Galbraith:

Just what a "Galbraithian" economist would do, however, is not clear. For Galbraith, there is no single market failure, no single serpent in the Eden of perfect competition. He starts from the ground and works up: What are the major forces and institutions in a given economy, and how do they interact? A graduate student cannot be taught to follow in Galbraith's footsteps. The only advice: Be supremely witty. Write very well. Read very widely. And master a terrifying amount of institutional detail.

This is not strictly a qual problem. Something similar happened with Feynman, who left no major students either (although note that this failure is regarded as exceptional). And there are a great many top-rank qualitative professors who have grown their own "trees" of students. But the distinction is that the qualitative apprenticeship model cannot scale, whereas you can easily imagine a very successful large-lecture approach to mastering the fundamental architecture of quant approaches or even a distance-learning class.

This is among the reasons I think that the Qual v Quant battle is being fought on terms that are often poorly chosen, both from the point of view of the qualitative researcher and also from the discipline. Quant researchers will simply be more productive than quant guys, and that differential will continue to widen. (This is a matter of differential rates of growth; qual guys are surely more productive now than they were, and their productivity growth will accelerate as they adopt more computer-driven workflows, as well. But there is no comparison between the way in which computing power increases have affected quallys and the way they have made it possible for even a Dummkopf like me to fit a practically infinite number of logit models in a day.) This makes revisions easier, by the way: a quant guy with domesticated datasets can redo a project in a day (unless his datasets are huge) but the qual guy will have to spend that much time pulling books off the shelves.

The qual-quant battles are fought over the desirability of the balance between the two fields. And yet the more important point has to do with the viability, or perhaps the "sustainability," of qualitative work in a world in which we might reasonably expect quant guys to generate three to five times as many papers in a given year as a qual guy. Over time, we should expect this to lead to first a gradual erosion of quallies' population, followed by a sudden collapse.

I want to make plain that I think this would be a bad thing for political science. The point of the DeLong obituary is that a discipline without Galbraiths is a poorer one, and I think the Galbraiths who have some methods training would be much better than those who simply mastered lots and lots of facts. But a naive interpretation of productivity ratios by university administrators and funding agencies will inevitably lead to qualitative work's extinction within political science.