Don't Hate the Player, Hate the Game
A new investigative report by journalists at Science and Retraction Watch lifts the lid on what appears to be a concerted effort by one university to game the current science publishing system.
The story focuses on Saveetha Dental College in Chennai, India, which requires hundreds of undergraduates to write manuscripts on research they’ve done as students. That sounds like a fine exercise for the class. The twist is that these student manuscripts then get published in bottom-of-the-barrel journals. While that alone generated over a thousand publications from the college last year, the real impact of this scheme comes from the students’ apparent habit - likely prompted - of including citations to large numbers of other studies published by researchers at the same school.
It’s hard to come up with a pedagogical justification for that, but very easy to see how it benefits the college; citation counts have become a standard tool for rating the quality of research at institutions worldwide.
Image courtesy Wikipedia.
While there are several approaches to quantifying academic productivity, most work in the same general way as the H-index, which weights the number of publications from an individual or organization by the number of times other researchers have cited them. That’s supposed to guard against the most obvious shortcut: shoveling huge numbers of low-quality papers into bottom-tier publications. Presumably, such worthless output won’t be cited by others.
By taking the strategy a step further and turning their undergraduate program into a citation factory, though, Saveetha has catapulted itself onto lists of India’s top dental schools.
Frederik Joelving’s well-written Science article details this scheme, and includes perspectives from understandably upset Indian academics, as well as some non-denials from university officials. It’s a straightforward story on the surface.
The deeper problem is that this is just the natural outcome of a brutal quagmire of perverse incentives. While Saveetha has apparently mined bogus citations on a larger scale than others, it’s far from the only offender, and the only reason this scheme works in the first place is that the current academic publishing system practically demands it.
Trying to quantify scientific accomplishment is a fool’s errand. The measure of science isn’t how many papers it generates, or even how many other people cite those papers within the entirely artificial world of academic publishing. What really matters is whether we’re making important discoveries, and there’s simply no way to put a number on that.
By nonetheless insisting on “metrics,” academic administrators and government agencies have created an inherently corruptible system. If you reward people based on a number, they’ll try to maximize that number however they can. Saveetha Dental College seems to have found a way to raise their score. Instead of clutching our pearls and accusing them of cheating, maybe we should ask why we’re playing this game.