Personal reputation in scientific publication

In Toward Author-Centric Science, Daniel Lemire, one of my favourite curmudgeons, amusingly describes the modern academic as treating publication as a kind of Turing test, where the challenge is to mimic all the characteristics of a worthwhile scientific paper without actually producing one. As an antidote, he proposes that we deprecate the broken peer-review system as a guarantor of value, and instead rely upon the reputation and popularity of the individual researcher. If scientists cared more about the number of people subscribing to their RSS feed, Lamire argues, and less about the number of articles they had published that year, they would have more of an incentive to produce interesting work.

I'm intrigued but not convinced. A concern for reputation amongst our peers is a useful counterbalance to the Stakhanovist incentives of modern academic life. And certainly the blog is a better outlet for the "look at me!" aspect of research writing than the formal publication is. However, being interesting is not the same as being worthwhile. Danny Calegari's blog has three times as many Google Reader subscribers as mine does, but does this mean his scientific contribution is three times as great as mine? Well, ok, in this case, yes it does. But in the general case, popularity is also Turing-test-passable. And indeed, as Socrates would remind us if he were still blogging, in the case of popularity and its kissing-cousin, interestingness, it is not clear what the underlying true expertise that is being mimiced, is.

That said, I agree with Lamire Lemire that the interesting chases out the formulaic, and in much computer science writing and research this would be no bad thing. Real science is formulaic because its standards are well-established and proven; diverging from the norms of method and exposition is almost invariably a sign of unsoundness; sticking to them, the surest way of contributing to science's steady progress. But while parts of computer science count as maths, very little of computer science qualifies as real science. One does not formulaically accumulate data about search algorithms, for instance, like one does about genes, stars, or pharmaceutical compounds; there is no MeSH for agent-oriented architectures; there are no objective natural phenomena to describe, explain, and predict in data compression. The affectation of being an enterprise in paradigmatic science that fields of computer science assume from time to time is more often than not a mere affectation; and following the formulae of the affected paradigm, while a good way to get past peer reviewers, is as much an obstacle to as an assurance of the value of your contribution.

5 Responses to “Personal reputation in scientific publication”

  1. Good points. Thank you.

    I'm not exactly sure though why making your research papers available through RSS or email notifications would, somehow negatively affect our work. You can subscribe to my research papers by a feed

    or by email

    Of course, nobody (but me) is subscribing to my research papers. But I would subscribe to your research paper feed. And maybe if 10 or 30 or 200 people did, you'd find value in having this implicit network of readers who like to receive notifications whenever you produce new work. I'd find value in being able to subscribe to your paper, that's for sure.

    And if more people did it, I think we would have an incentive to be more interesting, and still remain as significant as ever. You don't need to be boring to be do important work. The Theoretical Computer Science blogs have large readerships (more than my "softer" blog). So, the proof is there: interestingness does not kill significance.

    BTW I am subscribed to several arxiv feeds. It is not very valuable, but I like to receive a list of the latest preprints... it gives me a warm feeling.

    Note that you mispell my last name (but that's ok).

  2. william says:

    I agree with your general point that be aware and trying to cultivate a following for your publications would provide an incentive to maintain a high quality for them. I think though that publication feeds are not the most natural way of doing this, at least on an author-by-author basis, rather than through an aggregator. The natural mode for gathering a following and making others aware of your work is the one we're engaged in now -- blogging; although "a following" is not really the right word here, "a community" seems more appropriate.

    Blogs maintain a conversation between publications; give us an opportunity to try out new ideas, not so much research ideas (I agree with your post a little while back that research ideas need a protected space to develop in), but broader issues of understanding of the field; and, of course, provide a forum for discussing work once it has been published.

    I announce my publications (or, more precisely, my pre-prints) as they become available on my blog -- in fact, I have a CIKM pre-print that I'm waiting to find time to announce. I follow your blog (and Gene's, and Jeremy's, and Jeff's, and Panos's, and ...) and will certainly notice any publication notices you put up there.

    And sorry for the name mispelling; corrected.

  3. I shudder at the suggestion that there is a correlation between the value of my scientific work and the popularity of my blog. To get an idea of how small (or indeed negative) the correlation coefficient might be in certain cases, please note that the climate denialist website (scientific value < 0) won an award for "best science blog".

  4. william says:

    That could be your new by-line. "Danny Calegari -- almost as widely respected as a climate-change denialist".

    (BTW, I know it was a slip, but I like the idea of a "climate denialist"; a really no-compromise rejectionist of this whole "climate" nonsense.)

  5. Er, yes: climate-change denialist. (I look forward to Google correlating my name positively with "climate-change denialist" now . . . )

Leave a Reply