Incentives

From time to time I worry about the way that management by performance indicator can push those managed into playing the performance indicators, can result in the indicators being more important than the performance that they are trying to capture. A matter last touched on at the end of reference 1.

Which led to reference 2, which led to reference 3, with some background coming from references 4 and 5. A dark tale of the impact of performance indicators on university research, with one result being that fossil hunters can go too far in their hunts. With particular reference to the fossils to be found in Kachin amber from the Hukawng Valley in northern Myanmar. All of this material, while not exactly open access, is to be found on the Internet.

The short version seems to be that the citation statistics all too often used to measure research performance are not up to the job and lead to an unhealthy preoccupation with getting published in a small number of prestige journals such as Nature and Science. That these prestige journals are too keen on flashy and newsworthy results, leading on the one hand to the neglect of work which is worthy but neither flashy nor newsworthy and on the other to behaviour unbecoming to a university research department. Including the theft of fossils from their countries of origin, most of which have strict rules about such matters.

The long version

The first part of reference 2 is about the use of rather crude indicators to measure the performance of academics. Indicators which were easy to calculate and which save hard pressed adjudicators for academic plums much of the bother of actually looking at the work of those they were adjudicating. Life is too short. Must get on.

One of these indicators is the journal impact factor (JIF) which counts up the mean number of times that things published in the subject journal get cited. With Nature and Science being way out in front, as can be seen by the left hand side of the figure above. You then use this factor to weight recent publications of the person being adjudicated - effectively subcontracting that adjudication to the editors of the prestige academic journals. 

Another indicator is the h-index of reference 6, computed for an individual rather than a publication, which attempts to reward quantity as well as quality. A bold statistician might attempt to map h-score onto academic rank, membership of the Royal Society or even Nobel prizes.

It is alleged that high prestige journals have to retract more papers given the temptation to cheat to get to the top of the heap.

There is also the problem of journals citing themselves, prompting allegations of gaming the JIF system, certainly an issue with the journal at the top of the table snipped above. Also with the taxonomic journal 'Zootaxa' to be found at reference 8.

As far as palaeontology is concerned it is dinosaurs which are sexy, that is to say publishable in high prestige journals, and bits of small dinosaurs preserved in Burmese amber particularly so. This kind of stuff even leaks out into the Twittersphere, or at least it did before Musk got his paws on it.

Burmese amber is a particularly tricky one given the unpleasant military regime presently in power, busily commercialising the amber mines in Kachin State, with some paleontologists going so far as to want to ban the publication of work on Burmese amber acquired after the middle of 2017. Another complication being the value of jade as an antique or a gemstone: most fossils are just fossils. While the authorities in Mexico, Brazil, Mongolia and China are getting much stricter about the export of fossils and are likely to insist on local participation in the work.

The paper with a plea to tone down the culture of publish or perish which pervades academia. A culture which some young academics choose to put aside in favour of more informal channels of communication - but quite possibly putting their career progression on hold thereby. In which connection I might also point to the tendency of management of some tertiary institutions - of which I can claim some secondhand knowledge - to ignore other contributions to academic life, such as a commitment to teaching, to the support of societies and activities related to one's discipline and pastoral care. Or even just a commitment to the collegiate life.

A digression

A related phenomenon is the way in which it is easy to get seduced by the trappings of something, rather than the substance. T. E. Lawrence, for example, wrote of all the comforting paraphernalia of a brigade headquarters, very soothing to a middle ranking general. Or the full Treasury trappings of a minute or memorandum, possibly now swept away in the tidal wave of email. Or the suburban hostess of a hundred years ago caught up in a welter of elaborate (dining) table furniture and furnishings.

From which it is but a short jump to ritual in general. With another example from last summer to be found at reference 9.

Conclusions

I suppose that it is this sort of thing that drives the monster lists of references which one gets at the end of today's academic papers. You scratch my back, and I'll scratch yours. While I recall, on several occasions, being surprised by old papers by eminences from the past which manage with footnotes and just a few references at the end. With the paper at reference 7, read before the great and the good of the Royal Society, having few footnotes and no references at all.

Glad I am not a middle ranking academic! Not the comfortable life it was until, say, the mid 1970's. And it was not well paid, even then, with, job-for-job, academics getting significantly less than civil servants. Although, I speak as someone once in the statistical service which was aligned with the administrators and which did rather better for itself than the often more highly qualified scientists. Lots of whom were to be found, for example, in what was then the Ministry of Agriculture, Fisheries and Food (Ag & Fish or MAFF to its friends) or the Ministry of Defence.

PS: I thought it perhaps time to take another peek at the film about academic life noticed at reference 10. But I doubt whether ITVX will plug the hole left by the absence of both DVD and DVD player.

References

Reference 1: https://psmv5.blogspot.com/2023/02/in-praise-of-classifications.html.

Reference 2: Pressure to publish is ‘fuelling illegal practices in palaeontology’: More safeguards and stronger journal policies are needed to curb the problem, say authors of analysis on publication trends - Clare Watson, Nature - 2022.

Reference 3: Publication pressure threatens the integrity of palaeontological research - Raja, Nussaïbah B., Dunne, Emma M. - 2022.

Reference 4: A simple proposal for the publication of journal citation distributions - Vincent Larivière, Véronique Kiermer, Catriona J. MacCallum, Marcia McNutt, Mark Patterson, Bernd Pulverer, Sowmya Swaminathan, Stuart Taylor, Stephen Curry - 2016. 6 pages of paper, 16 pages of stuff at the end. A bonus being the new-to-me word 'bibliometrician'.

Reference 5: https://en.wikipedia.org/wiki/Burmese_amber.

Reference 6: https://en.wikipedia.org/wiki/H-index.

Reference 7: On governors - J. Clark Maxwell - 1867.

Reference 8: https://www.biotaxa.org/Zootaxa/index.

Reference 9: https://psmv5.blogspot.com/2022/07/mourning-form-and-content.html.

Reference 10: https://psmv2.blogspot.com/2014/01/film-week.html.