How to Judge Scientists

Well, my 300th scientific article was just accepted for publication (N. Smolin, R. Biehl, G.R. Kneller, D. Richter and J.C. Smith "Functional Domain Motions in Proteins on the ~1-100ns Timescale: Comparison of Neutron Spin Echo Spectroscopy of Phosphoglycerate Kinase with Molecular Dynamics Simulation, Biophysical Journal - good job, Nikolai!) and there will be a few beers in the Union Jack pub later on in the week. However, this kind of artificial milestone brings one to reflect on how really to judge scientists.

Clearly, although a large number of publications does point to some aspect of productivity, such as, possibly, getting involved in a lot of projects and helping bring them to fruition, it is a very one-dimensional metric and misses important elements of scientific life. Numbers of citations, h-factors and the like also have their problems (just as an anecdote, for example, a very famous physicist working at Saclay when I was there once said one of his most cited articles was one he got wrong - his rivals loved pointing this out in their own publications!).

So how can one judge scientists? Well, increasingly, discoveries result from the voluntary sharing and development of knowledge through collaboration, rather than individual discoveries, and so an intriguing recent article by Azoulay et al tries to quantitatively track effects on collaborations of the ideas that scientists create. The concept is that a scientist will influence the people with whom they work, by forming an "invisible college" of ideas. To quantify this influence they tracked the publication productivity of faculty-level collaborators of eminent scientists in the life sciences. They found that if an eminent scientist suddenly and tragically died before the end of their career (mostly of heart attacks, but in the sample studied three were actually murdered!) then the publication productivity of their collaborators subsequently irreversibly declined on average by 8%.

The authors concluded that the effects of, as they call it, "superstar extinction" appear to be driven by the loss of an irreplaceable source of scientific ideas. My own opinion is that while this may indeed account for some of their observed effect, the collaborative nature of science means that success depends on not only the exchange of scientific ideas, but also inevitably social aspects such as friendship, motivation, drive and team spirit. When sources of these are not replaced then productivity will decrease.