Research integrity

In the School of Life Sciences where I work we produce systems like the Image Data Resource which is full of strange pretty pictures acquired from expensive microscopes and used to justify scientific conclusions. There is also some initial proof-of-principle code for reproducing analyses via the IDR Jupyter Hub. OMERO.figure is also rather neat: turn the raw image files acquired from the microscope into figures ready for Adobe Illustrator to put into your paper; information in the figures like timepoints, scalebars, etc. is derived from the metadata encoded by the microscope as it acquired the images.

Modern academic life is highly competitive and journals are far more keen to publish interesting new discoveries, however lucky, so there is great career pressure report the right kinds of findings. Further, many of them turn out to be difficult to impossible to reproduce. Even despite this, my impression is that the kind of research misconduct I have in mind is, at its core, well-intentioned: the results may be a little doctored, or an unusually significant subsample, or whatever, but the researcher does generally believe the hypothesis that they are trying to prove, they are just exaggerating the evidence for it.

I figure that our work stuff might be useful if it helps to encourage a culture of sharing all the raw data and the procedures by which it was analyzed. But, I wonder if this papers over a more fundamental problem: that the people generating the hypotheses are also those testing them. I am amused to be thinking of this as a conflict of interest.

I can see why it happens. The people who have the idea are probably the more enthusiastic about testing it. Maybe not many labs are used to working with those cell lines or protocols or whatever at all so it is not like any lab could just pick up the work. And, even if we had a system where the people who generate hypotheses are separate from those who test them, one can see that there is still scope for mutual back-scratching and the like. One can imagine the specifics of the experimental design would be something of a negotiation between the hypothesizer and the tester.

So, I am not saying that even this pipedream idea of having researchers' hypotheses tested by third parties is a good one even if it were workable. But, I do wonder if there is some related but realistic way in which scientific research could be restructured to make it more trustworthy.

Busy people in "The West Wing"

In The West Wing (1999) the senior White House staffers chronically work very long hours. How does this make sense? Is it good for their cognition to be ongoingly compromised? Can there not be enough staff to take on the workload? Perhaps the problem is that they would not be able to leave notes for each other or that they do not have enough room to put all the people or something. It does seem as if there has to be a more effective way for them to work. Maybe we are supposed to believe that they are so superlatively good at their jobs that even half-asleep they outperform some next-best people who might instead be on duty to catch some acute situation.