In the School of Life Sciences where I work we produce systems like the Image Data Resource
which is full of strange pretty pictures acquired from expensive microscopes and used to justify scientific conclusions. There is also some initial proof-of-principle code for reproducing analyses
via the IDR Jupyter Hub. OMERO.figure
is also rather neat: turn the raw image files acquired from the microscope into figures ready for Adobe Illustrator to put into your paper; information in the figures like timepoints, scalebars, etc. is derived from the metadata encoded by the microscope as it acquired the images.
Modern academic life is highly competitive and journals are far more keen to publish interesting new discoveries, however lucky, so there is great career pressure report the right kinds of findings. Further, many of them turn out to be difficult to impossible to reproduce. Even despite this, my impression is that the kind of research misconduct I have in mind is, at its core, well-intentioned: the results may be a little doctored, or an unusually significant subsample, or whatever, but the researcher does generally believe the hypothesis that they are trying to prove, they are
exaggerating the evidence for it.
I figure that our work stuff might be useful if it helps to encourage a culture of sharing all the raw data and the procedures by which it was analyzed. But, I wonder if this papers over a more fundamental problem: that the people generating the hypotheses are also those testing them. I am amused to be thinking of this as a conflict of interest.
I can see why it happens. The people who have the idea are probably the more enthusiastic about testing it. Maybe not many labs are used to working with those cell lines or protocols or whatever at all so it is not like any lab could just pick up the work. And, even if we had a system where the people who generate hypotheses are separate from those who test them, one can see that there is still scope for mutual back-scratching and the like. One can imagine the specifics of the experimental design would be something of a negotiation between the hypothesizer and the tester.
So, I am not saying that even this pipedream idea of having researchers' hypotheses tested by third parties is a good one even if it were workable. But, I do wonder if there is some related but realistic way in which scientific research could be restructured to make it more trustworthy.