mtbc: maze C (black-yellow)
[personal profile] mtbc
In the School of Life Sciences where I work we produce systems like the Image Data Resource which is full of strange pretty pictures acquired from expensive microscopes and used to justify scientific conclusions. There is also some initial proof-of-principle code for reproducing analyses via the IDR Jupyter Hub. OMERO.figure is also rather neat: turn the raw image files acquired from the microscope into figures ready for Adobe Illustrator to put into your paper; information in the figures like timepoints, scalebars, etc. is derived from the metadata encoded by the microscope as it acquired the images.

Modern academic life is highly competitive and journals are far more keen to publish interesting new discoveries, however lucky, so there is great career pressure report the right kinds of findings. Further, many of them turn out to be difficult to impossible to reproduce. Even despite this, my impression is that the kind of research misconduct I have in mind is, at its core, well-intentioned: the results may be a little doctored, or an unusually significant subsample, or whatever, but the researcher does generally believe the hypothesis that they are trying to prove, they are just exaggerating the evidence for it.

I figure that our work stuff might be useful if it helps to encourage a culture of sharing all the raw data and the procedures by which it was analyzed. But, I wonder if this papers over a more fundamental problem: that the people generating the hypotheses are also those testing them. I am amused to be thinking of this as a conflict of interest.

I can see why it happens. The people who have the idea are probably the more enthusiastic about testing it. Maybe not many labs are used to working with those cell lines or protocols or whatever at all so it is not like any lab could just pick up the work. And, even if we had a system where the people who generate hypotheses are separate from those who test them, one can see that there is still scope for mutual back-scratching and the like. One can imagine the specifics of the experimental design would be something of a negotiation between the hypothesizer and the tester.

So, I am not saying that even this pipedream idea of having researchers' hypotheses tested by third parties is a good one even if it were workable. But, I do wonder if there is some related but realistic way in which scientific research could be restructured to make it more trustworthy.

Date: 2017-06-21 10:14 pm (UTC)
emperor: (Default)
From: [personal profile] emperor
I heard from someone who worked in a mouse facility, who said they observed more interesting phenotypes than they could work up, so they would sometimes call up researches in a relevant area and see if they were interested in investigating a particular genotype/phenotype. I imagine there there is less "I must find something publishable out of this" pressure there, but perhaps that's too optimistic...


mtbc: photograph of me (Default)
Mark T. B. Carroll

September 2017

      1 2
3 4 5 6 78 9
10 1112 13 14 15 16
171819 20212223

Most Popular Tags

Page Summary

Style Credit

Expand Cut Tags

No cut tags
Page generated Sep. 21st, 2017 10:28 am
Powered by Dreamwidth Studios