Characterizing the accuracy of medical tests

[mtbc]

As I laid in bed this morning in between waking up and getting up my thoughts drifted to the use of statistics in managing communicable disease in animal farming, specifically related to deciding to quarantine or cull. Typically we do know some kind of accuracy rate for a test for the disease of concern though a separate issue is the degree to which even the medical community act with enough understanding of how that statistic should affect their decisions.

I would like to imagine that we typically know both false positive and false negative rates for tests. However, I realized that, for the decision models I was considering, I also want to know the degree to which each inaccuracy is due to random error or something else, i.e. how much accuracy to the determination would a retest add? For instance, perhaps a false result is because the individual is a little atypical in its biology (baseline levels of this or that) or has some other condition so one would expect the retest to be as misleading as the first.

I am ignorant of both statistics and farm practice. Despite how much industry profit hinges on even small improvements I wonder if perhaps we mostly don't have empirical data on how much it might help to try a retest using a different batch of equipment or whatever or even to what degree false results from one test for a condition correlate with false results from different kinds of test for it. Maybe the tests are often significantly expensive even when compared with the cost of an animal making retests rarely worthwhile.

Comments

[emperor]

the other key thing is how rare or otherwise the disease you're testing for is - a 1% false positive rate for a disease that only occurs in 0.01% of the population would be entirely useless, for example.

[mtbc]

Aha, thank you, that might be part of what I was groping for with the medical community deciding well -- possibly want to limit interventions while the evidence remains rather weak!

[wpadmirer]

I can tell you from dealing with a lab being set up for blood donor testing that it can be horrendously expensive. The reagents for particular diseases also must be well maintained temperature-wise, and the lab always runs confirmation tests if there is any question about the results.

I would imagine that for animal testing it's the same.

The testing machines these days are incredible and massively sophisticated. If they are calibrated properly and well maintained, false testing is not common.

[mtbc]

That makes sense: the reagents can be a significant cost for our research labs at work too and there's endless fussing about keeping the various freezers and incubators and whatnot at the right temperature. Though, with blood donation the stakes are high indeed: miss one thing and a bad batch could infect a lot of people! I keep meaning to check how they are these days with prions and whatnot.