Lies, Damn Lies, and Statistics

I just read an interesting article on Slate regarding the use of statistics in medicine.

I took a class last semester in which we read a number of papers from medical journals instead of psychology journals like we usually do. The different ways in which statistics get used in the two fields was really striking.

In psychology, we like combinations of p-values and effect sizes. It gets drilled into our heads a lot that statistically significant does not necessarily mean practically significant. If I have a sample of 100 students and every single one of them does two points better on their final exam thanks to my intervention, that’s probably a statistically significant result, but it’s definitely not a practically significant result.

In medicine, it’s all about risk ratios, which I found terribly messy to interpret. And some of the results looked awfully sketchy from a more psychological point of view. In psychology, we avoid turning continuous variables—like age—into discrete variables because it violates certain statistical rules. In the medical papers we read, they did that all the time, and often created arbitrarily-sized age groups. In psychology, we call that “fishing for a result”.

What does it mean, practically speaking, that men aged 37-47 who got Treatment X had 2.3 times lower risk of complications than men aged 37-47 who got placebo, while men aged 48-52 who got Treatment X only had 1.1 times lower risk of complications than men aged 47-52 who got placebo? I haven’t the faintest.

| Subscribe to Feed

Yvonne posted this on September 27th, 2006 @ 12:46pm in Graduate School, Psychology/Neuroscience, Science | Permalink to "Lies, Damn Lies, and Statistics"