Archive for the 'numbers' Category

I’ve got your number

Found this on the EMA site, by Rick Bukata

NNT (number to needed to treat) is a great concept for getting some kind of message across to patients. That yes we have medicines to treat you but we cannot guarantee that you will personally benefit. Overall out of 200 or so people someone will benefit. I’m just not sure that’s you.

NNTs are generally lower for treating things that already exist. Eg most people with appendicitis will benefit from having their appendix removed.

NNTs are generally a lot higher when it comes to screening and preventing conditions developing.

He provides a few examples:

Mammographic Screening
How many women between the ages of 50-59 need to have a mammogram yearly for five years to prevent one death from breast cancer?  The number is about 2,500 such women.

Lipid Treatment
71 patients with known cardiovascular risk factors have to be treated with a statin for 3-5 years to prevent one serious adverse cardiovascular event.  But no increase in total mortality or total serious events can be expected.  To put this into perspective, at $1,000 per year per patient for statins, for 71 patients taking them for an average of four years, the cost to prevent one serious adverse cardiovascular event is 71 patients x $1,000 x 4 years = $284,000


Prevention of Colon Cancer Death with Occult Blood Testing
The number needed to screen for five years to prevent one death from colon cancer is 1,374 patients.

Prevention of Hip Fracture by Treatment of Osteoporosis
In women without risk factors, approximately 2,000 women between the ages of 60-64 need to be screened and subsequently treated for osteoporosis for five years to prevent one hip fracture (1,000 women if there is at least one risk factor).

Detection of Diabetes in Men With and Without Hypertension
The number is 13 in 55-year-old men with hypertension, and 19 in those without hypertension.

Simple Antihypertensive Treatment for Mild Hypertension
700 patients would need to be treated for one year to prevent one stroke, MI or death in that year.

Prophylactic Antibiotics for Dog Bites
Only about one in 16 patients will benefit.

Compression Stockings to Prevent Post-Op DVT
One in nine patients may be expected to benefit.

Antibiotics to Improve Short-Term Outcomes in Otitis Media
Only one in seven patients can expect to benefit from antibiotics (i.e., decreased symptoms at 7-14 days post onset of treatment).

Aspirin in Healthy Physicians to Prevent an MI or CV Death
The number is 500 over one year.

All of these numbers are only as good as the data that you’ve calculated it from but still it puts some things in perspective.

The magic numbers

This [Via wee Phil]

Experts in the math of probability and statistics are well aware of these problems and have for decades expressed concern about them in major journals. Over the years, hundreds of published papers have warned that science’s love affair with statistics has spawned countless illegitimate findings. In fact, if you believe what you read in the scientific literature, you shouldn’t believe what you read in the scientific literature.

I have a very dubious and incomplete knowledge of statistics. Unsurprisingly we had modules of it in medical school. Also unsurprisingly I didn’t go to any of the classes. They were were quite staggeringly dull and I had better things to do like read CS Lewis.

These days I spend a lot of time reading medical studies and trying to critique the numbers and methodologies found there in. There is a tremendous amount of Bad Science out there. And don’t be fooled into believing that it’s just the SCAMs (Supplements, Complementary and Alternative Medicine) – there’s plenty of badly done stuff in the professional and published medical literature.

People say statistics can be used to prove anything. People say that 62% of statistics are made up. These things may be true. I just can’t prove it.

Doctors bow before the throne of the randomised clinical trial. Where a bunch of people get one drug and a bunch of people get placebo and no one knows what anyone got till after the trial is over and the scientists go back and look at how people did on either the drug or the placebo.

If more people lived while on drug then they compare the numbers and calculate the probability that the observed positive results of the drug would be due to chance alone (this involves assuming a null hypothesis) and if this number is less than 5% (ie the chance of the observed results being to chance alone is less than 5%) then it is called a significant difference.

[Apparently this idea was birthed by a chap called Fisher in the 1920s who was looking at agricultural yields with fertiliser]

The problem being that statistical significance has little to do with clinical or actual significance.

If you do a big enough trial involving 1000s of people on a certain drug you can prove a stastistically significant benefit from your drug even if the actual difference between the two groups was 3 people out of 10000 dying in the placebo group and 2 people out of 10000 dying in the wonderdrug group (incidentally you could also quote a 50% relative reduction or a 0.01% absolute reduction in death in this trial).

So you see that the numbers appear very different depending on how you quote them.

The 5% number (p<0.05) is an arbitrary number. Let that sink in. Someone just decided that 5% defined statistical significance. In the name of objectivity and scientific rationalism surely that can’t be right!

But these are the tools we have and the tools we use to misunderstand and spin the medical literature.

This is how lots of people build their CVs and careers – you can’t get anywhere without publishing. And this is how lots of people make large amounts of money from new drugs that are basically stereo-isomers or other barely changed molecules from the previous ones.

Worth reading anyhow.


August 2022