Thursday, July 30, 2009

Alain de Botton examines our ideas of success and failure

reposted from:
Alain de Botton examines our ideas of success and failure -- and questions the assumptions underlying these two judgments. Is success always earned? Is failure? He makes an eloquent, witty case to move beyond snobbery to find true pleasure in our work.


Monday, July 20, 2009

False Positives

Did Rocky our cat have a false positive result for feline HIV? Rocky was diagnosed with feline HIV a few years ago but has had no symptoms. What was the accuracy of Rocky's HIV test?

reposted from: http://news.bbc.co.uk/go/rss/-/1/hi/magazine/8153539.stm



Michael Blastland
GO FIGURE
Different ways of seeing stats
If a screening test is 90% accurate, and your result comes back positive, what are the chances it is a false positive, asks Michael Blastland in his regular column.
Browsing the web recently, I found a fascinating article about screening for terrorists and it's made me think about accuracy and uncertainty.
Imagine you've invented a machine to detect terrorists. It's good, about 90% accurate. You sit back with pride and think of the terrorists trembling.
Conventional lie-detector or polygraph accuracy has been claimed to be 90% but this is doubtful. Most independent experts think it's more like 60% - not much better than tossing a coin.
But your invention is the real deal, it really is 90% accurate. It's quick, light, portable and works by detecting patterns of brain activity and facial movement known to match terrorist intent.
You're in the Houses of Parliament demonstrating the device to MPs when you receive urgent information from MI5 that a potential attacker is in the building. Security teams seal every exit and all 3,000 people inside are rounded up to be tested.
The first 30 pass. Then, dramatically, a man in a mac fails. Police pounce, guns point.
How sure are you that this person is a terrorist?
A. 90%
B. 10%
C. 0.3%
The answer is C, about 0.3%.
 Think about screening all the non-terrorists for innocence - and being wrong about 10 people in every 100 
If 3,000 people are tested, and the test is 90% accurate, it is also 10% wrong. So it will probably identify 301 terrorists - about 300 by mistake and 1 correctly. You won't know from the test which is the real terrorist. So the chance that our man in the mac is the real thing is 1 in 301.
That a good test can leave us so uncertain about any individual is a head-spinner to many. The problem is the false positives: tests that say you've found what you are looking for but are wrong, and which wreak particular havoc with the results when what you are looking for is rare. That means most of your mistakes apply to those you are not looking for.
Go Figure has been puzzling over how to make all this more intuitive and invites readers to send their own ideas, using the form at the bottom of this page.
Here are a couple of suggestions.
The first is to visualise the numbers. In the picture below, four pixels = 10,000 people. The whole area is the population of the United States - about 300m people. The dark blue area is roughly how many would be suspected of terrorism by a screening process with 90% accuracy - about 30m. On this scale, the area representing the number who are real terrorists - let's say 300 people, of whom 30 would be missed - is too small to see on screen so we've blown up one pixel to show the proportion.
Terror screening

The second suggestion is that whenever we discuss screening, be it of terrorists, HIV, cancer or anything else, we should try to refocus. Any mention of screening for terrorists causes all our attention immediately to zoom into those who really are terrorists. We think of the individuals and how a 90% accurate test would work on one of them. We zoom into the white area and forget the blue.
Refocus. Get into the habit of also thinking about screening the light blue area too.
How would this work in practice? Whenever we hear what's being screened for, we should switch it around to think about the opposite. So, screening for terrorists with 90% accuracy? Think about screening all the non-terrorists for innocence - and being wrong about 10 people in every 100. Imagine them all, virtually the whole population, 10% of whom might become suspects.
Screening for HIV with 99.9% accuracy? Switch it around. Think also about screening the millions of non-HIV people and being wrong about one person in every 1,000.
For another, visually-captivating method, try the brilliant animations on the Understanding Uncertainty website - read the first page then click on "testing" - which encourages us to think of real people rather than percentages.

• "We don't know."
It has become a refrain, the answer to almost every question. I'm discussing swine flu with someone who is looking at how it's spreading. Based at one of our leading medical institutions, highly experienced and capable, this is someone who might have been expected to know quite a lot.
"We don't know."
SWINE FLU SYMPTOMS
Swine flu leaflet with tissues, thermometer and anti-bacterial gel
If you have a temperature and two or more of the following, it may be swine flu:
Cough
Sore throat
Body aches
Chills
Fatigue
I'm not asking for clairvoyance, this is not about what will happen in future. All I want to know - all this researcher wants to know but can't find out - is simple stuff: how many people who had swine flu in the first few months since it emerged are believed to have caught it abroad? What proportion of people with reported swine flu have been hospitalised? Do they tend to be younger or older? How soon after the first symptoms do they start antiviral treatment?
Someone, somewhere might have a slightly better idea of some of this, but my academic friend, whose job it is to try to understand the illness, is exasperated by the difficulty of finding out the basics.
Statistical models of the spread of disease are never perfect, but they can help. If they are to be remotely useful, they need some reasonable numbers to start with. Otherwise, as the old adage has it: rubbish in, rubbish out.
Although there's a rough total of reported cases, we don't know how many there have really been, and how often these are serious, because we have little idea how many people have sub-clinical symptoms. Little idea how many treat themselves without reference to the health service. Little idea what proportion of the total finish up in hospital. Little idea how accurate the diagnoses are now that diagnosis is no longer confirmed with a blood test.
The numbers you see quoted in the media are bound to be crude. How crude, we don't know.
As so often with data, it is the simple business of counting things and keeping consistent, accurate records that turns out to be where the glitches occur. That's just a lot harder than it seems. Not only do we not know where we are going to be with swine flu in a few months time, we don't really know where we are.
But there's also a perverse comfort in some of this. If there is a huge amount of mild swine flu we don't know about, the proportion of cases that are serious is correspondingly reduced.

Add your comments on this story, using the form below.
The way I like to explain false positives to people is by what effect it would have on them - for 90% accuracy it means that everyone could have someone in their immediate family (parents, children, siblings) labelled as a terrorist. Make it personal and people start to really comprehend the problem.
Peter Clarke, Auckland, NZ
You could try the visualisation method from slide five of this presentation which deals with forecasting extreme rain events. Of course, if your event is rare enough, the most accurate forecast available will simply be to say "no" every time (e.g. no-one is a terrorist). More usefully, your forecasting method should be as accurate as the probability of NOT finding the event (e.g. 90% accurate is fine for finding something which has a probability of 10%).Candy Spillard, York, UK
Ninety per cent accuracy means that 90 times in 100, the machine will be correct ABOUT ANY ONE PERSON, 10 times in 100 it will be incorrect.
Jelani Crue, UK
You discuss the problems around using a detection system with a 90% accuracy. It should be noted that these problems diminish rapidly if you have a second independent system with a similar level of accuracy, and obviously concentrate on the double positives.
Lawson G, Taunton
Of course in reality you let the MPs, staff, families etc. go home, then just test the people that are left (the guys in the macs). Just as you don't subject random members of the public to lie detector tests, nor do OAP nuns seek HIV tests.
Finn, London, UK
The very term 90% accurate is the sort of thing that might get trumpeted in a media headline but is actually meaningless. Does this mean it catches 9 out of 10 terrorists? Or does it mean it catches 1 out of 10 innocent people? You have assumed the latter to make your point, but in real world situations the chance of false positives is not the same as the chance of false negatives. In fact great effort goes into minimising one and maximising the other. Didn't you make this point in a very early column, in a medical context? It should be made again here.
Ian Nartowicz, Stockport, England
Tests used in medicine often have a specificity and sensitivity rather than just "accuracy"; I think on the whole this is more useful, and reflects the trade-offs inherent in many such tests between being too sensitive (finding all your terrorists, but picking up a lot of innocent people) and too specific (only finding terrorists, but missing a few of them).
Matthew, Coventry, UK
The concept of false positive and false negative (and true positives and negatives) results from any test is hugely important and needs to be better explained to the general public. Only then can discussions about e.g. breast cancer screening or any mass testing process can be properly understood.
DavidF, Watford, England

Tuesday, July 14, 2009

Cats 'exploit' humans by purring

Cats 'exploit' humans by purring

By Victoria Gill
Science reporter, BBC News


See Pepo the cat employing his 'soliciting purr'

Cat owners may have suspected as much, but it seems our feline friends have found a way to manipulate us humans.
Researchers at the University of Sussex have discovered that cats use a "soliciting purr" to overpower their owners and garner attention and food.
Unlike regular purring, this sound incorporates a "cry", with a similar frequency to a human baby's.
The team said cats have "tapped into" a human bias - producing a sound that humans find very difficult to ignore.
Dr Karen McComb, the lead author of the study that was published in the journal Current Biology, said the research was inspired by her own cat, Pepo.
"He would wake me up in the morning with this insistent purr that was really rather annoying," Dr McComb told BBC News.

Impossible to resist: cats use sounds that humans are "highly sensitive" to
"After a little bit of investigation, I discovered that there are other cat owners who are similarly bombarded early in the morning."
While meowing might get a cat expelled from the bedroom, Dr McComb said that this pestering purr often convinced beleaguered pet lovers to get up and fill their cat's bowl.
To find out why, her team had to train cat owners to make recordings of their own cats' vocal tactics - recording both their "soliciting purrs" and regular, "non-soliciting" purrs.
"When we played the recordings to human volunteers, even those people with no experience of cats found the soliciting purrs more urgent and less pleasant," said Dr McComb.
How annoying?
She and her team also asked the volunteers to rate the different purrs - giving them a score based on how urgent and pleasant they perceived them to be.
"We could then relate the scores back to the specific purrs," explained Dr McComb. "The key thing (that made the purrs more unpleasant and difficult to ignore) was the relative level of this embedded high-frequency sound."


They learn how to do this, and then they do it quite deliberately
Karen McComb
University of Sussex
"When an animal vocalises, the vocal folds (or cords) held across the stream of air snap shut at a particular frequency," explained Dr McComb. The perceived pitch of that sound depends on the size, length and tension of the vocal folds.
"But cats are able to produce a low frequency purr by activating the muscles of their vocal folds - stimulating them to vibrate," explained Dr McComb.
Since each of these sounds is produced by a different mechanism, cats are able to embed a high-pitched cry in an otherwise relaxing purr.
"How urgent and unpleasant the purr is seems to depend on how much energy the cat puts into producing that cry," said Dr McComb.
Previous studies have found similarities between a domestic cat's cry and the cry of a human baby - a sound that humans are highly sensitive to.
Dr McComb said that the cry occurs at a low level in cats' normal purring. "But we think that (they) learn to dramatically exaggerate it when it proves effective in generating a response from humans."
She added that the trait seemed to most often develop in cats that have a one-on-one relationship with their owners.
"Obviously we don't know what's going on inside their minds," said Dr McComb. "But they learn how to do this, and then they do it quite deliberately."
So how does Dr McComb feel about Pepo now she knows he has been manipulating her all these years?
"He's been the inspiration for this whole study, so I'll forgive him - credit where credit's due."