The Shrine

After some random internet thread where guy claims I misused statistics (even though the computation involved no advanced statistical methods whatsoever) I started thinking what good is statistics anyway. I mean of course statistics has its use, but when's the last time you have seen a convincing argument based on standard deviation or chi-square or any of that stuff? If on average my Internet power level is 10 and yours is 3 does it really matter there's some weird statistics formula that may prove it otherwise? If I tossed a coin 20 times and it came up heads 16 times, do you really need statistics to suggest the coin is probably rigged?

Even in pretty technical papers, usually I notice people just use simple methods if that method supports whatever they're trying to claim. For example the global warming guys say temperature will raise on average X +- Y degrees in the next Z years. Assuming X is big enough, Y is small enough, and Z is soon enough, that certainly can tell me 'bad things are going to happen soon'. If my algorithm on average is 3 times as fast as yours I probably don't need anything else to show that I am better than you. Sure everything is not so clear cut so you need fancier methods when the differences are small, but I'm always of the opinion that if simple methods (assuming it doesn't omit glaring issues) shows what you need to show then you don't need advanced methods.

Another thing I noticed is that people confuse sample size with accurate statistical methods. Let's say I did the aforementioned 20 coin tosses and ended up with 16 heads, and assume those tosses were fair tosses as opposed to say me dropping the coin straight down with heads facing up 20 times. A lot of people will say 'you only did this 20 times! That's way too small!' Yet take a fair coin and try to toss it 20 times and get 16 heads, and I bet you'll be needing a lot of time (I estimate the chance is around 1 in 1000 using some simplifications). If I win 3 Mega Millions lotteries in a row, are people going to say well he only won 3 times and that's too small of a sample size? No people will be quite correct to assume there might be something weird going on.

Don wrote:If I tossed a coin 20 times and it came up heads 16 times, do you really need statistics to suggest the coin is probably rigged?

Probability theory tells you the coin is probably rigged, not statistics. And in that case, no our intuition serves us well. But our intuition is really, really bad when it comes to most questions related to probability. We just did not evolve to be able to make snap judgements related to probabilities. Hence, we need formal methods to help us make rational decisions.

Well I'm sure statistical methods will tell you how likely the coin is actually rigged if you get 16 heads out of 20 tosses, but it shouldn't take statistical methods to suggest that the coin is rigged. I can also just hand you the coin and you can try it yourself and assuming the coin is indeed rigged toward head showing up 80% of the time, it should not be very hard to see that anybody throwing the coin 20 times is likely to get an abrnomally high number of heads.

I mean these problems aren't exactly ones where intuition goes against what is actually true. Intuitively the event I described should be pretty rare and it is if you sat down and compute the odds. We're not exactly talking about the birthday paradox here.

The probability of the outcome of tossing a fair coin is probability theory, not statistics. You use statistics to summarize your results, but you must compare your statistical quantities to the probabilities associated with the events. I'm surprised how often even people with a good math background confuse the two, but I guess I did, too:

http://en.wikipedia.org/wiki/Probability
http://en.wikipedia.org/wiki/Statistics

Don wrote:I mean these problems aren't exactly ones where intuition goes against what is actually true. Intuitively the event I described should be pretty rare and it is if you sat down and compute the odds. We're not exactly talking about the birthday paradox here.

I can travel to campus by walking. This fact does not negate the need for airplanes.