SportsInsights.com Article:

Sports Investing and Statistical Significance

Financial traders often talk about last year’s returns – or results over the past three or five years. Investors review long-term stock and mutual fund performance. Investing in sports is a different animal – but can be approached in a similar manner. In fact, the sports world lends itself to statistical analysis in many ways. For starters, there are tons of stats in sports. More importantly, for the purposes of our research and article: sporting events can be broken down into independent events that we bet (invest!) on. At SportsInsights.com, we try to better quantify results and study various methods that can help serious sports investors improve their results.

We often hear about systems that are hot. “This system has been 8-1 since last week!” Does this mean anything? What if a system hit 56% over 100 games? Sounds decent – but is this “statistically significant?” And what does that phrase mean, anyway?

In this article, we want to cut through some of the cloudiness that surrounds mathematics and statistics and give our readers some guidelines to help evaluate systems. We also want to share some insight (no pun intended) on some of the tools and research that we use. The information on this site is for entertainment and educational purposes only. Use of this information in violation of any federal, state, or local laws is prohibited.

What does “Statistical Significance” Mean?
To “normal” people, “significant” means important. To statisticians, however, “significant” means “probably true.” Math people like to quantify things – and “statistically significant” is no different. If something is “statistically significant” at the 95% level, it means that there is a 95% probability that some hypothesis is true. Note that this STILL means that there is a 5% chance that this is false. We’ll use the 95% level as our definition of “statistically significant” for this article.

Applications to Sports Investing
Now, how can we use this? Statistics and math can help us determine if an approach is any good. They can tell us how long we need to study a system. They can also give us some guidelines on how long we might stick with a system.

Depending on the “vig” we pay, we need to win around 51% to 52.5% of our bets. Let’s say we want to test how viable a system is at the 55% winning percentage level. We’d like to win even more, but for the purposes of this article, we’ll use 55% as the threshold that we are testing. Moneylines are a different category, but the thinking is similar.

Proving Statistical Significance
Now, let’s cut to the chase and see what “statistics” can tell us. There’s a lot of “mumbo-jumbo textbook stuff” but let’s get “down and dirty” and see if we can get a better understanding of statistical significance. In life – and in many problems – it pays to “frame” the answer so you can see if your answer is reasonable. Let’s do that with some thoughts on imaginary sports betting systems.

• Let’s say that a system is producing better than 57% over a million games. We’d agree that was pretty good. Is that statistically significant? Yes, it is.
• What if, instead of a million games, this 57% was based on 100,000 games? Same thing: pretty good results – AND statistically significant.
• What does math tell us? It says that if a system is producing a better than 57% winning percentage, the cutoff is around 2,000 games to prove statistical significance (that the results will beat the 55% winning percentage we chose above).

That is, if a system produces a 57% winning percentage over 2,000 games, mathematicians say that there is a 95% chance that the results are true (results will be better than 55% in the long-run). Please see Graph 1 for a plot of “Winning Percentage versus Sample Size.” Below 2,000 games, the results are good, but statisticians wouldn’t say that results are “significant” enough.

Graph 1:  Statistical Significance (95% Level) -- 55% Winning Percentage
Winning Percentage to prove “Statistical Significance” versus Sample Size

Some Notes and Guidelines

• Note that there do NOT have to be hard and fast rules about statistics. Some mathematicians label results as “mildly” significant or “highly” statistically significant. Let’s just say that for us to consider a system, it should average greater than 57% or some other “hurdle-rate.” If the sample size is greater than 2,000, super! (If the sample size were a million games, just over 55% would be good enough!)
• From Graph 1, we can see that at a sample size of 20, you would need to hit around 80% to prove statistical significance. If a decent system is connecting at 67%, it doesn’t mean that it’s “no good.” It just means that there is too much randomness in the small sample size and that the system should be tested over more games (a longer time period or larger sample size). Don’t throw it out! Just give it time and watch how it performs in the long-term.
• At the 200-game sample size, you would need a winning percentage in the low 60% range to prove statistical significance. Again: you should use your judgment and consider variables such as luck (slow start for a system) and the long-term average.
• Over time, we know that various systems and approaches will have ups and downs. “System A has gone 7-2 since I tracked it.” Based on Graph 1, a sample size as small as 50-100 can start to tell us a story (10-20 is too small a sample size, unless results are extraordinary) – but 200-500 is even better.

SportsInsights.com
At SportsInsights.com, an important part of what we do is: maintain a clean database of the sports marketplace. We then analyze the data and try to help our members profit from the sports markets, just like investors profit from the stock market.

How does SportsInsights.com use “statistical significance?” Results for “Betting Against the Public” are fairly consistent across the major sports. Favorable results for some sports (that generate many games such as the NBA [2400 games] or MLB [4000 games]) can be shown to be statistically significant for that particular sport. When taken in total (our database includes more than 10,000 games!), we are pleased that “Betting Against the Public” results are robust and statistically significant. “Smart Money” methods also show good, robust, and statistically significant results, albeit over a smaller sample size.

Disclaimer
We do not guarantee that the trends and biases we’ve found will continue to exist. It is impossible to predict the future. Any serious academic research in the field of “market efficiencies” recognizes that inefficiencies may disappear or fade over time. Once inefficiencies are discovered, it is only a matter of time before the market corrects itself. We do not guarantee our data is error-free. However, we’ve tried our best to make sure every score and percentage is correct.