You may have read about statistical models that claim to predict the outcome of the upcoming Presidential election. It’s easy to imagine that these models are complicated and contain many demographic, sociological, economic, and political factors. However, I was surprised to read in this article that two simple models supposedly generate accurate predictions.
Both of these models use stock market data. One model is based on the Dow Jones and the other on the S&P 500. Statistics are best when they are a hands-on experience, so while neither study included the data, I obtained both the stock market data and election data so we can try these models ourselves using Minitab statistical software!
We’ll evaluate both models. If the models are satisfactory, we’ll use them to make predictions for the upcoming Presidential election. Today, we’ll evaluate the Dow Jones model and tomorrow the S&P model. You can get the worksheet for the Dow Jones model here.
Model 1: Three-Year Change in the Dow Jones Industrial Average
The first model comes from a recent study titled “Social Mood, Stock Market Performance and U.S. Presidential Elections” by Prechter, et al. You can download their paper here.
The researchers find a positive, significant relationship between several outcomes for presidential elections that have an incumbent and the percentage change in the Dow Jones over a 3 year period. Each three-year period extends from November 1 after the previous election through October 31 of the year of the election.
Their theory states that the stock market is a proxy variable for social mood, not that the stock market directly affects voting. The stock market is a good measure of social mood because if society feels positive enough to invest more money in the stock market, they are presumably happy with the status quo, which could favor the incubment.
The Dow Jones Industrial Average (DJIA) data back to 1897 was easy to obtain from the Federal Reserve Economic Data (FRED) web site. For elections prior to 1896, I use the Foundation for the Study of Cycles data set that the study used. This data set uses market data from earlier indices to create a longer DJIA.
The study looks at the percentage change in the DJIA over different lengths of time (2-4 years) and how it correlates to different election outcomes. The researchers also include the traditional big three predictors of Presidential elections: economic growth, inflation, and unemployment. The study concludes that the three-year change in the DJIA is the best predictor. Further, when the DJIA predictor is included in the model, the other “Big Three” predictors become insignificant.
I’m going to test the three-year model to determine if it can predict whether the incumbent wins or loses. I also include other election outcomes in the worksheet if you want to try those out.
Assessing the Model with Binary Logististic Regression
Because the election outcome for the incumbent only has two possible values (Win or Lose), we need to use Binary Logistic Regression. And, we have one predictor, the percentage DJIA change over three years. We get the results below.
The p-value for the Dow Jones changes is significant at 0.025. Further, the odds ratio is 1.10, which indicates that every 1% increase in the Dow Jones is associated with the incumbent being 1.10 times more likely to win. The goodness-of-fit tests (not shown) all have very high p-values (greater than 0.6), which suggests that the model fits.
We should also look at the concordant/discordant pairs in the output:
This portion of the output indicates whether the predicted event probabilities of the binary logistic regression model match the observed outcomes. To do this, Minitab compares all pairs of observations that have different response values (Win or Lose) and their predicted event probabilities.
- If the predicted probability of success is higher for the observation corresponding to a "success," the pair is considered concordant.
- If the predicted probability of success is higher for the observation corresponding to a "failure," the pair is considered discordant.
For the Dow Jones model, 86.3% of the pairs are concordant, which is excellent! There are nearly 7 times as many pairs that are concordant than are discordant. In other words, the predicted event probabilities are accurate.
Predicting the Election with Minitab
The authors of the study specifically don’t use the model to predict the election's outcome because, for them, it's a study designed to determine the stronger influence in voting behavior. But why let that stop us from using their model to make a prediction? We have determined that the three-year percentage change in the DJIA is a significant predictor and that the model produces accurate event probabilities.
So, we just need to enter the the percentage change in the Dow Jones from November 1, 2009 to October 31, 2012 (33.8%) in the Prediction subdialog for binary logistic regression. Enter 33.8 and we get the follow prediction output.
The model predicts that President Obama has a 95% chance of being re-elected. The confidence interval (59, 99) is very wide, but it is entirely above 50 percent. Further, from the concordant pairs, we know that the probability is generally correct. This probability of re-election may seem high given the tight race in the polls. However, this prediction is based entirely on the Dow Jones, which has increased significantly while President Obama has been in office.
I'll close with a table that puts this prediction in the context of all Presidential elections since 1828 with an incumbent candidate. The table is sorted by the probability that the incumbent is re-elected. You can see how the high probabilities are associated with "Won" and the low probabilities are associated with "Lost." The middle probabilities are a bit mixed up, as you'd expect. The green row indicates the prediction for the upcoming election.
I'm pretty impressed by the fact that a single predictor works so well for something as complex as a Presidential election.
In the next post, we'll look at a different model that uses the S&P 500 over a different length of time. I was surprised by those results!
Prev
Powered by Compendium
These may all be 'statistically significant' without having a real grounding in reality.
For one thing, the electoral precedents are not metrics (a metric is a standard of measurement for a quantitative analysis). Instead this type of argument is an informal statement along the lines of "no President has been re-elected if . . .". And, these are followed by some observation that supposedly determines the outcome of the election. In other words, if you just know this one fact, you'll know the outcome of the election.
The stock market data are different, particularly when you apply statistical analysis. We've measured changes in the stock market for nearly 200 years. We then statistically relate these changes to nearly two centuries of Presidential elections and their outcomes.
Further, the statistical model does not suggest that changes in the stock market entirely predict the outcome. In fact, good statistics quantify the uncertainty. In this case, we have both the confidence interval for the prediction and the concordant/discordant pair analysis which assesses how well the probabilies have matched historical reality.
The last table shows how the model produces probabilities, derived from changes in the Dow Jones, that generally match the actual outcomes. However, these are probabilities, not certainties. So, you'll also notice that there are candidates who have won when the probability was low and candidates who have lost when the probability was high.
However, the general trend for the probabilites is that higher probabilities correspond to a win and low probabilities correspond to a loss.
These assessments are how we ground the analysis in reality. It's possible that some day the Dow Jones model will no longer predict election outcomes. However, we have yet to see that in the historical record.
Jim
Thanks for your response. I certainly agree that the webcomic is not the same as the statistical analysis. It is kind of funny though in light of a lot of the predictions that a lot of people seem to make, and the importance many people seem to place on indicators that may not have much to do with the outcome.
At any rate, it seems that we can get similar results with other indicators as well, and they may be just as questionable as picking a semi-arbitrary timeframe and economic indicator.
For example, I looked up the average temperature for the month of October, and duplicated your analysis, with similar results. (data from here: http://www.ncdc.noaa.gov/oa/climate/research/cag3/na.html and it only goes back to 1895, so I only have 19 data points)
Year Winner %DJ Change IncumbentResult OctTemperature
1900 William McKinley 20.2199 Won 56.52
1904 Theodore Roosevelt -2.5360 Won 54.05
1912 Woodrow Wilson -8.7792 Lost 53.26
1916 Woodrow Wilson 34.5293 Won 52.20
1924 Calvin Coolidge 41.6939 Won 55.04
1932 Franklin Roosevelt -75.9780 Lost 52.42
1936 Franklin Roosevelt 97.6679 Won 53.51
1940 Franklin Roosevelt -0.9784 Won 56.16
1944 Franklin Roosevelt 23.2691 Won 55.35
1948 Harry Truman -0.2965 Won 53.24
1956 Dwight Eisenhower 73.4063 Won 56.24
1964 Lyndon Johnson 24.0452 Won 53.79
1972 Richard Nixon 11.8169 Won 52.36
1976 James Carter 1.6968 Lost 50.08
1980 Ronald Reagan 14.5716 Lost 52.67
1984 Ronald Reagan 39.2884 Won 53.74
1992 William Clinton 21.9351 Lost 54.49
1996 William Clinton 63.2823 Won 53.77
2004 George W. Bush 8.2424 Won 55.55
I don’t have data for Oct2012, but assuming that it’s the same 0.13F warmer that Sept2012 was compared to Sept2011, then I get similar results (though not quite as good of a p-value. Is that due to the fewer data points, or a slightly worse significance?)
Response Information
Variable Value Count
IncumbentResult Won 14 (Event)
Lost 5
Total 19
Logistic Regression Table
Odds 95% CI
Predictor Coef SE Coef Z P Ratio Lower Upper
Constant -50.6595 29.3071 -1.73 0.084
OctTemperature 0.966191 0.550237 1.76 0.079 2.63 0.89 7.73
Log-Likelihood = -8.282
Test that all slopes are zero: G = 5.337, DF = 1, P-Value = 0.021
Measures of Association:
(Between the Response Variable and Predicted Probabilities)
Pairs Number Percent Summary Measures
Concordant 55 78.6 Somers' D 0.57
Discordant 15 21.4 Goodman-Kruskal Gamma 0.57
Ties 0 0.0 Kendall's Tau-a 0.23
Total 70 100.0
Predicted Event Probabilities for New Observations
New Obs Prob SE Prob 95% CI
1 0.941616 0.0707179 (0.564475, 0.995042)
Values of Predictors for New Observations
New Obs OctTemperature
1 55.31
Perhaps, this means that it's another indicator that our current incumbent will be reelected, but it doesn't seem to match with the process knowledge that these would be related.
I didn’t spend much time trying to find more data, my guess for the October 2012 temperature is a stab in the dark, and I don’t really understand binary logistic regression, so obviously my comments and results are questionable at best. Even that makes me wonder what some TV analysts are doing and if any of this has any practical significance. I guess we’ll find out tomorrow!
The web comic was certainly funny! And, I think that type of analysis is easier for the media to present. It's simpler to present and the conclusions appear to be more definite. Unfortunately, things in the real world are often not simple or definite!
I ran your binary logistic model with the average temperature for October just to see what it produces. The temperature predictor is not signficant with a p-value of 0.079. While not significant, I'm impressed that it's anywhere near significance!
To get r-squared values to compare, I ran 2 more models with General Regression, both using the smaller dataset from 1900 to present for consistency. For the continuous response, I used the difference in the popular vote between the incumbent and closest challenger.
When I ran this with the change in Dow Jones, the DJ is significant (p=0.004) and the r-squared is 39.91%
When I use the October temperature as the predictor, the temperature is not significant (p=0.208) and the r-squared is only 9.15% The predicted r-squared is actually a -7.34%!
So, while I'm surprised that the binary logistic model for the October temperature has as low of a p-value as it does, it is still insignificant. And, the comparison of the two General Regression models also seems to reconfirm that the Dow Jones model is better (significant predictor and higher r-squared) than the October temperature model.
Whew! I'm not sure what I would've thought if the average October temperature predicted elections as well. I suppose temperature is related to turnout as Patrick Runkel writes about in his current Minitab blog post.
But, you raise a great point. You have to use your process knowledge to see what makes sense to include as variables. I only spent a couple of sentences explaining why the change in the Dow Jones might be a legitimate predictor for election outcomes. If you want a more complete rational, you should read the original study. There is a link to it in the blog post and authors write about it for 42 pages. Whether you agree or disagree with them, at least you'll get the full case!
Of course, in the end, the only statistic that matters for the election is how people vote. Well, technically it's how the Electoral College votes, but that's another issue!
Jim
Anyway, we could pick an alpha of 0.1 and then call the temperature data significant. We could also claim process knowledge that warmer temperatures are "consistent with socionomic voting theory, which includes the hypotheses that (1) social mood as reflected by the [ambient temperature] is a more powerful regulator of re-election outcomes than economic variables such as GDP, inflation and unemployment and (2) voters unconsciously credit or blame the leader for their mood."
I didn’t spend a ton of time looking for variables the do or don’t correlate, but I did look at a few other months of temperature data that didn’t, and could have made similar conclusions about why October’s temperature is more powerful than April’s. Part of the point is that we can always find something that correlates and they may or may not be actually related. If I look hard enough, and with at most 26 data points to compare to, I’ll find something.
The study seemed to look at several economic indicators, and found that one of them correlated. I looked at several months of temperature data, and found that one (almost) correlated.
To end on a humorous note again, here’s another XKCD comic that relates to the point:
http://www.explainxkcd.com/2011/04/06/significant/
Maybe there really is a common cause, or maybe not. How’s that for a political statement?
Thanks again for your blog! These topics really make the use of Minitab interesting and engaging. Now I wish it was more help to my fantasy football team, but that’s a different story entirely. ;-)
Funny comic! Ah, the very real perils of data mining.
You could raise the alpha to 0.1, but that's fairly uncommon and it does raise the risk of the concern that write about, a correlation by pure chance. Also, the General Regression results show a much higher p-value for temperature, which isn't the case for the Dow Jones. Collectively, the Dow Jones is a better predictor.
It's a great point, though. Is the correlation that you see by chance or not? This issue is something that you should consider for every analysis. I think the authors and analysis of the study make a good case, but your mileage may vary.
Thanks for reading! For your fantasy football team, you should check out Kevin Rudy's blog here at Minitab. He often writes about that.
Jim