Reassessing GDP Growth with Data and Statistics, part 1

If you combine tough economic times with a Presidential election year, you get a heightened interest in how the economy is changing. Is it growing faster or slowing down? Unsurprisingly, there are many contradictory predictions about what will happen over the longer term. You'll find countless TV pundits pushing their opinions.

A lot of attention focuses on the quarterly GDP growth. I’m going to explore what you can learn about this particular economic measure from statistical techniques and actual data. By the end of this blog, you’ll understand what you can and cannot infer from these economic growth estimates.

To do this, I’m going to use a tool near and dear to the hearts of quality improvement analysts: a control chart! This will be a three-part blog running on consecutive days. The final part will coincide with the latest announcement from the U.S. Bureau of Economic Analysis (BEA) about GDP growth. We’ll see what we can make of it using the insights we gain.

Why Revisit Data Analysis and GDP?

I wrote in an earlier post about statistical techniques to estimate the GDP and how the early estimates of the GDP growth are actually based on relatively small amounts of data. The hows and whys are pretty interesting, and you can read my blog about it here. I presented the graph below that compares the early estimates to the latest, gold-standard GDP values. They certainly look pretty similar. I also presented results from a July 2011 study which suggests that the early estimates are very good. Despite this evidence, there were differences that nagged at me and I wanted to dig deeper into them.

Times series plot of GDP growth

As I’ve written before, always investigate your data mysteries, because you're bound to learn something important! By part three of this series, we'll solve this GDP mystery. Specifically, we'll understand how the early estimates are different from the later estimates.

Varieties of GDP Growth Estimates

Here’s a quick refresher about BEA’s multiple GDP estimates. I’ll be focusing on the differences between the Third and Latest estimates (bold).

  • Advance: The very first estimate released near the end of the first month after the quarter.
  • Second: A revised estimate issued a month after the Advance estimate.
  • Third: Another revision delivered a month after the Second estimate, and the last of the early estimates.
  • Latest: Over the years, BEA continues to update and revise the GDP estimate. These are the gold standard estimates, although even these estimates continue to be revised as improved information and techniques become available. So they may be the gold standard GDP estimates, but not quite 24k gold!

Using I-MR Charts to Assess GDP Growth Estimates

According to the study about the revisions, it is not valid to use traditional statistical analyses on the estimates of GDP growth because the types of data, measures, definitions, and adjustments change over time. “Accuracy, as a result, cannot be assessed by conventional statistical measures, such as standard errors. It can, however, be assessed by examining magnitudes and patterns . . .”

Since my previous post about the GDP, I’ve been mulling over how to analyze these data with these restrictions in mind. Hmm, magnitudes and patterns...these are properties that a control chart assesses. In Quality Digest, Dr. Donald J. Wheeler wrote about using I-MR charts to detect changes in survey data about the percentage of high-school seniors who smoke. I’m going to use a similar approach to see what I-MR charts can tell us about tracking quarterly GDP growth over time. However, I need to expand on his approach a bit, because GDP estimates come in different quality levels, including a gold standard.

Wheeler writes about the need to filter out the noise whenever we attempt to interpret data. Otherwise, you risk interpreting the noise as the signal. GDP growth estimates will always vary from quarter to quarter. Some of this is unusual variation (the signal) and some of it is normal fluctuation and error (noise). For example, even if the economy grew at a perfectly constant rate, the GDP estimates will continue to vary because the estimates contain error. However, even looking at just the true variation, a lot of it is just the regular variability that you expect from a process and it doesn’t signify anything. We need to detect the unusual variation and patterns.
We want to filter out the noise in order to detect any real signal that might be present. To do this, we’ll use an I-MR chart because it assesses the changes between individual measurements. These charts determine the amount of variation between subsequent measurements that is inherent (common cause) and calculates the I-MR control limits accordingly. Consequently, points that exceed a control limit, or violate an I-MR test, signal a magnitude change or an unusual pattern (special cause). These are the signals that stand out from the noise.

BEA’s standard of accuracy for the early estimates is to compare them to the Latest estimates. I’m going to extend that and say that if the Latest estimates are the gold standard values, then the patterns evident in the Latest estimates are the gold standard patterns. We will assess how closely the earlier estimates exhibit the same patterns. The key here is that we're not just comparing individual points (more on why not later), but comparing patterns of points.

My approach will be to use an I-MR chart to look at the patterns that are both visually evident and that Minitab confirms as unusual. Specifically, I’ll compare 2 I-MR charts:

  • The Latest “gold standard” estimates to see what patterns are real
  • The Third estimates to see how well they reflect the gold standard patterns

Tomorrow, in part two, we’ll look at the gold standard patterns.

You can download the Minitab Statistical Software data file used in this series here.

7 Deadly Statistical Sins Even the Experts Make

Do you know how to avoid them?

Get the facts >


blog comments powered by Disqus