Beware of Commerce Department Statistics

November 20, 2007

One of my primary complaints about the media -- which is likely a result of editorial cuts in various newsrooms -- concerns those journalists who blindly report statistics without defining (or questioning) the underlying methodology. For example, only now are reporters even regularly questioning the methodology of RealtyTrac versus another source like Dataquick. It makes me wonder how many past headlines would've been printed in a smaller font had the sources been vetted better.

Take today's release from The Commerce Department; hey, it looks like starts are up by 3%! A rebound? Sure, for multi-family homes, which rose by 44% (after the previous month's 36% plunge), not so fast for single-family home starts, which fell by over 7%.

Of course we can expect revisions down the road, because the sample only includes about 900 permit-issuing jurisdictions out of a universe of 19,000 permit-issuing places -- or a sample size of less than 5% (anyone similarly fascinated with how they do this can find the methodologies here) -- these national numbers can swing wildly in successive months, especially when you realize that their field surveyors estimate total starts for buildings of 1-4 units at a 1/50 ratio and then count total units for buildings with 5 units or more. Clear as mud? Great! And the results of this monthly tally are quoted and published nationwide -- generally verbatim -- each and every month.

I'm not saying don't use the numbers at all - I'm just saying that any such quotations should note that they're (a) seasonally adjusted (and how that could impact the numbers); and (b) based on a relatively small sample size that requires a lot of guesswork. While they're claiming a 90% confidence level (which in statistics measures how right you think you might be based on the sample size), the constant revisions down the road would suggest otherwise to me. Call me a cynic!

A better indicator from Commerce might be building permits, because the sample size (8,500 permit-issuing places) is simply much higher -- well over 40%! In this case, U.S. applications for building permits are down by 6.6% for October and by over 24% from the same time last year. In the West, total permits for October were up by 4.5% from September but down by 2.6% for single-family homes. Since October of 2006, total permits in the West are down by 18% for all units and by 30.9% for single-family only.

If you'd like to see how individual markets of California are faring (at least for September, October numbers should be posted any day now), the CBIA compiles these numbers monthly with the help of the Construction Industry Research Board, which personally contacts each city/county statewide (more on how their data differs from the Commerce Department's in another post).