« And yet Another Iteration of Interactive Information Hypercube Viewer | Main | Possible Error in Census Bureau Data, an Update »

Census Bureau Confirms Error, Revises Data Set 

Well, it seems that there was in fact an error in the Census data set which I thought might have existed as I mentioned in this blog post.

You can see the Census Bureau's revised Excel data set here.  (Or you can grab it from the State Government Finances page here. See the "Summary Table" [Excel, 43KB] link on that page.)

So let me summarize the chain of events:

  1. I was creating an XBRL prototype I called the State Fact Book. The primary purpose of the prototype was to test the differences between XML, XHTML, XBRL and RDF.  See this blog post for more information. Here is the actual analysis.
  2. This spreadsheet shows the error. This screen shot also shows the error.
  3. For this prototype I used some publicly available data.  Part of that happened to be some data from the US Census Bureau (the link above).
  4. In creating the XBRL, of course I wanted to make sure I was doing everything correctly, so I created several XBRL formulas to verify that I was not making any mistakes.  Here is the XBRL Formula file.  Here are the human readable rendering of the validation results. This was generated by the UBmatrix XBRL processor which supports XBRL Formula. Note the "notSatisfied" results.
  5. Someone in government read my blog post and decided to investigate if this was in fact an error.  They wrote a letter to the Census Bureau.  Here is the letter.
  6. The letter and this story was reported in NextGov.
  7. This revised data set clearly shows the revision to the US Census data.

Could have this error been discovered using Excel formulas?  Sure it could have been discovered.  But, there are no formulas in that spreadsheet data set.  A business rule in a global standard format could have been created which is useable by both the creator of the data (here the Census Bureau) and the consumer of the data (in this case myself or anyone else using that data).  The business rules in a global standard format does not limit your use of those rules should you not use Excel.

The points here are as follows: (a) if you publish information and that information has numeric relations, it is important to publish business rules that both document how the information adds up and proves that the information adds up; (b) this is what XBRL Formula is for and as you can see it does work; (c) the key advantage of the global standard XBRL Formula is that the business rules are not application dependent, but rather can be exchanged between business systems independent of the system.

There is no intent here to beat up on the U.S. Census Bureau.  It just so happened that I stumbled upon this error because I wanted to be sure my system was not reporting incorrect information.  In my years of creating XBRL demos, I have run across this type of error before in SEC filings.  These information sets are complex. Other errors exist in publish information sets, you can count on that.  XBRL and XBRL Formula when properly used can improve data integrity.

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments

There are no comments for this journal entry. To create a new comment, use the form below.

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
All HTML will be escaped. Hyperlinks will be created for URLs automatically.