Digital Financial Reporting

The following is my list of the top ten things which someone using XBRL information should keep in the forefront of their mind.  This list might be looked at as the biggest challenges one needs to overcome when using SEC XBRL information.  These are all decisions which the domain stakeholders of financial reporting needs to make in order to get XBRL to work how they want it to work.  The best way to make these decisions is to understand how XBRL works for SEC XBRL flings today, what the alternatives are, and what alternative best fits your stakeholder group needs.

My definition of user of SEC XBRL information would include analysts, investors, regulators, data aggregators, and software developers building software for these types of users.

Keep something in the back of your mind as you take a look at this list.  Financial reporting practices used today were developed in an era where paper was the medium for distributing information.  There may have been minor inconsistencies in certain financial reporting practices, but these minor inconsistencies where easily overcome by the humans reading the financial reports.  In a new era where elimination of many of the mundane tasks such as re-keying information is possible, these minor inconsistencies become insurmountable issues as we try and get computers to do these mundane tasks.  Processes must be reliable.  This new electronic medium, XBRL, is different than the paper-based medium of the past.  Certain changes can make achieving some desired characteristics significantly easier.  These historical choices should be reconsidered as to not hold back the possibilities offered by the new mediums.

To understand what you want it is best to understand your options.  The options when it comes to XBRL are the characteristics of how XBRL works.  These are business domain stakeholder decisions.  Knowledge can give you the power to make a case for what your stakeholder group needs from XBRL.

One final thing to consider.  Keep in mind how computers "think".  This thought experiment can help you understand how to look at financial information.  There is a simple goal here as I see it: to make XBRL work the best it can.  Of course "best it can" is subjective.  Are certain changes in the interest of financial reporting?  Maybe they are, maybe they are not.  That is not for me to say.  All I am doing here is providing information so that the right discussion takes place.  All financial reporting domain stakeholders should push for what they want from the strong position of knowing how XBRL actually works.

In looking at SEC XBRL filings, this is what I see:

1. Accounting inconsistencies or errors: This analysis details my point here.  I don't want to make any judgements, I am just delivering a message.  Is it an accounting error to have two different ways of computing net change in cash, as pointed out by this set of SEC XBRL filings?  The answer could be "yes" or it could be "no".  That is not the point.  The point is this.  If the vast majority do it one way, why can't everyone do it the same way?  What is so unique for those 8 compaines that they need to calculate the value differently?  Of course you can see that if there are two ways, adjustments need to be made to convert one approach to the other approach when analysis uses these numbers.  Should analysts really need to do this?  What is the reason for not only using one way?  This issue is not just about this area of the XBRL taxonomy, there are other areas.  The fact patterns are different for different areas of the US GAAP taxonomy, but what rule do you apply to determine whether alternatives in different areas are helpful or hurtful?
2. Filers adding concepts unnecessarily: When should SEC XBRL filers add new concepts in their extension taxonomies?  If you look at the SEC XBRL filings being submitted, there are many inconsistencies in where and when concepts are added.  This analysis of the statement of cash flows shows many of the inconsistencies in this area, the same sort of issues exist in other areas of filings also. Unnecessarily added concepts can may querying the information significantly more challenging. What exactly the rule which determines if a concept should be added and who enforces that rule?
3. Filers not following the US GAAP Taxonomy Information Model: By information model I mean things such as the [Table] structure, the [Roll Forward] structure, and other such structures in the US GAAP Taxonomy.  The US GAAP Taxonomy is very consistent in how it uses, for example, [Table]s and [Roll Forward].  But company extensions to the US GAAP Taxonomy are not.  See this analysis of how filers are constructing [Table]s.  The reason is that there are no validation rules which point out inconsistencies when SEC XBRL filings are submitted. Is this randomness beneficial, or should the free-for-all be constrained in order to realize better comparability across SEC XBRL filings?
4. Locating networks to make comparisons can be challenging: SEC XBRL filings each use their own set of networks to identify sections of the financial filing such as the balance sheet, income statement, or a specific disclosure.  See this analysis for more information.  Now, you may look at the filings and say, "Hey, this is no problem, I can read the network descriptions just fine."  But can a computer read the networks and then make a comparison across multiple companies, say comparing the balance sheets of each company?  How will a computer know WHICH network to grab?  A computer might be able to determine from the concepts within the network what the network is to a degree.  Another way of understanding this is to say that if filings use common structures, in this case network roles, to identify pieces of a financial filing, finding those pieces and comparing them would be more reliable and certainly easier to program. Is it best if everyone uses different network identifiers or would it be better if specific network identifiers were used for common things such as "balance sheet", "income statement", certain specific disclosures in order to enhance cross filer comparability?  Or, are networks meaningless, and it is really the [Table] which provides the comparison point?
5. Inconsistent use of concepts and relations at "higher levels": Financial reporting in the US is dynamic meaning that financial report creators can create reports which reflect their unique organizations.  Basically, creating a US GAAP financial report is not filling out a form, nor should it be.  The dynamic nature makes US financial reporting better.  But, should dynamic mean "free-for-all"? SEC XBRL filings are many times, at least in my view, changing concepts and relations at high levels where changes probably should never occur.  Case in point is the cash flow statement, this analysis provides information about how SEC XBRL flings are changing concepts such as "us-gaap:CashAndCashEquivalentsPeriodIncreaseDecrease".  Another way to look at this is that if the higher level concepts and relations were more solid with extension points more clearly specified, then US financial reporting keeps it's dynamic nature, provides areas where extensions are quite reasonable, and avoids a free-for-all which a computer will simply not be able to deal with when trying to analyze SEC XBRL filings.  For example, a balance sheet has "Assets", "Liabilities" and "Equity" (and sometimes temporary equity).  Do you really want filers changing those high levels?  Why?  Be aware that they can at this point.
6. Calculation inconsistencies and unexpressed computations: Many SEC XBRL flings have calculations which don't properly "foot and cross-cast".  See this analysis which discusses this in more detail.  There are those who believe that calculation inconsistencies cannot be made to go away.  I personally don't buy that and that has not been my experience.  Think about this though.  Would you want to use a financial report where things don't add up?  Would you trust the information?  How would you distinguish something which does not add up because of an error and something which is not supposed to add up?  Further, many times there are important computations which are not articulated by the SEC XBRL filing.  These should be explicitly articulated for two reasons in my personal view: so the information creator knows that things add up and so that the consumer of the information knows what is supposed to add up and how.  What should the rules be for computations?
7. Improper modeling of information characteristics: People have different interpretations about what XBRL Dimensions provides in the US GAAP taxonomy and therefore they are used in different ways by different users.  Even the US GAAP Taxonomy itself has different interpretations, sometimes using XBRL Dimensions and sometimes not to model information.  People will eventually realize that consistently and correctly using XBRL Dimensions makes easier. There are four major classes of things to realize when trying to interpret the US GAAP Taxonomy relative to XBRL Dimensions: (a) Confusing and inconsistent: When SEC XBRL filers create their extension taxonomies their presentation relations and definition relations can be inconsistent, usually the definition linkbase is missing things.  This impacts how the dimensions work because the definition linkbase really controls the workings of the dimensions, it is not the presentation linkbase.  This can occur because there are two ways of articulating the same information: the presentation linkbase and the definition linkbase. The solution to this issue is have only one way of articulating the information model, not two.  (b) Calculations don't work: Improperly modeled information can result in computations which don't add up correctly or you may struggle to figure out exactly how things add up.  For example, consider this calculation validation report which shows that the balance sheet does not add up (look for the red in the last column on the right).  If you look at all of these calculation reports (use the links to the calculation reports on the right hand side of the page), you can see that each of these SEC filers was able to get its balance sheet to calculate properly. (c) Drill down: A financial statement generally consists of summarized information in the primary financial statements such as the balance sheet and detailed information from many of the line items on the primary financial statements within the disclosures.  If the dimensional information is created correctly, "drilling down" from the primary financial statements to details in the disclosures is easy to do.  If they are not constructed correctly, a computer cannot correctly understand the links and therefore cannot possibly navigate between the primary financial statements and the details in the disclosures. (d) Mixing dimensional and non-dimensional information: Mixing XBRL Dimensional and non-dimensional information simply will not work correctly.  For example, XBRL Formula has two aspect models: dimensional and non-dimensional.  People will figure these things out when software improves, but just be aware if these issues.
8. Irregular shaped hypercubes: Information reported in an SEC XBRL filing has different dimensions. For example, a consolidated balance sheet summarizes information for the consolidated entity for two period of time, the current period and the prior period.  Whereas the segment breakdown of the balance sheet information has a different shape because it has different dimensions, usually one period and the breakdown is by business segment.  The two dimensional nature of paper limits how certain information, which may have three or more dimensions, can be presented.  Mismatches between the dimensions of information (i.e. the shape of the information) can make information harder to view.  This situation can be rectified simply by reorganizing the information into the proper [Table]s. Exacerbating this situation even more is the fact that not all information is broken down into [Table]s and many of the "chunks" of information in the US GAAP Taxonomy are quite large.  It might be better to have many smaller chunks of information which fit together, rather than the huge pieces which exist currently in the US GAAP Taxonomy.  How the US GAAP taxonomy is modeled is a choice which you can influence.
9. Extension concepts with poor descriptions:  Not providing quality descriptions for concepts added by an SEC XBRL filer makes it harder to understand the filing and harder to determine if the filer is creating a concept which actually already exists in the US GAAP Taxonomy.  Part of the point of XBRL is to make the meaning of things explicit, rather than have the user of the information guess.
10. Fiscal periods:  This is more of a nuisance than a significant issue, but it is something to be aware of.  As you know, not all companies report on a calendar year end.  When you manually go through paper reports and do comparisons, humans can pick the right periods to compare quite easily.  When you try and automate this process, it becomes a little more challenging.  There is no fiscal period information in SEC XBRL flings, so computer programs have to figure this out in automated processes.  Different software may do this in different ways. Analysts have been dealing with this for years so they are use to it.  But be aware, this can be automated reliably, if fiscal period information is included in the meta data.

Understanding this list of ten things will help you better understand the SEC XBRL filing information that you are working with.  It will also help you understand what you should expect and therefore request from those trying to figure out how to make XBRL work within the financial reporting supply chain.  The points are made not to be value judgments about what XBRL should be, rather they try and point out the spectrum of options which are possible.  It is up the the domain of financial reporting to make the decisions.  If you don't understand how things work and you are a stakeholder in the financial reporting supply chain, how are you going to know what to ask for from SEC XBRL filings?

The CFA Institute published a white paper, eXtensible Business Reporting Language: A Guide for Investors. They outline five guiding principles for what the CFA Institute wants to see from XBRL.  The CFA Institute's white paper articulates what their stakeholder group desires from XBRL. What does your stakeholder group want to see?  How is your stakeholder group making sure you get what you want?

The issues pointed out here, while specifically relating to the SEC XBRL filings, are actually issues which any system which uses XBRL's extensibilty features will need to address. For processes to be automated, the information must be unambiguous and the semantics must be clear and consistent.  Even when you have unambiguous information and clear and consistent semantics, you still have some choices as to how you want your system to operate.  Understanding these points helps business domain users make these choices.

Several months ago I searched and searched, trying to find what I would consider a model SEC XBRL filing.  I did find one.  (Here is a blog post which summarizes the types of mistakes filers are making.)

In the latest set of SEC XBRL filings, I found 24 SEC XBRL filings which I gave a "5 Star Rating"!  That is quite a nice leap.  You  can see the list here on an HTML page (and here is the list in XML if you want to read the list programatically and perform your own analysis).

Model SEC XBRL Filings

While these filings may not be totally perfect, this is a very significant move in the right direction for SEC filings in my view.  There is one thing which is a bit disappointing about the list which is that one filing agent created all of the filings.  That vendor is EdgarOnline.  Nothing against EdgarOnline, I just would like to have seen more vendors on the list of those getting a 5 Star Rating.

An interesting point here is that there are at least three separate parties who say that the list of 24 filers did a good job.  First there is me, I say those are good filings.  See below how I define "good".  Second, EdgarOnline says they are good; otherwise they would have done them differently.  Third, XBRL Cloud says that they are good.  The public list of SEC filings validation results (called the "dashboard") made available show zero errors in all the categories I used.  (I excluded certain best practices and informational messages from my evaluation criteria.)

Conceivably, you could add another "party" to those who say that those SEC XBRL Filings are on target which is the US GAAP Taxonomy Architecture. That is where XBRL Cloud, EdgarOnline, and myself get the information to help us figure out how to properly create SEC XBRL filings.

5 Star Ratings

Here is a list of SEC XBRL filings which I gave a 5 Star Rating to.  I consider these filings models of what I consider a good SEC XBRL filing.  Can you consider these best practices?  I think so.  Show be better practices.  See a detailed listing of the characteristics below.  But here is the list of the 24 SEC filers who get my 5 Star Rating (again, here is the list on a web page where you can get to additional details):

• ALCOA INC
• BOEING CO
• CARDINAL HEALTH INC
• CISCO SYSTEMS INC
• CLIFFS NATURAL RESOURCES INC.
• CNX GAS CORP
• CONSOL Energy Inc
• EBAY INC
• ELECTRONIC ARTS INC.
• FASTENAL CO
• Google Inc.
• HOLOGIC INC
• INTERCONTINENTALEXCHANGE INC
• INTUITIVE SURGICAL INC
• OPEN TEXT CORP
• PARKER HANNIFIN CORP
• PPG INDUSTRIES INC
• PRECISION CASTPARTS CORP
• QWEST COMMUNICATIONS INTERNATIONAL INC
• SIGMA ALDRICH CORP
• UNITEDHEALTH GROUP INC
• VARIAN MEDICAL SYSTEMS INC
• WALT DISNEY CO/
• WELLPOINT, INC

Congratulations to these 24 filers and to EdgarOnline for helping to move the list of models from 1 to 24.  There is still a ways to go to get where we need to be, but this is certainly a big step in the right direction.

Criteria for Receiving 5 Star Rating

Here are my criteria for receiving a 5 Star Rating. First, let me say that the rating is not about perfection, it is about a minimum standard.  A lot of people tell me, "Well, it got into the SEC so it is good enough."  My quality standards are higher than the SEC.  They are where the SEC really needs to go to make the XBRL truly "interactive data" (as the SEC calls it), usable for analysis across companies and to allow for the HTML/ASCII filings to eventually be phased out.  Again, even my criteria are not sufficient to attain the goal of usable analysis or phasing out the legacy formats.  However, they are absolutely necessary to achieve that goal. 

The data set I am working with are 10-K and 10-Q SEC XBRL filings between 2010-01-12 and 2010-02-19 from the XBRL Cloud report.  (I use that and not the SEC RSS feed because I cannot get a what I need from the SEC RSS feeds.)

Here are the criteria:

1. No XML, XBRL, or EFM Errors:  The first criteria is no basic errors.  This is indicated by having a "0" in Column 8 which is marked with "E" under "Edgar Filing Manual" on the XBRL Cloud validation report.  There really should be no difference between what XBRL Cloud says is an error, what the SEC says is an error, and what any other vendor says is an error.  The fact that there are submissions with errors as indicated by XBRL Cloud means that there are disagreements between the SEC, XBRL Cloud, and other vendors as to what is an XML, XBRL or EFM error.  (i.e. interoperability issues)  Making these interoperability issues go away is key to making SEC XBRL filings usable.  EdgarOnline seems to believe that XBRL Cloud is correct.  I say that because they pass XBRL Cloud definition of "EFM Errors".  (An interesting side note here is that these issues seem to be going WAY down.  Analysis of prior quarters showed multiple vendors being inconsistent with XBRL Cloud validation.  In this last analysis there was only one vendor who was inconsistent (7 are consistent with XBRL Cloud) and there were only 25 inconsistencies (last quarter there were 255 inconsistencies spread among the 7 vendors).
2. Calculations consistent: Automated processing and calculation inconsistencies don't go together.  It is my view that if most filers can get rid of 100% of calculation inconsistencies, then ALL filers should be able to do so.  I will hold that view until I have evidence to support a different view.  All calculation inconsistencies which I have seen are, in my view, a result of sloppy preparation of the XBRL instance.  As such, if you have calculation inconsistencies, you get no 5 Star Rating.  See the calculation reports for the 24 companies on the list.  All the calculations are squeaky clean.  (Keep in mind though that I am not testing to be sure that 100% of computations have some test to be sure they are correct.  That is where things need to go, but first the existing inconsistencies need to be made to go away so that is the current focus.)
3. Pass one basic XBRL Formula test: The filings need to pass one basic XBRL Formula test or business rule. The business rule states that the beginning balance of cash plus total changes in cash equal the ending balance of cash.  EVERYONE has cash generally.  Cash changes for everyone.  Therefore EVERYONE should both have all these concepts and the computation needs to work.  For all these filers, those criteria are met.  There are many reasons one can come up with to argue that this is not a valid criteria.  But, the point here is to drive that discussion which needs to happen to truly resolve a number of real issues relating to using XBRL.  Some of these are accounting related such as the unnecessarily inconsistent way filers calculate this change.  See this blog post for more information.
4. [Table]s created correctly: The US GAAP Taxonomy Architecture calls for [Table]s to be constructed in a specific way.  Every [Table] in the US GAAP Taxonomy follows the US GAAP Taxonomy Architecture.  SEC filers likewise need to follow the rules for [Table]s.  To not do so will cause very serious comparability problems.  There are other areas where filers need to follow the US GAAP Taxonomy, but this area is a start.  I am not even testing the XBRL Dimensions aspect of [Table]s at this point, that will come.  First, the basics.  If you don't use [Axis] and [Line Items] properly in a [Table], your information will not be comparable to other information and therefore you don't get a 5 Star Rating.
5. Extension concepts make sense: This is a bit more subjective so I am actually cutting people a lot of slack here.  But,  if you provide an extension concept, it needs to make sence in the grand scheme of things and you need to provide a description of the concept.  Poor choices here reduce comparability of the information, which is not good for investors using the information, and therefore if you do this you don't get a 5 Star Rating.

There you have it, those are the criteria.  Again, pretty basic.  As the number of SEC XBRL filings pass these basic criteria, then one can start looking at the additional things which will need to be addressed to make the SEC XBRL filings useful for comparison and so they can completely replace the HTML/ASCII filings.

If you are creating an SEC XBRL filing, these are good examples to follow.  I consider these 24 SEC XBRL filings to be good models for others to follow.

To get a fuller appreciation of the filings, go look at the details on this page.  Click on the reports such as the calculation validation reports, the actual "Interactive data" at the SEC web site, and all other information which can help you understand the filing.  If you are a software vendor and you want to run your tools over this set of filings, use this XML document.

Want to get added to the list?  Should someone be removed?

If you believe that your filing or any filing should be added to this list let me know.  Point me to the filing and I will check it out.  I personally want the club of 5 Star Ratings to grow, the more the merrier.  If you think someone deserves to be removed from this list due to some fatal flaw that I am not checking, let me know about that also.  I want to point to good examples for others to follow.  No need to have poor XBRL filings listed here.