Digital Financial Reporting

There is a belief which has reached somewhat urban legend status about calculation inconsistencies in XBRL instances. The belief is that there are certain calculation inconsistencies which can never be made to go away no matter what you do, no matter how hard you try.

Is this belief that not all calculation inconsistencies can be made to go away truly a fact or is it a mistaken belief?

Let's look at some facts.

• A set of 403 SEC XBRL filings I looked at (the second quarter of XBRL filings with the SEC), 313 had no calculation inconsistencies. The 90 that have calculation inconsistencies are listed here.  That is about 22% of all XBRL filings.  That is an improvement from the first quarter of XBRL filings.
• The set of 408 SEC XBRL filings for the first filing period where XBRL was officially submitted there were 123 filings which had calculation inconsistencies.  That was 30% of all filings.
• Of the calculation inconsistencies which existed in the set of 403 filings mentioned above, when I try and figure out why the calculation inconsistencies exist; in 100% of the cases the inconsistencies resulted from either (a) an error in creation of the XBRL instance or (b) an error in modeling the XBRL taxonomy or (c) a combination of both.
• This is a PDF which shows two filings which I looked at and tried to figure out why the calculation inconsistencies existed.  It is not my intension to pick on these two filers.  I did need to show specific details to make my point, so it was necessary to pick specific filings.  It shows how I determined that the existing calculation inconsistencies, were in fact, unnecessary and could have been made to go away by correcting the XBRL instance or correcting the XBRL taxonomy.
• There seems to be a statistically significant relationship between the filing software used and the number of calculation inconsistencies.  If there were no relationship, then it seems to me that the inconsistencies would be evenly distributed between those creating XBRL filings.  But the data that I see does not show this.  Perhaps someone with a better background in statics can check into this further.  Here is the data from XBRL Cloud.  Here is my summary of this information. You look at the data and reach your own conclusions.
• There does seem to be a realization in the filings that there is a difference between the presentation of numbers in a rendering and how that number is put into an XBRL instance (i.e. mixing up the data and the presentation of the data).  This misunderstanding is NOT what is causing calcuation inconsistencies to the extent that it did before.  This is very, very good news.

Now, I would really like to make it through all 90 of the XBRL instances which have calculation inconsistencies to figure out why the calculation inconsistencies exist.  That would take a significant amount of time.  What I may do is wait another couple of quarters until filers fix what amount to obvious errors and then undertake that task, it would yield some good information.

In my personal experience, I have never found the need to create an XBRL instance that needed to contain calculation inconsistencies and have always found that my calculation errors resulted from one of two things: bad fact values in the XBRL instance or a poorly modeled XBRL taxonomy.

Now, don't get me wrong here.  Have I had inconsistencies in XBRL instances?  Absolutely.  Some inconsistencies were down right confounding.  This was particularly true when the validation reports were harder to read. (They are not that great today, but they are WAY better than they used to be!) For example, when I was helping to create the IFRS taxonomy I always tended to have problems getting the indirect cash flow statement and the income statement to both have zero calculation inconsistencies.  What I discovered is that I was making a modeling error in the taxonomy.  To make a long story short, this helped me reach the conclusion that it is critical to create XBRL instances when creating an XBRL taxonomy to be certain that you don't model the taxonomy so that it is impossible to create an XBRL instance which does not have calculation inconsistencies.  I learned a lot about creating XBRL instances from struggling to make calculation inconsistencies to go away.

However, I am not ready to say that it is possible to make all calculation inconsistencies to go away.  And, as such, I can certainly understand the risk management decision to allow calculation inconsistencies in SEC XBRL filings which the US SEC has decided take.  But at the same time I point out that allowing these inconsistencies with no ramifications for the inconsistencies promotes sloppiness.  Evidence of this are the 90 filings with inconsistencies, the two which I pointed out above which are clearly filer errors, and the 10 or so other filings I have been through and see no situation where I would not be able to make the calculation inconsistencies to go away.

Is more evidence necessary to determine if all inconsistencies can be made to go away for every filer in all situations?  I think so.  But should there be something to motivate filers to fix obvious inconsistencies?  Personally, I think that would be a good thing.

I know accountants.  I am an accountant. We accountants like things to add up, tick, tie, etc.  Personally I think that XBRL calculations are getting a bad rap for two reasons.  First, I have personally called XBRL calculations impotent.  Why? Well, they don't cover all the computational situations which exist in an XBRL instance for financial statements.  Roll forwards are an example of something they don't cover.  Cross dimensional aggregations is another.  Second, in the early days of XBRL there was a lack of understanding between calculation and other relations expressed in networks and the XBRL instance which caused poorly created XBRL taxonomies.  I know this because I made these mistakes myself.  A good example is hooking up both indirect and direct cash flow statement calculation networks to one XBRL instance which only has fact values for the indirect network of calculations.  I get that now, others do also.

So what is the conclusion here and what is the answer?  This is what I believe:

• All this will work itself out over time.  We clearly cannot have automated processes without things adding up correctly.  It would  be too difficult to create enough software which effectively overcomes the errors in numeric data where things need to add up to be useful.
• Filers should be more careful in creation of their filings.  Part of this is to become more familiar with modeling XBRL taxonomies.
• If you find yourself blaming XBRL for the calculation inconsistencies, you are missing the point and in my view are hiding behind XBRL in an effort to basically cover up your unfamiliarity with XBRL or sloppiness.  Don't fall into this trap.
• If you feel you cannot get the computations to be expressed correctly with XBRL calculations; fine, then use XBRL Formula.  Use something to help prove out your data.  You may not be able to submit the XBRL Formulas to the SEC, but what you will learn by going through this process are the taxonomy modeling errors you are making.
• Don't mix up poor XBRL taxonomy modeling practices and XBRL calculation inconsistencies.  For example, if you think you cannot make things add up because of "dimensional issues", ask yourself, "Well, do I have my dimensions expressed correctly?"  Generally you will discover that the problem is the taxonomy.
• Look at other SEC XBRL filings which have no calculation inconsistencies and see how they modeled their taxonomy.  Ask yourself, "If they can make the inconsistencies go away, why can't I?"
• This may be asking too much, but what the heck.  It is my belief that the SEC should not give filers carte blanc when it comes to calculation inconsistencies.  There should be something to provide an incentive to not have calculation inconsistencies.  Perhaps having to explain why they need to exist in an XBRL footnote in a filing or having to re-submit if the inconsistencies are subsequently found to be caused by an error would provide enough incentive.

If you do believe you have calculation inconsistencies which you simply cannot be made to go away, I would love to see the situation.  Knowing these situations will help figure out how to tame XBRL in this area.  Personally, I am taking the stance that until there is proof to the contrary, calculation inconsistencies can be made to go away if you try.

Continuing on with my analysis of the first set of 408 SEC XBRL filings, here I will summarize what I saw when I looked at XBRL calculations relating to those filings.  This web pagepoints provides a summary of these results and a viewer which lets you look at the results of checking these XBRL instances against the calculation relations provided by the filer.

• Of the 408 filings, every one provided a set of calculation relations.
• Of the 408 filings, 302 had no calculation inconsistencies. (Actually, in the validator I was using 285 had no inconsistencies and 18 had a reported inconsistency relating to an inferred precision issue being addressed by the XBRL specifications working group.  This was deemed not to be a calculation inconsistency.)
• That left 106 with calculation inconsistencies reported.  Of those: 12 filers had only one inconsistency; 38 filers had two inconsistencies (usually the same issue in two periods); 8 filers had 3 inconsistencies; 18 filers had 4 inconsistencies, and 49 filers had more than 5 inconsistencies.
• The maximum number of inconsistencies was 34.

You can get a PDF or an Excel spreadsheet with this information from that summary page.

On the report the generator software used for creating the filing is provided, this comes from XBRL Cloud (as does the list of filings).

So, an obvious question was WHY the inconsistencies existed and where the inconsistencies were within the XBRL instance.

As to where, this is the breakdown:

• Various: This means that the inconsistencies related to a number of different areas.  There were 29 of these which inconsistencies impacted multiple areas. Here is an example.
• Stray calculations: There were 2 of these. What this means is that calculations were being fired because of concepts provided by the XBRL instance.  The network did not have a full set of concepts.  This needs more analysis to see exactly why this is occurring, usually this relates to problems in constructing the XBRL taxonomy. Here is an example.
• Income statement: There were 9 relating to the income statement.
• Balance sheet: There were 22 relating to the balance sheet.  This is a little surprising, the balance sheet is generally pretty easy because all the concepts have a balance attribute (i.e. debit or credit).
• Cash flow statement: There were 28 which had cash flow statement inconsistencies.  Generally, these look like polarity issues.  This is where the filer entered a positive number but should have entered a negative number (or the other way around).  This is usually because the preparer is obsessed with presentation of the number and does not seem to understand that for comparability reasons one way needed to be agreed on.
• Changes in equity: There were 2 of these.  Frankly I would have expected more.
• EPS: There were 12 relating to EPS (earnings per share for you non-accountants) not footing.  Now this is interesting.  Seems like this should definitely foot.  Not sure what the deal is.  But, it this is NOT supposed to foot, then why are the calculation relations provided?

It has been my observation over the years, and I see nothing which changes this in the calculation validation results that I see, that if a filer cannot get to the point where they cannot get rid of all the calculation inconsistencies it is typically because the filer refuses to grasp how XBRL works.  Basically, if the XBRL taxonomy says things should add up, then they should add up.  If things are not supposed to add up, then don't put the calculation relations in the taxonomy.  That is pretty straight forward.  Presentation of the numbers in a rendering for humans is a totally separate issue and has absolutely nothing to do with calculations.

In general what I see is pretty encouraging.  For 302 filers to get this right in the first filing period with the SEC not even checking these calculation relations is pretty favorable.  Add the 76 who had less than 5 inconstencies which were very similar issues, that brings the total to 378, which is 92 percent of the total.

Here is a PDF which walks you through a few calculations inconsistencies and shows you how to compare the calculation report with the SEC rendering.

If you look at the consistency by generator you see that every generator application both has filings with inconsistencies and without inconsistencies.

In order for automated reuse of the XBRL information provided by the SEC EDGAR system, clearly the data cannot have calculation errors of any sort. XBRL calculations are not sufficient to prove that all computations in an XBRL instance work correctly, see the discussion of business rules in other posts on my blog such as this one.

I am pretty confident that accountants who create XBRL filings will eventually realize that they have a gold mine here in XBRL, rather than just yet another SEC rule that they need to comply with. Pressing one button to make sure that everything adds up, rather then have to put on those green eye shades! What is not to like about that.  Further, using business rules to automate the disclosure checklists used to create a financial statement.  These are not liabilities folks, they are assets.  Pretty soon software will exist which shows this to be the case. Then it will be much easier for business users to understand why XBRL is so useful for not only external reporting, but more importantly for internal reporting.