Digital Financial Reporting

I include the automated SEC Edgar Filer Manual (EFM) rules within my 7 minimum criteria for making use of information contained in SEC XBRL financial filings. However, I am not quite sure how to actually make use of this information.

On the one hand, automated EFM rules are critical to the process of making use of any SEC XBRL financial filings.  That is why the rules exist.  So for example, you can determine the current balance sheet date of a filing by checking for the value of the concept "dei:DocumentPeriodEndDate".  So in that regard, these rules are critical.

On the other hand, many of the EFM rules relate to things like how to properly format HTML within a text block and things which have no impact on the fundamental use of information within an SEC XBRL financial filing.

And so, I don't really use this rule category in my evaluation process; I mainly just track the error count.  But, I do evaluate key factors which are, in fact, required by the EFM such as that "dei:DocumentPeriodEndDate" value.  I do that within a separate criteria category though.

Secondly, I get this information from XBRL Cloud who evaluates SEC XBRL financial filings on their EDGAR Dashboard.  The terms of my agreement with XBRL Cloud for my use of this information preclude me from providing detailed information about automatable EFM errors because XBRL Cloud charges for that information.

What I can say is this: The number of EFM errors has dropped significantly over the years and the current data which I get from XBRL Cloud shows that 97.9% of all SEC XBRL financial filings have zero automated EFM rule violations.

Something that is worth pointing out which can be confusing is why does XBRL Cloud show any EFM rule violations at all?  I mean, isn't the point of the EFM to document how the SEC will be evaluating SEC filings when they are submitted to the SEC?

Well it does not quite work that way.  The SEC does not do a complete job of testing against their own EFM rules.  Further, XBRL Cloud has different interpretations of those rules than other software vendors or does a more complete job of implementing rules.  And so, because there are cracks in the SEC's inbound validation and because software vendors don't collaborate and try and make their software interoperable; XBRL Cloud points out these inconsistencies between the SEC, other software vendors, and XBRL Cloud's implementation of the EFM.  This is unfortunate because it is confusing, but that is what is going on.

I will call this good for automated EFM rule validation necessary to make use of information in SEC XBRL financial filings.  The specific things necessary will be covered within the other criteria I use in my evaluation process.  But I will summarize my insights as follows:

Insight #1: Automated EFM rules are fundamentally necessary to drive the usability of any of that public company financial information contained in SEC XBRL financial filings.  However, many rules such as HTML formatting do not relate to the minimum criteria necessary to make use of this information.

Insight #2: When the process works correctly, the SEC specifies an automated EFM rule, the SEC inbound validation verifies filings against those rules, SEC submissions comply with that rule because of the inbound validation, and things work.  For example, every SEC XBRL financial filing submitted provides a concept dei:EntityRegistrantName and SEC validation during submission checks to make sure that concept exists and if it does not then the submission is rejected.

Insight #3: Cracks in the process such as different implementations and interpretations of automated EFM rules by the SEC different software vendors cause inconsistencies in SEC XBRL financial filings. If you think about it, if things worked correctly the XBRL Cloud EDGAR Dashboard would always show ZERO errors.

What appears to be one of the most boring of the seven minimum criteria for evaluating an SEC XBRL financial filingis the category consistent XBRL technical syntax. First off, it really is not that important because 99.9% of all SEC XBRL financial filings have consistent XBRL technical syntax.  It is also "technical". No one even wants to deal with the "technical" stuff related to SEC XBRL financial filings.

But if one is curious and looks a little closer at WHY the consistency of SEC XBRL financial filings at the XBRL technical syntax level is almost perfect; then one would learn some important and useful things.

And so how is that consistency rate of 99.9% achieved?  Well, it is achieved because XBRL software, or good XBRL software anyway, supports the XBRL 2.1 conformance suite and other XBRL conformance suites such as XBRL Dimensions, XBRL Formula, etc. A conformance suite is a set of tests which helps software agree on how the XBRL technical syntax works.  These XBRL conformance suites have hundreds of tests which prove that software interprets the XBRL specifications consistently.  The conformance suites help software inter operate.

A good question might be: "Hey, can we use something like a conformance suite to help increase the consistency in the other six of the minimum criteria, to make SEC XBRL financial filings more consistent and more usable by investors and analysts?"

And the answer to the question is "Yes!"  If rules are created which define how SEC XBRL financial filings should be created, the rules are published for creators of those statements to follow, then fewer errors would result.  This is particularly true if the rules were used by the SEC inbound validation process.

For example, today every SEC XBRL financial filing reports a fact "dei:EntityRegistrantName".  EVERY SEC filing has that reported fact.  Why? The submission process verifies the existance of that fact before the filing is accepted by the SEC.

Why can't a test such as "Assets = Liabilities and Equity" be part of that inbound validation process?  That test might not require a filer to report "Assets" or "Liabilities and Equity"; but if they did, the two values should be equal for each context in which they exist.

Why would this be a good idea?  Well because:

The only way a meaningful exchange of information can occur is the prior existence of agreed upon technical syntax, domain semantics, and process/workflow rules.

These sorts of rules are the key to high-fidelity information exchange and quality, useful public company financial information. Filers don't want to make mistakes.  But there are so many details that it is hard to get all the details correct unless a computer helps the accountants who are creating these digital financial reports.

Although, what is truly amazing is how correct accountants are getting the information even without a complete, comprehensive conformance suite which covers all possible categories of the minimum criteria.

Finally, by "minimum criteria" I truly mean minimum.  There are thousands and thousands of relations within a financial report.  Computer processes can watch over these relations.  And I don't mean XBRL technical syntax.  That is trivial.  In fact, that is already happening, thus the 99.9% consistency in XBRL technical syntax.  I mean domain semantics.  Getting the accounting correct.  Digital financial reporting will automate much of what is done when something like a disclosure checklist is completed.

So the XBRL technical syntax being 99.9% consistent is not boring, it is a clue as to how all of the other criteria will be watched over in the future.  And these minimum criteria are the tip of a much bigger iceberg.

Almost forgot.  There are only three filings in the set of 6674 which had XBRL technical syntax errors:

• U. S. Premium Beef, LLC
• Atlas Resource Partners, L.P.
• RiceBran Technologies

Why would the SEC's XBRL processor not detect these errors and reject the filing, but some other XBRL processor(s) report these errors? The XBRL consistency suites are not perfect nor does XBRL software vendors implement the XBRL conformance suites perfectly.  After all, humans are involved in that process and humans make mistakes. However, all that said; arriving at 99.9% consistency shows the value of a conformance suite.

This is the summary of insights:

Insight #1: SEC XBRL financial filings are consistent at the XBRL technical syntax level because of the conformance suites which help software understand and implement the XBRL technical specifications correctly.

Insight #2: Conformance suites can be used to create business domain rules such as "Assets = Liabilities and Equity".  These rules are totally up to the business domain. It is these sorts of business rules which produce high-fidelity information exchanges.

The latest information on report level model structure greatly improves what was available in my prior analysis. One incredibly insightful and useful thing I was able to achieve is to access every relation between every report element for all 6674 SEC XBRL financial filings analyzed.  You can see the details of that analysis here.

One mistake I was making which came to light is that I was evaluating filing rather than actual relations.  This is important because while the data shows that 95.8% of SEC XBRL financial filings had unambiguous relations (i.e. if only ONE relation was ambiguous, the ENTIRE REPORT was said to be in error); if you looked at the RELATION LEVEL, 99.9% of the millions of relations were unambiguous.

Here are the report level and the relation level summaries:

Walking through an example is helpful in understanding the report level model structure relations.  So consider the information for relations where the parent is an [Axis].  In the graphic below you can see that there are 386,912 [Axis] in the set of 6,674 SEC XBRL financial filings analyzed.  As expected, an [Axis] had ZERO Networks as children, ZERO Tables, ZERO Axis, 450,091 Members, ZERO LineItems....BUT HOLD ON...what do we see...11 Concepts as children of an Axis! What is up with that?

The point here is that an [Axis] has Members, which is defined as a report element with a type attribute equal to 'nonnum:domainItemType', as children, nothing else.  What does it mean if a Concept is a child of a Member?  Concepts are used to report facts.  Members are used to describe the value of an Axis. Concepts are not intended as children of an Axis.  So, how should this be interpreted?

Below is a summary of the 14 most potentially ambiguous relations found in a set of 6,674 analyzed SEC XBRL financial filings.  Of the 6,442,922 relations in SEC XBRL financial filings, a total of only 344 relations were deemed odd and the interpretation of their relationships was ambiguous:

• 1 SEC filers have an [Axis] as the child of a Network.
• 3 SEC filers have a [Member] as the child of a Network.  [Member]s are generally children of an [Axis] or another [Member]
• 1 SEC filer has a [Table] that is the child of another [Table].  What is the meaning of this?
• 25 SEC filers have a Concept as a child of a [Table].  [Table]s generally have [Axis] and [LineItems] as children.  What does it mean if a Concept is a child of a [Table]?
• 22 SEC filers have an [Abstract] as a child of a [Table].  [Table]s generally have [Axis] and [LineItems] as children.  What does it mean if an Abstract is a child of a [Table]?
• 11 SEC filers have a Concept which is a child of an [Axis].  Only [Member]s should be children of an [Axis]
• 137 SEC filers have a Concept which is the child of a [Member].  Generally [Member]s are the only children of other [Member]s
• 1 SEC filer has an Abstract as the child of a [Member]. Generally [Member]s are the only children of other [Member]s
• 45 SEC filers have a [Table] as a child of a set of [LineItems].  Generally [LineItems] are children of [Table]s, not the other way around.
• 3 SEC filers have an [Axis] which is child of a set of [LineItems]s.  Generally Concepts and [Abstract]s are found in 99.99% as children of a LineItems of SEC filings.
• 1 SEC filer had a [Member] which is the child of a Concept.  How should this be interpreted?
• 2 SEC filers had [LineItems] which were the child of a Concept.  How should this be interpreted?
• 20 SEC filers had an [Axis] which is the child of an [Abstract].  How should this be interpreted?
• 72 SEC filers had a [Member] as the child of an [Abstract]. How should this be interpreted?

There are other illegal and odd relationships. For example, while 99.4% of Networks have a root child concept report element which is of the category Abstract; a handful use a Table as the root element (1261), or a Concept (46).  This may not be ambiguous.  But 183 use LineItems as the root concept.  LineItems are general used within a Table, but again; the information might not be ambiguous.  But 1 SEC filing as an [Axis] as the root and 3 have a Member as a root.  What the heck does that mean?

The point is that while there are 817 anomalies, the 344 relations summarized above are the most odd. However, it would be better if none of the 817 issues existed.

Identifying these report level model structure relations and attempting to extract this information from SEC XBRL financial filings provide a number of noteworthy insights:

• Insight #1: The report elements which make up an SEC XBRL financial filing can be grouped into categories: Network, Table, Axis, Member, LineItems, Concept, Abstract. This report element categories have relations with other report element categories. The vast majority of relations between report elements in SEC XBRL financial filings, 99.9%, and the vast majority of filings themselves, 95,8% of all such filings, follow these relations.
• Insight #2: Not following these report level model structure relations leads to potentially ambiguous representations of such digital financial filings.

The latest information related to fundamental accounting concepts is consistent with the prior analysis set. This latest information is based on a 100% automated assessment of 6,674 fiscal year end public company financial filings (10-Ks) submitted to the SEC between March 1, 2013 and February 28, 2014.

This is a summary of the results:

• 51 fundamental concepts were sought
• 21 relations between those concepts were tested, these relations are never expected to change
• On average 97.8% of reported facts within SEC XBRL financial filings fit within these concepts and relations (in the prior year's analysis it was 97.9%)
• For the total of 21 tests, 18 tests have passing rates between 94% and 99%
• For the total of 21 tests, 3 tests have passing rates between 87% and 89%
• For each of the 21 tests, there are specific reasons an SEC XBRL financial filing does not pass the test, for example this document explains the specific reasons 29 filings do not pass test B2 (Assets = Liabilities and Equity)
• At least 1,281 or 19% of all 6,674 SEC filings analyzed report all 51 fundamental accounting concepts and adhere to the 21 relations between those concepts
• This Excel application let you review all filings which pass and fail these tests.

This graphic breaks down the results into categories: (click image for larger view)

Click image for larger view

Identifying these fundamental accounting concepts and attempting to extract this information from SEC XBRL financial filings provide a number of noteworthy insights.

• Insight #1: If an SEC XBRL financial filing does not pass one of these tests exactly one of three things is incorrect: (a) the SEC XBRL financial filings, (b) the test, or (c) the software algorithm used to obtain the reported information.
• Insight #2: These fundamental accounting concepts and the relations between the concepts provide a "skeleton" into which all information must fit. If each of these fundamental concepts cannot be either specifically identified as being reported or imputed based on other information then the report is at best untrustworthy and likely ambiguous and at worst unsafe to use or completely unusable.
• Insight #3: Each and every one of these fundamental accounting concept and relations between the concepts appears necessary. What I mean by this is that a test of a relation cannot simply be dropped.  For example, I was tempted to drop the three tests which had passing rates of less than 90%.  But you cannot do that because if those relations are not intact the "skeleton" will fall apart.  The tests appear to be a set of simultaneous equations or linear equations which are tightly related.
• Insight #4: Related to insight #2, when an SEC XBRL financial filing uses a concept intended for one fundamental accounting concept category into some other category, bad things happen. Using the skeleton analogy it is like if someone takes a bone which was intended to be on the hand and uses it on the foot, things break. What this means is that categories of concepts should not be crossed.
• Insight #5: Understanding the reasons why SEC XBRL financial filings reported facts cannot be discovered is important for establishing both specific an general rules for reporting and consuming reported information to keep using reported information from being a guessing game.  For example, this document shows that of 29 SEC XBRL financial filings which have balance sheets that don't balance, 9 were caused by what could be considered "rounding errors" (i.e. perhaps an acceptable tolerance). This begs the question: Are rounding errors allowed?  6665 SEC filings don't have such rounding errors, are these 9 acceptable?  Are rounding errors, or tolerances, acceptable or appropriate in other areas or is it best that these rounding errors be eliminated when reports are created (as most filers seem to do)?

Check out all the data for yourself, see what conclusions you come up with. If you come up with additional insights please let me know what you come up with.

Here is a comparison of the fundamental accounting concepts of the DOW 30 provided by SECXBRL.info.

NOTE: Subsequent to this post I detected an error in a query and obtained more reliable information for XBRL technical syntax and EFM errors.  These changes yielded in increase in the number of digital financial all stars to a total of 1,281.  Supporting information has been updated to reflect these updates.

Using digital financial reports should not be a guessing game. I spent almost a year poking and prodding SEC XBRL financial filings (fiscal year 2012 10-Ks) trying to make use of the information these digital financial reports contain.  I arrived at a set of minimum criteria for making use of any reported information. There was one goal: safe, reliable, predictable, automated reuse of reported financial information.

I used these criteria against 10-Ks filed with the SEC between March 1, 2013 and February 28, 2014, basically for fiscal year 2013. There were a total of 6,674 SEC XBRL financial filings that I worked with after deleting a handful of trusts, funky CIK numbers, and a few other odd but rare things.

Of the total of 6,674 SEC XBRL financial filings, there were 915 digital financial reporting all stars which meet all seven of the minimum criteria. These digital financial reporting all stars represents about 14% of all SEC XBRL financial filings.

For the curious, those who want to see for themselves and those who might want to reverse engineer what is going on, I provided some additional stuff.

All the data for these SEC XBRL financial filings is made available in this ZIPPED Excel file. Also, another ZIPPED Excel file provides an application which extracts the fundamental accounting concepts from the SEC XBRL financial filing. A list of the digital financial reporting all stars which provides for safe, reliable, predictable, automated reuse is provides so you can check those out.

Also provided are lists of each filing which does NOT meet one or more specific criteria. This allows one to see the impact on trying to use this information.  Plus, I have provided one analysis thus far (I will provide more) to show the specific reasonsSEC XBRL financial filings do not pass these minimum criteria. This first analysis is basic, test BS2 - Assets = Liabilities and Equity (i.e. balance sheets balance). The analysis captures 8 specific reasons why balance sheets do not balance in SEC XBRL financial filings.

What is wrong if an SEC XBRL financial filing does not pass one of these rules?  There are exactly three possibilities:

1. The SEC XBRL financial filing has a mistake and should be adjusted.
2. The minimum criteria have a mistake, a rule is wrong, and the rule needs to be changed.
3. The software algorithm has a mistake and the algorithm must be changed to correct for the mistake.

What is the point? For every issue one of those three things needs to happen to bring the system into equilibrium, to bring the system into balance. Which is it? Which needs to be changed?  Well that is up to the financial reporting supply chain to determine.

While there are 915 digital reporting all stars which offer state-of-the-art digital financial reports, these numbers are even more interesting:

• 915 SEC XBRL financial filings have 0 errors as pointed out about, they are all stars. (This was revised upward to 1281)
• 2250 SEC filings have 1 error. (revised to 2293)
• 1382 have 2 errors.
• 714 have 3 errors. (revised to 700)
• 465 have 4 errors. (revised to 433)
• 275 have 5 errors. (revised to 255)
• 5086 SEC XBRL financial filings are 5 of fewer errors from becoming digital financial reporting all stars. That is 76% of the total.  (That would give us 14% plus 76% or 90% all stars.) (revised to be 6344 and 95% of filings have 5 of fewer errors)
• 66 SEC XBRL financial filings have between 10 and 45 errors. (revised to be 330 with between 6 and 316 errors)

There are a total of 13,181 specific errors/issues/anomalies, call them what you want. That is an average of about 2 per SEC filing.

Granted, this is not saying that correcting these 13,181 issues will create 100% correct SEC XBRL financial filings, that is not what I am saying.  This is only a minimum threshold level of information use.  But it will create a nice beach head: primary financial statement information for the default legal entity.

In my view establishing this beach head allows for two important things: (1) A base of information use, (2) a path forward, building on that base.