The following is a summary of verification/validation results for a set of 7,199 SEC XBRL financial filings. All filings are 10-Ks for the fiscal year 2012 filed with the SEC between March 1, 2012 and February 28, 2013.
Here is some information helpful in understanding these validation/verification results:
- XBRL technical syntax. This information comes from the XBRL Cloud EDGAR Dashboard. Basically 99.9% of all SEC XBRL financial filings in the set of filings I looked at were valid per the XBRL technical syntax specification. What is a little odd is that there were 8 filings which were not valid. What that means is that there are some XBRL interoperability issues. The XBRL validator(s) used by the SEC for inbound validation did not detect the 8 filings as not being valid against the XBRL specification. XBRL Cloud believes that there are XBRL technical syntax errors in these 8 filings. I validated each one against another XBRL processor and sure enough, I found errors in each of the 8 filings reported by that XBRL processor also. As such, I would concur with XBRL Cloud's assessment. (I also posted a message to the XBRL Specification Working Group informing them of these errors. Hopefully a conformance suite test will be added for each of the offending errors, which would improve consistency between XBRL processors.
- Edgar Filer Manual (EFM) automatible rules. Again, this information comes from XBRL Cloud. Why would XBRL Cloud's EFM validation differ from SEC EFM validation? Mainly because of different interpretation of the rules by software vendors (i.e. the ones with the errors), the SEC, and XBRL Cloud. It would be very helpful if all software vendors and the SEC reported the same consistent results. This is getting better, but still has a ways to go given that almost 20% of all filings have EFM rule violations. And these are only the automatable rules.
- US GAAP Taxonomy Architecture Model Structure. These rules relate to the relationships between [Table]s, [Axis], [Member]s, [Line Items], [Abstract]s, and Concepts within an XBRL taxonomy. They also relate to a degree to differences (i.e. ambiguity) between presentation, calculation, and definition relations. These relations are explained in the US GAAP Taxonomy Architecture (see section 4.5, Implementation of Tables). Of all the 10-K filers, 98% follow these rules.
- US GAAP Domain Level Rules. These are a small set of rules (i.e. there are many more) which every SEC XBRL financial filing must follow: assets must be reported, liabilities and equity must be reported, assets = liabilities and equity, equity must be reported, net income (loss) must be reported, and net cash flow must be reported. Why are these rules true? Well, given that 96% of all filers follow these rules and that every filing which I looked at clearly did NOT follow the rule (but should have); it is pretty safe to say that all filers should pass these rules. That is what "domain level" means, US GAAP specifies these sorts of things. Ever hear of the accounting equation: Assets = Liabilities + Equity?
- Balance sheet roll up rules missing. Balance sheets have "assets" and "liabilities and equity". Balance sheets also balance. "Assets" adds up, or "rolls up". Same for "Liabilities and equity". 93% of all filers provide business rules in the form of XBRL calculations which are used to prove that the roll ups are working correctly. 7% did not. If the rules are provided, then automated processes can be used to prove that the line items of the balance sheet roll up. If they are not provided, you can only resort to manual error to make sure things add up. A clue as to which process might work best: automated processes save time and money.
- Income statement roll up rules missing. As with the balance sheet, the income statement rolls up. It foots. Part of the reason why there are so many missing XBRL calculation relations is that filers put numbers in their XBRL filings backwards because they try to get the polarity (positive or negative) to match the presentation. I speculate that they give up in frustration or believe that they just cannot get them to add up correctly. A clue that you CAN make these add up is that about 80% of all filers seem to be able to provide these XBRL calculation rules.
- Cash flow statement roll up rules missing. Again, same deal for the cash flow statement; the line items add up and filers seem to be having problems with the polarity.
This is just a small set of the automatable rules which one can test for. Why would you do that, seems like more work? Well, it is actually less work. Automating the testing of correct modeling, computations, and other rules and relations makes it easier to prove to yourself that you did a good job creating your SEC XBRL financial filing.
As the automated rules grows the quality of SEC XBRL financial filings will improve. Why? Click here.