Digital Financial Reporting

By narrowing the focus of my analysis to commercial and industrial companies only (i.e. excluding banking and savings entities, insurance companies, brokers/dealers, real estate) and dropping two tests which I just cannot get get a high pass rate because of the variability within SEC filings; I was able to achieve a 98.4% accuracy rate.  Basically, 4,048 of 7,160 filings fit this criteria (57% of total filings).  In addition to that narrower set of 4,048 filings passing 98.4% of the tests, a total of 1,113 filings pass 100% of the tests.

Now, narrowing the testing to commercial and industrial companies seems very reasonable.  All I am saying by this is that if the total of 7,160 reporting entities were broken into more specific groups of reporting entities and targeting my information extraction algorithm to specific groups of reporting entities the algorithm will be more accurate.  Clearly it is easy to grasp why this is true.  For example, banking and savings entities report their revenues in different ways than commercial and industrial companies.

However, I don't know whether I should just be discarding two tests which I know should work.  The two tests are the following:

• Gross Profit = Revenues - Cost Of Revenue (IS1)
• Operating Income (Loss) = Gross Profit - Operating Expenses + Other Operating Income

The reason that SEC filings don't pass these tests is not that they don't follow the spirit of these equations.  Rather, they don't pass because I cannot find the facts which report that fundamental concept.  For example, Revenues is found for only 5,640 of the filers which is 79% of the total.  That is because of the variability of the US GAAP Taxonomy concepts used to express this fact.  I speculate that I can probably improve the success rate if I focused on trying to find more of the concepts used.  But, that may not be true.

Also, if I eliminate developing stage companies and companies which have going concern qualifications, then I get 97.3% accuracy for the overall set of 7,160.

Hey software developers! Want to create a nice little business?  Build tools which help accountants do digital financial reporting, such as SEC XBRL financial reports.  There is no need to even understand XBRL. XBRL Cloud has a nice little web service which hides XBRL in the background. There are other XBRL processors out there such as Arelle (open source), Semansys, ReportingStandards, CoreFilings, and others. Pick your poison.

Me, I used XBRL Cloud to create this working prototype tool using Excel.  Granted, I am not a programmer (i.e. that is why I am encouraging you to consider building tools).  But take a look at what I was able to do in a matter of about 10 minutes using my tool: compare the balance sheets of the most current 10-K (on the LEFT side) of a reporting entity with the prior 10-K (on the RIGHT side).  Here is the result:

• Apple: Note how they went from an extension concept to a US GAAP taxonomy concept for PPE.
• Boeing: They also went from an extension to a US GAAP taxonomy concept, a few other interesting things to notice.
• Home Depot: Moved the detail of PPE off the balance sheet, likely to the disclosures.  Tuned their inventory concept.
• IBM: Went from using an explicit [Table] to using an implied table.
• Intel: Looks like they have no accrued taxes in the current period.
• McDonalds:  Changed their network identifier for some reason.
• Microsoft: A few changes.
• Walt Disney: Interesting line item, Television Costs.

Easy to use, nothing really technical about this, most of XBRL is hidden in the background.  Accountants can relate to this.

Grab my Excel prototype, it works and all the code is there.  Clearly my code is not the best (not much error checking, nothing sophisticated, perhaps not efficient).  If you even just improve this working prototype to make it more stable, make it work for ANY SEC filing, make the comparison work to compare a pre-submission filing, or even better; compare the RENDERING (i.e. the model structure AND the facts not at the syntax level but rather at the SEMANTIC level) you will have created a very, very useful product.

Accountants need these niche tools for specific tasks like this.  Sure, business report creation tools are important but we need utility tools like this also. And this comparison utility is just one idea; I have hundreds of others. 

Digital financial reporting is just getting started, the window of opportunity is open. For some things, sure, you need to understand XBRL.  For others, it is far more important to understand the accounting aspects.  Don't understand accounting?  Ask an accountant.

If you create anything interesting please let me know.

In a prior blog post I summarized what I see that accountants need in order to review an SEC XBRL financial report.  I also provided more details on this topic in Chapter 10: Verification of Digital Financial Reports of Digital Financial Reporting.

This bog post assumes that you actually care to do a good job in the creation of your digital financial report, such as an SEC XBRL financial filing, rather then simply comply with the SEC mandate.  An XBRL-master craftsman understands that eventually "comply" will evolve to mean more that you can get away with today.  Buying inadequate software today can lead to problems down the road when the SEC tightens the screws when it comes to quality. You don't want to fall into this trap.

This blog post updates that information and tries to provide examples of software which performs these tasks.

Verification that a digital financial report such as an SEC XBRL financial filing is a true and fair representation of your financial information will always involve manual and automated processes.  It is not possible to automate 100% of this verification process.

Where the process can be automated by the creation of business rules which computer software can enforce depends on two things: (1) the probability that a computer can automate some process, (2) the existence of the rule so that aspect of the verification process can be automated.  Basically, no rule = no ability to automate.  Also, automation saves time and money plus increases quality. Full stop.

The best software will be both invisible to the user of the software but assist the user understand exactly what they are responsible for.  Creation of a digital financial report cannot be a "black box" or where a business user does his or her best and hopes the software will fulfill their obligation and meet their legal responsibility.  No paper-based financial report would ever have been released under those terms. Once accountants are held accountable, such as by the removal of the limited liability exemption by the SEC; they will care as much about their XBRL-based financial report as they do about their HTML or paper based report with which accountants are more comfortable.

The best digital financial report creation software will both assist the creator of the financial report to verify that all aspects of that financial report they are creating is, in fact, correct during the creation process within the creation software used to create the financial report.  The business user will have complete transparency into whatthe software is doing in the way of automated testing processes so that the manual processes can be established and integrated into one complete set of verification steps.

Other verification software will be stand-alone and be independent of the actual creation process. For example, it is likely that internal and external auditors will use software which is not integrated with the creation process.

In either case, all the automated and manual steps need to be organized and managed by one software interface rather than be a hodge-podge of different software and processes which causes an increased likelihood of things falling through the cracks.

I am not going to go into details about what needs to be verified.  You can get that information in this blog post, Understanding Quality of SEC XBRL Financial Filings. The XBRL Cloud Edgar Dashboardalso provides a fair amount of what is necessary, beyond what inboud SEC validation catches. But it still lets things fall through the cracks. Few commercial products today do things like verify the fundamental accounting concepts and the relations between those concepts. Eventually, meaningful reuse of all that public company information will drive verification requirements.

The following is a summary of functionality which is necessary within software designed to assist a business user in the process of verification of a digital financial report, such as an SEC XBRL-based financial filing, tells the story that the reporting entity truly wishes to tell:

• Technical syntax validation. Digital financial reports are ultimately expressed using some technical syntax.  For example, SEC XBRL-based financial reports are formatted in the XBRL technical syntax which uses XBRL 2.1 and XBRL Dimensions 1.0.  Filers might also choose to use XBRL Formula to express and test business rules.  As such, verification software clearly needs to be able to provide feedback to the user that XBRL which has been created complies with the XBRL technical specifications.  Such software should be shown to pass the XBRL 2.1, XBRL Dimensions 1.0, and the XBRL Formula conformance suite tests. As 100% of technical syntax verification can be automated, the business user really should never need to manually verify technical syntax.  Creation software should simply create nothing other than proper XBRL.
• Automated and manual SEC Edgar Filer Manual (EFM) rules validation. The SEC EFM places specific additional technical syntax restrictions on how you are allowed to use XBRL. The EFM also has a number of semantic rules and restrictions.  Business users will need to verify that their SEC XBRL-based financial report against these rules. Many EFM rules can be automatically verified but others need to be manually verified.
• Report logical structure verification. The US GAAP taxonomy specifies how to construct the model you use to represent your financial report. It uses report elements such as [Table]s, [Axis], [Member]s, [Line Items], networks, concepts, and abstracts in specific, logical and sensible ways. (See section 4.5 of the US GAAP Taxonomy Architecture for more information.)  Software must be able to help business user not create illogical, nonsensical, inconsistent, or ambiguous representations which can lead to misinterpretation of their digital financial report.  For example, inconsistencies between the XBRL definition relations, calculation relations, and presentation relations can cause illogical and misinterpret relations.  Software can easily verify that you are creating logical, sensible, consistent, and unambiguous information using automated testing.
• Business rules engine for processing business rules.  XBRL Formula is one type of global standard rules engine which can be used to express business rules which can be used to verify both mathematical type relations and other types of relations expressed in your digital financial reports. Using RDF and SPARQL is another approach. A business rules engine of some type is necessary because the relations expressed in digital financial reports go beyond the simplistic relations verified by XBRL calculations.  As such, digital financial report creation and verification software needs to provide this functionality.  Alternatively, creation software might take the approach of generating business rules from the actual representation which is created at the time of its creation, not allowing business users to make mathematical errors. CoreFilings Magnify and SpiderMonkeycan be used in combination to create busines rules using a proprietary syntax Sphinx. You can get additional information about business rules engines here.
• US GAAP domain level and industry/activity level rules validation. US GAAP had both domain level business rules, fundamental accounting relations which never change, which every digital financial report must follow, such as assets = liabilities and equity, and industry/activity level business rules, such as commercial and industrial companies must provide classified balance sheets which report current assets and current liabilities.  Verification software and creation software should support these sorts of business rule.
• Reporting entity specific business rules. Every digital financial report will have its own reporting entity specific business rules.  As such, creation software and/or verification software must enable the business users creating these digital financial reports to create and otherwise manage their unique business rules.  Alternatively, creation software could auto-generate these business rules.  Complex and difficult to use technical tools will not meet the needs of business users. Internal auditors and external auditors will also likely have additional business rules.
• Understandable and otherwise appropriate views of financial report semantic objects, relations, properties. Software needs to provide views of report element and their properties in ways that are understandable to business users.  XBRL technical syntax oriented views of report objects is not appropriate and unnecessary.  Business meaning is what is important to business users, not technical syntax which they will never understand, nor should they need to understand.  Business user oritented semantic views, are what is necessary.  The many views necessary must support both automated and manual verification of information within the SEC XBRL financial report.  The two best examples of understandable views is the XBRL Cloud Viewer and the XBRL Cloud Evidence Package. (You can see any SEC filing in the XBRL Cloud Viewer from the XBRL Cloud Edgar Dashboard.)
• Understandable and well organized navigation between financial report objects, relevant properties, and relevant relations. Financial report pieces are related to other pieces and have properties which business users need to examine.  As such, appropriate navigation between the sometimes thousands of report objects, their relations and their properties is necessary. Again, the report objects necessary are the ones which provide meaning to business users, not the technical syntax objects.  Mixing semantic type objects and syntax type objects is likewise inappropriate. A well-organized interface both exposes and leverages the intersections between the many financial report pieces.
• Understandable, sensible semantic renderings. For renderings of the information to be understandable, they must be shown correctly to the business user. There are exactly two reasons for bad renderings: (1) bad models, (2) bad rendering engines.  If the rendering engine is good, then the only reason for a bad rendering is a bad model which can be both identified and then fixed.  One thing most rendering engines do not do well at this time is leverage the well known, common accounting concept arrangement patterns being represented. Roll ups, roll forwards, adjustments, variances, hierarchies, and other accounting concept arrangement patterns have characteristics which software vendors could leverage in their rendering software to improve what business users see and have to therefore work with. Also, all information should be shown by software, not just some information.  Appropriate semantic renderings are a basis for appropriate views of information, appropriate navigation between components, and the ability to effectively verify a digital financial report. Again, the XBRL Cloud Viewer has a very good rendering engine; not perfect but better than anything else I have run across. (Try looking at the Apple balance sheet using the XBRL Cloud Viewer.)
• SEC interactive data rendering. While it is much more likely that an SEC XBRL financial filing will be viewed within software other than the SEC interactive data viewer; business users still want to understand how their filing will look on the SEC web site when rendered by the SEC interactive data viewer. As such, verification software should include an SEC interactive data rendering within the verification software.
• Support for tracking/managing both automated and manual validation tasks and steps. Not all verification tasks or steps can be automated.  As such, verification of a digital financial report, such as an SEC XBRL-based financial filing, will always be a combination of automated and manual tasks/steps.  As such, software supporting a business user in the verification process needs to help the business user manage both automated and manual verification tasks/steps.  Automated rules can be managed by just correcting mistakes which makes the automated test not result in an error.  Manual steps are similar to the "to dos" or open items which must be manually resolved and tracked to be sure any issues have been dealt with appropriately.
• Comprehensive and useful set of verification reports and appropriate verification evidence package. Business users need to be able to print many of the same views provided by software applications used to visualize a digital financial report.  Quantity of reports is not what is important, quality is what is important.  Well-thought-out and well organized reports which can be used both for verification of digital financial reports and for providing a historical archive, or evidence package, for a financial filing. Again, the XBRL Cloud Evidence Package provides a good example of useful reports. For example, take a look at this report which allows a business user to compare the labels within an SEC XBRL financial filing.
• Transparency into what the software is automatically verifying so manual verification work can be properly planned. Verification of a digital financial report such as an SEC XBRL-based financial report should not be a "black box". At the very least, business users need to understand exactly what automated verification steps software performs so that they can properly plan their manual tasks/steps required to supplement automated verification. As such, business users need to be able to see the specific automated verification rules software is performing. Today this is of particular importance as software may perform different sets of validation rules.
• Comparison between multiple versions of a report in order to understand differences. A necessary feature of a digital financial report creation or verification application is to manage last minute changes safely.  The ability to perform automated comparisons between different versions of an SEC XBRL financial report to understand changes between the two versions of the same report is crucial. For example, if you did extensive work in verifying your SEC XBRL financial report and then there are a few last minute changes which need to be made, how can you be sure some other change was not unintentionally or maliciously introduced? Here is information about a prototype comparison utility which I created. Take a look at this screen shotwhich compares the model structure of CostCo and Walmart. Imagine being able to compare the 10 AM version of a filing with the 11:30 AM version for ANY changes.  How useful would that be? Comparisons between versions of the same report and between a reporting entity and its peers are both very useful.
• Managing workflow.  Creation of a financial report is a set of tasks which could involve a specific workflow.  Managing the workflow of creating a financial report can be beneficial to users, but is not absolutely required from a software application. For enterprise users though, this is a must-have feature.
• Collaborative, multi-user. Creation of a financial report is a collaboration.  Verification is likewise a collaboration.  Although not required by everyone; the ability to effectively collaborate with others during the verification process can be a desirable feature to some, a required feature for others.

Regretfully, there is no software that I have seen which fits this bill.  As such, most digital financial report creation processes have holes in them which result in error found in SEC XBRL financial filings. But eventually, good software will appear which provides a formal framework which leads the user through defined processes with the rigor to help them prove to themselves that their financial report is a true and fair representation of their financial information and that the financial information tells the story that they are trying to tell.

My first attempt to test a fundamental set of accounting concepts and relations showed an accuracy rate of 95%. The second attempt tuned the algorithms and metadata and increased that accuracy rate to 96.6%. I have further tuned the algorithms and metadata and have achieved an accuracy rate of 97.3%. You can see the detailed computation here.

Now, I don't know if I am computing this correctly because there are two tests, IS1 and IS2, which do not apply to all filers.  However, I am leaving these tests in the computation because they do point out two problems: finding revenues reliably and differentiating nonoperaing income (loss) and interest and debt expense.

Want to try an Excel-based version of the tool used to grab this information from SEC XBRL Financial filings?  Well, here you go.

• Download working prototype: This is an Excel-based application written in VBA which grabs information from my set of 7160 SEC XBRL financial filings (10-Ks).
• VBA Code: This is the VBA code which grabs the information from the flings.  (Please don't use this to judge my programming skills, rather use it to reverse engineer the process.
• Sample output: This is sample output of the code which generates results directly to the debug window rather than populating the Excel spreadsheet. (Point is, there are two algorithms)
• Python code: Someone is converting my code into Python.  You can get to that code here.
• Raw data: Here is the raw data in Excel which was created using the same algorithm applied to each of the 7160 SEC 10-K filings.
• Prototype Microsoft Access Database Application with Data: This is a prototype database application created using Microsoft Access which provides an interface and metadata for working with the fundamental accounting information for the set of 7160 SEC filers.

Here is a screen shot of the Excel prototype populated with the data for Hewlett Packard:

Excel Interface

 Here is a dump of other information which I learned from this process:

• Of the set of 7160 SEC 10-K filings, 1071 (15%) pass all checks.  Here is the list.  The list is also provided in the Excel prototype.
• If I leave out checks IS1 and IS2, then 3325 (47%) of all SEC filers pass all of these checks.
• There are two areas which stick out has challenging to grab information: revenues and differentiating nonoperating income (loss) and interest and debt expense.
• Services which provide information which rely on only the XBRL concept will not generally allow those who want to use the information create useful comparisons.  The reason is that there are many different concepts which are used to express the same fundamental accounting concept.  Look through the algorithm which grabs the data.  Often, there are alternatives which filers are using to express key information, for example net income (loss) could be any of 7 different concepts.
• At this level of information, the vast majority of SEC filings are comparable.  Some are not.  For example, a filer providing a statement of net assets rather than a balance sheet does not totally fit into this representation. Situations like that are just additional edge cases which need to be provided for.

This is a boatload of information!  If you have good reverse-engineering skills, all these samples provide ideas on how accountants can make use of this financial information of public company SEC XBRL filers.

I am trying to get commercial software vendors interested in creating tools such as this for accountants and other business uers.