BLOG:  Digital Financial Reporting

This is a blog for information relating to digital financial reporting.  This is my brain storming platform.  This is where I think out loud (i.e. publicly) about digital financial reporting. It is for innovators and early adopters who are ushering in a new era of digital financial reporting.

Much of the information contained in this blog is synthasized, summarized, condensed, better organized and articulated in my book XBRL for Dummies and in the chapters of Intelligent XBRL-based Digital Financial Reporting. If you have any questions, feel free to contact me.

USAspending Data Lab

USAspending has published a BETA DataLab page. They also have an API but I cannot figure out how to use it.  It is bound to get better.  It will be interesting to see what people come up with.

Posted on Wednesday, April 18, 2018 at 09:12AM by Registered CommenterCharlie | CommentsPost a Comment | EmailEmail | PrintPrint

Logic's Role in Digital Mirror Worlds Representing Reality

In their paper Imagineering Audit 4.0 (figure 3 on page 10), Jun Dai and Miklos Vasarhelyi of Rutgers University use the term mirror world to describe the use of technology to create a virtual copy of the real world. A mirror world is defined as an ''informationally-enhanced virtual models or reflections of the physical world.''

The transaction cycles of a company and the interaction between the systems of the company and its customer, supplier, and bank are shown in the physical world on the bottom and the mirror world on the top in the cloud.

(Click image for larger view)

The graphic describes the relationships between inter-business parties in a supply chain and represents the connections between the real world in which such businesses operate and the mirror world within the accounting information systems of an enterprise resource planning (ERP) system of those economic entities.

These mirror worlds exist today in the databases that hold transaction information.  Within a company there tends to be fairly good integration.  Between companies in a supply chain is a different story.

The extent that the real world can be represented within some mirror world and automation can be achieve is open for debated.  However, what cannot be debated is that the real world and the mirror world need to use the same logic to explain each world.  Said another way, the business logic of the mirror world needs to match the business logic of the real world.

Logic a set of principles that forms a framework for correct reasoning. Logic is a process of deducing information correctly.  Logic is about the correct methods that can be used to prove a statement is true or false.  Logic tells us exactly what is meant.  Logic allows systems to be proven.

A logical statement, or a statement of fact, is a sentence that carries factual information that is either true or false. A statement is a piece of information.  A conditional statement is a type of statement that has an IF…THEN type format. Look at a statement as being a piece of information that is either correct or incorrect. A statement, a statement of fact, and a fact mean the same thing. (There are two assumptions that are held to be true.  First, a statement MUST be either true or false.  Second, a statement cannot be both true and false at the same time in the same context.)  Questions, commands, and opinions are not logical statements.

A model, or conceptual model, describes a possible world.  There exists some set of all possible models that can be used to describe real worlds that could exist.

A rule, or business rule or assertion, is a true statement with respect to some model of the real world that could possibly exist. (i.e. you cannot create rules that are true in worlds that can never exist)  A rule can be a mathematical expression.  A rule is a type of logical statement.

Logical entailment, or logical consequent, is when a logical statement follows from another statement or set of statements. Another name for logical entailment is inference.  The rules of inference provide a system in which we can produce new information (statements) from known information (statements).

A statement has a truth value with respect to some model.

The connectors AND, OR, and NOT are used to combined single statements to create compound statements.

Logic helps us understand the meanings of statements and produce new meaningful statements. Logic is the glue that holds strings of statements together and pins down the exact true unambiguous meaning of those statements.

Logic is the process of deducing information correctly; logic is not about deducing correct information.  Suppose you were give incorrect information.  Your statements can be false but the reasoning behind conclusion sound.  Other means are necessary to determine if a statement is true or false.

Facts, rules, and the possible models they represent can be stored in a knowledge based system if the facts and rules are put into machine-readable form. The machine-readable system can apply a problem solving logic using a problem solving method of a business rules engine to solve problems that normally would require human effort and thought to solve. The knowledge based system supplies an explanation and justification mechanism to help system users to understand the line of reasoning used and support conclusions reached by the machine-based knowledge based system, presents that information to the user of the system, and provide an audit trail. (i.e. how the system works cannot be a black box).

The fact database and knowledge base can be employed to effectively automate certain tasks involved in accounting processes and financial report creation processes.

The knowledge based system keeps track of things.  You can TELL the knowledge based system facts or rules; you can ASK for facts that exist in the knowledge based system or new facts that can be inferred using the rules of logic, existing facts, and existing rules. For example,

  • TELL statement "Current assets for the economic entity ABC Company as of December 31, 2017 is the amount $200,000 (USD)."
  • TELL statement "Assets for the economic entity ABC Company as of December 31, 2017 is the amount $1,000,000 (USD)."
  • TELL statement "For commercial and industrial companies; Assets = Current Assets + Noncurrent Assets."
  • TELL statement "ABC Company is a commercial and industrial company".
  • ASK fact "What is the amount of Noncurrent Assets for ABC Company, a commercial and industrial company, as of December 31, 2017 in USD?"

The notions of TELLING and ASKING are guided by the rules of logic.  To understand the importance of these rules of logic, think about how such a system would work without such rules.  The result would be anarchy.

It is the rules of logic and the ability to represent facts and rules in machine-readable form but also in human-readable form that can enable the creation of a digital mirror world that can be used to automate accounting and reporting processes. Some call this "robotic finance" .  Others call this "machine intelligence". Still others refer to this as the "artificial intelligence revolution".

It is important to remember that logic is the process of deducing information correctly; logic is not about deducing correct information.  Deciding which information is correct and which is not correct is the responsibility of the model you choose to make use of to describe your reality.

Digital mirror worlds can be created that leverage the nature of double-entry bookkeeping using the XBRL technical syntax.  To do this, one needs to consider computer empathy to understand how to do this and the extent that accounting process automation can realistically be achieved."


It was pointed out to me that another term to describe a digital mirror world is "digital twin". For more information see this video introduction to digital twin, this Wikipedia description, or search on "digital twin".

Posted on Saturday, April 14, 2018 at 05:04PM by Registered CommenterCharlie in | CommentsPost a Comment | EmailEmail | PrintPrint

Separate the Wheat from the Chaff: More Brainstorming on Implementing Robotic Finance

Here is more information related to implementing robotic finance and an updated version of my document Guide to Implementing Robotic Finance.

I think that Inline XBRL will help enable a transition from current accounting and financial report creation processes and robotic finance. In addition, there is a boatload to be learned form the Inline XBRL financial reports of public companies that have been submitted to the SEC. While it can be a challenge today to "separate the wheat from the chaff", here is why I think what I think.

First, you have to "look around" the quality issues, the information representation issues, the issues with the US GAAP XBRL Taxonomy, and other factors related to the XBRL-based financial reports that public companies submit to the SEC.  Don't let those things distract you.

If you look at the reports of public companies that don't have errors and if you look at disclosures that do not have errors there is a lot that can be learned.  There are many, many reasons why the XBRL-based reports of public companies have issues.  But there are just as many of good examples that provide excellent learning opportunities.

Second, you have to "look around" the software issues of the software vendors and filing agents that have created software to support the creation of XBRL-based financial reports.  Most have not taken a long-term view of the opportunity.  Most have built "bolt on" solutions that add to existing processes that make XBRL seem like more work rather than expose the true opportunity.  Don't let that distract you.

Third, you have to "look around" the fact that XBRL-based reports of public companies are not leveraging metadata appropriatly and the general lack of metadata.

If you don't understand what I am talking about, I would encourage you to read the document Closing the Skills Gap.  That will help you get better dialed into what is really going on.

Now, I did some experimenting with Inline XBRL (see points 1 through 6).  I am continuing that experimentation here on public company financial reports submitted to the SEC.  The prior examples where testing smaller test documents that I had created to examine specific things.  So here I am looking at the same idea of contrasting raw XBRL to Inline XBRL; but I am doing this experiment with a 10-K filed with the SEC by a public company. Here is the filing I am looking at.

  1. Raw XBRL: I start here with the raw XBRL. Now, this raw XBRL was not created by the filer, it was auto-generated by the SEC.  This company submitted their report in the Inline XBRL format.
  2. Inline XBRL (in Inline XBRL Viewer): This is the Inline XBRL as viewed using the SEC provided Inline XBRL viewer.
  3. Inline XBRL (not in viewer):  This is the same Inline XBRL document but not viewed using the SEC Inline XBRL viewer; this is the document you would work with to extract information from the Inline XBRL using a computer based process.
  4. Inline XBRL auto-generated from Raw XBRL: This is Inline XBRL that was auto-generated by XBRL Cloud from the raw XBRL auto-generated by the SEC from the Inline XBRL submitted by the filer.  This rendering is pretty readable but does have some readability issues.
  5. Extraction of Facts: This is a ZIP archive that contains an Excel spreadsheet with macros that extract facts from raw XBRL or inline XBRL. The tool will extract facts from #1, #3, and #4 above. (Note that there appears to be a bug in the XBRL Could auto-generated inline XBRL.  The first two documents report 2,182 facts but the third has 1,860 facts returned.  I am looking into why this is happening.)

So the point here is that extracting information from Inline XBRL is just as easy as extracting information from Raw XBRL.  You should get the exact same results when information is queried in either syntax; which syntax you use does not impact the meaning of the information represented.

A public company financial reoprt filed as part of a 10-K is about as complex document that I can imagine.  What might be more complex is the management discussion and analysis document which is also part of an SEC 10-K but does not need to be represented using XBRL currently.

In this prior analysis I was using smaller documents than a 10-K financial report.  I did that to make creating all the business rules to verify the facts reported in the document easier.  The only difference between the documents used in the prior analysis and the 10-K financial report is the volume of disclosures.  The fact is that the smaller test documents that I used are more complex than the 10-K financial report.  Why?  Because I included some additional information patterns that are not used in the typical 10-K financial report.

Quarterly XBRL-based Public Company Financial Report Quality Measurement (Mar 2018) 

The following is a summary of the quality measurements of XBRL-based financial reports submitted to the SEC as of March 31, 2018.  The following Excel spreadsheets and other documents provide details related to these quality measurements:

In summary, fundamental accounting concept relations quality has steadily improved each year as can be seen by the three year comparison.  On an individual test basis, each of the 22 relations tests is 98% consistent with what is expected.  88.5% of all XBRL-based financial reports are consistent with all 22 relations tests as would be expected.  The goal as I see it is that all XBRL-based financial filings are 99.99966% consistent with expectations (six sigma).

I have added measurements of what I call the disclosure mechanics of 65 disclosures.  Disclosure mechanics quality has increased from 88% to 90% for the 65 disclosures I am measuring.  Again, the goal is six sigma for every disclosure reported.

The following are graphical summaries:

Fundamental accounting concept relations for last 10-K or 10-Q filed by generator of the report:

(Click for larger image)

Fundamental accounting concept relations by relation test (same filings as above):

(Click image for larger view)

Comparison of 2016, 2017, and 2018 fundamental accounting relations results: 

(Click image for larger view)

Disclosure mechanics by generator results from analysis of last 10-K filed with SEC as of March 31, 2018:

(Click image for larger view)

Disclosure mechanics results by disclosure:

(Click image for larger view)

I will likely put together a narrative that explains these measurements in more detail. 


Guide to Implementing Robotic Finance (DRAFT)

I am continuing on with my brainstorming related to robotic finance and accounting process automation using XBRL.

While a lot of people are talking in general high-level terms about "robotic finance" or "process robotics" or "accounting process automation" or "financial report creation automation"; fewer are talking about these things at the level of detail that it actually takes to implement these sorts of things.

So here are details.  I created the DRAFT document Guide to Implementing Robotic Finance. This is mostly brainstorming and collecting my thoughts at this point.  If you are interested in this sort of stuff the document is definitely worth reading through.

I am contemplating two XBRL formats, trying to figure out which is the best format all things considered. I don't think this is an "either/or" type of question.  The two formats are:

  • Raw XBRL
  • Inline XBRL

Each format has its pros and cons.  Both formats are worth considering.  To add some contrast to this discussion, consider the W3C working draft HTML Microdata.  This is someone similar to Inline XBRL but it has one fundamental flaw: it does not support multidimensional information.

Now, the W3C has a recommendation The RDF Data Cube Vocabulary. (They seem to be working on updating this recommendation.)  That seems to focus on SDMX.

The first point that I want to make here is that (a) you will ultimately realize that you need a dimensional model and (b) XBRL has a solid, tested multidimensional model that works for the sorts of things you want to do in financial reports.  XBRL Dimensions does have a few issues, but within the set of 6,000 XBRL-based financial filings submitted to the SEC you can pretty much understand that dimensions work and how to represent things if you understand what you are looking at.  Be careful; there is a lot of garbage out there you have to wade through.  Dimensions work with either raw XBRL or inline XBRL.

The second point is that as far as I can tell, it does not matter whether you use raw XBRL or inline XBRL.  Both will work.  Inline XBRL appears to have some advantages but all things considered it really distills down to two things: (1) having a sophisticated rendering engine and (2) the complexity of the information that you want to or need to represent.

Here is what I mean.  First, download this Excel spreadsheet which lets you extract information from Raw XBRL or Inline XBRL documents.  The demonstration is not sexy or anything, it simply pulls information from a raw XBRL document and five different versions of inline XBRL documents.  Extracting information from either is rather easy.

So now look at each of the documents:

  1. Raw XBRL: So raw XBRL is not understandable to humans. But you can convert the raw XBRL into something that is readable by humans.  Here is an example. So, you can turn machine-readable raw XBRL into human-readable form to the extent that you have a sophisticated rendering engine and up to the point of complexity supported by the rendering engine functionality.
  2. Hidden facts in inline XBRL: This document has all but one fact represented as "hidden" from the HTML rendering.  Not much of a point in doing this.
  3. Unformatted facts in inline XBRL: This document basically has a list of raw facts, basically unformatted.
  4. High-quality formated facts in inline XBRL: This document has formatting of the quality of the HTML documents of financial reports submitted to the SEC.  It takes a lot of work to create that formatting.
  5. High-quality formated facts with business rule validation results in inline XBRL: This document has the same high-quality, precise formatting as above; but added to it is information about the validation results of business rules.  The information in the document needs to be consistent with business rules to make sure information quality is intact.
  6. Good-quality auto-generated formated facts in inline XBRL: This document is of good quality and the formatting is auto-generated by machine-based processes which outputs inline XBRL. Now, this version does not include the "green" cells which show that the information is consistent with applicable business rules; but that could be added with little effort really.

So what is my point here?  My point is that for many use cases, machine-generated formatting will work just fine and there is no need to have humans create the human readable formatting.  Now, you WILL ALWAYS need both human-readable and machine-readable information when you are doing "robotic finance".  There is no way around that.  Financial reporting CANNOT be a black box.  And quality does matter.  The information that is processed by the finance function is very, very high quality.  Due to the nature of this information, there can be no compromising on quality.  Low quality is a non-starter.

Here is a more complex XBRL-based report, the XASB reference implementation. That inline XBRL was auto-generated.

As such, as I pointed out above, the extent that auto-generated inline XBRL can be generated is directly dependent on:

  • the sophistication of the rendering engine
  • the complexity of the information you are working with

There are a lot of tricks that you can use to increase the sophistication of the rendering capabilities of a rendering engine. Hard coded information and metadata-based information are both helpful.  Again, information quality and the ability for an accounting professional to understand what is going on are paramount.  No black boxes.  Not technical oriented renderings or informational messages.

More coming soon so stay tuned.

Posted on Friday, March 30, 2018 at 03:14PM by Registered CommenterCharlie in | CommentsPost a Comment | EmailEmail | PrintPrint