Digital Financial Reporting

I wish I knew 15 years ago what I know now.  When I learned some things about the IT world, I made two mistakes.  First, I thought these things had been around for quite some time.  Object Oriented Programming is actually relatively new.  Second, not everyone in the IT world understands this stuff.  There are people who are good at object oriented programming, and there are those who are not.

This document, Object Oriented Programming by Carl Erickson, provided some enlightening information about the creation of software.

On page 40 of the PDF:

A study in 1999 by Elemer Magaziner of requirement specifications found that they are typically only 7% complete and 15% correct.

Except for very constrained domains, re-implementations of existing functionality, or trivial applications, the process of defining requirements is almost guaranted to result in:

incomplete specifications
incorrect specifications

That is pretty low.  Even the statistic was three or even four times what it is, it would still be low. Most of the stuff that I work on are things which have never existed before.  It is no wonder that they tend to not work as I would expect.

Grady Booch is the chief scientist at Rational (now part of IBM) since its founding in 1980. (This is Grady Booch's blog)

On page 7 of the PDF:

Booch identifies five common characteristics to all complex systems:

1. There is some hierarchy to the system. A minute ago we viewed hierarchy as something we did to cope with complexity. This view says that a complex system is always hierarchic. Perhaps people can only see complex systems in this way.

2. The primitive components of a system depend on your point of view. One man’s primitive is another complexity.

3. Components are more tightly coupled internally than they are externally. The components form clusters independent of other clusters.

4. There are common patterns of simple components which give rise to complex behavior. The complexity arises from the interaction or composition of components, not from the complexity of the components themselves.

5. Complex systems which work evolved from simple systems which worked. Or stated more strongly: complex systems built from scratch don’t work, and can not be made to work.

Also on page 40:

Generality is good, but comes at a cost. Some balance must be met. Developers who really become OO (object oriented) are particularly prone to this problem. The elegant, complete, general solution to a class design becomes a goal unto itself. You’ve got to know when good is good enough.

I cannot tell you how many conversations that I have had with software developers about them over-generalizing something.  Take an "element" in XBRL.  When you generalizing everything down to an element, you totally lose important differences between the categories of elements: hypercube, dimension, member, primary items, concept, abstract.

To create good software you need three things:

1. Extremely deep business domain expertise for the business domain for which the software will be developed.
2. Extremely deep technical expertise and at least some of this expertise should be in the form of a technical architect/engineer.  Now, I don't mean engineer how many IT people use the term engineer.  Many IT people tend to think that everyone who understands how to program is an engineer.  This is NOT the case.
3. Extremely good communication between the deep domain experts and the deep technical experts.  This is extremely hard.

Developing good software is hard work.  It is no wonder that business users don't particularly care for software which is used to create SEC XBRL financial filings.  Not only is the software hard to use but they don't even work correctly for their intended purpose. Now I understand why this is the case.

So don't get me wrong here.  I am NOT saying that software vendors are the problem.  I mention THREE points above. You need all three.  Business domain professionals are just as culpable as the technical experts.  Clearly the requirements or purpose of the software are not being communicated effectively.

This presentation points out the different ways OOP uses the term "class" as compared to those to create expert systems.

This article also points out that OOP classes and logic classes are not the same types of things.

More to come...