Bear with me as I walk through several different explanations of what is necessary to achieve meaningful interoperabilty. If you want to scroll directly to the final result, go to the bottom of this post.
One of the most helpful things that I had run across which helped me understand issues XBRL was having was a video published by the HL7 people. The HL7 video outlined three key things needed to achieve meaningful interoperability:
- Technical interoperability is concerned with the conveyance of payload. (moving data from system A to system B)
- Semantic interoperability communicates meaning unambiguously, using Codes, Identifiers and Context. (ensuring that system A and system B understand the information in the same way)
- Process interoperability enables shared human understanding that is needed to coordinate work processes and enable business systems to interoperate. (enable business processes housed at system A and system B to work together)
A great graphic that I found which discussed semantic clarity had a slightly different list and did not provide details of each item on the list: (but note that this does not have process interoperability)
- Syntactic interoperability
- Structural interoperability
- Semantic interoperability
This description of interoperability has the same set if items above, but provides explanations of each of the line items; but I don't particularly care for the explanations: (likewise this does not have process interoperability)
- Syntactic interoperability is just the ability to exchange information. It requires agreement or compatibility at the transport and application layers of the communications protocol stack and with the messaging protocol and encoding format
- Structural interoperability means that all of the expected information components are present with the same arrangement and granularity
- Semantic interoperability requires that the content of the message be understood by the recipient application or process
Another list (page 21) provided a different set of items, still similar: (likewise this does not have process interoperability)
- System
- Syntactic
- Structural
- Semantic
First, why interoperability? Easy. As the HL7 video puts it "Interoperability = Business benefits" (read this as better, faster, and/or cheaper). Second, likewise from the HL7 video: "Standards enable interoperability".
And so here is my list which steals the best ideas of each of the lists above and add some things that none of the other lists have. These are the levels of interoperabilty as I see the world:
- System interoperability: System interoperabilty relates to moving something from one system to another system. While system interoperability is not really an issue any longer, I will include it in order to make a point. The common denominator these days to achieve system interoperability is the internet. I am specifically saying internet and not the "World Wide Web". There are two keys here: TCP/IP and HTTP. Standard: TCP/IP; HTTP. Benefit: Any system can easily communicate with any other system. (Personally, I think that the internet is more important than the Web. Consider this Wired article: The Web is Dead. Long Live the Internet)
- Syntactic interoperability: Syntactic interoperabilty relates to the technical syntax being exchanged between systems. To be included as a candidate syntax, the syntax must work within TCP/IP and HTTP. There is another little piece that is related to syntax but not a lot of people get, another standard called Unicode. Unicode is actually extremely important. The short explanation is that not every language in the world uses the same 26 characters as the English alphabet (i.e ASCII characters). I don't want to get into a discussion of syntax, see the Understanding Syntax blog post for more detail. But, I will say this. If I were asked 15 years ago what the most important syntax is, I would have said XML hands down. But there are other syntaxes being used, such as JSON, which are giving XML a run for the money. But today XML is still king of the hill. Standard; XML, Unicode, JSON. Benefit: Global standard, easy to use machine readable structured formats which are also readable (sort of) by humans.
- Structural interoperability: Structural interoperability means that all of the expected information components of the information model are present with the same arrangement. Structural interoperabiilty relates to the information model metadata, the structure of the concent, not the content itself. Structure is content independent. If I understand this correctly a really good example of this is XBRL. While XBRL is a global standard syntax for business reporting, it was pointed out by Rene van Egmond and my self in the document, XBRLS: How a Simpler XBRL can be a Better XBRL, that different XBRL taxonomies were not interoperable. I would not position XBRLS as "the answer" today as I did back in 2008. The answer is the application profile or "the model". I would position XBRLS, or rather something like XBRLS, as "an answer". Here is another explanation of this using XML. XML is great, but the real power of XML is not really XML itself; it is standard languages created using XML. Each XML language is a standard structure. Now, this next thing may seem a little odd. XML languages are not generally extensible. XML itself is extensible. But once you create a language, that language is fixed, generally NOT extensible. Now, this helps one understand the difference between a "tree" and a "graph" (see syntax). Finally, XBRL has created a "structure" model, the XBRL Abstract Model 2.0. The bottom line here is that syntaxes like XBRL, EVA and RDF are extensible. Most XML languages are not. Standard: Application profile or "the model". Benefit: Flexibility.
- Semantic interoperability: Semantic interoperability ensures that each system making use of information is consistent. Semantic interoperability relates to the content which is being exchanged, the business domain. Semantics are content dependent. For example, financial reporting under US GAAP is a business domain. IFRS is a different business domain. There could be some similarities and there for different levels might exist, but generally the business rules used to express the allowed and disallowed content and the relations between the content are expressed via business rules specific to the domain of the content.
- Process interoperability: Process or work flow interoperability enables shared understanding that is needed to coordinate work processes and enable business systems to interoperate. Process interoperabiity enable business processes housed at system A and system B to work together correctly and effectively. For example, SEC XBRL financial filings can be ammended if, for example, there is an error discloved in a filing. Companies submitting information to the SEC need to understand the rules of resubmitting a filing, analysts need to understand what the SEC does when a filing is ammended so they don't extract the incorrect information from financial reports. That is an example of process or work flow.
Seem complicated? It is complicated. Very complicated. But software will hide this complexity from the user to the extent that "patterns" or standards exist. All humans need to do is express this information in an organized manner which computers can read and use. Consider the OSI model. The OSI model has seven layers, very complicated. Frankly, I don't understand how the OSI model works, all I care about is THAT it works. The OSI model enables system level interoperabilty. It is a smorgasbord of standards. Digital financial reporting can be complicated under the hood, but easy for business users to make use of; it things are set up correctly.
Two final thoughts.
No matter how different information might look (different syntaxes, different names, different structures); if their conceptual model is the same, it is possible to transform one implementation model to another implementation model using completely automated processes.
Automated validation/verification many times may not be sufficient to guarantee complete interoperability. Human judgment and manual effort can be necessary in addition to automated validation/verification to effectively communicate information and thefore achieve interoperability. Where automated validation/verification can be created, it is more reliable, cheaper than human validation steps, and faster.
The following are additional helpful resources: