This article by Narayanan Vaidyanathan, Explainable AI: Putting the User at the Core, published by the Association of Chartered Certified Accountants (ACCA), is spot-on when AI is used in accounting, reporting, auditing, and analysis. Don't miss the full report which you can get here.
What I have created is 100% explainable. That is what I mean by "justification and explanation mechanism" (see here). The user of the software should be able to follow the chain of reasoning used by any software application helping them. SBRM is likewise 100% explainable.
Here is part of the executive summary of this article:
Explainable artificial intelligence (XAI) emphasises the role of the algorithm not just in providing an output, but also in sharing with the user the supporting information on how the system reached a particular conclusion. XAI approaches aim to shine a light on the algorithm’s inner workings and/or to reveal some insight into the factors that influenced its output. Furthermore, the idea is for this information to be available in a user-readable way, rather than being hidden within code.
Historically, the focus of research within AI has been on developing and iteratively improving complex algorithms, with the aim of improving accuracy. Implicitly, therefore, the attention has been on refining the quality of the answer, rather than explaining the answer. But as AI is maturing, the latter is becoming increasingly important for enterprise adoption. This is both for decision making within a business, and post-fact audit of decisions made. Auditable algorithms are essentially ones that are explainable.
The complexity, speed and volume of AI decision-making obscure what is going on in the background, the so-called ‘black box’ effect, which makes the model difficult to interrogate. Explainability, or any deficit thereof, affects the ability of professional accountants to display scepticism. In a recent survey of members of ACCA and IMA (Institute of Management Accountants), those agreeing with this view, 54%, were more than twice the number who disagreed. It is an area that is relevant to being able to trust the technology, and to being confident that it is being used ethically. XAI can help in this scenario with techniques to improve explainability. It may be helpful to think of it as a design principle as much as a set of tools. This is AI designed to augment the human ability to understand and interrogate the results returned by the model.
If you are unclear about AI, I would recommend this document.