The income statement is a bear. The balance sheet and cash flow statement were rather straight forward as compared to the income statement. Might need to take a different approach.
There are two primary reasons the income statement is a bear. The first is that there really is no prescribed form or set of totals which one can latch onto. For the balance sheet and cash flow statement there are "buckets" which everything fits into, totals are generally reported for these buckets or enough totals exist so that the totals (such as liabilities) which are not reported can be easily imputed (i.e. because equity is required to be reported and liabilities and equity is reported, finding total liabilities is a cake walk). The second issue is the number of permutations/combinations which are possible due to the existence or non-existence of equity method investments, taxes, discontinued operations, extraordinary items, noncontrolling interest, preferred shares. Picking the correct "subtotals" can be a challenge.
What I can say about the income statement for the 5525 filings in my test set is this:
- Net income (loss): 5390 of the set (98%) reported one of three concepts to express what amounts to Net Income (Loss): us-gaap:ProfitLoss, us-gaap:NetIncomeLoss, us-gaap:NetIncomeLossAvailableToCommonStockholdersBasic. What I am saying is that the US GAAP term "Net Income (Loss)" is being reported using three DIFFERENT concepts. Basically, there is ambiguity as to which of the three of those concepts to use in different reporting scenarios. I think I have the right algorithm for detecting the correct concept but I cannot be sure as I have not tied out the income statement like I have tied out the upper level of the balance sheet which proves my balance sheet numbers are correct.
- Income (loss) from continuing operations before taxes: So far I have 3938 of the set (71%) who report what amounts to "Income (loss) from continuing operations before taxes". A wide variety of labels are used by filers for this concept, but the essence of what is being reported is precicely that: income or loss from continuing operations before taxes. Interestingly, lots of filers are creating an extension concept for this and even calling the concept exactly this. There seems to be two reasons for this. First, as I understand it in an earlier version of the US GAAP taxonomy did not have this concept, filers created an extension concept, and then they did not realize that a new concept has been added. Second, the names of the concept are so strange and which concept is used is hard to figure out because of the different permutations/combinations (whether you do or do not have certain line items) that filers seem to simply give up and create an extension. That is the sense I get from looking at filings. Of the 30 companies which make up the Dow Industrials, 100% report this line item, 22 use the same concept, and 8 created an extension concept for what appears to be exactly the same line item.
- Revenues: I was able to locate the line item(s) which make up revenues for 4624 (84%) of the filings, but that is not to say that revenues does not exist for others. Understanding which concepts make up revenues is another story, this is likely the hardest task in reading an SEC XBRL financial filing. This is because there are so many different ways filers report revenues. Some provide a helpful total for revenues, some do not. There are all sorts of combinations so making sure you have total revenues can be a challenge, particularly if there is no total. One company of the 30 companies which make up the Dow Industrials, Exxon Mobil, saw fit to create an extension concept for revenue. Yet Exxon Mobil did not create an extension for cost of revenues.
- Cost of revenues: Same deal as revenues, all over the board.
- Gross profit: While reporting gross profit is used by some industries, other industries do not report it. Of my set of filings 1846 (33%) do report gross profit. Having the concept gross profit can be quite helpful in figuring out which concepts make up revenues and cost of revenues.
- Operating income (loss): 3797 (69%) of filers report operating income (loss), all of these use the exact same concept.
- Nonoperating income (expenses): Did not look for this yet, I suspect that not a lot of filers provide this total and there is lots of variety in how the details are reported.
- Interest and debt expense: Did not look for this yet, again, I suspect a fair amount of variety here making this challenging to detect.
- Income (loss) from equity method investments: 877 (16%) of filers report equity method investment income. Clearly if a filer does not have equity method investments, which is often the case, this line item would not be reported.
- Income tax expense (benefit): Gotta pay the tax man! 3732 (68%) do so, I guess the rest have moved off shore to avoid paying taxes (humor).
- Income (loss) from discontinued operations: I found income from discontinued operations for 3811 (69%) filings. That could be off though, I was only looking for the total, might be higher. Not every filer has discontinued operations.
- Extraordinary items: Only 6 of the filings reported extraordinary items. As expected, this is very rare.
- Income (loss) from noncontrolling interest: 1139 (20%) of filers reported income or loss from a noncontrolling interest. Not every filer has a noncontrolling interest.
- Preferred stock dividends and other adjustments: Did not look at this yet.
I have not given up on being able to sort this out and automate the comparison of these high-level items between filers, clearly where it makes sense. Comparing a retailer who uses gross profit to a bank who does not use gross profit can be done, it is just done at a different level of the income statement, say income (loss) from continuing operations before taxes, which they both will have.
Personally, what I would like to see done is certain line items should be reported whether you have them or not. What I am trying to achieve is two things: automatable comparison and/or aggregation across all filings and "ease".
For example, if a filer does not have any revenue, from my vantage point I feel that it is better to report revenue of zero than it is to drive software vendors nuts trying to write reliable algorithms to accurately compute say an aggregation of revenues across all SEC filings. Same deal for a few other line items, maybe "income (loss) from continuing operations before taxes", "net income (loss)", perhaps a few others.
Trying to do this today is possible, but it will be brittle and fragile, not robust, reliable, accurate, and predictable which is what we need.
Filers are not giving up any flexibility to report their operations how they feel they should be reported within US GAAP. All I am talking about here is a few subtotals, make sure they are there. It is not disputable where income from equity investments, income from discontinued operations, income from noncontrolling interest, nonoperating income and expenses, revenues, gross profit, taxes, extraordinary items are reported. That does not change. All I am saying is that adding a few subtotals will make it easier for those trying to grab this data, expanding the set of those who can make use of this information.
From a creation perspective, my real interest really, these subtotals do exist in the taxonomy you are working with. The creators of information know the relations which they want to express and can go into the taxonomy to find the pieces, leveraging these semantic relations which do exist. That is not an issue. But for those using the information, if they don't have something to grab ahold of, they will have to unravel the income statements of filers one-by-one. This very possible, but time consuming, expensive, and brittle.