« Understanding the Need for a Framework and Theory | Main | Artificial intelligence to be 'the norm' in law firms in 2020 »

Understanding 'Deep Learning' and 'Neural Networks'

Something that I have been trying to figure out is how good machine can be at figuring something out.  In trying to understand true capabilities I run up against terms such as 'artificial intelligence' and 'deep learning' and 'neural networks'.

Artificial intelligence I already discussed (see the link above).  Basically artificial intelligence is about getting a machine to mimic or simulate the behavior of a human.  For example, a calculator be it mechanical (hardware) or software is a machine that can mimic human behavior.

'Deep learning' and 'neural networks' seem to be related to ways of creating artificial intelligence.  Here are definitions of each of those terms:

  • Deep learning: Wikipedia says: Deep learning (deep machine learning, or deep structured learning, or hierarchical learning, or sometimes DL) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using model architectures, with complex structures or otherwise, composed of multiple non-linear transformations. (O'Reilly Media is publishing a book Fundamentals of Deep Learning)
  • Neural network: A neural network is basically a computer system modeled after how the human brain and nervous system work.

If you look at the last sentence of the first section of the Wikipedia definition of 'deep learning' it says: "Alternatively, deep learning has been characterized as a buzzword, or a rebranding of neural networks."  Also, go look at Chapter 1 of the O'Reilly Media book, the chapter title is "The Neural Network".

This Basic Introduction to Neural Networks provides this explanation which sheds a lot of light on what neural networks are:

What Applications Should Neural Networks Be Used For?

Neural networks are universal approximators, and they work best if the system you are using them to model has a high tolerance to error. One would therefore not be advised to use a neural network to balance one's cheque book! However they work very well for:

•capturing associations or discovering regularities within a set of patterns;
•where the volume, number of variables or diversity of the data is very great;
•the relationships between variables are vaguely understood; or,
•the relationships are difficult to describe adequately with conventional approaches.

It seems to be the case that a 'neural network' is an architecture or approach to getting work done.  It is an alternative or compliment to currently used techniques.


People seem to have these delusions related to what you can do with a neural network.  Basically, there seems to be a lot of hype going around.  Slick marketing gimmicks lead you to believe that neural networks will do magical things for you without you having to put in any effort.  Their neural network is best.  But this is only hype.

Which type of network means far, far less than the model that you create and feed the neural network with.  That is the key, the metadata.

This SlideShare presentation, Deep neural networks, discusses the importance of unsupervised pre-training (slide 19) and supervised fine-tuning (slide 26) when constructing a system.

McKinsey published an article Artificial intelligence meets the C-suite.  That article makes the statement,

Many of the jobs that had once seemed the sole province of humans—including those of pathologists, petroleum geologists, and law clerks—are now being performed by computers.

(Don't forget this article about how artificial intelligence will impact law firms.)

Very interesting stuff.  Clearly there is a lot of hype and over-stating what a machine will be capable of doing.  But you cannot write off 100% of what people are purporting.  There is some truth buried in there.  Understanding how to make use of this technology is really important so you don't get fooled by those slick marketing gimmicks.

Stay tuned!

Posted on Thursday, September 17, 2015 at 01:19PM by Registered CommenterCharlie in | Comments1 Comment | References1 Reference

PrintView Printer Friendly Version

EmailEmail Article to Friend

References (1)

References allow you to track sources for this article, as well as articles that were written in response to this article.

Reader Comments (1)

Thanks for sharing this nice resource. It's really interesting. Keep up the good work and keep updating the same.
February 10, 2017 | Unregistered CommenterRoman Walker

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
All HTML will be escaped. Hyperlinks will be created for URLs automatically.