A form of machine learning that excels at pattern recognition
Deep learning is a form of artificial neural network used for machine learning. This is a way of storing and processing information that is very different to conventional computers. Deep learning allows a computer to learn a task without a human first defining the algorithm, the computer only needs to know the objective and be shown examples of correct inputs and outputs. Tasks such as controlling complex systems and pattern recognition are often performed best by deep learning.
A deep neural net
Artificial neural networks are simplified mathematical representations of the neural networks found in biological brains. They have sets of numerical inputs and outputs, connected by a network of connections. The ease with which information can flow along a particular pathway through this network is determined by adjustable weightings at the hidden nodes between the input and output layers. Information is processed as it flows through the network. Information is stored, or the network is trained, by adjusting the weightings of the nodes. Training algorithms are used to adjust nodes, strengthening a pathway when the network gives a correct output and weakening it when the network makes a mistake. Training datasets with inputs and known correct outputs are used.
Early neural networks were shallow, typically with only one hidden layer of nodes between the inputs and outputs. Although these can theoretically solve any problem, they generally require a lot of input parameters and can be very computationally inefficient. When the training datasets are small, shallow networks can give better results.
Deep learning uses a neural network that is deep — with more hidden layers of nodes between the inputs and outputs. This can create a more sophisticated model, which often requires fewer input parameters and can find a solution more quickly. However, it also often requires a larger training dataset before the model becomes accurate.
Mathematics of deep learning
Conventional regression fits a curve to data in two dimensions or a surface to data in three dimensions. Deep learning can fit a model to data with many, sometimes even hundreds of, dimensions.
Applications
A person is able to make a good approximation to a two dimensional regression, simply sketching a curve through data points on a graph. However, humans can find it extremely difficult to find more complex patterns in data with many dimensions. In some areas, such as recognising faces or voices, humans have natural abilities to solve such problems. However, humans are not good at recognizing complex patterns when the data relates to a less familiar problem, such as control parameters in a complex system like a distributed renewable power grid or a financial market. Deep learning can, therefore, tackle problems that are conventionally seen as requiring a human, as well as new classes of problem that humans cannot solve.
