项目作者: rasmusbergpalm

项目描述 :
用于深度学习的Matlab / Octave工具箱。包括深信仰网,堆叠自动编码器,卷积神经网络,卷积自动编码器和香草神经网络。每种方法都有一些示例可以帮助您入门。
高级语言: Matlab
项目地址: git://github.com/rasmusbergpalm/DeepLearnToolbox.git
创建时间: 2011-10-27T13:15:57Z
项目社区:https://github.com/rasmusbergpalm/DeepLearnToolbox

开源协议:BSD 2-Clause "Simplified" License

下载


Deprecation notice.

This toolbox is outdated and no longer maintained.

There are much better tools available for deep learning than this toolbox, e.g. Theano, torch or tensorflow

I would suggest you use one of the tools mentioned above rather than use this toolbox.

Best, Rasmus.

DeepLearnToolbox

A Matlab toolbox for Deep Learning.

Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.
It is inspired by the human brain’s apparent deep (layered, hierarchical) architecture.
A good overview of the theory of Deep Learning theory is
Learning Deep Architectures for AI

For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.

If you use this toolbox in your research please cite Prediction as a candidate for learning deep hierarchical models of data

  1. @MASTERSTHESIS\{IMM2012-06284,
  2. author = "R. B. Palm",
  3. title = "Prediction as a candidate for learning deep hierarchical models of data",
  4. year = "2012",
  5. }

Contact: rasmusbergpalm at gmail dot com

Directories included in the toolbox

NN/ - A library for Feedforward Backpropagation Neural Networks

CNN/ - A library for Convolutional Neural Networks

DBN/ - A library for Deep Belief Networks

SAE/ - A library for Stacked Auto-Encoders

CAE/ - A library for Convolutional Auto-Encoders

util/ - Utility functions used by the libraries

data/ - Data used by the examples

tests/ - unit tests to verify toolbox is working

For references on each library check REFS.md

Setup

  1. Download.
  2. addpath(genpath(‘DeepLearnToolbox’));

Example: Deep Belief Network

  1. function test_example_DBN
  2. load mnist_uint8;
  3. train_x = double(train_x) / 255;
  4. test_x = double(test_x) / 255;
  5. train_y = double(train_y);
  6. test_y = double(test_y);
  7. %% ex1 train a 100 hidden unit RBM and visualize its weights
  8. rand('state',0)
  9. dbn.sizes = [100];
  10. opts.numepochs = 1;
  11. opts.batchsize = 100;
  12. opts.momentum = 0;
  13. opts.alpha = 1;
  14. dbn = dbnsetup(dbn, train_x, opts);
  15. dbn = dbntrain(dbn, train_x, opts);
  16. figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights
  17. %% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NN
  18. rand('state',0)
  19. %train dbn
  20. dbn.sizes = [100 100];
  21. opts.numepochs = 1;
  22. opts.batchsize = 100;
  23. opts.momentum = 0;
  24. opts.alpha = 1;
  25. dbn = dbnsetup(dbn, train_x, opts);
  26. dbn = dbntrain(dbn, train_x, opts);
  27. %unfold dbn to nn
  28. nn = dbnunfoldtonn(dbn, 10);
  29. nn.activation_function = 'sigm';
  30. %train nn
  31. opts.numepochs = 1;
  32. opts.batchsize = 100;
  33. nn = nntrain(nn, train_x, train_y, opts);
  34. [er, bad] = nntest(nn, test_x, test_y);
  35. assert(er < 0.10, 'Too big error');

Example: Stacked Auto-Encoders

  1. function test_example_SAE
  2. load mnist_uint8;
  3. train_x = double(train_x)/255;
  4. test_x = double(test_x)/255;
  5. train_y = double(train_y);
  6. test_y = double(test_y);
  7. %% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN
  8. % Setup and train a stacked denoising autoencoder (SDAE)
  9. rand('state',0)
  10. sae = saesetup([784 100]);
  11. sae.ae{1}.activation_function = 'sigm';
  12. sae.ae{1}.learningRate = 1;
  13. sae.ae{1}.inputZeroMaskedFraction = 0.5;
  14. opts.numepochs = 1;
  15. opts.batchsize = 100;
  16. sae = saetrain(sae, train_x, opts);
  17. visualize(sae.ae{1}.W{1}(:,2:end)')
  18. % Use the SDAE to initialize a FFNN
  19. nn = nnsetup([784 100 10]);
  20. nn.activation_function = 'sigm';
  21. nn.learningRate = 1;
  22. nn.W{1} = sae.ae{1}.W{1};
  23. % Train the FFNN
  24. opts.numepochs = 1;
  25. opts.batchsize = 100;
  26. nn = nntrain(nn, train_x, train_y, opts);
  27. [er, bad] = nntest(nn, test_x, test_y);
  28. assert(er < 0.16, 'Too big error');

Example: Convolutional Neural Nets

  1. function test_example_CNN
  2. load mnist_uint8;
  3. train_x = double(reshape(train_x',28,28,60000))/255;
  4. test_x = double(reshape(test_x',28,28,10000))/255;
  5. train_y = double(train_y');
  6. test_y = double(test_y');
  7. %% ex1 Train a 6c-2s-12c-2s Convolutional neural network
  8. %will run 1 epoch in about 200 second and get around 11% error.
  9. %With 100 epochs you'll get around 1.2% error
  10. rand('state',0)
  11. cnn.layers = {
  12. struct('type', 'i') %input layer
  13. struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer
  14. struct('type', 's', 'scale', 2) %sub sampling layer
  15. struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer
  16. struct('type', 's', 'scale', 2) %subsampling layer
  17. };
  18. cnn = cnnsetup(cnn, train_x, train_y);
  19. opts.alpha = 1;
  20. opts.batchsize = 50;
  21. opts.numepochs = 1;
  22. cnn = cnntrain(cnn, train_x, train_y, opts);
  23. [er, bad] = cnntest(cnn, test_x, test_y);
  24. %plot mean squared error
  25. figure; plot(cnn.rL);
  26. assert(er<0.12, 'Too big error');

Example: Neural Networks

  1. function test_example_NN
  2. load mnist_uint8;
  3. train_x = double(train_x) / 255;
  4. test_x = double(test_x) / 255;
  5. train_y = double(train_y);
  6. test_y = double(test_y);
  7. % normalize
  8. [train_x, mu, sigma] = zscore(train_x);
  9. test_x = normalize(test_x, mu, sigma);
  10. %% ex1 vanilla neural net
  11. rand('state',0)
  12. nn = nnsetup([784 100 10]);
  13. opts.numepochs = 1; % Number of full sweeps through data
  14. opts.batchsize = 100; % Take a mean gradient step over this many samples
  15. [nn, L] = nntrain(nn, train_x, train_y, opts);
  16. [er, bad] = nntest(nn, test_x, test_y);
  17. assert(er < 0.08, 'Too big error');
  18. %% ex2 neural net with L2 weight decay
  19. rand('state',0)
  20. nn = nnsetup([784 100 10]);
  21. nn.weightPenaltyL2 = 1e-4; % L2 weight decay
  22. opts.numepochs = 1; % Number of full sweeps through data
  23. opts.batchsize = 100; % Take a mean gradient step over this many samples
  24. nn = nntrain(nn, train_x, train_y, opts);
  25. [er, bad] = nntest(nn, test_x, test_y);
  26. assert(er < 0.1, 'Too big error');
  27. %% ex3 neural net with dropout
  28. rand('state',0)
  29. nn = nnsetup([784 100 10]);
  30. nn.dropoutFraction = 0.5; % Dropout fraction
  31. opts.numepochs = 1; % Number of full sweeps through data
  32. opts.batchsize = 100; % Take a mean gradient step over this many samples
  33. nn = nntrain(nn, train_x, train_y, opts);
  34. [er, bad] = nntest(nn, test_x, test_y);
  35. assert(er < 0.1, 'Too big error');
  36. %% ex4 neural net with sigmoid activation function
  37. rand('state',0)
  38. nn = nnsetup([784 100 10]);
  39. nn.activation_function = 'sigm'; % Sigmoid activation function
  40. nn.learningRate = 1; % Sigm require a lower learning rate
  41. opts.numepochs = 1; % Number of full sweeps through data
  42. opts.batchsize = 100; % Take a mean gradient step over this many samples
  43. nn = nntrain(nn, train_x, train_y, opts);
  44. [er, bad] = nntest(nn, test_x, test_y);
  45. assert(er < 0.1, 'Too big error');
  46. %% ex5 plotting functionality
  47. rand('state',0)
  48. nn = nnsetup([784 20 10]);
  49. opts.numepochs = 5; % Number of full sweeps through data
  50. nn.output = 'softmax'; % use softmax output
  51. opts.batchsize = 1000; % Take a mean gradient step over this many samples
  52. opts.plot = 1; % enable plotting
  53. nn = nntrain(nn, train_x, train_y, opts);
  54. [er, bad] = nntest(nn, test_x, test_y);
  55. assert(er < 0.1, 'Too big error');
  56. %% ex6 neural net with sigmoid activation and plotting of validation and training error
  57. % split training data into training and validation data
  58. vx = train_x(1:10000,:);
  59. tx = train_x(10001:end,:);
  60. vy = train_y(1:10000,:);
  61. ty = train_y(10001:end,:);
  62. rand('state',0)
  63. nn = nnsetup([784 20 10]);
  64. nn.output = 'softmax'; % use softmax output
  65. opts.numepochs = 5; % Number of full sweeps through data
  66. opts.batchsize = 1000; % Take a mean gradient step over this many samples
  67. opts.plot = 1; % enable plotting
  68. nn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)
  69. [er, bad] = nntest(nn, test_x, test_y);
  70. assert(er < 0.1, 'Too big error');

Bitdeli Badge