使用找到的第二个函数库，然后实现一下随机森林。

未编译的压缩包

randomforest-matlab.zip
这个压缩包中包含：回归和分类两部分的代码，具体的文件如下图1所示
image.png
图1. 未编译文件列表
在使用这部分文件的时候，直接运行图中的tutorial_ClassRF.m这个教程文件，但是会出现mex编译文件的错误，这部分目前还没有解决。

解决办法

mex编译文件是c/c++与matlab混编过程中用到的，主要是让matlab能够调用c/c++的文件，加快运行速度
在Microsoft Visual Studio中开发Matlab的mexw文件，生成mexw64或mexw32文件，教程

已编译的压缩包

所以直接使用已经预编译完成的文件夹，Windows-Precompiled-RF_MexStandalone-v0.02-.zip
其中的文件列表如图2所示：
image.png
图2. 预编译完成的文件夹
直接进入tutorial_ClassRF.m文件，代码如下

  1. % A simple tutorial file to interface with RF
  2. % Options copied from http://cran.r-project.org/web/packages/randomForest/randomForest.pdf
  4. %run plethora of tests
  5. clc
  6. close all
  8. %compile everything
  9. if strcmpi(computer,'PCWIN') |strcmpi(computer,'PCWIN64')
  10.    compile_windows
  11. else
  12.    compile_linux
  13. end
  15. total_train_time=0;
  16. total_test_time=0;
  18. %load the twonorm dataset 
  19. load data/twonorm
  21. %modify so that training data is NxD and labels are Nx1, where N=#of
  22. %examples, D=# of features
  24. X = inputs';
  25. Y = outputs;
  27. [N D] =size(X);
  28. %randomly split into 250 examples for training and 50 for testing
  29. randvector = randperm(N);
  31. X_trn = X(randvector(1:250),:);
  32. Y_trn = Y(randvector(1:250));
  33. X_tst = X(randvector(251:end),:);
  34. Y_tst = Y(randvector(251:end));
  38. % example 1:  simply use with the defaults
  39.     model = classRF_train(X_trn,Y_trn);
  40.     Y_hat = classRF_predict(X_tst,model);
  41.     fprintf('\nexample 1: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));
  43. % example 2:  set to 100 trees
  44.     model = classRF_train(X_trn,Y_trn, 100);
  45.     Y_hat = classRF_predict(X_tst,model);
  46.     fprintf('\nexample 2: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));
  48. % example 3:  set to 100 trees, mtry = 2
  49.     model = classRF_train(X_trn,Y_trn, 100,2);
  50.     Y_hat = classRF_predict(X_tst,model);
  51.     fprintf('\nexample 3: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));
  53. % example 4:  set to defaults trees and mtry by specifying values as 0
  54.     model = classRF_train(X_trn,Y_trn, 0, 0);
  55.     Y_hat = classRF_predict(X_tst,model);
  56.     fprintf('\nexample 4: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));
  58. % example 5: set sampling without replacement (default is with replacement)
  59.     extra_options.replace = 0 ;
  60.     model = classRF_train(X_trn,Y_trn, 100, 4, extra_options);
  61.     Y_hat = classRF_predict(X_tst,model);
  62.     fprintf('\nexample 5: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));
  64. % example 6: Using classwt (priors of classes)
  65.     clear extra_options;
  66.     extra_options.classwt = [1 1]; %for the [-1 +1] classses in twonorm
  67.     % if you sort the labels in training and arrange in ascending order then
  68.     % for twonorm you have -1 and +1 classes, with here assigning 1 to
  69.     % both classes
  70.     % As you have specified the classwt above, what happens that the priors are considered
  71.     % also is considered the freq of the labels in the data. If you are
  72.     % confused look into src/rfutils.cpp in normClassWt() function
  74.     model = classRF_train(X_trn,Y_trn, 100, 4, extra_options);
  75.     Y_hat = classRF_predict(X_tst,model);
  76.     fprintf('\nexample 6: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));
  78. % example 7: modify to make class(es) more IMPORTANT than the others
  79.     %  extra_options.cutoff (Classification only) = A vector of length equal to
  80.     %                       number of classes. The 'winning' class for an observation is the one with the maximum ratio of proportion
  81.     %                       of votes to cutoff. Default is 1/k where k is the number of classes (i.e., majority
  82.     %                       vote wins).    clear extra_options;
  83.     extra_options.cutoff = [1/4 3/4]; %for the [-1 +1] classses in twonorm
  84.     % if you sort the labels in training and arrange in ascending order then
  85.     % for twonorm you have -1 and +1 classes, with here assigning 1/4 and
  86.     % 3/4 respectively
  87.     % thus the second class needs a lot less votes to win compared to the first class
  89.     model = classRF_train(X_trn,Y_trn, 100, 4, extra_options);
  90.     Y_hat = classRF_predict(X_tst,model);
  91.     fprintf('\nexample 7: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));
  92.     fprintf('   y_trn is almost 50/50 but y_hat now has %f/%f split\n',length(find(Y_hat~=-1))/length(Y_tst),length(find(Y_hat~=1))/length(Y_tst));
  95. %  extra_options.strata = (not yet stable in code) variable that is used for stratified
  96. %                       sampling. I don't yet know how this works.
  98. % example 8: sampsize example
  99.     %  extra_options.sampsize =  Size(s) of sample to draw. For classification, 
  100.     %                   if sampsize is a vector of the length the number of strata, then sampling is stratified by strata, 
  101.     %                   and the elements of sampsize indicate the numbers to be drawn from the strata.
  102.     clear extra_options
  103.     extra_options.sampsize = size(X_trn,1)*2/3;
  105.     model = classRF_train(X_trn,Y_trn, 100, 4, extra_options);
  106.     Y_hat = classRF_predict(X_tst,model);
  107.     fprintf('\nexample 8: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));
  109. % example 9: nodesize
  110.     %  extra_options.nodesize = Minimum size of terminal nodes. Setting this number larger causes smaller trees
  111.     %                   to be grown (and thus take less time). Note that the default values are different
  112.     %                   for classification (1) and regression (5).
  113.     clear extra_options
  114.     extra_options.nodesize = 2;
  116.     model = classRF_train(X_trn,Y_trn, 100, 4, extra_options);
  117.     Y_hat = classRF_predict(X_tst,model);
  118.     fprintf('\nexample 9: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));
  121. % example 10: calculating importance
  122.     clear extra_options
  123.     extra_options.importance = 1; %(0 = (Default) Don't, 1=calculate)
  125.     model = classRF_train(X_trn,Y_trn, 100, 4, extra_options);
  126.     Y_hat = classRF_predict(X_tst,model);
  127.     fprintf('\nexample 10: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));
  129.     %model will have 3 variables for importance importanceSD and localImp
  130.     %importance = a matrix with nclass + 2 (for classification) or two (for regression) columns.
  131.     %           For classification, the first nclass columns are the class-specific measures
  132.     %           computed as mean decrease in accuracy. The nclass + 1st column is the
  133.     %           mean decrease in accuracy over all classes. The last column is the mean decrease
  134.     %           in Gini index. For Regression, the first column is the mean decrease in
  135.     %           accuracy and the second the mean decrease in MSE. If importance=FALSE,
  136.     %           the last measure is still returned as a vector.
  137.     figure('Name','Importance Plots')
  138.     subplot(2,1,1);
  139.     bar(model.importance(:,end-1));xlabel('feature');ylabel('magnitude');
  140.     title('Mean decrease in Accuracy');
  142.     subplot(2,1,2);
  143.     bar(model.importance(:,end));xlabel('feature');ylabel('magnitude');
  144.     title('Mean decrease in Gini index');
  147.     %importanceSD = The ?standard errors? of the permutation-based importance measure. For classification,
  148.     %           a D by nclass + 1 matrix corresponding to the first nclass + 1
  149.     %           columns of the importance matrix. For regression, a length p vector.
  150.     model.importanceSD
  152. % example 11: calculating local importance
  153.     %  extra_options.localImp = Should casewise importance measure be computed? (Setting this to TRUE will
  154.     %                   override importance.)
  155.     %localImp  = a D by N matrix containing the casewise importance measures, the [i,j] element
  156.     %           of which is the importance of i-th variable on the j-th case. NULL if
  157.     %          localImp=FALSE.
  158.     clear extra_options
  159.     extra_options.localImp = 1; %(0 = (Default) Don't, 1=calculate)
  161.     model = classRF_train(X_trn,Y_trn, 100, 4, extra_options);
  162.     Y_hat = classRF_predict(X_tst,model);
  163.     fprintf('\nexample 11: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));
  165.     model.localImp
  167. % example 12: calculating proximity
  168.     %  extra_options.proximity = Should proximity measure among the rows be calculated?
  169.     clear extra_options
  170.     extra_options.proximity = 1; %(0 = (Default) Don't, 1=calculate)
  172.     model = classRF_train(X_trn,Y_trn, 100, 4, extra_options);
  173.     Y_hat = classRF_predict(X_tst,model);
  174.     fprintf('\nexample 12: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));
  176.     model.proximity
  179. % example 13: use only OOB for proximity
  180.     %  extra_options.oob_prox = Should proximity be calculated only on 'out-of-bag' data?
  181.     clear extra_options
  182.     extra_options.proximity = 1; %(0 = (Default) Don't, 1=calculate)
  183.     extra_options.oob_prox = 0; %(Default = 1 if proximity is enabled,  Don't 0)
  185.     model = classRF_train(X_trn,Y_trn, 100, 4, extra_options);
  186.     Y_hat = classRF_predict(X_tst,model);
  187.     fprintf('\nexample 13: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));
  190. % example 14: to see what is going on behind the scenes    
  191. %  extra_options.do_trace = If set to TRUE, give a more verbose output as randomForest is run. If set to
  192. %                   some integer, then running output is printed for every
  193. %                   do_trace trees.
  194.     clear extra_options
  195.     extra_options.do_trace = 1; %(Default = 0)
  197.     model = classRF_train(X_trn,Y_trn, 100, 4, extra_options);
  198.     Y_hat = classRF_predict(X_tst,model);
  199.     fprintf('\nexample 14: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));
  201. % example 14: to see what is going on behind the scenes    
  202. %  extra_options.keep_inbag Should an n by ntree matrix be returned that keeps track of which samples are
  203. %                   'in-bag' in which trees (but not how many times, if sampling with replacement)
  204. %
  205.     clear extra_options
  206.     extra_options.keep_inbag = 1; %(Default = 0)
  208.     model = classRF_train(X_trn,Y_trn, 100, 4, extra_options);
  209.     Y_hat = classRF_predict(X_tst,model);
  210.     fprintf('\nexample 15: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));
  212.     model.inbag
  214. % example 16: getting the OOB rate. model will have errtr whose first
  215. % column is the OOB rate. and the second column is for the 1-st class and
  216. % so on
  217.     model = classRF_train(X_trn,Y_trn);
  218.     Y_hat = classRF_predict(X_tst,model);
  219.     fprintf('\nexample 16: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));
  221.     figure('Name','OOB error rate');
  222.     plot(model.errtr(:,1)); title('OOB error rate');  xlabel('iteration (# trees)'); ylabel('OOB error rate');
  225. % example 17: getting prediction per tree, votes etc for test set
  226.     model = classRF_train(X_trn,Y_trn);
  228.     test_options.predict_all = 1;
  229.     [Y_hat, votes, prediction_pre_tree] = classRF_predict(X_tst,model,test_options);
  230.     fprintf('\nexample 17: error rate %f\n',   length(find(Y_hat~=Y_tst))/length(Y_tst));

image.png
图3. example10结果
image.png
图4. example16结果