Smart trees are collections of Classification Trees with binary splits.
For a given observation, expressed as an input vector, each tree makes a decision or provides classification probabilities. Decisions (probabilities) from all trees are treated as votes and the final answer is the decision which obtained the most votes (obtained the highest average probability). Votes may be weighted or not, depending on the selected voting method.
The building algorithm grows the required number of classification trees using the training data set. In order for the Smart Trees module to have an advantage over a single classification tree some variance among the trees comprising the model is required. This variance is introduced in the following ways:
By randomly selecting the training data for each tree. The training subset for each tree is obtained in the following manner: let N denote the number of samples in the training data set. The training data for a single tree is obtained by drawing at random N samples with replacement. The unused data samples are referred to as out-of-bag (oob) data, and are used later on to obtain classification error.
By (randomly or deterministically) selecting the variables on which splits are based, either for the whole tree or for individual splits.
By randomly selecting the split evaluation method for each tree.
By introducing random variance of the minimal node size (within the given bounds) for each tree.
The user controls how variance is introduced into the trees via model parameters. The construction algorithm builds each tree individually according to the provided parameters, as described in the Classification trees – Tree building algorithm section. All trees are binary and no pruning is performed.
For each tree an oob classification error estimate is computed. For each tree approximately 1/3 of training data samples were left out (the oob data). The oob data set is used to obtain an unbiased classification error estimate for the tree.
The accuracy of a particular tree on the oob data is also used to compute its weight in the final voting process, according to the weighting statistic parameter (link).
Variable importance estimate for each tree is calculated according to the method used by the Classification Trees module.