Decision Trees and Random Forest analysis, R template of Decision tree

Decision trees and random forests are data driven methods, which can be used both for classification (called Classification tree) and for prediction (called Regression tree). Trees are wildly spread because it is easy to interpret for non-professionals in Data mining and they provide accurate results. Decision trees algorithm based on separating observations into smaller groups creating smaller splits. These splits create rules, which are used to classify or predict a desirable outcome. For example, IF Income > 500$ and Age < 23 than the observation belongs to class 1.

Here is an example of a decision tree, which helps us to predict approve the loan or not.

There are two major ideas, which help us to construct a tree: Recursive partitioning and Pruning.

Recursive partitioning:  Repeatedly split the records into two parts to achieve maximum homogeneity within new parts.

Pruning the tree: Simplify the tree by pruning peripheral branches to avoid overfitting.

Recursive partitioning steps:

1.      Pick one of the predictor variable Xi

2.

Logistic regression explained and example of Logistic regression created by R.

Logistic regression explained: Linear regression very looks like Logistic regression by the idea but the Logistic regression expands the area where regression can be used. The expanded area is a Y dependent variable, which in Logistic regression can be categorical variable. For example when Y dependent variable donates a recommendation on holding, selling, buying a stock we can create an ordinal logistic regression with three categorical variables. Logistic regression uses predictor variables  to classify an observation where its class is unknown.

This regression is used in applications such as:

1.  Classifying customers as returning or churning                           (classification)

2.  Finding factors that differentiate between male and female         top executives (profiling)

3. Predicting approval or disapproval of a loan based on                 customer information

Logistic regression is used in different fields, especially when we need to classify a category (in particular a binary).

            Logistic regression Model: