Machine Learning with Coffee

We introduce the concept of a perceptron as the basic component of a neural network. We talk about how important is to understand the concept of backpropagation applied to a single neuron.

XGBoost is an open-source software library which has won several Machine Learning competitions in Kaggle. It is based on the principles of gradient boosting, which is based on the ideas of the Leo Breiman, the creator of Random Forest. The theory behind gradient boosting was later formalized by Jerome H. Friedman. Gradient boosting combines weak learners just as Random Forest. XGBoost is an engineering implementation which includes a clever penalization of trees and a proportional shrinking of leaf nodes.

Random Forest is one of the best out-of-the-shelf algorithms. In this episode we try to understand the intuition behind the Random Forest and how it tries to leverage the capabilities of Decision Trees by aggregating them using a very smart trick called “bagging”. Variable Importance and out-of-bag error are two of the nice capabilities of Random Forest which allow us to find the most important predictors and compute a good generalization error, respectively. 

Logistic regression is a very robust machine learning technique which can be used in three modes: binary, multinomial and ordinal. We talk about assumptions and some misconceptions. For example, people believe that because logistic regression fits only a linear separator in the expanded dimensional space it wouldn’t be able to fit a complex boundary in the original space. Also, people normally use either linear regression or multinomial logistic regression when they should be using ordinal logistic regression. 

In this episode I will try to convince you that Linear Regression is one of the most powerful Machine Learning algorithms. We will talk about common misconceptions, especially that Linear Regression is not able to model non-linear relationships. We also discuss how the myth of normality encourages many people to completely discard Linear Regression on non-normal data, when in reality, normality of the data has nothing to do with this assumption. Finally, I provide advice in how to check, but most importantly, how to fix any violated assumption in Linear Regression.