One interesting thing that I have found is that the linear regression model, Lasso, can be used to select features when making predictions on a dataset. The reason for this is because Lasso puts a constraint on the sum of the absolute values of the model parameters: the sum has to be less than a fixed value (upper bound). In order to do this, the method applies a shrinking (regularisation) process where it penalises the coefficients of the regression variables, shrinking some of them to zero.
The regularisation process is controlled by the alpha parameter in the Lasso model. The higher the value of alpha, the feature coefficients are zero. When alpha is set to zero, Lasso regression produces the same coefficients as linear regression.
Regression coefficients are estimates of the unknown population parameters and describe the relationship between a predictor variable and the response. In linear regression, coefficients are the values that multiply the predictor values.
According to Python’s main machine learning library, sklearn, Lasso’s alpha parameter is the constant that multiplies the L1 term. The default of the alpha parameter is 1.0. When alpha is set to zero then it is equivalent to an ordinary least square, and behaves like the Linear Regression model. It is for that reason that alpha…
