Logistic regression is a very basic model that is perhaps one of the first algorithms a student of data science learns. The mechanics of this algorithm are likely to come up in a job interview, so it is important to study it and become familiar with how it works in a machine learning project.
Logistic regression is a type of statistical model that is often used for classification and predictive analytics. It is used to estimate the relationship between a dependent variable and one or more independent variables, making a prediction about a categorical value. Logistic regression estimates the probability of an event occurring based on a given dataset of independent values. Since the outcome is a probability, the dependent variable is a value between 0 and 1.
In logistic regression, a logic transformation is applied to the odds, which means the probability of success is divided by the probability of failure. The result of this calculation is called the log odds or the natural logarithm of odds.
The formula for logistic regression is:-
The formula for the natural logarithm of odds is:-
For binary classification, a probability less than 0.5 will result in a 0, while a probability equal to or greater than 0.5 will result in a 1.
There are three types of logistic regression, being:-
- Binary logistic regression
- Multinomial logistic regression
- Ordinal logistic regression
In binary logistic regression the dependent variable has only two possible outcomes, being either 0 or 1. An example of this is a model to determine whether an email is spam or not. Within logistic regression, binary logistic regression is the most used approach and one of the most common classifiers used for binary classification.
Multinomial logistic regression is a type of regression model where the dependent variable has three or more possible outcomes.
Ordinal logistic regression is a type of regression where the dependent variable has three or more outcomes in a defined order. An example of ordinal logistic regression would be a grading scale from A to F.
Sklearn is Python’s machine learning library and it has the logistic regression function that can be used to train and fit the data into the model, and then make…