A comparison of Pytorch and sklearn’s Linear Regression models

For the past several days I have been working on the California House Prices dataset, initially as a favour to someone, but later to see if I could use different models to improve the accuracy of predictions.

When I was experimenting on the models that I can use on the California House Prices dataset, I tried out sklearn’s LinearRegression and then PyTorch’s Linear Regression model because I wanted to see if I could achieve a greater accuracy with a neural network. The reason for this is because neural networks are supposed to be superior to other models. What occurred was quite surprising and that is why I decided to write a post on my findings.

I used the California House Prices dataset that comes with Google Colab, which is a free online Jupyter Notebook.

The code I used for sklearn’s LinearRegression model can be found in the screenshot below. The error I achieved using this model was 59,248:-

I then used PyTorch’s Linear Regression model. As can be seen from the code below, this model is much more complex than sklearn’s model. I had to initially convert the data to PyTorch tensors before it could be used in the model:-

I then had to define the model:-

Once the tensors and the model had been defined, I had to train the data into the model. In this instance I used 20 million epochs to get the highest level of accuracy possible:-

I then predicted on the validation set and imagine my surprise when I achieve an error of 59,697, which is 449 greater than sklearn’s model!

I then plotted the predictions against the validation set, and it can be seen that PyTorch was unable to improve upon sklearn’s values:-

The moral of the story is that neural networks are not always better than more traditional methods of making predictions. I spent a lot of time training PyTorch’s Linear Regression model and it still was unable to adequately compete against sklearn’s LinearRegression.