Training Deep Neural Networks on a GPU with PyTorch
Part 4 of "PyTorch: Zero to GANs"
This notebook is the fourth in a series of tutorials on building deep learning models with PyTorch, an open source neural networks library. Check out the full series:
- PyTorch Basics: Tensors & Gradients
- Linear Regression & Gradient Descent
- Image Classfication using Logistic Regression
- Training Deep Neural Networks on a GPU
- Image Classification using Convolutional Neural Networks
- Data Augmentation, Regularization and ResNets
- Generating Images using Generative Adverserial Networks
In the previous tutorial, we trained a logistic regression model to identify handwritten digits from the MNIST dataset with an accuracy of around 86%.
However, we also noticed that it's quite difficult to improve the accuracy beyond 87%, due to the limited power of the model. In this post, we'll try to improve upon it using a feedforward neural network.
This tutorial takes a code-first approach, and you should try to follow along by running and experimenting with the code yourself. The easiest way to start executing this notebook is to click the "Run" button at the top of this page, and select "Run on Kaggle". This will run the notebook on Kaggle, a free online service for running Jupyter notebooks (you might need to create an account).
Running on your computer locally
(Skip this if you are running on Kaggle) You can clone this notebook, install the required dependencies using conda, and start Jupyter by running the following commands on the terminal:
pip install jovian --upgrade # Install the jovian library
jovian clone aakashns/04-feedforward-nn # Download notebook
cd 04-feedforward-nn # Enter the created directory
conda create -n 04-feedfoward-nn python=3.8 # Create a conda environment
conda activate 04-feedforward-nn # Activate virtual environment
conda install jupyter # Install Jupyter
jupyter notebook # Start Jupyter
On older versions of conda, you might need to run
source activate 04-feedfoward-nn to activate the virtual environment. For a more detailed explanation of the above steps, check out the System setup section in the first notebook.
# Uncomment and run the commands below if imports fail
# !conda install numpy pandas pytorch torchvision cpuonly -c pytorch -y
# !pip install matplotlib --upgrade --quiet
import numpy as np
import matplotlib.pyplot as plt
import torch.nn as nn
import torch.nn.functional as F
from torchvision.datasets import MNIST
from torchvision.transforms import ToTensor
from torchvision.utils import make_grid
from torch.utils.data.dataloader import DataLoader
from torch.utils.data import random_split