Learn practical skills, build real-world projects, and advance your career

Classifying CIFAR10 images using a ResNet and Regularization techniques in PyTorch

Training an image classifier from scratch to over 90% accuracy in less than 5 minutes on a single GPU

Part 6 of "PyTorch: Zero to GANs"

This post is the sixth in a series of tutorials on building deep learning models with PyTorch, an open source neural networks library. Check out the full series:

  1. PyTorch Basics: Tensors & Gradients
  2. Linear Regression & Gradient Descent
  3. Image Classfication using Logistic Regression
  4. Training Deep Neural Networks on a GPU
  5. Image Classification using Convolutional Neural Networks
  6. Data Augmentation, Regularization and ResNets
  7. Generating Images using Generative Adverserial Networks

This notebook is an extension to the tutorial Image Classification using CNNs in PyTorch, where we trained a deep convolutional neural network to classify images from the CIFAR10 dataset with around 75% accuracy. Here are some images from the dataset:

cifar10

In this tutorial, we'll use the following techniques to achieve over 90% accuracy in less than 5 minutes:

  • Data normalization
  • Data augmentation
  • Residual connections
  • Batch normalization
  • Learning rate scheduling
  • Weight Decay
  • Gradient clipping
  • Adam optimizer

System Setup

This notebook is hosted on Jovian.ml, a platform for sharing data science projects. If you want to follow along and run the code as you read, you can choose the "Run on Kaggle" option from the "Run" dropdown above. Remember select "GPU" as the accelerator and turn on internet from the sidebar within the Kaggle notebook.

Otherwise, to run the code on your machine, you can clone the notebook, install the required dependencies using conda, and start Jupyter by running the following commands:

pip install jovian --upgrade               # Install the jovian library 
jovian clone aakashns/05b-cifar10-resnet   # Download notebook & dependencies
cd 05b-cifar10-resnet                      # Enter the created directory 
conda create -n 05b-cifar10-resnet         # Create virtual env
conda activate 05b-cifar10-resnet          # Activate virtual env
conda install jupyter                      # Install Jupyter
jupyter notebook                           # Start Jupyter

For a more detailed explanation of the above steps, check out the System setup section in the first notebook. Before you start executing the code below, you may want to clear the cell outputs by selecting "Kernel > Restart and Clear Output" from the Jupyter notebook menu bar, to avoid confusion.

We begin by importing the required modules & libraries.

# Uncomment and run the commands below if imports fail
# !conda install numpy pandas pytorch torchvision cpuonly -c pytorch -y
# !pip install matplotlib --upgrade --quiet
import os
import torch
import torchvision
import tarfile
import torch.nn as nn
import numpy as np
import torch.nn.functional as F
from torchvision.datasets.utils import download_url
from torchvision.datasets import ImageFolder
from torch.utils.data import DataLoader
import torchvision.transforms as tt
from torch.utils.data import random_split
from torchvision.utils import make_grid
import matplotlib.pyplot as plt
%matplotlib inline
project_name='05b-cifar10-resnet-live'