Building your Recurrent Neural Network - Step by Step
Welcome to Course 5's first assignment! In this assignment, you will implement key components of a Recurrent Neural Network in numpy.
Recurrent Neural Networks (RNN) are very effective for Natural Language Processing and other sequence tasks because they have "memory". They can read inputs (such as words) one at a time, and remember some information/context through the hidden layer activations that get passed from one time-step to the next. This allows a unidirectional RNN to take information from the past to process later inputs. A bidirectional RNN can take context from both the past and the future.
Notation:
-
Superscript denotes an object associated with the layer.
-
Superscript denotes an object associated with the example.
-
Superscript denotes an object at the time-step.
-
Subscript denotes the entry of a vector.
Example:
- denotes the activation of the 2nd training example (2), 3rd layer [3], 4th time step <4>, and 5th entry in the vector.
Pre-requisites
- We assume that you are already familiar with
numpy
. - To refresh your knowledge of numpy, you can review course 1 of this specialization "Neural Networks and Deep Learning".
-
Specifically, review the week 2 assignment "Python Basics with numpy (optional)".
-
Be careful when modifying the starter code
- When working on graded functions, please remember to only modify the code that is between the
#### START CODE HERE
and
#### END CODE HERE
- In particular, Be careful to not modify the first line of graded routines. These start with:
# GRADED FUNCTION: routine_name
- The automatic grader (autograder) needs these to locate the function.
- Even a change in spacing will cause issues with the autograder.
- It will return 'failed' if these are modified or missing."
Updates for 3b
If you were working on the notebook before this update...
- The current notebook is version "3b".
- You can find your original work saved in the notebook with the previous version name ("v3a")
- To view the file directory, go to the menu "File->Open", and this will open a new tab that shows the file directory.
List of updates
rnn_cell_backward
- fixed error in equations
- harmonize rnn backward diagram with rnn_forward diagram and fixed Wax multiple (changed from at to xt).
- clarified dba batch as summing 'm' examples
- aligned equations
lstm_cell_backward
- aligned equations
lstm_forward
- fixed typo, Wb to bf
lstm_cell_forward
- changed c_next_tmp.shape to a_next_tmp.shape in test case
- clarified dbxx batch as summing 'm' examples
Let's first import all the packages that you will need during this assignment.
import numpy as np
from rnn_utils import *
1 - Forward propagation for the basic Recurrent Neural Network
Later this week, you will generate music using an RNN. The basic RNN that you will implement has the structure below. In this example, .