Creating a Siamese model using Trax: Ungraded Lecture Notebook

import trax
from trax import layers as tl
import trax.fastmath.numpy as np
import numpy

# Setting random seeds
trax.supervised.trainer_lib.init_random_number_generators(10)
numpy.random.seed(10)

INFO:tensorflow:tokens_length=568 inputs_length=512 targets_length=114 noise_density=0.15 mean_noise_span_length=3.0

L2 Normalization

Before building the model you will need to define a function that applies L2 normalization to a tensor. This is very important because in this week's assignment you will create a custom loss function which expects the tensors it receives to be normalized. Luckily this is pretty straightforward:

def normalize(x):
    return x / np.sqrt(np.sum(x * x, axis=-1, keepdims=True))