In this homework, you should try to implement a three layer artificial neural network -threelayernet based one twolayernet. note that since you have three layers in the network, there should be three fc (fully connected) layers and two activation layers (use relu). the h2 in __init__ function is the output dimension of the second fc layer and the input dimension of the third fc layer.

(a) train the twolayernet with default parameter setting, set iteration=25000, try both optimizers ‘sgd’ and ‘sgdmomentum‘, report the training and testing accuracy. plot the training loss over iterations like the below image. (5 points)
training loss over iterations. x-axis represents the training iterations/100, y is the training loss.

(b) implement the threelayernet by yourself. set h1=300, h2=100, and train the threelayernet model 25000 iterations. report accuracy on both training and testing dataset and draw the loss figure. (10 points)
(c) train lenet5 model 2000 iterations, report the accuracy on testing dataset. (5 points)