Consider the following 2 Neutral Networks (NN), a 3-layer NN and a 2-layer NN.
(2)
W11
Y
Y1
n
X
Y
X1
Y1
11
Y2
X2
Y2
U
X
X1
X2
Xn-1
W22
(1)
W21
W
,(1)
W11
Z
,(2)
W12
(1)
(2)
W
n-1
Xn-1
(1)
w
n2
Z2
(2)
W2n
X
?
n
"x
??-1
n-1
n
Unn
??
a) (6 pts) Assuming the activation function at Z is a linear function, i.e., g(z) = z.
Show that the 3-layer network can be replaced by the 2-layer network.
(Hint: Express the output Y in the 3-layer network in terms of X and W(1) and W(2) and
assign weights in U).
b) (6 pts) Is it always advantageous to replace a 3-layer network by 2-layer network?
Give justifications for your answer (at least 3 justifications in order to get full marks).
c) (8 pts) Consider the following neural network with the sign function as the activation
function, where sign(x)=1 if x>0 and 0 otherwise
b
Wo
W
W
X2
Ex
W3
Sign
Determine whether it can compute the following functions f and g.
Justify your "yes" answer(s) by giving the weights (Wo, W1, W2, W3) of the network and
your "no" answer(s) by supporting arguments.
X1
X2
X3
f
g
0
0
0
0
1
0
0
1
0
0
0
1
0
0
0
1
1
0
1
1
0
0
1
0
1
0
1
1
1
1
1
0
0
