316,614 views
7 votes
7 votes
Consider a neural network with two hidden layers: p= 4 input units, 2 units in the first hidden layer, 3 units in the second hidden layer, and a single output. (a) Draw a picture of the network, similar to Figures 10.1 or 10.4.

(b) Write out an expression for f(x), assuming ReLU activation functions. Be as explicit as you can! (c) Now plug in some values for the coefficients and write out the value of f(X).
(d) How many parameters are there?

User Maximilian Peters
by
3.1k points

1 Answer

13 votes
13 votes

Final answer:

The neural network in question can be visualized as a series of connected units across layers, with the ReLU activation function applied to a weighted sum at each layer. The expression for f(x) can be defined using weights and biases and evaluated by plugging in values for them. The total number of parameters in this network is 20.

Step-by-step explanation:

For part (a), imagine a diagram with four input units at the bottom. These then connect to the first hidden layer with two units, which in turn connects to the second hidden layer with three units. Finally, these three units connect to a single output unit, completing the network's architecture. For part (b), the expression for f(x), using ReLU activation functions, would be a combination of weighted sums and the application of the ReLU function. ReLU(x) is defined as max(0, x), meaning that the activation is the input itself if positive, and zero otherwise. If we denote Wij as the weight from unit i to unit j and bj as the bias for unit j, the expression can be written as: f(x) = ReLU(ReLU(W11x + b1) * W21 + ReLU(W12x + b2) * W22 + b3).

For part (c), plugging in arbitrary values for the coefficients and given inputs, you could compute the value of f(X) by applying the weights and biases sequentially through the layers of the network. For part (d), the number of parameters in the neural network includes the weights and biases for each connection. With 4 input units, 2 units in the first hidden layer, 3 units in the second hidden layer, and one output unit, the number of parameters would be the sum of the weights and biases across all these connections. This can be calculated as (4 * 2 + 2) + (2 * 3 + 3) + (3 * 1 + 1) = 20.

User Michael Ellick Ang
by
3.2k points