Final answer:
The first-order derivative of E(w) is computed using the chain rule, and the logistic regression maintains a linear decision boundary because classification is based on whether wTx crosses a threshold, irrespective of the value of the threshold. Changing the threshold value shifts the boundary but does not affect its linearity.
Step-by-step explanation:
The calculation of the first-order derivative of the function E(w) involves applying the chain rule and the properties of the logistic function. Let's denote the logistic function as θ(z) = (1 + e-z)-1. The derivative of E(w) with respect to w is ∇E(w) = ∂E/∂w. Using the chain rule:
∇E(w) = ΣNn=1(-ynxn(1 - θ(ynwTxn)))
This takes into account the derivative of the logarithm and the logistic function.
(b) Even though θ(wTx) is a nonlinear function, the decision boundary of logistic regression remains linear because it is determined by the set of points for which wTx = 0. This can be visualized as a line (or hyperplane in higher dimensions) in the input space. The nonlinearity of θ does not affect this boundary because it only scales the output between 0 and 1.
(c) The decision boundary is still linear if the prediction rule changes to use a threshold of 0.9 instead of 0.5. This is because the decision boundary is still determined by wTx being equal to the log-odds corresponding to θ-1(0.9), which is a constant. Essentially, changing the threshold shifts the boundary in input space but does not change its linearity.
(d) The essential property of logistic regression that results in a linear decision boundary is that the classification decision is made based on whether the linear combination of input features wTx crosses a threshold. Regardless of the value of this threshold, the boundary is defined by a set of points with equal linear score, which forms a hyperplane when graphed in the input feature space.