Question

Don't forget the regularization loss! For ( L_{2} ) regularization, ( R(w)=r e g * w^{2} ), what is ( {d R(w)}/{d w} ? ) How about for ( L_{1} ) regularization?|

Answer 1

In L2 regularization, the derivative of R(w) with respect to w is 2 * reg * w. In L1 regularization, the derivative depends on the sign of w.

Step-by-step explanation:

For L2 regularization, the regularization loss is defined as R(w) = reg * w2. To find the derivative of R(w) with respect to w, we can use the power rule in calculus. Taking the derivative, we get dR(w)/dw = 2 * reg * w.

For L1 regularization, the regularization loss is defined as R(w) = reg * |w|. The derivative of the absolute value function is dependent on the sign of w. When w > 0, dR(w)/dw = reg. When w < 0, dR(w)/dw = -reg. At w = 0, we say the derivative is undefined.