First: the homogeneous solutions: the characteristic equation is4r^2 - 4r - 3 = 0which has roots r = 3/2, -1/2 hence the homogeneous solution isy = c1.exp(-x/2) + c2.exp(3x/2)
next you need the general form for the guess for yp and that isyp = A1cos(2x) + A2sin(2x)
Now substitute that into the equation and solve for A1, A2.