Final answer:
Using the method of undetermined coefficients involves finding the complementary solution from the characteristic equation and assuming a form for the particular solution, which after substitution, gives the general solution.
Step-by-step explanation:
The method of undetermined coefficients is used to solve nonhomogeneous linear differential equations with constant coefficients. When the nonhomogeneous term is x e^{2x}, a particular solution is assumed to be of the form Ax e^{2x}. In this case, the given differential equation is y''' - 3y'' + 4y = x e^{2x}. The characteristic equation associated with the corresponding homogeneous equation y''' - 3y'' + 4y = 0 is r^3 - 3r^2 + 4 = 0. The roots of this equation provide the complementary solution. After finding the complementary solution, substitute the assumed form of the particular solution into the original differential equation and equate the coefficients of like terms to solve for A. Combine the complementary and particular solutions to get the general solution to the original equation.