First check the characteristic solution: the characteristic equation for this DE is
r ² - 3r + 2 = (r - 2) (r - 1) = 0
with roots r = 2 and r = 1, so the characteristic solution is
y (char.) = C₁ exp(2x) + C₂ exp(x)
For the ansatz particular solution, we might first try
y (part.) = (ax + b) + (cx + d) exp(x) + e exp(3x)
where ax + b corresponds to the 2x term on the right side, (cx + d) exp(x) corresponds to (1 + 2x) exp(x), and e exp(3x) corresponds to 4 exp(3x).
However, exp(x) is already accounted for in the characteristic solution, we multiply the second group by x :
y (part.) = (ax + b) + (cx ² + dx) exp(x) + e exp(3x)
Now take the derivatives of y (part.), substitute them into the DE, and solve for the coefficients.
y' (part.) = a + (2cx + d) exp(x) + (cx ² + dx) exp(x) + 3e exp(3x)
… = a + (cx ² + (2c + d)x + d) exp(x) + 3e exp(3x)
y'' (part.) = (2cx + 2c + d) exp(x) + (cx ² + (2c + d)x + d) exp(x) + 9e exp(3x)
… = (cx ² + (4c + d)x + 2c + 2d) exp(x) + 9e exp(3x)
Substituting every relevant expression and simplifying reduces the equation to
(cx ² + (4c + d)x + 2c + 2d) exp(x) + 9e exp(3x)
… - 3 [a + (cx ² + (2c + d)x + d) exp(x) + 3e exp(3x)]
… +2 [(ax + b) + (cx ² + dx) exp(x) + e exp(3x)]
= 2x + (1 + 2x) exp(x) + 4 exp(3x)
… … …
2ax - 3a + 2b + (-2cx + 2c - d) exp(x) + 2e exp(3x)
= 2x + (1 + 2x) exp(x) + 4 exp(3x)
Then, equating coefficients of corresponding terms on both sides, we have the system of equations,
x : 2a = 2
1 : -3a + 2b = 0
exp(x) : 2c - d = 1
x exp(x) : -2c = 2
exp(3x) : 2e = 4
Solving the system gives
a = 1, b = 3/2, c = -1, d = -3, e = 2
Then the general solution to the DE is
y(x) = C₁ exp(2x) + C₂ exp(x) + x + 3/2 - (x ² + 3x) exp(x) + 2 exp(3x)