In general, for a problem like this, you make the substitution indicated in the problem statement, then find the value(s) of the variable (a, in this case) that make the equations match the way the problem statement tells you.
Here, we can take a shortcut, because we recognize that both f(x) and f(x+a) are perfect squares.
It is convenient to rewrite the function f(x+a) to match the form of f(x).
... f(x+a) = (x-3)²
Now, we can compare this to f(x+a) using the definition of f(x).
... f(x+a) = (x+a+2)² = (x-3)²
Comparing these forms, we see that
... a+2 = -3
... a = -5 . . . . . . subtract 2 to isolate a
This matches your first choice, ...
.... (A) -5