Answer: No, it is not a right triangle.
===================================================
Let
Let's see if a^2 + b^2 = c^2 is a true equation. If so, then we have a right triangle.
a^2 = 5^2 = 25
b^2 = 8^2 = 64
a^2+b^2 = 25+64 = 89
c^2 = 12^2 = 144
We see that a^2+b^2 = 89, but c^2 = 144. So a^2+b^2 = c^2 is a false equation when (a,b,c) = (5,8,12). Therefore, this is not a right triangle.
Side note: Because c^2 is larger than a^2+b^2 in this case, this means we have an obtuse triangle.