Answer:
Matrices are a useful way to represent, manipulate and study linear maps between finite-dimensional vector spaces (if you have chosen basis).
Matrices can also represent quadratic forms (it's useful, for example, in analysis to study hessian matrices, which help us to study the behavior of critical points).