Notice that the subject - madre - is singular. This means the verb will also be singular. Let's take a look at our options.
Estamos - "we are"
Example: Estamos ocupados. "We are busy."
Está - "is" or "this"
Example 1: Esta mujer. "This woman."
Example 2: Ella esta aquí. "She is here."
Están - "they are"
Example: Están en la escuela. "They are at school."
From all of this, you can see that the only option that makes sense here is está - Mi madre no está en el garaje. "My mother is not in the garage."
I hope this helps you!