Answer:
Let
The subset S is a subgroup of GL(n,R) if satisfies:
1. The identity matrix
belong to S.
2. If A and B are in S then AB is in S.
3. If A is belong to S then
belongs to S.
Let's see if S satisfies these conditions.
1. We know that
, then
.
2. Let A and B in S.
Then AB is in S.
3. Let
,
, then
.
Since S satisfies all conditions then S is a subgroup of GL(n,R).