It all comes down to antibodies.
Antibodies are specialized proteins that are produced in response to anything that your immune system might need to fight off, such as bacteria and viruses. Consider them the first ‘identification’ step of the immune system, trying to find anything that doesn’t belong.
Despite the similar name, antibodies are not to be confused with antigens. An antigen is any kind of molecule, such as a protein or a carbohydrate, that can be recognized by the immune system—the antibodies target whichever antigens it identifies as being foreign invaders.
It is important though for the antibodies to not identify antigens that do belong. They’re also produced based on the antigens that are not already present on your red blood cells. For example:
A type blood has anti-B antibody in the plasma
B type blood has anti-A antibody in the plasma
AB has neither A nor B antibody in the plasma
O has both A and B antibody in the plasma.