Not always... The kinds of acid-fast organisms that do cause disease that are serious, like tuberculosis, are more difficult to treat because there are large amounts of lipids in their cell walls.
Those characteristics can make the organism serious and dangerous.