Not as bad as Japanese-Americans, but they were still treated with suspicion.
Part of this was due to the Black Tom explosion that scared the living snot out of the government. This played a major role in the decision to intern the Japanese-Americans in World War II. FDR made an explicit statement to the effect that we "weren't going to have any more Black Toms." Of course, the whole concept of taking away the civil liberties of Americans is nuts, but it's a form of nuts that didn't happen in a vacuum.