Answer: Yes. Workers were worried because many immigrants were willing to work for low wages.
Americans want to restrict immigration because Americans were scared of the competition it would bring with regards to housing and jobs. Since immigrants are unskilled and poor they would work for less or lower wages.