This is simply not true. A person can be arrested at anytime in public by any officer of the law. If this weren't the case many people could avoid arrest and escape.
An arrest warrant simply means that the police are looking for that person and they will be arrested if they're found. Usually given to petty criminals were there is no active search for them.
A search warrant is given to police from a judge. It allows them to search private property without the consent of the owner. Usually given when someone is suspected of having incriminating evidence or illegal possessions.