The answer is The Whig Party. The north voter base mostly settled to the new Republican Party. In the South, most merged the Know Nothing Party, which ineffectively ran Fillmore in the 1856 presidential election, by which time the Whig Party had turn out to be virtually non-operational. Some past Whigs became Democrats.