He believed it could reduce the debt of the country. There's a lyric from the musical Hamilton that helps describe his view on the bank.
"If we assume the debts the union gets a new line of credit a financial diaretic, how do you not get it? If we're aggressive and competitive, the Union gets a boost, you'd rather give it a sedative."
This lyric occured in the song where he was battling with Thomas Jefferson over the pros and cons of a national bank.