The word that best matches the meaning of the expression "went to the big ocean" is died.
The goldfish died, which is why the father looks at the children mournfully. The "big ocean" is the father's way of saying fish heaven. He could have just said that the goldfish died, but this way, the children imagine the fish in a bigger, better place instead of just picturing a dead fish.