The doctor will advise the client to Keep the intake of dairy products and grains lesser or avoid them altogether. Dairy products like milk, cheese, ice-cream, yoghurt can worsen all cases of gallbladder disease, liver disease and they are very difficult to digest. Food intolerance is a usual cause of gallbladder problems.