The speaker is not concerned with the missing grain.
In the poem the speaker understands that the mouse needs the grain to live so he is not concerned at all about the missing grain. There are many grains left so it makes no difference to him that the mouse stole one.