So, you know that phrase "I'm afraid"? It can actually show that someone really wants to do something, but there are reasons holding them back. Like if someone says "Much as I would love to visit him, I'm afraid." It means they really want to see someone, but for some reason they can't. It's kind of a sad feeling because they're missing out on something they really want to do. It's like having a strong desire to do something, but not being able to because of circumstances.
But, even though your afraid, you should put your fears aside and do what is right.