there is both. the persons body heat transfers to the ice while the cold transfers off the ice to the persons hand. thats why when you hold and ice cube for a long enough time it melts because your body heat melts it, while at the same time your hand starts to get colder.