The imagery of "the heart" suggests that the relationship between the society the narrator plans to build and the outside world will be the following: membership for the new society on the mountain will be open to anyone.
In chapter twelve of Anthem, which is the chapter from which the excerpt was extracted, Equality 7-2521 and the Golden One decide that they will launch a new race in the abandoned house they found from the Unmentionable Times. This new race that they vow to create will accept individualism, and they intend to make it the heart of the earth, i.e., the central piece of the planet, the core that will keep life flowing for humanity (much like a heart keeps the body alive by pumping blood). A humanity that believes in individualism, the word "I", and the supremacy of the ego.