The environment can act directly on phenotypes, which are, of course the variations presented by genotypes. Genotype does determine phenotype but it is the phenotype which is exposed to the external environment so selection pressures can only act on the traits individuals have, not what gives them the trait i.e. if the same trait is produced by two different genotypes selection will act equally on both.