Phenotypic plasticity is known to arise in varying habitats where it diminishes harmful environmental effects. How plasticity shapes genetic architecture of traits under varying selection is unknown. Using an analytic approximation and Monte Carlo simulations, we show that balanced polymorphism and recombination modification arise simultaneously as a consequence of epistatic plastic modification in periodic environments. Under this novel finite-population scenario of balancing selection, recombination arises between a plasticity modifier locus and its target coding-locus in the absence of typically assumed antagonistic co-evolution or constant influx of mutation. Moreover, even in the absence of epistasis or initial physical linkage between the co-modified coding loci, they cluster together such that alleles with aligned effects associate in supergenes. In turn, diversity increases due to both recombination between modifier and target loci and recombination suppression between additively acting target loci. This study uncovers the role of phenotypic plasticity in the maintenance of genetic variation and in the evolution of recombination rates.