I will show how the study of non-tautological classes on the moduli space of abelian varieties helps explain the structure of the tautological ring of the moduli space of curves of compact type. On the curves side, this is joint work with Hannah Larson and Johannes Schmitt, and on the abelian varieties side, it is joint with Dragos Oprea and Rahul Pandharipande.