The physicists' notion of energy is derived from Newtonian mechanics. The theory of thermodynamics is developed based on that notion, and the realization of mechanical energy dissipation in terms of heat. Since the work of L. Boltzmann, who trusted that atoms were real as early as in 1884, the heat became intimately related to the stochastic motion of the invisible atoms and molecules. In this talk, starting from a stochastic description of a class of rather general dynamics that is not limited to mechanics, we show a notion of energy can be derived mathematically, in the limit of vanishing stochasticity, based on the Kullback-Leibler divergence, or relative entropy associated with the stochastic, Markov processes. With the emergent notion of an energy function, e.g., "landscape", a mathematical structure inherent to the stochastic dynamics, which is akin to thermodynamics, is revealed. This analysis implies that an abstract "thermodynamic structure" exists, and can be formulated, for dynamics of complex systems independent of classical thermal physics, for example, in ecology.