We consider a directed polymer model in dimension 1+1, where the disorder is given by the occupation field of a Poisson system of independent random walks on Z. In a suitable continuum and weak disorder limit, we show that the family of quenched partition functions of the directed polymer converges to the Stratonovich solution of a multiplicative stochastic heat equation with a Gaussian noise whose space-time covariance is given by the heat kernel.