Here we establish femtosecond covariance spectroscopy as a technique that uses ultrashort stochastic light pulses to measure nonlinear material responses. By using pulses with spectrally uncorrelated fluctuations we can leverage on the noise and consider each repetition of the experiment as a measurement under different conditions. In this limit, we demonstrate that nonlinear processes in the sample can be retrieved by measuring the spectral correlations in different pulses. We validate the approach by studying stimulated Raman scattering in α-quartz. This concept can be applied to reveal low-energy modes of electronic, spin, and vibrational origin and adapted to different techniques and wavelength ranges, from optical to X-ray free-electron lasers, where strong stochastic fluctuations are unavoidable.