We present a search for dark photon dark matter that could couple to gravitational-wave interferometers using data from Advanced LIGO and Virgo's third observing run. To perform this analysis, we use two methods, one based on cross-correlation of the strain channels in the two nearly aligned LIGO detectors, and one that looks for excess power in the strain channels of the LIGO and Virgo detectors. The excess power method optimizes the Fourier transform coherence time as a function of frequency, to account for the expected signal width due to Doppler modulations. We do not find any evidence of dark photon dark matter with a mass between mA∼10-14-10-11 eV/c2, which corresponds to frequencies between 10-2000 Hz, and therefore provide upper limits on the square of the minimum coupling of dark photons to baryons, i.e., U(1)B dark matter. For the cross-correlation method, the best median constraint on the squared coupling is ∼1.31×10-47 at mA∼4.2×10-13 eV/c2; for the other analysis, the best constraint is ∼2.4×10-47 at mA∼5.7×10-13 eV/c2. These limits improve upon those obtained in direct dark matter detection experiments by a factor of ∼100 for mA∼[2-4]×10-13 eV/c2, and are, in absolute terms, the most stringent constraint so far in a large mass range mA∼2×10-13-8×10-12 eV/c2.