We present a molecular dynamics simulation study that focuses on the formation and growth of nanoscale droplets inside polymer networks. Droplet formation and growth are investigated by the liquid-vapor phase separation of a dilute Lennard-Jones (LJ) fluid inside regularly crosslinked, polymer networks with varying mesh sizes. In a polymer network with small mesh sizes, droplet formation can be suppressed, the extent of which is dependent on the attraction strength between the LJ particles. When droplets form in a polymer network with intermediate mesh sizes, subsequent growth is significantly slower when compared with that in bulk without a polymer network. Interestingly, droplet growth beyond the initial nucleation stage occurs by different mechanisms depending on the mesh size: droplets grow mainly by diffusion and coalescence inside polymer networks with large mesh sizes (as observed in bulk), whereas Ostwald ripening becomes a more dominant mechanism for droplet growth for small mesh sizes. The analysis of droplet trajectories clearly reveals the obstruction effect of the polymer network on the movement of growing droplets, which leads to Ostwald ripening of droplets. This study suggests how polymer networks can be used to control the growth of nanoscale droplets.