But you are also looking at a universe that was much more compact then, with galaxies and structures bunched much closer together than now. It may be a narrow field of vision compared to the current size of the universe, but that narrow field of vision has also expanded in the subsequent 13+ billion years.