|MadSci Network: Astronomy|
The radiation that we see as the Cosmic Microwave Background wasn't created at the instant of the Big Bang but about 300,000 years afterwards. This corresponds to the time at which the expanding Universe would have been cool enough for neutral hydrogen to form from the protons and electrons that existed in the hot early Universe, at which point the Universe became transparent to radiation.
It's not clear to astronomers when galaxies, such as the Milky Way, formed but we see galaxies today at redshifts large enough to imply that galaxies existed around about 250 million or 500 million years after the Big Bang.
When we look out into the Universe, we are looking back in time, so we have to see the remnant radiation of the Big Bang at some point as we look further and further out. It's important to note that although we can only see for 12 billion ly, because of the finite age of the Universe, the Universe itself may be much larger than that. According to a theory known as inflation theory, which was proposed to explain some problems with Big Bang theory, the very early Universe may have gone through a phase during which it expanded much faster than the speed of light. This is fine from the point of view of relativity because matter isn't actually moving, space itself is expanding.
It's also important to note that the Big Bang didn't take place at one particularly point in the Universe, with radiation and matter then moving out from that point. What happened was that the singularity of the Big Bang was the whole Universe, which then expanded. At first it was full of radiation but as it expanded and cooled, matter formed from the photons of radiation. As it continued to get larger, the matter and radiation decoupled (this is the point at which the CMB was formed), then the matter clumped under the influence of gravity, forming galaxies and stars.
This is only a very brief description of modern cosmology. For more information, you should probably take a look at Ned Wright's cosmology tutorial.
Try the links in the MadSci Library for more information on Astronomy.