|MadSci Network: Astronomy|
The sun's gravity does attract everything that comes near it, but sometimes a comet will have enough kinetic energy to escape the sun's gravitational pull, and leave the solar system after only a partial-orbit.
If you understand the difference between potential (stored) energy and kinetic (moving) energy, you can see how this works. Gravitational potential energy is a negative number (or zero, for something infinitely far away). If the kinetic energy is less than the potential energy, than the object is "trapped" around the sun -- it doesn't have enough kinetic energy to escape. It will therefore orbit the sun in an elliptical orbit. Some comets, Like Halley's comet, orbit the sun over and over, in a highly elliptical orbit, because the potential energy is just a little bit bigger than the kinetic energy -- the sun barely has enough pull to bring the comet back every seventy-some years. Comets in stable orbits like this may fly very far away from the sun, but they always come back again.
But if the kinetic energy is bigger than the gravitational potential energy, then the comet can escape. It will swing by the sun and then fly out of the solar system, making not an ellipse-shaped orbit, but instead a path shaped like a parabola or a hyperbola.
The comet will not come back to the sun if this is the case, and it may very well fly toward another star system somewhere in the galaxy. No one knows how many "rogue" comets are currently flying through interstellar space, between the stars.
If that didn't make any sense, here's a good webpage explaining this in more detail.
Try the links in the MadSci Library for more information on Astronomy.