|MadSci Network: Astronomy|
In principle the accretion of mass to the Earth should affect its rotation rate, but there are other things going on that swamp this effect. (This question has sort of been addressed here and here , but not quite with this emphasis.)
The tidal effects of the Moon, and to a lesser extent the Sun and other planets, are now much more important for influencing the motions of the Earth than the bombardment of matter from space. There's a long, slow decrease in rotation speed that has been going on since the Moon formed, due to the tides raised by the Moon on the Earth, and the subsequent loss of rotational kinetic energy from the Earth by frictional heating.
Try the links in the MadSci Library for more information on Astronomy.