|MadSci Network: Astronomy|
I'm assuming by non-gaseous planets you are asking how large a terrestrial (Earth-like) planet can be, before it inevitably becomes a gas giant such as Uranus or Jupiter. It's a good question, as we have no direct experience of terrestrial planets bigger than the Earth, or gas giants less than the mass of Uranus (about 17 Earth-masses). In golfing terms, we might well wonder where the "cut" should be made. In science terms we might wonder if there's a cut at all, and whether our definition of planets as terrestrial or gas giant, is merely an artifact of our experience with the solar system.
To help explain why planets have atmospheres in the first place, we should note that the ability for any planet to hold on to a gaseous atmosphere for long periods depends on:
the amount of mass and density of the planet;
the gas being considered; and
the solar energy present at the planet.
Let's take each of these in turn.
Due to the Earth's mass and density, the velocity required by an object to escape the planet is approximately 11 km/s. It's clear that any gas particle (or space probe for that matter) which exceeds 11 km/s will be permanently lost from the Earth. The less massive a body, the lower this escape velocity. Notice that the Moon, eighty times less massive than the Earth, and of a similar sort of density, has virtually no atmosphere.
Similarly, as the mass of a planet increases, the escape velocity increases (again dependent on similar density). Uranus, for example, has a lower density than that of the Earth, but its great mass means that the Uranian escape velocity is over 21 km/s. We shouldn't be surprised that it has a lot of gas in its atmosphere.)
Different gases have different masses. Hydrogen molecules (H2) make up the lightest gas, while Helium (He) gas contains atoms that are twice as massive. Nitrogen (N2) and Oxygen (O2) are fourteen and sixteen times more massive than H2. Just as it's easier to push a bike than to push a car, it's easier (that is, it takes less energy) to speed up a low-mass gas molecule compared to a heavy one. We might, therefore, expect to find Earth's atmosphere lacking in lighter elements, as those could achieve escape velocity. Indeed, our atmosphere is 99% N2 and O2, with tiny traces of He, and it's wholly lacking in H2. On the other hand, Uranus atmosphere is almost 98% Hydrogen and Helium gas---that planet's escape velocity is higher than the escape velocity for these low-mass gases, so they have been retained.
Solar energy provides the bulk of warming to all the planets. What we call the temperature of a gas is a direct reflection of the amount of kinetic energy held in that gas, and kinetic energy is dependent on the velocity of the particles in it (it equals m*v*v/2). The higher temperature a gas has, the higher are its particles' velocities, and vice versa. If we could move Uranus closer to the Sun (heating it up) we would find a distance at which it would start to lose Hydrogen and Helium gas into space. Eventually a rocky, terrestrial-type core would be left.
Recent discoveries of some extrasolar planets known as "hot Jupiters" suggests that they are losing their light gasses in just such a manner, and will one day be lava-covered "terrestrial" planets of up to 15 Earth masses. Possibly these mark the largest sizes non-gaseous worlds can be.
As with the recent discussions over whether Pluto should be justifiably called a planet, I suspect there is no obvious cut-off between terrestrials and gas-giants, and that (as you've seen) the mass, density, location and initial materials to be found in a protostar's dust clouds will all affect a planet's eventual fate.
[This previous MadSci answer on the formation of the solar system may also be of interest. Moderator]
Try the links in the MadSci Library for more information on Astronomy.