|MadSci Network: Physics|
To quote a former president... It depends what the meaning of "is" is. :-)
According to general relativity (Einstein's beautiful masterpiece, which has passed every experimental test so far), the best way to think of our universe is as a 4-dimensional "block" of space-time. (I can't think in 4D myself, but feel free to drop one of the spatial dimensions and picture a structure that has two dimensions in space and one in time.) In this "block universe" framework, instants of "now" are just particular slices of the 4D space-time block. And just as you can take different slices of a 3D block (revealing different internal 2D surfaces), one can take different slices of our 4D spacetime to define particular versions of "now". But according to relativity, all of these "nows" are equally valid, even if they don't agree! So this block-universe view strongly indicates that our concept of "now" is just a subjective slicing of our 4D universe - - a well-defined "now" doesn't exist, any more than there is a single well-defined 2D surface inside a featureless 3D block.
In general relativity, what you call "matter-energy" is defined in terms of a complicated mathematical structure called the "stress energy tensor"; I'll call it "T". This T has values everywhere throughout our 4D space time. But if you made a particular 3D slice, called it "now", and asked about the energy "now", you would only consider the values of T on that 3D surface. If general relativity is correct, then the answer to your question is that the past and future are indeed "filled" with matter/energy, in the form of this stress-energy tensor.
That said, there is another branch of physics, seemingly incompatible with general relativity (although tested to the same order of precision) called quantum theory. And quantum theory is built up from the idea that the 3D surfaces are fundamental (quantum states, they're called), and one stitches them together to create our 4D spacetime. People who are steeped in quantum theory often dislike the block universe framework I described above, because quantum theory itself is not really compatible with it. Several leading quantum theorists (Lee Smolin, for one) have recently speculated that we *don't* live in a block universe, and there is somehow a "now" that exists separately from everything else. So your question is a very good one -- even top physicists don't agree as to the answer.
Myself, I vote for the block universe. In fact, I've recently written an essay summarizing why it's the best viewpoint, and why the arguments against it fall flat. If you're interested, you can find the pdf file at this link.
Try the links in the MadSci Library for more information on Physics.