Even in space, Big Brother is watching. And as far as space debris is concerned, that’s a good thing. The U.S. tracked more than 19,000 pieces of debris orbiting Earth this year, according to a State Department report.

While it’s an impressive number, those remnants of defunct satellites and other spacecraft represent just a small fraction of the orbital debris actually circling Earth. Some NASA estimates put the true tally at more than 500,000. The trouble is the tracking devices can’t follow fragments of debris smaller than 10 centimeters.

“By and large, we know exactly where the bigger parts of space debris are,” says Frans von der Dunk, a space law professor at the University of Nebraska College of Law. “And as a consequence, if they come too close to either a valuable satellite or the space station, it is possible to temporarily move [the spacecraft] a little out of harm’s way and then, after the big chunk has passed, to move it back.”

But even debris as small as 1 cm–too small to track–can have a devastating effect on spacecraft. Every time a shuttle returns to Earth, for example, engineers must replace a number of its heat shield tiles because of damage from tiny bits of debris, von der Dunk says. The space shuttle Columbia’s catastrophic combustion upon re-entry in 2003 resulted from a single damaged tile, though the shuttle’s own foam insulation breaking off during liftoff, rather than space debris, created that wound.

Even if a piece of debris is initially large enough to track, over time it can fragment to the point it literally falls off the radar. Not only does that make the debris more dangerous, it also makes liability more difficult to pinpoint if a collision does occur, von der Dunk says. The U.N. Outer Space Treaty holds the launching country responsible for all harm caused by debris resulting from spacecraft licensed to it.

“The problem is not so much in the liability as such, but in making it stick–being able to label the liable state, in case the accident happens long after the original collision,” von der Dunk says.