What goes up, must come down (especially if it explodes halfway up).
Range security and minimising the risk of raining debris down on populated areas is one of the reasons the US launches satellites into low-inclination orbits from the eastern seaboard, particularly sites like Cape Canaveral, while satellites going into polar orbits launch from sites like Vandenburg AFB where they can safely track southwards.
Not only do many multi-stage rockets drop their lower stages into the sea, but in the case of a launch failure (such as happened to SpaceX's Falcon 9 launch in June) the debris can scatter over a wide, uncontrolled area. NASA operates recovery ships to bring back spent solid rocket boosters, but obviously debris is harder to recover.
A large chunk of an American space rocket has been found in the sea off the Isles of Scilly. A large section of a spacecraft, measuring about 10m (32ft) by 4m (13ft), was spotted on the surface between Bryher and Tresco. Coastguards believe it is from the unmanned SpaceX Falcon 9 which exploded after take-off in Florida in June.