You will eventually end up closing off a port that something else needs to work properly, and now you have to figure out how to resolve that. 

Packet filtering is more computationally taxing than just routing is. Your edge equipment is likely going to be built for maximum routing efficiency. Trying to bite off too much filtering there increases your risk of legit traffic being tossed on the floor.