Filter NTP traffic by packet size?

Curious if anyone else thinks filtering out NTP packets above a certain packet size is a good or terrible idea.

From my brief testing it seems 90 bytes for IPv4 and 110 bytes for IPv6 are typical for a client to successfully synchronize to an NTP server.

If I query a server for it's list of peers (ntpq -np <ip>) I've seen packets as large as 522 bytes in a single packet in response to a 54 byte query. I'll admit I'm not 100% clear of the what is happening protocol-wise when I perform this query. I see there are multiple packets back forth between me and the server depending on the number of peers it has? Would I be breaking something important if I started to filter NTP packets

200 bytes into my network?