NAT devices not translating privileged ports
Folks, While discussing port randomization (in the context of https://www.ietf.org/archive/id/draft-ietf-ntp-port-randomization-06.txt ), it has been raised to us that some NAT devices do not translate the source port if the source port is a privileged port (<1024). Any clues/examples of this type of NATs? Thanks! Regards, -- Fernando Gont Director of Information Security EdgeUno, Inc. PGP Fingerprint: DFBD 63E3 B248 AE79 C598 AF23 EBAE DA03 0644 1531
I believe all devices will translate a privileged ports, but it won't translate to the same number on the other side. It will translate to an unprivileged port. Is it what you meant or really there are some devices that will not translate at all a privileged port? What are you trying to achieve? Jean -----Original Message----- From: NANOG <nanog-bounces+jean=ddostest.me@nanog.org> On Behalf Of Fernando Gont Sent: June 4, 2021 3:00 AM To: nanog@nanog.org Subject: NAT devices not translating privileged ports Folks, While discussing port randomization (in the context of https://www.ietf.org/archive/id/draft-ietf-ntp-port-randomization-06.txt ), it has been raised to us that some NAT devices do not translate the source port if the source port is a privileged port (<1024). Any clues/examples of this type of NATs? Thanks! Regards, -- Fernando Gont Director of Information Security EdgeUno, Inc. PGP Fingerprint: DFBD 63E3 B248 AE79 C598 AF23 EBAE DA03 0644 1531
Current gen Cisco ASA firewalls have logic so that if the connection from a private host originated from a privileged source port, the NAT translation to public IP also uses an unprivileged source port (not necessarily the same source port though). I found out that this behavior can cause issues when you have devices on your network that implement older DNS libraries or configs using UDP 53 as a source and destination port for their DNS lookups. Occasionally the source port gets translated to one that ISC BIND servers have in a blocklist (chargen, echo, time, and a few others) and the query is ignored. As I recall, this behavior is hard coded so patching and recompiling BIND is required to work around it. I forget what the older ASA behavior was. It may have been to leave the source port unchanged through the NAT process (I think this is what you mean by "not translated"). In that case the client doesn't implement source port randomization and the NAT doesn't "upgrade" the connection to a random source port so I don't really see it as an issue. Ideally the client would implement source port randomization itself so it would be using source ports within its ephemeral port range for outgoing connections. --Blake On 6/4/2021 7:36 AM, Jean St-Laurent via NANOG wrote:
I believe all devices will translate a privileged ports, but it won't translate to the same number on the other side. It will translate to an unprivileged port. Is it what you meant or really there are some devices that will not translate at all a privileged port?
What are you trying to achieve?
Jean
-----Original Message----- From: NANOG <nanog-bounces+jean=ddostest.me@nanog.org> On Behalf Of Fernando Gont Sent: June 4, 2021 3:00 AM To: nanog@nanog.org Subject: NAT devices not translating privileged ports
Folks,
While discussing port randomization (in the context of https://www.ietf.org/archive/id/draft-ietf-ntp-port-randomization-06.txt ), it has been raised to us that some NAT devices do not translate the source port if the source port is a privileged port (<1024).
Any clues/examples of this type of NATs?
Thanks!
Regards, -- Fernando Gont Director of Information Security EdgeUno, Inc. PGP Fingerprint: DFBD 63E3 B248 AE79 C598 AF23 EBAE DA03 0644 1531
For Linux iptables SNAT (used with --to-source), the default is to change the packet as little as possible. https://linux.die.net/man/8/iptables "If no port range is specified, then source ports below 512 will be mapped to other ports below 512: those between 512 and 1023 inclusive will be mapped to ports below 1024, and other ports will be mapped to 1024 or above. Where possible, no port alteration will occur." So, if there are no "collisions", the same src port will be used. If there are "collisions" (multiple flows with the same src port and dst IP/port), then another src port within its "range" will be used. But it can be configured, for example, to use ports 1024-65535, in which case flows with src port < 1024 will endup using ports > 1024 after they are NATed. https://datatracker.ietf.org/doc/html/rfc6335#section-6 is also a good reference. Alvaro On Fri, Jun 4, 2021 at 10:14 AM Blake Hudson <blake@ispn.net> wrote:
Current gen Cisco ASA firewalls have logic so that if the connection from a private host originated from a privileged source port, the NAT translation to public IP also uses an unprivileged source port (not necessarily the same source port though).
I found out that this behavior can cause issues when you have devices on your network that implement older DNS libraries or configs using UDP 53 as a source and destination port for their DNS lookups. Occasionally the source port gets translated to one that ISC BIND servers have in a blocklist (chargen, echo, time, and a few others) and the query is ignored. As I recall, this behavior is hard coded so patching and recompiling BIND is required to work around it.
I forget what the older ASA behavior was. It may have been to leave the source port unchanged through the NAT process (I think this is what you mean by "not translated"). In that case the client doesn't implement source port randomization and the NAT doesn't "upgrade" the connection to a random source port so I don't really see it as an issue. Ideally the client would implement source port randomization itself so it would be using source ports within its ephemeral port range for outgoing connections.
--Blake
I believe all devices will translate a privileged ports, but it won't
On 6/4/2021 7:36 AM, Jean St-Laurent via NANOG wrote: translate to the same number on the other side. It will translate to an unprivileged port. Is it what you meant or really there are some devices that will not translate at all a privileged port?
What are you trying to achieve?
Jean
-----Original Message----- From: NANOG <nanog-bounces+jean=ddostest.me@nanog.org> On Behalf Of
Sent: June 4, 2021 3:00 AM To: nanog@nanog.org Subject: NAT devices not translating privileged ports
Folks,
While discussing port randomization (in the context of https://www.ietf.org/archive/id/draft-ietf-ntp-port-randomization-06.txt ), it has been raised to us that some NAT devices do not translate the
Fernando Gont source port if the source port is a privileged port (<1024).
Any clues/examples of this type of NATs?
Thanks!
Regards, -- Fernando Gont Director of Information Security EdgeUno, Inc. PGP Fingerprint: DFBD 63E3 B248 AE79 C598 AF23 EBAE DA03 0644 1531
Hi, Blake, Thanks a lot for your comments! In-line.... On Fri, 2021-06-04 at 11:13 -0500, Blake Hudson wrote:
Current gen Cisco ASA firewalls have logic so that if the connection from a private host originated from a privileged source port, the NAT translation to public IP also uses an unprivileged source port (not necessarily the same source port though).
Did you actaully mean "...also uses a *privileged port*"?
I found out that this behavior can cause issues when you have devices on your network that implement older DNS libraries or configs using UDP 53 as a source and destination port for their DNS lookups. Occasionally the source port gets translated to one that ISC BIND servers have in a blocklist (chargen, echo, time, and a few others) and the query is ignored. As I recall, this behavior is hard coded so patching and recompiling BIND is required to work around it.
I forget what the older ASA behavior was. It may have been to leave the source port unchanged through the NAT process (I think this is what you mean by "not translated"). In that case the client doesn't implement source port randomization and the NAT doesn't "upgrade" the connection to a random source port so I don't really see it as an issue.
The issue would be that if the port is not translated, and multiple systems in the internal real of the NAT try to use the same privileged port (say, 123) simultaneously, things wouldn't work. Thanks, -- Fernando Gont Director of Information Security EdgeUno, Inc. PGP Fingerprint: DFBD 63E3 B248 AE79 C598 AF23 EBAE DA03 0644 1531
On 6/10/2021 4:04 AM, Fernando Gont wrote:
Hi, Blake,
Thanks a lot for your comments! In-line....
On Fri, 2021-06-04 at 11:13 -0500, Blake Hudson wrote:
Current gen Cisco ASA firewalls have logic so that if the connection from a private host originated from a privileged source port, the NAT translation to public IP also uses an unprivileged source port (not necessarily the same source port though). Did you actaully mean "...also uses a *privileged port*"?
Yes I did. Thanks.
I found out that this behavior can cause issues when you have devices on your network that implement older DNS libraries or configs using UDP 53 as a source and destination port for their DNS lookups. Occasionally the source port gets translated to one that ISC BIND servers have in a blocklist (chargen, echo, time, and a few others) and the query is ignored. As I recall, this behavior is hard coded so patching and recompiling BIND is required to work around it.
I forget what the older ASA behavior was. It may have been to leave the source port unchanged through the NAT process (I think this is what you mean by "not translated"). In that case the client doesn't implement source port randomization and the NAT doesn't "upgrade" the connection to a random source port so I don't really see it as an issue. The issue would be that if the port is not translated, and multiple systems in the internal real of the NAT try to use the same privileged port (say, 123) simultaneously, things wouldn't work.
Not quite. If multiple devices behind a NAT use SRC=123 & DST = 123 for connections, their connections will still work most of the time. First, if the connections are to different destinations there would be no NAT conflict. Second, if the connections occur at different times, there would be no NAT conflict. Third, if there was a NAT conflict (meaning connections using the same SRC port, DST port, and DST host at the same time) the NAT device would see this and would either adjust the translation to use a different SRC port (in which case the connection succeeds) or may drop/reject the connection (in which case the client would eventually retry). So saying "things wouldn't work" or "you won't be able to have multiple NTP clients behind the same firewall" is not true. Saying that "the number of simultaneous connections is limited" when behind a NAT would be more accurate. _But this statement is true of any one-to-many NAT_. Most NTP clients utilize multiple NTP servers, often from a pool of available servers, and initiate connections rather infrequently so I do not expect this to be a problem in practice unless there are thousands of NTP clients behind a single NAT accessing a common NTP server (and that NAT does not do a good job of dealing with collisions). I do, however, agree that clients should probably use ephemeral ports when making any outbound connections as this provides more entropy for NAT as well as for connection security. This extends to NTP.
Hi, Jean, On Fri, 2021-06-04 at 08:36 -0400, Jean St-Laurent wrote:
I believe all devices will translate a privileged ports, but it won't translate to the same number on the other side. It will translate to an unprivileged port. Is it what you meant or really there are some devices that will not translate at all a privileged port?
What has been reported to us is that some boxes do not translate the src port if it's a privileged port. IN such scenarios, NTP implementations that always use src port=123, dst port=123 might be in trouble if there are multiple NTP clients behind the same NAT device.... Thanks! Regards, -- Fernando Gont Director of Information Security EdgeUno, Inc. PGP Fingerprint: DFBD 63E3 B248 AE79 C598 AF23 EBAE DA03 0644 1531
Fernando Gont via NANOG <nanog@nanog.org> writes:
What has been reported to us is that some boxes do not translate the src port if it's a privileged port.
IN such scenarios, NTP implementations that always use src port=123, dst port=123 might be in trouble if there are multiple NTP clients behind the same NAT device....
This problem used to be very common for 500/udp. Ref https://datatracker.ietf.org/doc/html/rfc3715#section-2.3 Bjørn
Hi, Bjørn, On Thu, 2021-06-10 at 12:10 +0200, Bjørn Mork wrote:
Fernando Gont via NANOG <nanog@nanog.org> writes:
What has been reported to us is that some boxes do not translate the src port if it's a privileged port.
IN such scenarios, NTP implementations that always use src port=123, dst port=123 might be in trouble if there are multiple NTP clients behind the same NAT device....
This problem used to be very common for 500/udp. Ref https://datatracker.ietf.org/doc/html/rfc3715#section-2.3
THanks a lot for the link! -- this is indeed a good read. I'm curious if there exists something similar for UDP/123? FWIW, we have this IETF I-D on NTP port randomization: https://datatracker.ietf.org/doc/html/draft-ietf-ntp-port-randomization-06 , which has this section on the same kind of behavior, but for the NTP port: ---- cut here ---- 3.4. Effect on NAT devices Some NAT devices will not translate the source port of a packet when a privileged port number is employed. In networks where such NAT devices are employed, use of the NTP well-known port for the client port will essentially limit the number of hosts that may successfully employ NTP client implementations. In the case of NAT devices that will translate the source port even when a privileged port is employed, packets reaching the external realm of the NAT will not employ the NTP well-known port as the local port, since the local port will normally be translated by the NAT device possibly, but not necessarily, with a random port. ---- cut here ---- So I'm trying to find some reference that documents such behavior for the NTP case.... Thanks! Regards, -- Fernando Gont Director of Information Security EdgeUno, Inc. PGP Fingerprint: DFBD 63E3 B248 AE79 C598 AF23 EBAE DA03 0644 1531
Hi Fernando, NTP sounds simple but it could be very complex when you dig deep down and/or get lost in details. Here are 2 things to consider: 1. NTP clients can query NTP servers by using SRC UDP ports > 1024. 2. NTP servers cannot query/sync/communicate to another NTP server when using SRC UDP port > 1024. In short, server to server wants SRC and DST UDP 123. The query and the response will be fully 123 when server to server. In your case, it sounds like you want to achieve NTP server to NTP server, but you mention NTP clients behind NAT devices. Because multiple clients behind the same NAT devices should work. Multiple NTP servers behind the same NAT and wanting to use other NTP server *should* not work. Can you give us more details on what kind of communication you need here? From what I understand client to server should work just fine with any NAT devices. Maybe you meant multiple NTP servers behind the same NAT to external NTP server? Thanks Jean
Hi, Jean, On Thu, 2021-06-10 at 06:54 -0400, Jean St-Laurent via NANOG wrote:
Hi Fernando,
NTP sounds simple but it could be very complex when you dig deep down and/or get lost in details. Here are 2 things to consider:
1. NTP clients can query NTP servers by using SRC UDP ports > 1024.
This is indeed the case we're addressing. The NTP spec mandates srt port=123, even for client-to-server cases.
In your case, it sounds like you want to achieve NTP server to NTP server, but you mention NTP clients behind NAT devices.
Nope. We simply recommend to randomize the source port for client-to- server cases. So in the quoted section we make the case that requiring src port=123 clients doesnt really make sense: 1) if the NAT translates the port, the server won-t see src 123 anyway 2) if the NAT doesn't translate the port, you won't be able to ahve multiple NTP clients behind the same firewall.
Can you give us more details on what kind of communication you need here? From what I understand client to server should work just fine with any NAT devices.
Maybe you meant multiple NTP servers behind the same NAT to external NTP servers
Please let me know if what I wrote above clarifies our intent. Thanks! Regards, -- Fernando Gont Director of Information Security EdgeUno, Inc. PGP Fingerprint: DFBD 63E3 B248 AE79 C598 AF23 EBAE DA03 0644 1531
Let's start with this example. When I click sync my clock in windows, this happened. On the inside or Private side 08:15:07.434344 IP 192.168.254.205.123 > 13.86.101.172.123: NTPv3, Client, length 48 08:15:07.473681 IP 13.86.101.172.123 > 192.168.254.205.123: NTPv3, Server, length 48 You are indeed right that the client must use UDP port 123. Is the RFC saying must or should on the client SRC port? I'm not sure. But, on the Public, this happened. 08:15:07.434381 IP 192.2XX.XXX.58291 > 13.86.101.172.123: NTPv3, Client, length 48 08:15:07.473656 IP 13.86.101.172.123 > 192.2XX.XXX.58291: NTPv3, Server, length 48 // Public ip obfuscated. I know, it indeed starts with 192.2. It's EBOX in Canada. What we see on the public side, is that a network device did a NAT translation of the SRC UDP port to 58921. My clock synced perfectly. So your goal is to find the devices that don't follow this behaviour, right? Jean -----Original Message----- From: Fernando Gont <fernando.gont@edgeuno.com> Sent: June 10, 2021 7:09 AM To: jean@ddostest.me; nanog@nanog.org Subject: Re: NAT devices not translating privileged ports Hi, Jean, On Thu, 2021-06-10 at 06:54 -0400, Jean St-Laurent via NANOG wrote:
Hi Fernando,
NTP sounds simple but it could be very complex when you dig deep down and/or get lost in details. Here are 2 things to consider:
1. NTP clients can query NTP servers by using SRC UDP ports > 1024.
This is indeed the case we're addressing. The NTP spec mandates srt port=123, even for client-to-server cases.
Hi, Jean, On Thu, 2021-06-10 at 08:23 -0400, Jean St-Laurent wrote:
Let's start with this example. When I click sync my clock in windows, this happened.
On the inside or Private side 08:15:07.434344 IP 192.168.254.205.123 > 13.86.101.172.123: NTPv3, Client, length 48 08:15:07.473681 IP 13.86.101.172.123 > 192.168.254.205.123: NTPv3, Server, length 48
You are indeed right that the client must use UDP port 123. Is the RFC saying must or should on the client SRC port? I'm not sure.
Section 9.1 ("Peer Process Variables") of [RFC5905] SAYS: dstport: UDP port number of the client, ordinarily the NTP port number PORT (123) assigned by the IANA. This becomes the source port number in packets sent from this association. That said, as noted in https://datatracker.ietf.org/doc/html/draft-ietf-ntp-port-randomization-06#s... , other client implementations don't bind the local port to 123, and hence assign an ephemeral port.
But, on the Public, this happened. 08:15:07.434381 IP 192.2XX.XXX.58291 > 13.86.101.172.123: NTPv3, Client, length 48 08:15:07.473656 IP 13.86.101.172.123 > 192.2XX.XXX.58291: NTPv3, Server, length 48
// Public ip obfuscated. I know, it indeed starts with 192.2. It's EBOX in Canada.
What we see on the public side, is that a network device did a NAT translation of the SRC UDP port to 58921. My clock synced perfectly.
So your goal is to find the devices that don't follow this behaviour, right?
No. The goal of our I-D is that NTP clients randomize their source port -- there's no need for clients to use port 123, and using that port on the client side has negative security implications.
Thanks, -- Fernando Gont Director of Information Security EdgeUno, Inc. PGP Fingerprint: DFBD 63E3 B248 AE79 C598 AF23 EBAE DA03 0644 1531
participants (5)
-
Alvaro Pereira -
Bjørn Mork -
Blake Hudson -
Fernando Gont -
Jean St-Laurent