Hello NANOG community. I was directed here by our network administrator since she is on vacation. Luckily, I minored in Computer Science so I have some familiarity. We have a small satellite campus of around 170 devices that share one external IPv4 and IPv6 address via NAT for internet traffic. Internal traffic is over an MPLS. We're having problems where viruses are getting through Firefox, and we think it's because our Palo Alto firewall is set to bypass filtering for IPv6. Unfortunately, the network admin couldn't give me the password since a local consultant set it up, and it seems they went out of business. I need to think outside the box. Is there some kind of NAT-based IPv6 firewall I can setup on the router that can help block viruses? I figure that's the right place to start since all the traffic gets funneled there. We have a Cisco Catalyst as a router. Or, ideally, is there an easy way to turn off IPv6 completely? I really don't see a need for it, any legitimate service should have an IPv4 address. I'd really appreciate your advice. I plan to drive out there tomorrow, where I can get the exact model numbers and stuff. Regards, Dr. Edgar Carver
You emailed the wrong list to say this "Or, ideally, is there an easy way to turn off IPv6 completely? I really don't see a need for it, any legitimate service should have an IPv4 address." Turning off IPv6 is not the right solution, nor will it magically fix your issues. Fix the Palo Alto, either hire another consultant or just erase it and start over. Although even PA's Layer7 inspection won't catch everything and you should have antivirus/antimailware software on the end user computers. *Spencer Ryan* | Senior Systems Administrator | sryan@arbor.net *Arbor Networks* +1.734.794.5033 (d) | +1.734.846.2053 (m) www.arbornetworks.com On Fri, Jul 1, 2016 at 10:28 PM, Edgar Carver <dredgarcarver@gmail.com> wrote:
Hello NANOG community. I was directed here by our network administrator since she is on vacation. Luckily, I minored in Computer Science so I have some familiarity.
We have a small satellite campus of around 170 devices that share one external IPv4 and IPv6 address via NAT for internet traffic. Internal traffic is over an MPLS.
We're having problems where viruses are getting through Firefox, and we think it's because our Palo Alto firewall is set to bypass filtering for IPv6. Unfortunately, the network admin couldn't give me the password since a local consultant set it up, and it seems they went out of business. I need to think outside the box.
Is there some kind of NAT-based IPv6 firewall I can setup on the router that can help block viruses? I figure that's the right place to start since all the traffic gets funneled there. We have a Cisco Catalyst as a router. Or, ideally, is there an easy way to turn off IPv6 completely? I really don't see a need for it, any legitimate service should have an IPv4 address.
I'd really appreciate your advice. I plan to drive out there tomorrow, where I can get the exact model numbers and stuff.
Regards, Dr. Edgar Carver
Hard to know where to begin with this one, but let me take a shot at it. 1. My top priority would be to get into that Palo Alto firewall. Get Palo Alto on the phone and figure out password recovery with them. Since you don’t have the password it is possible that firewall is compromised. Do not be surprised if you have to jump through some hoops with Palo Alto to prove that you own it and what has happened. Remember their job is to keep people out of your network. They are probably also going to want you to be current on support. If you have to pay to get current on support, do it. You need that help right now badly. You could ask Palo Alto how to block the v6 while you are at it or even better set up a rules that mirror your v4 protection. I cannot stress enough how big a security issue it is to not have access to your firewall and not know who does. 2. There are lots of ways to shut off ipv6 but my suggestion would be to just secure the Palo Alto firewall, to say that any legitimate service should have a ipv4 address is not quite true now and will definitely not be true in the near future. 3. Just about any kind of firewall or router CPE device can block or firewall ipv4 and ipv6 as long as its firmware is fairly recent. However, you would most likely have to replace the Palo Alto with it. You DO NOT WANT THEM BOTH INLINE! Most likely they are both configured to do ipv4 NAT out of the box and that will not work correctly to have them both inline together. While it is possible to set up that sort of thing to work correctly, it’s a bad idea and pretty advanced configuration for a temporary network admin. The interaction of one firewall fronting another can be very difficult to troubleshoot without a deep understanding of what is going on. Referring back to item 1, you are probably going to need to get the configuration of the current firewall if you seek to replace it (there will be rules in the Palo Alto that you would want to replicate if you are going to replace it). 4. Cisco Catalyst as the router.....there could be a lot of things going on in there. The Catalyst is primarily a switch with routing functionality. It can definitely block ipv6 if configured to do so but we would need to know a lot more about its current configuration to give you the best way to do that. It could just be a service providers switch on your premise in which case you can't do much with it. Again, much easier to accomplish Item 1 with Palo Alto and let your firewall do what it is supposed to do. Steven Naslund Chicago IL -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Edgar Carver Sent: Friday, July 01, 2016 9:29 PM To: nanog@nanog.org Subject: NAT firewall for IPv6? Hello NANOG community. I was directed here by our network administrator since she is on vacation. Luckily, I minored in Computer Science so I have some familiarity. We have a small satellite campus of around 170 devices that share one external IPv4 and IPv6 address via NAT for internet traffic. Internal traffic is over an MPLS. We're having problems where viruses are getting through Firefox, and we think it's because our Palo Alto firewall is set to bypass filtering for IPv6. Unfortunately, the network admin couldn't give me the password since a local consultant set it up, and it seems they went out of business. I need to think outside the box. Is there some kind of NAT-based IPv6 firewall I can setup on the router that can help block viruses? I figure that's the right place to start since all the traffic gets funneled there. We have a Cisco Catalyst as a router. Or, ideally, is there an easy way to turn off IPv6 completely? I really don't see a need for it, any legitimate service should have an IPv4 address. I'd really appreciate your advice. I plan to drive out there tomorrow, where I can get the exact model numbers and stuff. Regards, Dr. Edgar Carver
On 7/5/16, Naslund, Steve <SNaslund@medline.com> wrote:
Hard to know where to begin with this one, but let me take a shot at it.
1. My top priority would be to get into that Palo Alto firewall. Get Palo Alto on the phone and figure out password recovery with them. Since you don’t have the password it is possible that firewall is compromised. Do not be surprised if you have to jump through some hoops with Palo Alto to prove that you own it and what has happened. Remember their job is to keep people out of your network. They are probably also going to want you to be current on support. If you have to pay to get current on support, do it. You need that help right now badly.
You could ask Palo Alto how to block the v6 while you are at it or even better set up a rules that mirror your v4 protection. I cannot stress enough how big a security issue it is to not have access to your firewall and not know who does.
2. There are lots of ways to shut off ipv6 but my suggestion would be to just secure the Palo Alto firewall,
Right. But how long is it going to take to secure the Palo Alto firewall? If the central Cisco Catalyst really is an IPv6 router, doing a conf t ipv6 access-list denyIPv6 deny ipv6 any any interface [whatever connects to the ISP] ipv6 traffic-filter denyIPv6 in ipv6 traffic-filter denyIPv6 out end would be a quick fix for the firewall not doing any ipv6 filtering. It could also break ipv6 enabled web sites or even internal connectivity, so it'd be better to get someone on the phone w/ Cisco tech support and have Cisco figure out the best way to block IPv6 for you.
... to say that any legitimate service should have a ipv4 address is not quite true now and will definitely not be true in the near future.
True. But they're in "stop the bleeding" mode and disabling ipv6 is just a temp work-around until the firewall is fixed. Regards, Lee
3. Just about any kind of firewall or router CPE device can block or firewall ipv4 and ipv6 as long as its firmware is fairly recent. However, you would most likely have to replace the Palo Alto with it. You DO NOT WANT THEM BOTH INLINE! Most likely they are both configured to do ipv4 NAT out of the box and that will not work correctly to have them both inline together. While it is possible to set up that sort of thing to work correctly, it’s a bad idea and pretty advanced configuration for a temporary network admin. The interaction of one firewall fronting another can be very difficult to troubleshoot without a deep understanding of what is going on. Referring back to item 1, you are probably going to need to get the configuration of the current firewall if you seek to replace it (there will be rules in the Palo Alto that you would want to replicate if you are going to replace it).
4. Cisco Catalyst as the router.....there could be a lot of things going on in there. The Catalyst is primarily a switch with routing functionality. It can definitely block ipv6 if configured to do so but we would need to know a lot more about its current configuration to give you the best way to do that. It could just be a service providers switch on your premise in which case you can't do much with it. Again, much easier to accomplish Item 1 with Palo Alto and let your firewall do what it is supposed to do.
Steven Naslund Chicago IL
-----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Edgar Carver Sent: Friday, July 01, 2016 9:29 PM To: nanog@nanog.org Subject: NAT firewall for IPv6?
Hello NANOG community. I was directed here by our network administrator since she is on vacation. Luckily, I minored in Computer Science so I have some familiarity.
We have a small satellite campus of around 170 devices that share one external IPv4 and IPv6 address via NAT for internet traffic. Internal traffic is over an MPLS.
We're having problems where viruses are getting through Firefox, and we think it's because our Palo Alto firewall is set to bypass filtering for IPv6. Unfortunately, the network admin couldn't give me the password since a local consultant set it up, and it seems they went out of business. I need to think outside the box.
Is there some kind of NAT-based IPv6 firewall I can setup on the router that can help block viruses? I figure that's the right place to start since all the traffic gets funneled there. We have a Cisco Catalyst as a router. Or, ideally, is there an easy way to turn off IPv6 completely? I really don't see a need for it, any legitimate service should have an IPv4 address.
I'd really appreciate your advice. I plan to drive out there tomorrow, where I can get the exact model numbers and stuff.
Regards, Dr. Edgar Carver
Did you get the impression that this person asking for help was going to be able to set that up? I didn't (if he was he would probably already know what an ACL is). I do not know if the Catalyst he is looking at is his or his service providers edge devices (or maybe the consultants didn't give them access to that either), I don't know that that Catalyst is the primary router for their network (could be an L2 switch behind the firewall). I also doubt the problem stems from ipv6 as much as it comes from having an out of control firewall. Given what I am hearing about this network I am kind of doubting that it is really ipv6 enabled in any case so your fix prevents ipv6 traffic that is probably not even being routed in the first place. In my opinion not having control of your own firewall is the five alarm emergency in that network right now. If the network is ipv6 enabled, blocking all ipv6 traffic at that router is probably not a good idea without knowing more. If it is not ipv6 enabled then it will have no effect on the reported issue (malware). Steven Naslund Chicago IL
Right. But how long is it going to take to secure the Palo Alto firewall? If the central Cisco Catalyst really is an IPv6 router, doing a conf t ipv6 access-list denyIPv6 deny ipv6 any any
interface [whatever connects to the ISP] ipv6 traffic-filter denyIPv6 in ipv6 traffic-filter denyIPv6 out end would be a quick fix for the firewall not doing any ipv6 filtering. It could also break ipv6 enabled web sites or even internal connectivity, so it'd be better to get someone on the phone w/ Cisco tech support and have Cisco figure out the best way to block IPv6 for you.
True. But they're in "stop the bleeding" mode and disabling ipv6 is just a temp work-around until the firewall is fixed.
On 7/5/16, Naslund, Steve <SNaslund@medline.com> wrote:
Did you get the impression that this person asking for help was going to be able to set that up?
Yes, I think the OP could create & apply the acl. Which is why I said it could break their network & suggested they get Cisco tech support on the phone to figure out how to safely turn off IPv6. I'm also giving them the benefit of the doubt that IPv6 really is the malware infection vector.
I didn't (if he was he would probably already know what an ACL is). I do not know if the Catalyst he is looking at is his or his service providers edge devices (or maybe the consultants didn't give them access to that either), I don't know that that Catalyst is the primary router for their network (could be an L2 switch behind the firewall). I also doubt the problem stems from ipv6 as much as it comes from having an out of control firewall. Given what I am hearing about this network I am kind of doubting that it is really ipv6 enabled in any case so your fix prevents ipv6 traffic that is probably not even being routed in the first place. In my opinion not having control of your own firewall is the five alarm emergency in that network right now.
Maybe I wasn't clear that the call to Cisco tech support should be a parallel effort?
If the network is ipv6 enabled, blocking all ipv6 traffic at that router is probably not a good idea without knowing more.
Which is why I suggested getting Cisco tech support involved. A mailing list is not where they should be going for help right now. Best Regards, Lee
... If it is not ipv6 enabled then it will have no effect on the reported issue (malware).
Steven Naslund Chicago IL
Right. But how long is it going to take to secure the Palo Alto firewall? If the central Cisco Catalyst really is an IPv6 router, doing a conf t ipv6 access-list denyIPv6 deny ipv6 any any
interface [whatever connects to the ISP] ipv6 traffic-filter denyIPv6 in ipv6 traffic-filter denyIPv6 out end would be a quick fix for the firewall not doing any ipv6 filtering. It could also break ipv6 enabled web sites or even internal connectivity, so it'd be better to get someone on the phone w/ Cisco tech support and have Cisco figure out the best way to block IPv6 for you.
True. But they're in "stop the bleeding" mode and disabling ipv6 is just a temp work-around until the firewall is fixed.
On 5 July 2016 at 17:40, Lee <ler762@gmail.com> wrote:
Right. But how long is it going to take to secure the Palo Alto firewall? If the central Cisco Catalyst really is an IPv6 router, doing a conf t ipv6 access-list denyIPv6 deny ipv6 any any
interface [whatever connects to the ISP] ipv6 traffic-filter denyIPv6 in ipv6 traffic-filter denyIPv6 out end would be a quick fix for the firewall not doing any ipv6 filtering.
Nope, that is not going to stop his IPv6 address from appearing, which I will bet you good money is in the range of fe80::/64.
Hi,
Right. But how long is it going to take to secure the Palo Alto firewall?
around 5 minutes? recover password, restart, log in, fix rules. https://live.paloaltonetworks.com/t5/Management-Articles/How-to-Reset-the-Ad... obviously the firewall is also blocking google access! ;-) alan
FYI There is no way to reset the password on a PAN without doing a factory reset if you do not know the password of any previous config release version. If you do a reset then you will have to reconfigure the fw rules, ip addresses, routes, nat, inspection policy's, and other basic functions depending on if it in layer3 mode or layer2 from scratch. Also are you sure the exploit vector is from ipv6 and not from traffic that the PAN cannot see such as TLS traffic? Also are you sure IPv6 is working? You can test connectivity over IPv6 here http://test-ipv6.com/ On Tue, Jul 5, 2016 at 12:47 PM, Octavio Alvarez <octalnanog@alvarezp.org> wrote:
On 07/01/2016 07:28 PM, Edgar Carver wrote:
Is there some kind of NAT-based IPv6 firewall I can setup on the router that can help block viruses?
You need layer-7 firewalls for this. NAT-based "firewalls" (pseudo-firewalls, really) are layer-4 only. Those will not help you block typical viruses, as people will usually get infected from connecting to a compromised Website, or from an e-mail attachments. And even more, if connections are encrypted, an L7 firewall will not be able to do anything (whether IPv4 or v6) unless... better not open a can of worms.
They will just help you block *some* attack vectors, though: those that rely on starting connections to your hosts from the outside.
My guess is that, with regard to e-mail attachments and compromised Websites, IPv4 hosts are still attacked more than IPv6 ones, so, even if you turn off IPv6 you will still get attacked through IPv4.
Everything else has been already said by others: fixing the Palo Alto is still your best bet.
Good luck!
On Tue, Jul 5, 2016 at 12:45 PM, <A.L.M.Buxey@lboro.ac.uk> wrote:
Hi,
Right. But how long is it going to take to secure the Palo Alto firewall?
around 5 minutes?
recover password, restart, log in, fix rules.
https://live.paloaltonetworks.com/t5/Management-Articles/How-to-Reset-the-Ad...
obviously the firewall is also blocking google access! ;-)
alan
On another note, using a firewall to stop viruses is probably not going to work in general (unless the firewall has some additional malware detection engine). Here is the issue in a nutshell. A firewall primarily controls where people can connect to and from on a network. The problem with that is that a lot of malware is received from sites that your users intended to go to. People click on links without knowing where they go and people go to less than reputable web sites (or reputable sites that we recently compromised). If you, by default, allow your users to access the Internet with a browser they are vulnerable to malware. Even with malware detection capability you are still vulnerable to signatures and attacks that are not yet able to be detected. Even if filtering was enabled on your Palo Alto for ipv6 it would not help at this point because you have no idea what signatures it is using to filter with and when the last time those were updated I doubt your v4 filtering is of much use either at this point. URL filtering is largely a big game of whack a mole that you will lose eventually. Malware filtering is based on one or both of the following methods. 1. You filter URLs known to be bad players (you are vulnerable until your protection vendor realizes they are bad players). 2. You filter based on adaptive detection of code that looks suspicious. This is a bit better but still vulnerable because the bad guys are always innovating to pass through these devices. My recommendation would be network malware detection (possibly through a firewall add-on) as well as good virus/malware detection on the client computers. Sometimes the malware is easier to detect at the client because it reveals itself by trying to access unauthorized memory, processes, or storage. Steven Naslund Chicago IL -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Edgar Carver Sent: Friday, July 01, 2016 9:29 PM To: nanog@nanog.org Subject: NAT firewall for IPv6? Hello NANOG community. I was directed here by our network administrator since she is on vacation. Luckily, I minored in Computer Science so I have some familiarity. We have a small satellite campus of around 170 devices that share one external IPv4 and IPv6 address via NAT for internet traffic. Internal traffic is over an MPLS. We're having problems where viruses are getting through Firefox, and we think it's because our Palo Alto firewall is set to bypass filtering for IPv6. Unfortunately, the network admin couldn't give me the password since a local consultant set it up, and it seems they went out of business. I need to think outside the box. Is there some kind of NAT-based IPv6 firewall I can setup on the router that can help block viruses? I figure that's the right place to start since all the traffic gets funneled there. We have a Cisco Catalyst as a router. Or, ideally, is there an easy way to turn off IPv6 completely? I really don't see a need for it, any legitimate service should have an IPv4 address. I'd really appreciate your advice. I plan to drive out there tomorrow, where I can get the exact model numbers and stuff. Regards, Dr. Edgar Carver
You may want to look into a new product by Ixia https://www.ixiacom.com/products/threatarmor (seems their site is under maint atm). On Tue, Jul 5, 2016 at 10:31 AM, Naslund, Steve <SNaslund@medline.com> wrote:
On another note, using a firewall to stop viruses is probably not going to work in general (unless the firewall has some additional malware detection engine).
Here is the issue in a nutshell. A firewall primarily controls where people can connect to and from on a network. The problem with that is that a lot of malware is received from sites that your users intended to go to. People click on links without knowing where they go and people go to less than reputable web sites (or reputable sites that we recently compromised). If you, by default, allow your users to access the Internet with a browser they are vulnerable to malware. Even with malware detection capability you are still vulnerable to signatures and attacks that are not yet able to be detected.
Even if filtering was enabled on your Palo Alto for ipv6 it would not help at this point because you have no idea what signatures it is using to filter with and when the last time those were updated I doubt your v4 filtering is of much use either at this point. URL filtering is largely a big game of whack a mole that you will lose eventually. Malware filtering is based on one or both of the following methods.
1. You filter URLs known to be bad players (you are vulnerable until your protection vendor realizes they are bad players).
2. You filter based on adaptive detection of code that looks suspicious. This is a bit better but still vulnerable because the bad guys are always innovating to pass through these devices.
My recommendation would be network malware detection (possibly through a firewall add-on) as well as good virus/malware detection on the client computers. Sometimes the malware is easier to detect at the client because it reveals itself by trying to access unauthorized memory, processes, or storage.
Steven Naslund Chicago IL
-----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Edgar Carver Sent: Friday, July 01, 2016 9:29 PM To: nanog@nanog.org Subject: NAT firewall for IPv6?
Hello NANOG community. I was directed here by our network administrator since she is on vacation. Luckily, I minored in Computer Science so I have some familiarity.
We have a small satellite campus of around 170 devices that share one external IPv4 and IPv6 address via NAT for internet traffic. Internal traffic is over an MPLS.
We're having problems where viruses are getting through Firefox, and we think it's because our Palo Alto firewall is set to bypass filtering for IPv6. Unfortunately, the network admin couldn't give me the password since a local consultant set it up, and it seems they went out of business. I need to think outside the box.
Is there some kind of NAT-based IPv6 firewall I can setup on the router that can help block viruses? I figure that's the right place to start since all the traffic gets funneled there. We have a Cisco Catalyst as a router. Or, ideally, is there an easy way to turn off IPv6 completely? I really don't see a need for it, any legitimate service should have an IPv4 address.
I'd really appreciate your advice. I plan to drive out there tomorrow, where I can get the exact model numbers and stuff.
Regards, Dr. Edgar Carver
On Fri, 01 Jul 2016 21:28:54 -0500, Edgar Carver said:
We're having problems where viruses are getting through Firefox, and we think it's because our Palo Alto firewall is set to bypass filtering for IPv6.
Do you have any actual evidence (device logs, tcpdump, netflow, etc) that support that train of thought? Remember that your Palo Alto isn't stopping 100% of the icky stuff on the IPv4 side either - the sad truth is that most commercial security software is only able to identify and block between 30% and 70% of the crap that's out in the wild. There's also BYOD issues where a laptop comes in and infects all your systems from behind the firewall (as Marcus Ranum says: "Crunchy on the outside, soft and chewy inside"). In any case,your first two actions should be to recover the password for the Palo Alto, and make sure it has updated pattern definitions in effect on both IPv4 and IPv6 connections. And your third should be to re-examine your vendor rules of engagement, to ensure your deliverables include things like passwords and update support so you're not stuck if your vendor goes belly up..
On Jul 5, 2016, at 9:33 AM, Valdis.Kletnieks@vt.edu wrote:
On Fri, 01 Jul 2016 21:28:54 -0500, Edgar Carver said:
We're having problems where viruses are getting through Firefox, and we think it's because our Palo Alto firewall is set to bypass filtering for IPv6.
Do you have any actual evidence (device logs, tcpdump, netflow, etc) that support that train of thought?
Remember that your Palo Alto isn't stopping 100% of the icky stuff on the IPv4 side either - the sad truth is that most commercial security software is only able to identify and block between 30% and 70% of the crap that's out in the wild.
That is only the percentage that it identifies from what it can see. It most likely can not see viruses in encrypted traffic. " • A forecast that 70% of global Internet traffic will be encrypted in 2016, with many networks exceeding 80%” https://www.sandvine.com/pr/2016/2/11/sandvine-70-of-global-internet-traffic... "In the fourth quarter of 2015 nearly 65 percent of all web connections that Dell observed were encrypted, leading to a lot more under-the-radar attacks, according to the company. Gartner has predicted that 50 percent of all network attacks will take advantage of SSL/TLS by 2017." http://www.darkreading.com/attacks-breaches/when-encryption-becomes-the-enem... This article mentions how difficult is it for Sandboxes to detect malware. https://www.fireeye.com/content/dam/fireeye-www/global/en/current-threats/pd... This article mentions malware that changes it’s download image every 15 seconds. http://www.darkreading.com/vulnerabilities---threats/cerber-strikes-with-office-365-zero-day-attacks/d/d-id/1326070?_mc=NL_DR_EDT_DR_weekly_20160630&cid=NL_DR_EDT_DR_weekly_20160630&elqTrackId=1d7f1b5bcdb24c469164471a423f746b&elq=01e6838c279149a08e460cdbe3b8b54a&elqaid=70982&elqat=1&elqCampaignId=21896
There's also BYOD issues where a laptop comes in and infects all your systems from behind the firewall (as Marcus Ranum says: "Crunchy on the outside, soft and chewy inside”).
In any case,your first two actions should be to recover the password for the Palo Alto, and make sure it has updated pattern definitions in effect on both IPv4 and IPv6 connections.
And your third should be to re-examine your vendor rules of engagement, to ensure your deliverables include things like passwords and update support so you're not stuck if your vendor goes belly up..
--- Bruce Curtis bruce.curtis@ndsu.edu Certified NetAnalyst II 701-231-8527 North Dakota State University
That is a good point. In order for your PCs to be compromised via ipv6, they would have to be able to establish ipv6 connectivity to each other or to an internet location. If your network is not configured to support ipv6 it will probably only be possible for your clients to communicate with each other via ipv6 on the local LAN meaning they could only be infecting each other. In order for your clients to be receiving traffic from the Internet via ipv6 would probably require routing and ipv6 configuration support that it sounds like your network does not have. If your firewall is passing v6 traffic, it must understand it enough to forward it across interfaces. At this point it does not much matter whether the transport layer is v4 or v6 because this problem is higher up the protocol stack. Setting up your firewall to bypass v6 (i.e. just pass it) was a huge tactical error (might be why your consultant is out of business :) and a bit hard for me to understand. If you want v6 then you would apply the same policies that you do to v4 traffic and if you don't want v6 you would just tell the firewall to drop it. I think it is much more probable that you are receiving malware via ipv4 or even executable attachments that the out of control firewall is not detecting. I can tell you that we use the most current versions of Checkpoint firewalls with all of the malware bells and whistles (megabucks) and they are not still 100% effective all of the time. We stop thousands of hacking and malware attempts per hour but it only takes one to become a big pain to deal with. Steven Naslund Chicago IL -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Valdis.Kletnieks@vt.edu Sent: Tuesday, July 05, 2016 9:33 AM To: Edgar Carver Cc: nanog@nanog.org Subject: Re: NAT firewall for IPv6? On Fri, 01 Jul 2016 21:28:54 -0500, Edgar Carver said:
We're having problems where viruses are getting through Firefox, and we think it's because our Palo Alto firewall is set to bypass filtering for IPv6.
Do you have any actual evidence (device logs, tcpdump, netflow, etc) that support that train of thought? Remember that your Palo Alto isn't stopping 100% of the icky stuff on the IPv4 side either - the sad truth is that most commercial security software is only able to identify and block between 30% and 70% of the crap that's out in the wild. There's also BYOD issues where a laptop comes in and infects all your systems from behind the firewall (as Marcus Ranum says: "Crunchy on the outside, soft and chewy inside"). In any case,your first two actions should be to recover the password for the Palo Alto, and make sure it has updated pattern definitions in effect on both IPv4 and IPv6 connections. And your third should be to re-examine your vendor rules of engagement, to ensure your deliverables include things like passwords and update support so you're not stuck if your vendor goes belly up..
On 7/1/16 8:28 PM, Edgar Carver wrote:
Unfortunately, the network admin couldn't give me the password since a local consultant set it up, and it seems they went out of business. I need to think outside the box.
So your network admin didn't bother to get the login/enable password for a device that is an integral part of your network? That's... a very big lapse in their responsibilities. I had a consultant recently in CO try to pull the same stunt on me for one of the companies I consult for - stalling, giving bullshit reasons, etc on why they couldn't just hand over the administrative passwords to the actual IT people in the company. Why were we demanding admin access? Because the company we were paying to maintain it we suspected weren't doing their job. We figured they knew exactly why we were asking for the information, and were buying time. IIRC, we were right about the condition of the firewall, switches, etc. Anyways, moral of the story, don't let a consultant hold any and all the keys to the castle for exactly the situation you have right now. -- Brielle Bruns The Summit Open Source Development Group http://www.sosdg.org / http://www.ahbl.org
Not to belabor the point, because it will likely be made frequently in responses, but every legitimate service _should_ have both IPv4 and IPv6 addresses. Get Palo Alto on the horn, and get access to that box. Get it configured properly. I won't hammer you since you're just trying to solve a problem, but v6 is not a second class citizen. You must consider v4 and v6 for these types of issues, and making one or the other 'go away' is simply collecting some tech debt that you'll have to eventually pay off. On Friday, July 1, 2016, Edgar Carver <dredgarcarver@gmail.com> wrote:
Hello NANOG community. I was directed here by our network administrator since she is on vacation. Luckily, I minored in Computer Science so I have some familiarity.
We have a small satellite campus of around 170 devices that share one external IPv4 and IPv6 address via NAT for internet traffic. Internal traffic is over an MPLS.
We're having problems where viruses are getting through Firefox, and we think it's because our Palo Alto firewall is set to bypass filtering for IPv6. Unfortunately, the network admin couldn't give me the password since a local consultant set it up, and it seems they went out of business. I need to think outside the box.
Is there some kind of NAT-based IPv6 firewall I can setup on the router that can help block viruses? I figure that's the right place to start since all the traffic gets funneled there. We have a Cisco Catalyst as a router. Or, ideally, is there an easy way to turn off IPv6 completely? I really don't see a need for it, any legitimate service should have an IPv4 address.
I'd really appreciate your advice. I plan to drive out there tomorrow, where I can get the exact model numbers and stuff.
Regards, Dr. Edgar Carver
Hi, I would go through the password recovery options on the PaloAlto. as a next gen firewall you need to ensure you are getting all the latets rulesets and detection code through - check your subscription with them once you've sorted out access you can look at the policies and ensure that the IPv6 AV filtering rules match that for IPv4 - fairly easy with their interface. (check your codebase version for feature abilities....once again, you may need to deal with PA to ensure your codebase is current. these things get OLD quickly as for NAT for IOV6. nope. and turning it off ISNT the answer (yes, its an answer...just the wrong one! ;-) ) alan
The Palo-Alto's also don't support anything but NAT64, so depending on what you meant by the IPv6 side is sharing "one address" might not be correct. *Spencer Ryan* | Senior Systems Administrator | sryan@arbor.net *Arbor Networks* +1.734.794.5033 (d) | +1.734.846.2053 (m) www.arbornetworks.com On Tue, Jul 5, 2016 at 11:40 AM, <A.L.M.Buxey@lboro.ac.uk> wrote:
Hi,
I would go through the password recovery options on the PaloAlto.
as a next gen firewall you need to ensure you are getting all the latets rulesets and detection code through - check your subscription with them
once you've sorted out access you can look at the policies and ensure that the IPv6 AV filtering rules match that for IPv4 - fairly easy with their interface. (check your codebase version for feature abilities....once again, you may need to deal with PA to ensure your codebase is current. these things get OLD quickly
as for NAT for IOV6. nope. and turning it off ISNT the answer (yes, its an answer...just the wrong one! ;-) )
alan
NAT64 is the only type of IPv6 NAT they support. *Spencer Ryan* | Senior Systems Administrator | sryan@arbor.net *Arbor Networks* +1.734.794.5033 (d) | +1.734.846.2053 (m) www.arbornetworks.com On Tue, Jul 5, 2016 at 12:18 PM, <Valdis.Kletnieks@vt.edu> wrote:
On Tue, 05 Jul 2016 11:54:14 -0400, Spencer Ryan said:
The Palo-Alto's also don't support anything but NAT64,
They don't support proper dual-stack?? Or NAT64 is the only NAT flavor they support on the v6 side?
Hi,
The Palo-Alto's also don't support anything but NAT64,
They don't support proper dual-stack?? Or NAT64 is the only NAT flavor
of course they support native IPv6 ...or IPv4 with IPv6 in dual-stack. i believe the comment was related to the 6/4 xlat stuff - ie just NAT64 and not 464XLAT etc - I've not looked into that myself as we do dual stack alan
On 07/01/2016 07:28 PM, Edgar Carver wrote:
Is there some kind of NAT-based IPv6 firewall I can setup on the router that can help block viruses?
You need layer-7 firewalls for this. NAT-based "firewalls" (pseudo-firewalls, really) are layer-4 only. Those will not help you block typical viruses, as people will usually get infected from connecting to a compromised Website, or from an e-mail attachments. And even more, if connections are encrypted, an L7 firewall will not be able to do anything (whether IPv4 or v6) unless... better not open a can of worms. They will just help you block *some* attack vectors, though: those that rely on starting connections to your hosts from the outside. My guess is that, with regard to e-mail attachments and compromised Websites, IPv4 hosts are still attacked more than IPv6 ones, so, even if you turn off IPv6 you will still get attacked through IPv4. Everything else has been already said by others: fixing the Palo Alto is still your best bet. Good luck!
On 5 July 2016 at 21:47, Octavio Alvarez <octalnanog@alvarezp.org> wrote:
Everything else has been already said by others: fixing the Palo Alto is still your best bet.
No while that is also needed, it is very unlikely to fix his issue. The issue at hand is that some of their computers have become virus infected. The fix for that is to upgrade the virus scanner and making sure that all software upgrades are done. Someone comes to you and says his Firefox is getting infected through IPv6. If your support is worth anything, you will not take that at face value and bill him for a ton work related to IPv6. No, you will go find out what the real issue is and solve that. The only thing we know right now is that he is confused. Regards, Baldur
It is all about defense in depth. The engineers here are speaking to the network pieces (the second N in NANOG is network, right :) and we have told this person that it is unlikely that v6 in the only vector and I myself talked about malware handling on the clients themselves. From a network engineering perspective many of us agreed that the biggest single threat to his network was a firewall in an unknown state with an unknown administrator password that could be owned by anyone on earth at this point. That single piece threatens the entire network as a whole and is a ticking time bomb ready to blow his entire LAN off the Internet if it fails. He probably does not own the entire environment himself, he is filling in for a vacationing network engineer. So he is working on the network piece and is probably not responsible for the anti-malware software on the clients (if anyone is, see below). Our "support" as you call it was a response to this person questions about blocking v6 as an attack vector in the first place. We answered his question but then told him that was unlikely to be the problem and what he should do about taking back his firewall, securing v6 via the firewall, and handling the malware at the client. Seems solid advise to me so far. BTW we did not bill him for anything. He got a lot of free advice from a lot of people he could not even begin to afford to employ, so not a bad deal for him. You also have to understand that this gentleman seems to be in an educational environment which usually means lots of clients he does not have control over so having some kind of network based malware control is helpful. Clients in this type of environment have to defend themselves from each other and he will likely have stuff brought in from the outside. Good malware detection in the network can help identify clients that contain malware and are a threat to other devices. Fancier network gear/IDS/IDP would actually remove offending clients from the network or at least segments them into an isolation area. Let me re-iterate: 1. Take back ownership of your firewall and bring it up to date including new malware signatures. If you don't have current support, get it...........directly so if your consultant bails you are not dead meat. This will ensure that the outside world will not own or control stuff inside your network while you put the fires out. At the very least it can help malware infected machines from phoning home to their command and control servers which sometimes prevents a lot of damage. 2. Make your v6 rules mirror at least the security level of your v4 rules. Passing v6 unchallenged is unacceptable. If your firewall won't do it replace it with one that will. 3. Ensure all clients under your control have current anti-virus/anti-malware detection. Clients have to defend themselves from threats internal to the firewall as well as ones outside. Don't be hard on the outside with a soft chewy center. 4. Never, ever accept anything less than full administrative control passwords and accounts from your consultants, before you give them final payment. I actually prefer to lock them out when they complete an install until I need them to help with something. This prevents them from holding you hostage or one of their "postal" employees from wiping you out as well as preventing them from using your network for experimentation without you knowing it. It is an important part of change control to ensure that outsiders cannot modify your configuration without contacting you first. We usually give our consultants highly logged VPN accounts that we can disable or enable as needed. Steven Naslund Chicago IL
No while that is also needed, it is very unlikely to fix his issue. The issue at hand is that some of their computers have become virus infected. The fix for that is to upgrade the virus scanner and making sure that all software upgrades are done.
Someone comes to you and says his Firefox is getting infected through IPv6. If your support is worth anything, you will not take that at face value and bill him for a ton work related to IPv6. No, you will go find out what the real issue is and solve that. The only thing we know right now is that he is >>confused.
Regards,
Baldur
The original email was not a serious question, but a joke: https://twitter.com/SwiftOnSecurity/status/749059605360062464 https://twitter.com/SwiftOnSecurity/status/749062835687174144 https://twitter.com/SwiftOnSecurity/status/749068172460847105 On Tue, Jul 5, 2016 at 1:41 PM, Naslund, Steve <SNaslund@medline.com> wrote:
It is all about defense in depth. The engineers here are speaking to the network pieces (the second N in NANOG is network, right :) and we have told this person that it is unlikely that v6 in the only vector and I myself talked about malware handling on the clients themselves. From a network engineering perspective many of us agreed that the biggest single threat to his network was a firewall in an unknown state with an unknown administrator password that could be owned by anyone on earth at this point. That single piece threatens the entire network as a whole and is a ticking time bomb ready to blow his entire LAN off the Internet if it fails.
He probably does not own the entire environment himself, he is filling in for a vacationing network engineer. So he is working on the network piece and is probably not responsible for the anti-malware software on the clients (if anyone is, see below).
Our "support" as you call it was a response to this person questions about blocking v6 as an attack vector in the first place. We answered his question but then told him that was unlikely to be the problem and what he should do about taking back his firewall, securing v6 via the firewall, and handling the malware at the client. Seems solid advise to me so far.
BTW we did not bill him for anything. He got a lot of free advice from a lot of people he could not even begin to afford to employ, so not a bad deal for him. You also have to understand that this gentleman seems to be in an educational environment which usually means lots of clients he does not have control over so having some kind of network based malware control is helpful. Clients in this type of environment have to defend themselves from each other and he will likely have stuff brought in from the outside. Good malware detection in the network can help identify clients that contain malware and are a threat to other devices. Fancier network gear/IDS/IDP would actually remove offending clients from the network or at least segments them into an isolation area.
Let me re-iterate:
1. Take back ownership of your firewall and bring it up to date including new malware signatures. If you don't have current support, get it...........directly so if your consultant bails you are not dead meat. This will ensure that the outside world will not own or control stuff inside your network while you put the fires out. At the very least it can help malware infected machines from phoning home to their command and control servers which sometimes prevents a lot of damage. 2. Make your v6 rules mirror at least the security level of your v4 rules. Passing v6 unchallenged is unacceptable. If your firewall won't do it replace it with one that will. 3. Ensure all clients under your control have current anti-virus/anti-malware detection. Clients have to defend themselves from threats internal to the firewall as well as ones outside. Don't be hard on the outside with a soft chewy center. 4. Never, ever accept anything less than full administrative control passwords and accounts from your consultants, before you give them final payment. I actually prefer to lock them out when they complete an install until I need them to help with something. This prevents them from holding you hostage or one of their "postal" employees from wiping you out as well as preventing them from using your network for experimentation without you knowing it. It is an important part of change control to ensure that outsiders cannot modify your configuration without contacting you first. We usually give our consultants highly logged VPN accounts that we can disable or enable as needed.
Steven Naslund Chicago IL
No while that is also needed, it is very unlikely to fix his issue. The issue at hand is that some of their computers have become virus infected. The fix for that is to upgrade the virus scanner and making sure that all software upgrades are done.
Someone comes to you and says his Firefox is getting infected through IPv6. If your support is worth anything, you will not take that at face value and bill him for a ton work related to IPv6. No, you will go find out what the real issue is and solve that. The only thing we know right now is that he is >>confused.
Regards,
Baldur
You know the cosmological model that the earth is balanced on the back of a giant turtle, which is supported by successive lower tiers of other turtles? https://en.wikipedia.org/wiki/Turtles_all_the_way_down It's like that, except it's trolls all the way down. On Tue, Jul 5, 2016 at 3:24 PM, Chase Christian <madsushi@gmail.com> wrote:
The original email was not a serious question, but a joke:
https://twitter.com/SwiftOnSecurity/status/749059605360062464 https://twitter.com/SwiftOnSecurity/status/749062835687174144 https://twitter.com/SwiftOnSecurity/status/749068172460847105
On Tue, Jul 5, 2016 at 1:41 PM, Naslund, Steve <SNaslund@medline.com> wrote:
It is all about defense in depth. The engineers here are speaking to the network pieces (the second N in NANOG is network, right :) and we have told this person that it is unlikely that v6 in the only vector and I myself talked about malware handling on the clients themselves. From a network engineering perspective many of us agreed that the biggest single threat to his network was a firewall in an unknown state with an unknown administrator password that could be owned by anyone on earth at this point. That single piece threatens the entire network as a whole and is a ticking time bomb ready to blow his entire LAN off the Internet if it fails.
He probably does not own the entire environment himself, he is filling in for a vacationing network engineer. So he is working on the network piece and is probably not responsible for the anti-malware software on the clients (if anyone is, see below).
Our "support" as you call it was a response to this person questions about blocking v6 as an attack vector in the first place. We answered his question but then told him that was unlikely to be the problem and what he should do about taking back his firewall, securing v6 via the firewall, and handling the malware at the client. Seems solid advise to me so far.
BTW we did not bill him for anything. He got a lot of free advice from a lot of people he could not even begin to afford to employ, so not a bad deal for him. You also have to understand that this gentleman seems to be in an educational environment which usually means lots of clients he does not have control over so having some kind of network based malware control is helpful. Clients in this type of environment have to defend themselves from each other and he will likely have stuff brought in from the outside. Good malware detection in the network can help identify clients that contain malware and are a threat to other devices. Fancier network gear/IDS/IDP would actually remove offending clients from the network or at least segments them into an isolation area.
Let me re-iterate:
1. Take back ownership of your firewall and bring it up to date including new malware signatures. If you don't have current support, get it...........directly so if your consultant bails you are not dead meat. This will ensure that the outside world will not own or control stuff inside your network while you put the fires out. At the very least it can help malware infected machines from phoning home to their command and control servers which sometimes prevents a lot of damage. 2. Make your v6 rules mirror at least the security level of your v4 rules. Passing v6 unchallenged is unacceptable. If your firewall won't do it replace it with one that will. 3. Ensure all clients under your control have current anti-virus/anti-malware detection. Clients have to defend themselves from threats internal to the firewall as well as ones outside. Don't be hard on the outside with a soft chewy center. 4. Never, ever accept anything less than full administrative control passwords and accounts from your consultants, before you give them final payment. I actually prefer to lock them out when they complete an install until I need them to help with something. This prevents them from holding you hostage or one of their "postal" employees from wiping you out as well as preventing them from using your network for experimentation without you knowing it. It is an important part of change control to ensure that outsiders cannot modify your configuration without contacting you first. We usually give our consultants highly logged VPN accounts that we can disable or enable as needed.
Steven Naslund Chicago IL
No while that is also needed, it is very unlikely to fix his issue. The issue at hand is that some of their computers have become virus infected. The fix for that is to upgrade the virus scanner and making sure that all software upgrades are done.
Someone comes to you and says his Firefox is getting infected through IPv6. If your support is worth anything, you will not take that at face value and bill him for a ton work related to IPv6. No, you will go find out what the real issue is and solve that. The only thing we know right now is that he is >>confused.
Regards,
Baldur
Wonderfully crafted, too. Great work. S. On 5 July 2016 at 15:39, Seth Mattinen <sethm@rollernet.us> wrote:
On 7/1/16 19:28, Edgar Carver wrote:
Hello NANOG community. I was directed here by our network administrator since she is on vacation. Luckily, I minored in Computer Science so I have some familiarity.
This is not legit, ya'll are being trolled.
~Seth
My how the world has changed! On 7/1/2016 21:28, Edgar Carver wrote:
Hello NANOG community. I was directed here by our network administrator since she is on vacation.
I am Old School, I guess. In my day Step One would be "Fire the administrator." The job is by nature a 24 X 7 X 52 job and "On Call" the rest of the time. "Vacation" is never a reason to leave your assignment insecure. "NAT-based firewall"? Really? How long has the consultant been out of business? Luckily, I minored in Computer Science so I have
some familiarity.
We have a small satellite campus of around 170 devices that share one external IPv4 and IPv6 address via NAT for internet traffic. Internal traffic is over an MPLS.
We're having problems where viruses are getting through Firefox, and we think it's because our Palo Alto firewall is set to bypass filtering for IPv6. Unfortunately, the network admin couldn't give me the password since a local consultant set it up, and it seems they went out of business. I need to think outside the box.
Is there some kind of NAT-based IPv6 firewall I can setup on the router that can help block viruses? I figure that's the right place to start since all the traffic gets funneled there. We have a Cisco Catalyst as a router. Or, ideally, is there an easy way to turn off IPv6 completely? I really don't see a need for it, any legitimate service should have an IPv4 address.
I'd really appreciate your advice. I plan to drive out there tomorrow, where I can get the exact model numbers and stuff.
Regards, Dr. Edgar Carver
-- "Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid." --Albert Einstein From Larry's Cox account.
My how the world has changed! On 7/1/2016 21:28, Edgar Carver wrote:
Hello NANOG community. I was directed here by our network administrator since she is on vacation.
I am Old School, I guess. In my day Step One would be "Fire the administrator." The job is by nature a 24 X 7 X 52 job and "On Call" the rest of the time. "Vacation" is never a reason to leave your assignment insecure. "NAT-based firewall"? Really? How long has the consultant been out of business? Luckily, I minored in Computer Science so I have
some familiarity.
I have no idea how I fat-fingered a "send" at this point/ I started to write that you have an emergency on your hands and you need to focus your attention of finding a person or firm that can take charge and fix problems you don't even know about yet. A "Dear Abby" approach is going do way more harm than good. -- "Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid." --Albert Einstein From Larry's Cox account.
On Fri, Jul 01, 2016 at 09:28:54PM -0500, Edgar Carver wrote:
Hello NANOG community. I was directed here by our network administrator since she is on vacation. Luckily, I minored in Computer Science so I have some familiarity.
Well played, Tay. Well played. For everyone else: https://twitter.com/SwiftOnSecurity/status/749062835687174144 - Matt
On 7/5/2016 18:46, Matt Palmer wrote:
On Fri, Jul 01, 2016 at 09:28:54PM -0500, Edgar Carver wrote:
Hello NANOG community. I was directed here by our network administrator since she is on vacation. Luckily, I minored in Computer Science so I have some familiarity.
Well played, Tay. Well played.
I was suspicious at the "minored" announcement, but it looked so much like traffic here..... I guess the reality is that for legitimate traffic, this list is used only as a "calling frequency" with the "working frequency" being somewhere secret. Sad.
For everyone else:
https://twitter.com/SwiftOnSecurity/status/749062835687174144
-- "Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid." --Albert Einstein From Larry's Cox account.
participants (19)
-
A.L.M.Buxey＠lboro.ac.uk -
Baldur Norddahl -
Brielle Bruns -
Bruce Curtis -
Chase Christian -
Dovid Bender -
Edgar Carver -
Eric Kuhnke -
Jason R -
Larry Sheldon -
Lee -
Matt Palmer -
Naslund, Steve -
Octavio Alvarez -
Seth Mattinen -
Spencer Ryan -
Stephen Strowes -
Tom Beecher -
Valdis.Kletnieks＠vt.edu