Re: AWS Web Application Firewall blocks ISP ranges?
Endured a similar problem last year. The company I was with migrated prod to Global Accelerator, ALBs, and WAF with Shield Advanced on an accelerated timeline. Several of our customer networks were blocked in AWS's default blocklists. The path to resolution was meeting with our AWS account executive and their internal WAF team to expedite the removal of the addresses. -G On 2024-03-21 18:44, Aaron Wendel wrote:
Yes.
our network is a mix of content and eyeballs and they listed the whole thing. This has prevented the local school district from using their text to speech application (for their deaf students) as well as others. This has also affected the local library and residences.
It's a PITA.
Thanks Amazon.
Aaron
On 3/21/2024 12:16 PM, Jonathan Kalbfeld via NANOG wrote:
Hi All,
I just became aware that AWS has a list of hosting IP providers and that list is blocked by their WAF? (!?!?). None of my VM or colo customers can reach anything in AWS, such as Docker, Twilio, etc. I confirmed through source routing that when I access it using one of my peering partners as a source IP it is reachable, but using one of my net blocks, it is not reachable and times out. Checked all of my routing tables and those AWS blocks are definitely visible. Also confirmed from looking glass that my IP ranges are showing up.
Has anyone else encountered that? If so, is there a way to get removed from that list? I have a very curated list of clients and I know all of them personally and none of them have been abusing AWS, so I was wondering if it was some kind of blanket ban?
If you're internal to AWS, my ASN is 54380, IP ranges affected are 199.33.244.0/24, 199.79.202.0/24, 199.188.96.0/22, 45.59.144.0/22 and 206.197.110.0/24
Feel free to reach out off-list.
Thanks,
Jonathan Kalbfeld
Jonathan Kalbfeld
office: +1 310 317 7933 <tel:%28310%29%20317-7933> fax: +1 310 317 7901 <tel:%28310%29%20317-7901> home: +1 310 317 7909 <tel:%28310%29%20317-7909> mobile: +1 310 227 1662 <tel:%28310%29%20227-1662>
ThoughtWave Technologies, Inc. Studio City, CA 91604 https://thoughtwave.com
View our network at https://bgp.he.net/AS54380
+1 844 42-LINUX
Hi. Our experience from last month was quite similar to what others have already described. Based on our tests all the allocations associated with our ASN were added to "HostingProviderIPList". At first we tried to reach the AWS by e-mail, but received no response. Once we opened a ticket(we are both the AWS customer and a Direct Connect Delivery partner) and explained that our address space is not solely used for hosting services, then all our networks were whitelisted within few days. We even had a call with AWS support team and they seemed to be well aware about false positives in the "HostingProviderIPList". In other words, we were clearly not a first ISP complaining. Final update to our ticket from the AWS Support Engineering department was following: "I would like to inform you that as per the update I got from our internal team, the IP Data which we use for our rules is provided by a third party and changes frequently. Moving ahead I would like to request you to kindly check your setup and see if you can separate the network ranges reserved for any hosting type services in your ASN and provide that to us so that we can check if we can allow the range which is not used for hosting type services." It's unfortunate that Amazon continues to offer this service despite their underlying IP address data having a low level of granularity, and they are well aware of the problems it causes for address owners. AWS Web Application Firewall for testing purposes can be build for example like this: 1) created a VPC("VPC only") with "IPv4 CIDR" 10.0.0.0/24 2) created a subnet 10.0.0.0/28 in one availability zone(for example "eu-north-1a") and a subnet 10.0.0.16/28 in another availability zone(for example "eu-north-1b") 3) launched an Ubuntu EC2 instance("t3.micro", free tier eligible). Auto-assigned the public IP. Allowed SSH and HTTP traffic from anywhere. 4) created an "Internet gateway" and attached it to VPC 5) added a default route to route table and pointed it to Internet gateway 6) installed(sudo apt update && sudo apt install apache2 -y) a web server in Ubuntu VM 7) tested with "curl http://51.20.94.186" where 51.20.94.186 is the public IP address automatically assigned to EC2 instance 8) created a "Target group" for "Instances" type target which is associated with VPC created earlier. Protocol:Port configuration has to be HTTP:80. 9) created an Internet-facing "Application Load balancer" and mapped it with two subnets created previously in two different availability zones. Load balancer has to forward to target group created in the previous step. 10) created a WAF and associate it with Application Load Balancer 11) used an AWS managed rule group named "Anonymous IP list" in WAF. This includes the notorious "HostingProviderIPList" with a default rule action of "Block". 12) "curl http://test-213161342.eu-north-1.elb.amazonaws.com" returns "HTTP/1.1 403" from the networks included in the "HostingProviderIPList". DNS name is seen on the load balancer details page. P.S. Previous thread on AWS WAF was https://mailman.nanog.org/pipermail/nanog/2024-February/224931.html Martin
participants (2)
-
G -
Martin Tonusoo