Re: black listing of web traffic
Can't find my IP on any of the black lists. Don't have any proxies. Sites that behave poorly are consistent. That is to say that facebook.com, apple.com would always come up without an issue, but cnn.com, forever21.com(i know, don't ask, students), store.apple.com would consistently take forever to come up. Just wanted to check of rate-limiting web clients is a common practice nowdays in the industry. If it's not, it's probably an unlikely cause of my troubles... Thanks, Andrey ----- Andrey Gordon [andrey.gordon@gmail.com] On Tue, Feb 9, 2010 at 4:40 PM, Geoffrey Keating <geoffk@geoffk.org> wrote:
Andrey Gordon wrote:
Other things you might want to check out include whether your NAT gateway is well-behaved in the presence of PMTU discovery, TCP timestamps, and ECN. The web sites your students are having trouble with may share some property that, correctly or not, is interacting poorly with your NAT implementation. (I remain astonished at the number of "big name" web sites out there that send out their content with the DF bit set, then drop the "fragmentation required" ICMP packets they get back on the floor.) Jim Shankland
Andrey Gordon wrote:
It could be that the problem sites have some form of load balancer that has an issue keeping state on multiple sessions from the same IP. You mentioned that changing the source IP fixed it. Is this a temporary fix that breaks after several users access the sites from the new IP? -- Jay Hennigan - CCIE #7880 - Network Engineering - jay@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
Thx to all the folks replying off the list. The more I trouble shoot the more I'm convinced that it's not the sites that are doing rate-limiting. I went to a website of one of my previous employers (a small company). Chances of them having a fancy reverse proxy with some sort of black list filtering are slim to none, yet their site barely opens up as well. Must be something that either my firewall device is doing (which is what is doing the NATting) or I don't' know what else. I'm working with my firewall guy since f/w is his domain and I have no clue about that vendor of the firewalls (PaloAlto). Thanks all for the suggestions. I'll keep digging. ----- Andrey Gordon [andrey.gordon@gmail.com] On Tue, Feb 9, 2010 at 4:56 PM, Jay Hennigan <jay@west.net> wrote:
On Tue, 2010-02-09 at 17:04 -0500, Andrey Gordon wrote:
A few months ago I was involved in a hard-to-troubleshoot intermittent problems similar to yours. I finally diagnosed a faulty or overloaded state table somewhere in one of the cheap plastic routers they were using. All problems ended when I replaced the cheap plastic stuff with a x86 hardware running pf or iptables, I forget exactly which (irrelevant). Could it be that you have some arp-poisoning going on? That was my first thought in the above situation, but Wireshark showed otherwise. The clue to the state tables - it was mainly SSL/TLS that was getting expired/dropped. Gord
By changing my outbound IP address to a different one (i suspect effectively resetting sessions) the problem was solved. So, after that I set it back to the original source NAT. And the sites open up just fine still. It really behaves like a NAT table exhaustion, but the firewall only reports 13000 sessions in progress for all the NAT addresses on that firewall. I'm thinking memory leak or something. We only put that device in place this winter break and this is the second time this is happening. Last time was about 2-3 weeks ago. Seems to be fixed for now and the f/w dude is opening a ticket with the f/w vendor. ----- Andrey Gordon [andrey.gordon@gmail.com]
That's not surprising behaviour on a PaloAlto unit, they are still very young in the market and my colleagues have had issues with NAT and proxy arp in the recent past. Chris Campbell --------------------- On 9 Feb 2010, at 22:31, "Andrey Gordon" <andrey.gordon@gmail.com> wrote:
participants (6)
-
Andrey Gordon
-
Chris Campbell
-
gordon b slater
-
Jay Hennigan
-
Jim Shankland
-
Rogelio