I don't need no stinking firewall!
Security Gurus, et al, I have my own idea of what a firewall is and what it does. I also understand what statefull packet inspection is and what it does. Given this information, and not prejudging any responses, exactly what is a firewall for and when is statefull inspection useful? Please respond on-list as I want to have some useful discourse and discussion in the clear. Flamers and Trolls will be disregarded. :) Thank you. - Brian CONFIDENTIALITY NOTICE: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, copying, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Thank you.
On Jan 6, 2010, at 3:16 AM, Brian Johnson wrote:
Given this information, and not prejudging any responses, exactly what is a firewall for and when is statefull inspection useful?
In the most basic terms, a stateful firewall performs bidirectional classification of communications between nodes, and makes a pass/fail determination on each packet based on a) whether or not a bidirectional communications session is already open between the nodes and b) any policy rules configured on the firewall as to what ports/protocols should be allowed between said nodes. Stateful firewalls make good sense in front of machines which are primarily clients; the stateful inspection part keeps unsolicited packets away from the clients. Stateful firewalls make absolutely no sense in front of servers, given that by definition, every packet coming into the server is unsolicited (some protocols like ftp work a bit differently in that there're multiple bidirectional/omnidirectional communications sessions, but the key is that the initial connection is always unsolicited). Putting firewalls in front of servers is a Really Bad Idea - besides the fact that the stateful inspection premise doesn't apply (see above), rendering the stateful firewall superfluous, even the biggest, baddest firewalls out there can be easily taken down via state-table exhaustion; an attacker can craft enough programmatically-generated, well-formed traffic which conforms to the firewall policies to 'crowd out' legitimate traffic, thus DoSing the server. Addtionally, the firewall can be made to collapse far quicker than the server itself would collapse, as the overhead on the state-tracking is less than what the server itself could handle on its own. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
On 1/5/10 1:29 PM, Dobbins, Roland wrote:
Putting firewalls in front of servers is a Really Bad Idea - besides the fact that the stateful inspection premise doesn't apply (see above), rendering the stateful firewall superfluous, even the biggest, baddest firewalls out there can be easily taken down via state-table exhaustion; an attacker can craft enough programmatically-generated, well-formed traffic which conforms to the firewall policies to 'crowd out' legitimate traffic, thus DoSing the server. Addtionally, the firewall can be made to collapse far quicker than the server itself would collapse, as the overhead on the state-tracking is less than what the server itself could handle on its own.
The trick is to not track ports/IPs that do not need it. On my combo firewalls (that handle both NATing and serving websites, dns, etc) for example, I'll do a NOTRACK on the LAN side to prevent connections to the firewall itself from taking up valuable table space. It's all how you configure and tweak the firewall. Recommending people run servers without a firewall is bad advice - do you really want your Win2k3 server exposed, SMB, RPC, and all to the world? -- Brielle Bruns The Summit Open Source Development Group http://www.sosdg.org / http://www.ahbl.org
On Tue Jan 05, 2010 at 01:58:52PM -0700, Brielle Bruns wrote:
It's all how you configure and tweak the firewall. Recommending people run servers without a firewall is bad advice - do you really want your Win2k3 server exposed, SMB, RPC, and all to the world?
I have an answer to that problem, but not everyone would agree with it [1]. Simon -- Simon Lockhart | * Sun Server Colocation * ADSL * Domain Registration * Director | * Domain & Web Hosting * Internet Consultancy * Bogons Ltd | * http://www.bogons.net/ * Email: info@bogons.net * [1] That said, when I've had no choice but to use a Win2k3 web server, I've proxy-passed it behind an Apache server.
On 1/5/10 2:06 PM, Simon Lockhart wrote:
I have an answer to that problem, but not everyone would agree with it [1].
One of my biggest beefs with some people is that they'll stand there with their fingers in their ears yelling LA LA LA if you point out to them that not every person in the world can use Linux/UNIX/etc. Some businesses _must_ use Windows to support the applications they use.
[1] That said, when I've had no choice but to use a Win2k3 web server, I've proxy-passed it behind an Apache server.
Except that it requires yet another machine to be sold to a customer who already laid out however many thousands of dollars for a server + licenses + CALs + applications. If we already have the firewall, its _very_ hard for me to justify to the customer another chunk of cash for a second firewall just for the server. -- Brielle Bruns The Summit Open Source Development Group http://www.sosdg.org / http://www.ahbl.org
On Jan 5, 2010, at 3:58 PM, Brielle Bruns wrote:
It's all how you configure and tweak the firewall. Recommending people run servers without a firewall is bad advice - do you really want your Win2k3 server exposed, SMB, RPC, and all to the world?
Some people think that exposing any functionality by default such as that is a poor security practice :) My biggest issue is that people think that Firewalls, AV, etc are a catch-all for any network/user/security badness. The real world is more complex than that. Most people make poor security choices and this creates much larger issues. "I thought the firewall would protect me". "I thought my IPS would protect me" "I thought my AV would protect me" Most of these technologies create a truly false sense of security. I'm once again reminded of many people who do technically "silly" things like block TCP/53, packets over 512 bytes, port 587, ssl imap ports, etc. It's frustrating and sad because it's not an effective security strategy and frustrates grumpy old-school users as myself that used odi drivers w/ ka9q to multitask over our CSLIP networks. - Jared
From: Jared Mauch <jared@puck.nether.net> Date: Tue, 5 Jan 2010 16:20:56 -0500
On Jan 5, 2010, at 3:58 PM, Brielle Bruns wrote:
It's all how you configure and tweak the firewall. Recommending people run servers without a firewall is bad advice - do you really want your Win2k3 server exposed, SMB, RPC, and all to the world?
Some people think that exposing any functionality by default such as that is a poor security practice :)
My biggest issue is that people think that Firewalls, AV, etc are a catch-all for any network/user/security badness. The real world is more complex than that.
Most people make poor security choices and this creates much larger issues.
"I thought the firewall would protect me". "I thought my IPS would protect me" "I thought my AV would protect me"
Most of these technologies create a truly false sense of security.
I'm once again reminded of many people who do technically "silly" things like block TCP/53, packets over 512 bytes, port 587, ssl imap ports, etc.
It's frustrating and sad because it's not an effective security strategy and frustrates grumpy old-school users as myself that used odi drivers w/ ka9q to multitask over our CSLIP networks.
I suspect at least part of this will soon get fixed due to DNSSEC. Blocking tcp/53 and packets over 512 bytes will cause user complaints and, after enough education, the problem will get fixed. I had a problem with a large US government site due to tcp/53 blocking and had no luck getting it fixed. The "Security Officer" informed me that tcp/53 was only ever needed for zone transfer and any other use was clear evidence of abuse. RFCs meant nothing to him. (I don't know if he knew what an RFC was.) Now that gov domains are mandated to be signed, seems like he learned that tcp/53 could be used for normal operations. "You can get more with a kind word and a two-by-four than you can with just a kind word." J. Michael Straczynski from Ceremonies of Light and Dark Babylon 5 -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: oberman@es.net Phone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751
On Tue, 5 Jan 2010, Kevin Oberman wrote:
I suspect at least part of this will soon get fixed due to DNSSEC. Blocking tcp/53 and packets over 512 bytes will cause user complaints and, after enough education, the problem will get fixed.
Yes. Remember the root zone is due to be signed within the next six months, and many nameservers (BIND in particular) request DNSSEC data by default. You WILL have to deal with large DNS replies SOON - the first ones from the root servers will appear this month. http://www.root-dnssec.org/ Tony. -- f.anthony.n.finch <dot@dotat.at> http://dotat.at/ GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS. MODERATE OR GOOD.
On Jan 6, 2010, at 3:58 AM, Brielle Bruns wrote:
It's all how you configure and tweak the firewall. Recommending people run servers without a firewall is bad advice - do you really want your Win2k3 server exposed, SMB, RPC, and all to the world?
Nope - I use stateless ACLs in hardware, capable of handling mpps. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
Stateful firewalls make absolutely no sense in front of servers, given that by definition, every packet coming into the server is unsolicited (some protocols like ftp work a bit differently in that there're multiple bidirectional/omnidirectional communications sessions, but the key is that the initial connection is always unsolicited).
I'm interested by this assertion; surely Stateful Inspection is meant to facilitate the blocking of out-of-sequence packets, ones which aren't part of valid + recognised existing sessions - whilst of course allowing valid SYN session-starters, etc? So thus, there may still be some value in catching 'injected' packets which don't actually belong in a session... ?
Putting firewalls in front of servers is a Really Bad Idea - besides the fact that the stateful inspection premise doesn't apply (see above), rendering the stateful firewall superfluous, even the biggest, baddest firewalls out there can be easily taken down via state-table exhaustion; an attacker can craft enough programmatically-generated, well-formed traffic which conforms to the firewall policies to 'crowd out' legitimate traffic, thus DoSing the server. Addtionally, the firewall can be made to collapse far quicker than the server itself would collapse, as the overhead on the state-tracking is less than what the server itself could handle on its own.
Some might argue that DoS is preferred to the other degrees of risk that many webservers hold... (trying not to point the finger in any one specific direction.) Mark.
On Jan 6, 2010, at 4:07 AM, Mark Foster wrote:
I'm interested by this assertion; surely Stateful Inspection is meant to facilitate the blocking of out-of-sequence packets, ones which aren't part of valid + recognised existing sessions - whilst of course allowing valid SYN session-starters, etc?
So thus, there may still be some value in catching 'injected' packets which don't actually belong in a session... ?
Nope - the hosts handle this better on their own.
Some might argue that DoS is preferred to the other degrees of risk that many webservers hold... (trying not to point the finger in any one specific direction.)
Except that the firewalls don't mitigate any of the other degrees of risk, either. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
On Tue, 5 Jan 2010, Dobbins, Roland wrote:
In the most basic terms, a stateful firewall performs bidirectional classification of communications between nodes, and makes a pass/fail determination on each packet based on a) whether or not a bidirectional communications session is already open between the nodes and b) any policy rules configured on the firewall as to what ports/protocols should be allowed between said nodes.
Stateful firewalls make good sense in front of machines which are primarily clients; the stateful inspection part keeps unsolicited packets away from the clients.
Stateful firewalls make absolutely no sense in front of servers, given that by definition, every packet coming into the server is unsolicited (some protocols like ftp work a bit differently in that there're multiple bidirectional/omnidirectional communications sessions, but the key is that the initial connection is always unsolicited).
Putting firewalls in front of servers is a Really Bad Idea - besides the
Hi Roland. I disagree strongly with this position.
fact that the stateful inspection premise doesn't apply (see above),
The problem is that your premise is wrong. Stateful firewalls (hereafter just called firewalls) offer several advantages. This list is not necessarily exhaustive. (1) Security in depth. In an ideal world every packet arriving at a server would be for a port that is intended to be open and listening. Unfortunately ports can be opened unintentionally on servers in several ways: sysadmin error, package management systems pulling in an extra package which starts a service, etc. By having a firewall in front of the server we gain security in depth against errors like this. (2) Centralised management of access controls. Many services should only be open to a certain set of source addresses. While this could be managed on each server we may find that some applications don't support this well, and management of access controls is then spread across any number of management interfaces. Using a firewall for network access control reduces the management overhead and chances of error. Even if network access control is managed on the server, doing it on the firewall offers additional security in depth. (3) Outbound access controls. In many cases we want to stop certain types of outbound traffic. This may contain an intrusion and prevent attacks against other hosts owned by the organisation or other organisations. Trying to do outbound access control on the server won't help as if the server is compromised the attacker can likely get around it. (4) Rate limiting. The ability to rate limit incoming and outgoing data can prevent certain sorts of DoSes. (5) Signature based blocking. Modern firewalls can be tied to intrusion prevention systems which will 'raise the shields' in the face of certain attacks. Many exploits require repeated probing and this provides a way to stop the attack before it is successful.
rendering the stateful firewall superfluous, even the biggest, baddest firewalls out there can be easily taken down via state-table exhaustion;
Do you have any evidence to support this assertion? You've just asserted that all firewalls have a specific vulnerability. It isn't even possible to know the complete set of architectures (hardware & software) used for firewalls so I don't see how you can assert they all have this vulnerability. In any case, my experience tells me that a DDoS will be successful due to exhaustion of available network capacity before it exhausts the state table on the firewall.
an attacker can craft enough programmatically-generated, well-formed traffic which conforms to the firewall policies to 'crowd out' legitimate traffic, thus DoSing the server. Addtionally, the firewall can be made to collapse far quicker than the server itself would collapse, as the overhead on the state-tracking is less than what the server itself could handle on its own.
Again, I don't believe such a global claim can be made given the wide variety of architectures used for firewalls. Cheers, Rob -- I tried to change the world but they had a no-return policy http://www.practicalsysadmin.com
On Jan 6, 2010, at 4:24 AM, Robert Brockway wrote:
Hi Roland. I disagree strongly with this position.
You can disagree all you want, but it's still borne out by real-world operational experience. ;>
The problem is that your premise is wrong.
Just what about my premise is wrong? Nothing in your message repudiates - or even addresses - a single thing I've said. ;>
Stateful firewalls (hereafter just called firewalls) offer several advantages.
Not in front of servers, they don't. Clients, yes. Servers, no.
This list is not necessarily exhaustive.
No, it doesn't include all - or even any - of the reasons firewalls are inappropriate for placement in front of servers, not by a long shot. ;>
(1) Security in depth. In an ideal world every packet arriving at a server would be for a port that is intended to be open and listening. Unfortunately ports can be opened unintentionally on servers in several ways: sysadmin error, package management systems pulling in an extra package which starts a service, etc. By having a firewall in front of the server we gain security in depth against errors like this.
Stateless ACLs in router/switch hardware capable of handling mpps takes care of this.
(2) Centralised management of access controls. Many services should only be open to a certain set of source addresses. While this could be managed on each server we may find that some applications don't support this well, and management of access controls is then spread across any number of management interfaces. Using a firewall for network access control reduces the management overhead and chances of error. Even if network access control is managed on the server, doing it on the firewall offers additional security in depth.
Stateless ACLs in router/switch hardware capable of handling mpps takes care of this.
(3) Outbound access controls. In many cases we want to stop certain types of outbound traffic. This may contain an intrusion and prevent attacks against other hosts owned by the organisation or other organisations. Trying to do outbound access control on the server won't help as if the server is compromised the attacker can likely get around it.
Stateless ACLs in router/switch hardware capable of handling mpps takes care of this.
(4) Rate limiting. The ability to rate limit incoming and outgoing data can prevent certain sorts of DoSes.
Rate-limiting during a DDoS - i.e., an attack against state and *capacity* - is absolutely the *worst* thing one can possibly do, in almost all circumstances. Yet another security myth which is easily disproved via hands-on operational experience.
(5) Signature based blocking.
Signatures are obsolete before they're ever created; 15 years of firewalls and so-called IDS/'IPS', and the resultant hundreds of millions of botted hosts, pretty much put paid to this canard.
Modern firewalls can be tied to intrusion prevention systems which will 'raise the shields' in the face of certain attacks.
These systems are worse than useless, they're actively harmful; they form DDoS chokepoints themselves, drastically increase the attack surface, and can also be gamed.
Many exploits require repeated probing and this provides a way to stop the attack before it is successful.
In the same way that powering off the server ensures perfect confidentiality and integrity of its data, applications, and services. ;>
Do you have any evidence to support this assertion?
Yes - many years of watching the biggest, baddest firewalls choke and die under trivial DDoS. Including the better part of a decade working for the largest firewall vendor in the world.
You've just asserted that all firewalls have a specific vulnerability.
No, I've asserted that all stateful firewalls created in the history of the world to date, commercial or open-source, are based upon a specific *fundamental architectural premise* which precludes their placement in front of servers. This isn't the same as saying they've a *vulnerability*. I'm saying they're unsuited to be placed in front of servers.
I don't see how you can assert they all have this vulnerability.
Again, I'm not asserting they've a vulnerability. I'm asserting that it doesn't make much sense to pound nails with a monkey-wrench. ;>
In any case, my experience tells me that a DDoS will be successful due to exhaustion of available network capacity before it exhausts the state table on the firewall.
My experiences defending against DDoS attacks from the 1990s to the present nanosecond indicate otherwise. ;>
Again, I don't believe such a global claim can be made given the wide variety of architectures used for firewalls.
Again, *all* stateful firewalls produced to date share a fundamental architectural premise which renders them unsuitable for placement in front of servers. In fact, one major firewall vendor recently implemented a 'stateful inspection bypass' feature as a direct result of this issue; apparently, one is supposed to purchase their $100KUSD 10gb/sec stateful firewall, wedge it into one's network (forcing artificial and undesirable traffic symmetry in order to do so) . . . and then *disable* the stateful inspection one supposedly purchased it to perform in the first place, heh. ;> ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
Excerpts from Dobbins, Roland's message of Tue Jan 05 20:23:28 -0800 2010: Roland, On many of the points you've made, I totally agree. Well-managed hardware routers that have support for ACLs in hardware are a great firewall for things that have a relatively small set of rules (e.g. "any:any -> server:80", "server:80 -> any:any"), and come with the added bonus of being able to both firewall and route traffic at whatever speed it forwards at. However, the "well managed" part seems to be a sticking point for most organizations I've seen. No doubt, shops that use this effectively have some sort of homebrew or commercial firewall management platform that let's you place policy in one place and make sure that it's pushed out properly.
Rate-limiting during a DDoS - i.e., an attack against state and *capacity* - is absolutely the *worst* thing one can possibly do, in almost all circumstances.
Why so? Because of something this does to the device doing the rate limiting (I assume an upstream router of some sort), or because it renders the attack successful?
No, I've asserted that all stateful firewalls created in the history of the world to date, commercial or open-source, are based upon a specific *fundamental architectural premise* which precludes their placement in front of servers.
I'm not so sure I follow you here. How does a "fundamental architectural premise" (I assume you mean keeping track of application-layer session state) *preclude* it from being placed in front of a server? Sure, it's a poor use of raw silicon and electrical power, but why does that rule out in advance placing it in front of a server? In theory though, someone could construct a massive state-tracking machine that can still keep track of stateful traffic, Mpps and above. Cheers, jonathan
On Jan 6, 2010, at 11:52 AM, Jonathan Lassoff wrote:
However, the "well managed" part seems to be a sticking point for most organizations I've seen. No doubt, shops that use this effectively have some sort of homebrew or commercial firewall management platform that let's you place policy in one place and make sure that it's pushed out properly.
Concur 100% - all the commercial systems which have purported to do this to date have been unusable, miserable failures. Folks tend to use homegrown systems, starting with something as basic as RCS. Tom Ptacek over at Matasano is working on a firewall/ACL rules management system which, given his track record of innovation, one hopes may buck this trend.
Why so? Because of something this does to the device doing the rate limiting (I assume an upstream router of some sort), or because it renders the attack successful?
The latter. DDoS attacks are attacks against capacity and/or state. Start reducing capacity, and you end up making it even easier for the attacker's programatically-generated attack traffic to 'crowd out' the legitimate user traffic. It's self-defeating. ;>
I'm not so sure I follow you here. How does a "fundamental architectural premise" (I assume you mean keeping track of application-layer session state) *preclude* it from being placed in front of a server? Sure, it's a poor use of raw silicon and electrical power, but why does that rule out in advance placing it in front of a server?
Because, by definition, all incoming packets to the server are unsolicited. Therefore, it's a waste of money, and also forms a DDoS chokepoint due to the non-infinite state-table which forms the basis for said stateful firewalling. It will fall over and die under any kind of serious attack.
In theory though, someone could construct a massive state-tracking machine that can still keep track of stateful traffic, Mpps and above.
See above; in front of the server, there's no state to track in the first place, heh. Fish, meet bicycle. ;> Additionally, it becomes an impractical physics problem (physical dimensions, logic density, power consumption, heat dissipation, et. al.) to construct a device which could even plausibly attempt this due to the extreme capacity/resource asymmetry which favors the attacker (i.e., botnets with thousands, tens of thousands, hundreds of thousands, and even millions of compromised hosts). ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
See above; in front of the server, there's no state to track in the first place, heh.
Fish, meet bicycle.
I think that is the part that some people aren't getting. You have a network just sitting there. A syn packet arrives for port 80 to an http server. You ARE going to allow it because that is what a web server does. Now if you have a firewall in front of the load balancer you have a three-way handshake that goes on with the firewall. Then another one between the firewall and the load balancer. And then possibly yet another one between the balancer and the server if you aren't using persistent connections in that part of the network. So now you get a DoS request that is as simple as "GET /index.html" or worse, some huge file, which you are also going to allow anyway because there is no way to tell a legitimate request from a flood of requests from a bot net or someone posted your link on Slashdot or whatever. So now your web server is flooded with "legitimate" requests that pass all of your policy but you are being overwhelmed by the sheer volume of them and they are originating from thousands of IP addresses from all around the world. They are all getting through your firewall. So now it is just a matter of which is the weakest link in the chain. If you have enough servers and bandwidth, the firewall is often that weakest link.
On Tue, Jan 5, 2010 at 11:41 PM, Dobbins, Roland <rdobbins@arbor.net> wrote:
On Jan 6, 2010, at 11:52 AM, Jonathan Lassoff wrote: DDoS attacks are attacks against capacity and/or state. Start reducing
DDoS, by its very nature is a type of attack that dances around common security measures like conventional firewalls, by its very nature. The possibility of someone dropping a nuke on your facility, shouldn't stop you from locking your doors at night. If necessary, use another arrangement to detect that threat, and protect firewall+servers from it. Having no 'firewall' type safeguard at all (stateless or otherwise) would appear pretty risky.
Because, by definition, all incoming packets to the server are unsolicited.
For UDP servers sure.. not for TCP.. the initial SYN is unsolicited, for inbound TCP connections. Once the server acknowledges the connection by invoking accept(), the rest of it the packets are solicited, the packets are either part of an active connection, or unwanted. There are other types of noise, 'stealth port scans', port 25, 445, 139, 22 probes, DNS cache poisoning attempts, and sequence prediction, TCP connection hijacking attempts, TCP Reset attempts, LAND attacks, that firewalls protect against (through port randomization), etc.
[...]and also forms a DDoS chokepoint due to the non-infinite state-table which >forms the basis for said stateful firewalling.
The number of states which can be required is not infinite, it's really only a question of how efficient your equipment can be, if you can find suitable choices for your stateful gear. Even if your firewall has a whole 1 gigabit connection, find some stateful firewall, that can efficiently track a maximum of at least 22,321,500 X2 states. (For 100 megabits, a much smaller table will do) Set the connection timeout to an idle time of 15 seconds, so a connection expires if a packet is not received in 15 seconds. The line rate of Gigabit Ethernet is < 1/((0.096+0.064+0.512)*10^-6) => 1488096 pps in each direction. So, even if every single packet is the SYN for a valid new connection, you will not exceed the maximum size of the table based on that rate. In the worst case, you receive 22321440 packets over 15 seconds. On the 16th second, 1488096 connections expire and 1488096 connections are added, since every packet was a new connection. Now relax your timeout reqs according to your preferences _real world_ estimates of maximum connection rate.... "Overflowing the state table" then becomes only a possible outcome that has some acceptable level of probability, assuming that your other protections have already failed... -- -J
On Wed, 2010-01-06 at 01:47 -0600, James Hess wrote:
On Tue, Jan 5, 2010 at 11:41 PM, Dobbins, Roland <rdobbins@arbor.net> wrote:
On Jan 6, 2010, at 11:52 AM, Jonathan Lassoff wrote: DDoS attacks are attacks against capacity and/or state. Start reducing
DDoS, by its very nature is a type of attack that dances around common security measures like conventional firewalls, by its very nature.
The possibility of someone dropping a nuke on your facility, shouldn't stop you from locking your doors at night. If necessary, use another arrangement to detect that threat, and protect firewall+servers from it.
DDoS mitigation gear tends to choke up in my experience. It's a really touchy subject.
Having no 'firewall' type safeguard at all (stateless or otherwise) would appear pretty risky.
Not really, because firewalls don't do anything useful. Stateless ACL policies do something useful, and usually that is handled in the router in a modern network. The other features of a firewall range from not so useful to actively harmful.
Because, by definition, all incoming packets to the server are unsolicited.
For UDP servers sure.. not for TCP.. the initial SYN is unsolicited, for inbound TCP connections. Once the server acknowledges the connection by invoking accept(), the rest of it the packets are solicited, the packets are either part of an active connection, or unwanted.
Wrong. You seem to assume that TCP stacks are well-behaved, or that botnets aren't just synthesizing junk. I've seen unsolicited ACK floods before. They are quite real. So, in fact, all incoming packets should be considered unsolicited until proven otherwise. It should be mentioned that DDoS mitigation gear in use on that network let those packets through without even alerting us about it. William
On Jan 6, 2010, at 3:03 PM, William Pitcock wrote:
So, in fact, all incoming packets should be considered unsolicited until proven otherwise.
Concur - it works this way, as well. At one extreme, completely pathological, at the other extreme, perfectly normal - just faux. ;>
It should be mentioned that DDoS mitigation gear in use on that network let those packets through without even alerting us about it.
This is where baselining and anomaly-detection can come into play, along with an understanding of valid/invalid states, if the system is designed correctly, heh. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
On Jan 6, 2010, at 2:47 PM, James Hess wrote:
"Overflowing the state table" then becomes only a possible outcome that has some acceptable level of probability, assuming that your other protections have already failed...
Wrong. The attacker just programmatically generates semantically-valid traffic which is indistinguishablle from real traffic, and crowds out the real traffic. All those fancy timers and counters and what-not don't matter. I've seen it done over and over again. Why some folks seem to think this is theoretical or that I somehow haven't thought of something they think will prove to be a magic solution is really beyond me, heh. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
On Jan 6, 2010, at 3:12 AM, Dobbins, Roland wrote:
Wrong. The attacker just programmatically generates semantically-valid traffic which is indistinguishablle from real traffic, and crowds out the real traffic.
All those fancy timers and counters and what-not don't matter.
I've seen it done over and over again. Why some folks seem to think this is theoretical or that I somehow haven't thought of something they think will prove to be a magic solution is really beyond me, heh.
The reality is they just have not been attacked yet, and hence have no experience in what to do about the problem... - Jared
On Jan 6, 2010, at 8:42 PM, Jared Mauch wrote:
The reality is they just have not been attacked yet, and hence have no experience in what to do about the problem...
And they've been bombarded with misinformation for years by 'security' vendors, wildly unrealistic certification training courses, and the 'compliance' mafia; you're right, of course. ;> ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
I will not argue the more complete statement about the architectural premise that statefull firewalls are being produced under. That would be fruitless and I would concede to Roland and his statements on that. It appears that the real argument is whether statefull inspection is useful, and whether the firewall causes other issues to the network design. If this is so, then I would say that it depends on the network and it's design as to whether a statefull firewall is useful. One could put ACLs in routers and switches, but when you break it down and turn off statefull inspection, that is what a firewall is. As always, you should always consider your network design before implementing any network appliance that will/may affect traffic. I don't think that discarding ideas like signature based analysis and DPI are wise. Depending on the network, the staff running the network, the users using the network, external exposure and many other metrics, I don't think that anyone should be making broad statements on equipment decisions. I'm glad that I can go to lists like NANOG with this type of question and not get the clue bat across the head. Like Roland, I've been doing this for over a decade as well, and I have seen some pretty strange things, even a statefull firewall in front of servers with IPS actually work. This thread is a tribute to different ideas and beliefs as well as experience on this topic. Please keep up the conversation and down the condescension and rhetoric. Thank you. - Brian
-----Original Message----- From: Dobbins, Roland [mailto:rdobbins@arbor.net] Sent: Wednesday, January 06, 2010 7:52 AM To: NANOG list Subject: Re: I don't need no stinking firewall!
On Jan 6, 2010, at 8:42 PM, Jared Mauch wrote:
The reality is they just have not been attacked yet, and hence have no experience in what to do about the problem...
And they've been bombarded with misinformation for years by 'security' vendors, wildly unrealistic certification training courses, and the 'compliance' mafia; you're right, of course.
;>
-----------------------------------------------------------------------
Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>
Injustice is relatively easy to bear; what stings is justice.
-- H.L. Mencken
CONFIDENTIALITY NOTICE: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, copying, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Thank you.
On Jan 6, 2010, at 6:51 AM, Brian Johnson wrote:
Like Roland, I've been doing this for over a decade as well, and I have seen some pretty strange things, even a statefull firewall in front of servers with IPS actually work.
What do you mean by "work"? If you mean "all three pieces ran for years without being seriously attacked", then that's really not the same thing as "continued to perform assigned duties effectively in the face of a determined DDoS". I'd venture to say the vast majority of network operators, including myself, have never faced a DoS worse than a miscreant kid with a cable modem. The few customers I've talked to who have been DDoS'd have all said the firewall died first. It's pretty simple. Of the devices on your network that have to keep state, a firewall has to maintain far more of them, since it's the aggregate of many down-stream hosts. The resources to maintain state are finite. At some point, those finite resources will be exceeded, and that will happen to a device holding the aggregate before any other device succumbs to the same problem. If the firewall goes down, that DoS's everything behind it. Is that really better than having only a portion of the down-stream hosts unavailable? IMO firewalls have been a crutch for far too long. They're an excuse for not having tight host-based security and (more importantly) good patch-management. There really isn't a network perimeter any more any way. If any of your hosts gets infected, they're going to attempt to infect their neighbors. Worms have been doing this since they were invented and a network firewall offers very little protection against it. Put another way: Is it clear that spending money on fancy network firewalls and IPS is more effective at mitigating risk than investing the same money in patch-management and host-hardening? I don't think so. I'd also like to add a +1 to the statement "firewalls break things in subtle and hard-to-debug ways". The longest support calls are always those trying to figure out how the customer's firewall is breaking things, and then how to prove this to their $management so they'll approve disabling the offending "feature". Speaking of which, there are about 700MB of PCAPs that I'm supposed to be looking at right now... -- bk
As long as you raise the level of CAIN (Confidentiality, Availability, Integrity, Non-Repudiation) that your mission requires and funding permits, you can do it anywhere you like, with whatever you like, and call it whatever you like. David On Wed, Jan 6, 2010 at 9:38 AM, Brian Keefer <chort@smtps.net> wrote:
On Jan 6, 2010, at 6:51 AM, Brian Johnson wrote:
Like Roland, I've been doing this for over a decade as well, and I have seen some pretty strange things, even a statefull firewall in front of servers with IPS actually work.
What do you mean by "work"? If you mean "all three pieces ran for years without being seriously attacked", then that's really not the same thing as "continued to perform assigned duties effectively in the face of a determined DDoS".
I'd venture to say the vast majority of network operators, including myself, have never faced a DoS worse than a miscreant kid with a cable modem. The few customers I've talked to who have been DDoS'd have all said the firewall died first.
It's pretty simple. Of the devices on your network that have to keep state, a firewall has to maintain far more of them, since it's the aggregate of many down-stream hosts. The resources to maintain state are finite. At some point, those finite resources will be exceeded, and that will happen to a device holding the aggregate before any other device succumbs to the same problem.
If the firewall goes down, that DoS's everything behind it. Is that really better than having only a portion of the down-stream hosts unavailable?
IMO firewalls have been a crutch for far too long. They're an excuse for not having tight host-based security and (more importantly) good patch-management. There really isn't a network perimeter any more any way. If any of your hosts gets infected, they're going to attempt to infect their neighbors. Worms have been doing this since they were invented and a network firewall offers very little protection against it.
Put another way: Is it clear that spending money on fancy network firewalls and IPS is more effective at mitigating risk than investing the same money in patch-management and host-hardening? I don't think so.
I'd also like to add a +1 to the statement "firewalls break things in subtle and hard-to-debug ways". The longest support calls are always those trying to figure out how the customer's firewall is breaking things, and then how to prove this to their $management so they'll approve disabling the offending "feature". Speaking of which, there are about 700MB of PCAPs that I'm supposed to be looking at right now...
-- bk
- Brian
-----Original Message----- From: Brian Keefer [mailto:chort@smtps.net] Sent: Wednesday, January 06, 2010 11:38 AM To: Brian Johnson Cc: NANOG list Subject: Re: I don't need no stinking firewall!
On Jan 6, 2010, at 6:51 AM, Brian Johnson wrote:
Like Roland, I've been doing this for over a decade as well, and I have seen some pretty strange things, even a statefull firewall in front of servers with IPS actually work.
What do you mean by "work"? If you mean "all three pieces ran for years without being seriously attacked", then that's really not the same thing as "continued to perform assigned duties effectively in the face of a determined DDoS".
By work I mean that it held-up under DDoS attack. The size of a DDoS attack is the question. If I have enough resources a person can DDoS an entire network, irrelevant of its equipment, that will make the network un-usable and unreachable. Statefull firewall or not. They simply need to fill up the inbound connection with traffic so that nothing else gets through. If your point is given unlimited inbound bandwidth that a stateful firewall will fail (not work correctly), I can say that about any piece of equipment. And even if it does fail, does it matter if your connection is full of useless traffic? DDoS attacks are not designed to compromise or gather data about networks. DDoS is the sledge hammer of the dubious to cause disruption. It doesn't matter what you put in there (Statefull Firewall, IDS, IPS, Router ACLS, et al...), if the connection is flooded, the network will be unreachable. Does it matter if the equipment can't handle it if no good traffic, that would need to be statefully inspected, is traversing the connection? - Brian CONFIDENTIALITY NOTICE: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, copying, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Thank you.
On Jan 6, 2010, at 11:29 AM, Brian Johnson wrote:
If your point is given unlimited inbound bandwidth that a stateful firewall will fail (not work correctly), I can say that about any piece of equipment. And even if it does fail, does it matter if your connection is full of useless traffic?
It's a lot easier to fill up a state table than to fill up a pipe, which I believe was Roland's point. It's quite possible to flood the state table on a device with a fraction of the pipe's capacity, in which case a stateful device will fall over where a stateless device would not have. This type of attack will definitely degrade the service it's aimed at, and probably degrade other services sharing the same pipe, but won't _necessarily_ kill them as is the case when a stateful gateway falls over. Typical scenario is $badguys DDoS one of your webservers. If the gateway is stateless, your webservers grind to a crawl, but your DNS, e-mail, VOIP, etc probably still function to a degree. Contrast that with site-wide outage if your gateway was stateful and crashed/rebooted/refused to pass traffic due to having the state table filled. You're not going to be able to stop $sophisticated_badguy from enumerating your services no matter how fancy your gear is. Could you detect a distributed portscan that looks at 5000 proto/IP/port combos per day, across your IP space, each probe coming from a different IP? I really doubt it. If you have services listening, someone is going to find them. IMO you're better off making sure only the services you intend to provide are listening, and that those services are hardened appropriately for public exposure. This topic has probably run it's course; everyone has different opinions and takes away different lessons from their experience. I think it's valuable to challenge the common assumptions (everyone knows you need a stateful firewall!) now and then to make sure they actually make sense. -- bk
-----Original Message----- From: Brian Keefer [mailto:chort@smtps.net] Sent: Wednesday, January 06, 2010 3:12 PM To: Brian Johnson Cc: NANOG list Subject: Re: I don't need no stinking firewall!
<SNIP>
It's quite possible to flood the state table on a device with a fraction of the pipe's capacity, in which case a stateful device will fall over where a stateless device would not have. This type of
will definitely degrade the service it's aimed at, and probably degrade other services sharing the same pipe, but won't _necessarily_ kill
attack them
as is the case when a stateful gateway falls over.
This would depend on the nature of the DDoS (how it was crafted). I would agree that a DDoS designed to fill a state table would do this. However, a DDoS can also be designed with large packets to fill up the capacity of the connection. In this case it would depend on the amount of bandwidth available. If total bandwidth / packet size > state table size (rough formula), then your assertion is valid. But if not, then it may be flawed. This goes back to the idea that the network design/goals/intent is an unknown quantity in every statement made on this matter.
Typical scenario is $badguys DDoS one of your webservers. If the gateway is stateless, your webservers grind to a crawl, but your DNS, e-mail, VOIP, etc probably still function to a degree. Contrast that with site-wide outage if your gateway was stateful and crashed/rebooted/refused to pass traffic due to having the state table filled.
I'll give this to you, but this still depends on the previous point. I know that under minimum packet sizes and using a pipe with > 200 Mbps capacity, I have seen a statefull firewall handle a large DDoS just fine. The problem was that the packets that needed to get in to the host couldn't, because the bandwidth was fully utilized. I also know that if the state table on said device were to be filled, the box wouldn't crash or reboot. It would just queue or drop the packets until a state slot became available. The scope of the state table was so large though that it would take a lot more traffic than the operation would ever purchase to fill it up. Remember YMMV. :)
You're not going to be able to stop $sophisticated_badguy from enumerating your services no matter how fancy your gear is. Could you detect a distributed portscan that looks at 5000 proto/IP/port combos per day, across your IP space, each probe coming from a different IP?
I
really doubt it. If you have services listening, someone is going to find them.
So the port scan would tell them about services I want there to be access to. I'm OK with that to a large extent. In practice if I do detect a port scan (usually from a single IP address), this would lead me to take prerequisite steps to block a future attack.
IMO you're better off making sure only the services you intend to provide are listening, and that those services are hardened appropriately for public exposure.
OK. This is obvious to anyone with experience in these things. But I also believe in a layered approach. It never hurts to add more layers to prevent human error or even internal breaches as the different systems are under the control of different equipment (servers, routers, switches, security devices). It's like two supports holding up something without knowing if the other one is doing its job. Both need to pull the full weight in case the other fails.
This topic has probably run it's course; everyone has different opinions and takes away different lessons from their experience. I think it's valuable to challenge the common assumptions (everyone
knows
you need a stateful firewall!) now and then to make sure they actually make sense.
I agree. I just don't want anyone to go throw out their statefull firewalls because you, I or somebody else maybe would do it differently. Or because we have a different type of network or size of network or even goals of our network. So here's the brunt of it... Understand what statefull firewalls are and what they do, every network is different and YMMV. Now, let's go get a beer. :) - Brian CONFIDENTIALITY NOTICE: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, copying, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Thank you.
On Jan 6, 2010, at 3:56 PM, Brian Johnson wrote:
-----Original Message----- From: Brian Keefer [mailto:chort@smtps.net] Sent: Wednesday, January 06, 2010 3:12 PM To: Brian Johnson Cc: NANOG list Subject: Re: I don't need no stinking firewall!
<SNIP>
<SNIP>
IMO you're better off making sure only the services you intend to provide are listening, and that those services are hardened appropriately for public exposure.
OK. This is obvious to anyone with experience in these things. But I also believe in a layered approach. It never hurts to add more layers to prevent human error or even internal breaches as the different systems are under the control of different equipment (servers, routers, switches, security devices). It's like two supports holding up something without knowing if the other one is doing its job. Both need to pull the full weight in case the other fails.
I disagree. "Never" is pretty absolute. If that were true there would be no limit to the number of layers. Realistically I have experienced the harm from having firewalls in the network path. I have witnessed too many video sessions that either couldn't be started or had the sessions dropped prematurely because of firewalls. When the worms were infecting machines a couple of years ago our network was robust and stable and I identified and blocked infected machines quickly. Other universities shut down their residence halls or large portions of their network because their firewalls rolled over and died otherwise from all of the scanning from inside their network. I have talked to universities who consider the firewall the canary of the network world, its the first box in the network to cease functioning when there is a problem. Others have already mentioned the troubleshooting nightmares that firewalls generate, I would consider that a harm also. --- Bruce Curtis bruce.curtis@ndsu.edu Certified NetAnalyst II 701-231-8527 North Dakota State University
-----Original Message----- From: Bruce Curtis [mailto:bruce.curtis@ndsu.edu] Sent: Tuesday, January 12, 2010 5:14 PM To: NANOG list Subject: Re: I don't need no stinking firewall!
<SNIP>
IMO you're better off making sure only the services you intend to provide are listening, and that those services are hardened appropriately for public exposure.
OK. This is obvious to anyone with experience in these things. But I also believe in a layered approach. It never hurts to add more layers to prevent human error or even internal breaches as the different systems are under the control of different equipment (servers, routers, switches, security devices). It's like two supports holding up something without knowing if the other one is doing its job. Both need to pull the full weight in case the other fails.
I disagree. "Never" is pretty absolute. If that were true there would be no limit to the number of layers.
I'm with you, but you get my sentiment without being too literal. :)
Realistically I have experienced the harm from having firewalls in the network path.
I've experienced harm from routers in the network path. If you use the tool correctly and with full knowledge of its limitations, then you will be able to avoid "harm" and add functionality/security/value... whatever the goal is.
I have witnessed too many video sessions that either couldn't be started or had the sessions dropped prematurely because of firewalls.
So putting a firewall that can't handle your traffic in your network path sounds like a bad idea FOR YOU. :)
When the worms were infecting machines a couple of years ago our network was robust and stable and I identified and blocked infected machines quickly. Other universities shut down their residence halls or large portions of their network because their firewalls rolled over and died otherwise from all of the scanning from inside their network.
I remember hearing about this type of thing. I'm sorry for this learning lesson, but that doesn't mean that firewalls are "bad" or that stateful inspection is "bad". It means that it was a bad choice for your environment.
I have talked to universities who consider the firewall the canary of the network world, its the first box in the network to cease functioning when there is a problem.
I think this type of assertion is just folly. I would say that some universities (full of all those really smart people ;) should be able to discern that a monkey wrench was being used to do the job of a hammer, or vice versa. The problem was not the tool, but the person who used the wrong tool for the job at hand.
Others have already mentioned the troubleshooting nightmares that firewalls generate, I would consider that a harm also.
I've had one of those "troubleshooting nightmares" before. It was due to MY IGNORANCE of what I was doing. The firewall is not causing the nightmare. Ignorance is. My last statement on this thread is that if you use a tool in the wrong way, you will either break the tool, or the item you are using it on. If you don't know how to use a tool, learn before you try. If you try first, you will learn later (Here comes that nightmare) how the tool does/doesn't work. Specific examples of failure are not failures of the device, but failures of the implementer(s) to correctly use the tool with the obvious exception of vendors not being truthful about the tools capabilities. Please no more examples of specific failures of firewalls. We all know that they were designed by Satan himself to destroy our networks and bring about the Antichrist. ;) - Brian CONFIDENTIALITY NOTICE: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, copying, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Thank you.
Lots of interesting technical information in this thread. Mixed with a healthy dose of religion/politics :-) I suspect that most people are going to keep doing what they are doing. In our environment, at the transport level, we have moved from stateful towards stateless, as it has proved to be operationally simpler and more resilient. At the same time some of our application people have seen the need to put their servers behind stateful "Layer 7" firewalls (I say why stop at Layer 7?) Here is a thought experiment: Replace all the routers on the Internet with stateful firewalls. What happens? Replace all the stateful firewalls on the Internet with stateless packet filters. What is the result? -- Tim:> Sent from Brooklyn, NY, United States
-----Original Message----- From: nanog-bounces@nanog.org [mailto:nanog-bounces@nanog.org] On Behalf Of Robert Brockway Sent: Tuesday, January 05, 2010 1:25 PM To: NANOG list
On Tue, 5 Jan 2010, Dobbins, Roland wrote:
Putting firewalls in front of servers is a Really Bad Idea - besides the
Hi Roland. I disagree strongly with this position.
fact that the stateful inspection premise doesn't apply (see above),
The problem is that your premise is wrong. Stateful firewalls (hereafter just called firewalls) offer several advantages. This list is not necessarily exhaustive.
(1) Security in depth. In an ideal world every packet arriving at a server would be for a port that is intended to be open and listening. Unfortunately ports can be opened unintentionally on servers in several ways: sysadmin error, package management systems pulling in an extra package which starts a service, etc. By having a firewall in front of the server we gain security in depth against errors like this.
Most large operations don't have individual servers accepting incoming traffic. They generally have a farm of servers behind a load balancer. The load balancer itself provides this layer of security. You configure a service on a specific address/port/protocol on the load balancer and bind that to a group of servers on the other side of the balancer. It doesn't matter what you have open on the server, the server is not directly accessible from the Internet. If you don't have the service configured on the load balancer, traffic simply gets dropped on the floor at the front door. Most load balancers these days are even configurable to handle (or not) specifically formatted requests so you can craft ACLs based on the actual requests if you need to. In other words, for all practical purposes these days, the load balancer IS a stateful firewalling proxy. Having another one in front of the load balancer that is simply configured to pass your production traffic to the load balancer is a waste of money and resources in addition to a potential bottleneck. This is particularly true if you are a client/server operation where you are handling traffic on custom ports using custom protocols that a firewall isn't going to know how to inspect anyway.
(2) Centralised management of access controls. Many services should only be open to a certain set of source addresses. While this could be managed on each server we may find that some applications don't support this well, and management of access controls is then spread across any number of management interfaces. Using a firewall for network access control reduces the management overhead and chances of error. Even if network access control is managed on the server, doing it on the firewall offers additional security in depth.
(3) Outbound access controls. In many cases we want to stop certain types of outbound traffic. This may contain an intrusion and prevent attacks against other hosts owned by the organisation or other organisations. Trying to do outbound access control on the server won't help as if
In the case where a service is open to only a certain set of source addresses such as when providing a specific service to a business partner, the ACL can just as easily be configured on the routers at the ingress to your network. Any traffic that you KNOW you are going to drop should be dropped as soon as possible so you don't waste resources forwarding the traffic to the firewall. Ingress ACLs are the best place for that, in my opinion. A modern layer2/3 switch can drop traffic in hardware without a lot of resource consumption. the
server is compromised the attacker can likely get around it.
Yes. Outbound access control IS a valid location for a firewall but that is a separate path from your production service traffic, or at least it probably should be. A modern load balancer is capable of source NATing the traffic so the connections to your server appear to come from the load balancer itself and they can insert a header in the transaction that includes the original connecting IP address of the client so the server still has access to that information if it needs it. When the server needs to initiate an outbound connection, it has a default gateway to a path that eventually leads to a firewall on the way out.
(4) Rate limiting. The ability to rate limit incoming and outgoing data can prevent certain sorts of DoSes.
This can be done on both the load balancer or the routers depending on the kind of traffic you want to limit.
(5) Signature based blocking. Modern firewalls can be tied to intrusion prevention systems which will 'raise the shields' in the face of certain attacks. Many exploits require repeated probing and this provides a way to stop the attack before it is successful.
Modern load balancers are capable of this. You can program it for a particular query string, for example, and if you see it, you can ignore it, redirect it, log it, whatever. The line is also blurring in many routers these days between what is a firewall function and what is a router function. IOS, for example, has a large array of firewalling features available. Many of the problems associated with firewalls in your production path aren't going to show themselves if you have less than say a million or two users. When you get to tens of millions of users with literally billions of transactions, things get a little different. I have run a lot of firewalls out of resources over the years ... Netscreens, PIX/ASA, even special purpose units hand-built using OS kernel firewalling. And that is not even in the face of a DoS, that is with normal production traffic. In practically every case the response is the same ... start disabling features on the firewall for the inbound production traffic flow because you are going to allow that anyway. Put any "blocks" on the provider edge ingress ports and drop the traffic before it even gets to the firewall or load balancer. At that point the majority of the traffic through the firewall becomes "allow <protocol> any <service IP> eq <service port>" for the address/port/protocol of your service VIPs so if you are allowing "any", why are you paying for an extra colo heater? The load balancers these days have methods just as effective as firewalls for dealing with things like syn attacks, they can rate limit requests, they can buffer them, they can drop them, routers can limit icmp. There are a lot of tools in the drawer. Yes, you have to take some of the things that were done in one spot and do them in different locations now, but the results are an amazing increase in service capacity per dollar spent on infrastructure.
On Jan 6, 2010, at 11:43 AM, George Bonser wrote:
Yes, you have to take some of the things that were done in one spot and do them in different locations now, but the results are an amazing increase in service capacity per dollar spent on infrastructure.
I strongly agree with the majority of your comments, with the caveat that I've seen many, many load-balancers fall over due to state-exhaustion, too; load-balancers need northbound protection from DDoS (S/RTBH, flow-spec, IDMS, et. al.), as well. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
-----Original Message----- From: Dobbins, Roland [mailto:rdobbins@arbor.net] Sent: Tuesday, January 05, 2010 8:53 PM To: NANOG list Subject: Re: I don't need no stinking firewall!
On Jan 6, 2010, at 11:43 AM, George Bonser wrote:
Yes, you have to take some of the things that were done in one spot and do them in different locations now, but the results are an amazing increase in service capacity per dollar spent on infrastructure.
I strongly agree with the majority of your comments, with the caveat that I've seen many, many load-balancers fall over due to state- exhaustion, too; load-balancers need northbound protection from DDoS (S/RTBH, flow-spec, IDMS, et. al.), as well.
Yes, I have seen load balancers fall over, too. I have some interesting stories of how those problems have been solved. Sometimes it relies on using a feature of one vendor to leverage a feature of another vendor. But I generally agree with you. There is a lot that can be done ahead of the load balancers.
What is nice about load balancers is that if you design your solution correctly, you can scale them in a very nice way. Things like direct server return, where only the requests hit the load balancer, but the replies (which are usually larger) just route back directly to the client can free up resources on the load balancer to handle more complex policies. This also reduces the imposed symmetry for routing that firewalls bring to the table. Further on, if you want to really protect against a real DDoS you would most likely would have to look at a really distributed solution, where the different geographical load balancing solutions come into play. Arie On Wed, Jan 6, 2010 at 7:03 AM, George Bonser <gbonser@seven.com> wrote:
-----Original Message----- From: Dobbins, Roland [mailto:rdobbins@arbor.net] Sent: Tuesday, January 05, 2010 8:53 PM To: NANOG list Subject: Re: I don't need no stinking firewall!
On Jan 6, 2010, at 11:43 AM, George Bonser wrote:
Yes, you have to take some of the things that were done in one spot and do them in different locations now, but the results are an amazing increase in service capacity per dollar spent on infrastructure.
I strongly agree with the majority of your comments, with the caveat that I've seen many, many load-balancers fall over due to state- exhaustion, too; load-balancers need northbound protection from DDoS (S/RTBH, flow-spec, IDMS, et. al.), as well.
Yes, I have seen load balancers fall over, too. I have some interesting stories of how those problems have been solved. Sometimes it relies on using a feature of one vendor to leverage a feature of another vendor. But I generally agree with you. There is a lot that can be done ahead of the load balancers.
On Jan 8, 2010, at 3:21 PM, Arie Vayner wrote:
Further on, if you want to really protect against a real DDoS you would most likely would have to look at a really distributed solution, where the different geographical load balancing solutions come into play.
GSLB or whatever we want to call it is extremely useful from a general availability standpoint; however, the attackers can always scale up and really distribute their already-DDoS even further (they learned about routeservers and DNS tinkering years ago). Architecture, visibility, and control are key, as are vendor/customer/peer/upstream/opsec community relationships. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
All, This thread certainly has been educational, and has changed my perception of what an appropriate outward facing architecture should be. But seldom do I have the luxury of designing this from scratch, and also the networks I administer are "small business's". My question is at what size connection does a state table become vulnerable, are we talking 1mb dsl's with a soho firewall? Or as I suspect we are talking about a larger scale? I know there are variables, I am just looking for a "rule of thumb". I would not want to recommend a change if it is not warranted. But when fatter and fatter pipes become available at what point would a change be warranted. Thanks Bill Kruchas Dobbins, Roland wrote:
On Jan 8, 2010, at 3:21 PM, Arie Vayner wrote:
Further on, if you want to really protect against a real DDoS you would most likely would have to look at a really distributed solution, where the different geographical load balancing solutions come into play.
GSLB or whatever we want to call it is extremely useful from a general availability standpoint; however, the attackers can always scale up and really distribute their already-DDoS even further (they learned about routeservers and DNS tinkering years ago).
Architecture, visibility, and control are key, as are vendor/customer/peer/upstream/opsec community relationships.
----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>
Injustice is relatively easy to bear; what stings is justice.
-- H.L. Mencken
On Jan 8, 2010, at 8:22 PM, bill from home wrote:
Or as I suspect we are talking about a larger scale?
Even an attacker with relatively moderate resources can succeed simply by creating enough well-formed, programatically-generated traffic to 'crowd out' legitimate traffic. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
Roland, I understand, but at the site we are protecting, at what point is the bottleneck the connection speed, and at what point is the state table the bottle neck. It saves me the following uncomfortable conversation. ME> Mr customer, remember that firewall you bought a couple of years ago for $$$$. Customer> Yes... ME> We might better throw it out. And then you can pay me to harden your hosts. Or I could just re cable, and leave it turned on, they would never know (just kidding). And maybe there is no way to tell, but I feel I need to ask the question. Thanks Bill Kruchas Dobbins, Roland wrote:
On Jan 8, 2010, at 8:22 PM, bill from home wrote:
Or as I suspect we are talking about a larger scale?
Even an attacker with relatively moderate resources can succeed simply by creating enough well-formed, programatically-generated traffic to 'crowd out' legitimate traffic.
----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>
Injustice is relatively easy to bear; what stings is justice.
-- H.L. Mencken
On Jan 8, 2010, at 9:02 PM, bill from home wrote:
And maybe there is no way to tell, but I feel I need to ask the question.
Situationally-dependent; the only way to really tell, not just theorize, is to test the firewall to destruction during a maintenance window (or one like it, in the lab). ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
Dobbins, Roland wrote:
On Jan 8, 2010, at 9:02 PM, bill from home wrote:
And maybe there is no way to tell, but I feel I need to ask the question.
Situationally-dependent; the only way to really tell, not just theorize, is to test the firewall to destruction during a maintenance window (or one like it, in the lab).
see my post in the subject, a reasonably complete performance report for the device is a useful place to start. if you know what the maximum session rate and state table size for the device are, you have a pretty good idea at what rate of state instantiation it will break. rather frequently it's more than two orders of magnitude lower than the peak forwarding rate.
----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>
Injustice is relatively easy to bear; what stings is justice.
-- H.L. Mencken
On Jan 9, 2010, at 7:52 AM, Joel Jaeggli wrote:
see my post in the subject, a reasonably complete performance report for the device is a useful place to start.
The problem is that one can't trust the stated vendor performance figures, which is why actual testing is required. I've seen instances in which actual performance is 5% or less of vendor assertions (i.e., vendor constructed a highly artificial scenario in order to be able to make a specific claim which doesn't hold up in real life). Also note that most vendors don't perform negative testing, astonishing though that may seem. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
Dobbins, Roland wrote:
On Jan 9, 2010, at 7:52 AM, Joel Jaeggli wrote:
see my post in the subject, a reasonably complete performance report for the device is a useful place to start.
The problem is that one can't trust the stated vendor performance figures, which is why actual testing is required. I've seen instances in which actual performance is 5% or less of vendor assertions (i.e., vendor constructed a highly artificial scenario in order to be able to make a specific claim which doesn't hold up in real life).
I'll go out on a limb and just quote myself since you didn't fully.
if you know what the maximum session rate and state table size for the device are, you have a pretty good idea at what rate of state instantiation it will break. rather frequently it's more than two orders of magnitude lower than the peak forwarding rate.
two orders of magnitude lower is 1% of forwarding rate. It could be lower but it's probably not 3 orders of magnitude. rfc 3511 testing won't tell you that much that's useful in the real world. but it will tell you how many concurrent sessions you can establish (which is almost purely a function of the amount of ram for that data strcture) through the DUT and how quickly you can establish them (which may vary with your rule base but will almost certainly never get better). vendors are mostly honest about that becuase you can trivially replicate that test even with fairly low end equipment on all but the biggest stateful devices.
Also note that most vendors don't perform negative testing, astonishing though that may seem.
----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>
Injustice is relatively easy to bear; what stings is justice.
-- H.L. Mencken
I think we are over looking what an enterprise class firewall accomplishes from a security perspective and what a firewalls function is in the overall security posture of a network. First, statefull inspection by itself is not the only security feature of a firewall, it is one security feature of a firewall. Couple that with other security features and now you have a security device. Other security features in an Enterprise Class firewall; -Inside source based NAT, reinforces secure traffic flow by allowing outside to inside flows based on configured translations and allowed security policies -TCP sequence number randomization (to prevent TCP seq number guessing) -Intrusion Detection and Prevention (subset of most common signatures) recognize scanning attempts and mitigate recognize common attacks and mitigate -Deep packet inspection (application aware inspection for common network services) - Policy based tools for custom traffic classification and filtering -Layer 3 segmentation (creates inspection and enforcement points) -Full/Partial Proxy services with authentication - Alarm/Logging capabilities providing info on potential attacks -etc ...... Statefull inspection further enhances the security capabilities of a firewall. Another point is that a firewall by itself is not security, "Defense in Depth" means that every node on the network has it's role in the overall security architecture, no one or two devices is security in itself. You may choose not to use a firewall or implement a sound security posture utilizing the "Defense in Depth" philosophy, however you chances of being compromised are dramatically increased. So, I would be more interested in implementing a sound security architecture than whether or not a firewall provides security to my networks. my two cents, mike On Fri, Jan 8, 2010 at 11:18 PM, Joel Jaeggli <joelja@bogus.com> wrote:
Dobbins, Roland wrote:
On Jan 9, 2010, at 7:52 AM, Joel Jaeggli wrote:
see my post in the subject, a reasonably complete performance report for the device is a useful place to start.
The problem is that one can't trust the stated vendor performance figures, which is why actual testing is required. I've seen instances in which actual performance is 5% or less of vendor assertions (i.e., vendor constructed a highly artificial scenario in order to be able to make a specific claim which doesn't hold up in real life).
I'll go out on a limb and just quote myself since you didn't fully.
if you know what the maximum session rate and state table size for the device are, you have a pretty good idea at what rate of state instantiation it will break. rather frequently it's more than two orders of magnitude lower than the peak forwarding rate.
two orders of magnitude lower is 1% of forwarding rate. It could be lower but it's probably not 3 orders of magnitude.
rfc 3511 testing won't tell you that much that's useful in the real world. but it will tell you how many concurrent sessions you can establish (which is almost purely a function of the amount of ram for that data strcture) through the DUT and how quickly you can establish them (which may vary with your rule base but will almost certainly never get better). vendors are mostly honest about that becuase you can trivially replicate that test even with fairly low end equipment on all but the biggest stateful devices.
Also note that most vendors don't perform negative testing, astonishing though that may seem.
----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>
Injustice is relatively easy to bear; what stings is justice.
-- H.L. Mencken
On Jan 10, 2010, at 5:51 AM, harbor235 wrote:
Other security features in an Enterprise Class firewall; -Inside source based NAT, reinforces secure traffic flow by allowing outside to inside flows based on configured translations and allowed security policies
Terrible from an availability perspective, troubleshooting perspective, too. Just dumb, dumb, dumb - NATted servers fall over at the drop of a hat due to the NAT device choking.
-TCP sequence number randomization (to prevent TCP seq number guessing)
Server IP stack does this itself just fine.
-Intrusion Detection and Prevention (subset of most common signatures) recognize scanning attempts and mitigate recognize common attacks and mitigate
Snake-oil.
-Deep packet inspection (application aware inspection for common network services)
Terrible from an availability perspective, snake-oil.
- Policy based tools for custom traffic classification and filtering
Can be done statelessly, no firewall required.
-Layer 3 segmentation (creates inspection and enforcement points)
Doesn't require a firewall.
-Full/Partial Proxy services with authentication
If needed, can be better handled by transparent reverse-proxy farms; auth handled on the servers themselves.
- Alarm/Logging capabilities providing info on potential attacks -etc ......
NetFlow from the network infrastructure, the OS/apps/services on the server itself do this, etc.
Statefull inspection further enhances the security capabilities of a firewall.
No, it doesn't, not in front of servers where there's no state to inspect, in the first place, given that every incoming packet is unsolicited.
You may choose not to use a firewall or implement a sound security posture utilizing the "Defense in Depth" philosophy, however you chances of being compromised are dramatically increased.
Choosing not to make the mistake of putting a useless, counterproductive firewall in front of a server doesn't mean one isn't employing a sound, multi-faceted opsec strategy. I know that all the firewall propaganda denoted above is repeated endlessly, ad nauseam, in the Confused Information Systems Security Professional self-study comic books, but I've found that a bit of real-world operational experience serves as a wonderful antidote, heh. ;> ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
Other security features in an Enterprise Class firewall; -Inside source based NAT, reinforces secure traffic flow by allowing outside to inside flows based on configured translations and allowed security policies
Terrible from an availability perspective, troubleshooting perspective, too. Just dumb, dumb, dumb - NATed servers fall over at the drop of a hat due to the NAT device choking.
How is that possible with inside source NATing? You must mean a misconfigured outside source NATing
-TCP sequence number randomization (to prevent TCP seq number guessing)
Server IP stack does this itself just fine.
What server randomizes TCP sequence numbers? servers randomize initial >>> sequence numbers!, regardless, the FW will accept and randomize again making sure the endpoints get the correct TCP seq numbers, again more secure
-Intrusion Detection and Prevention (subset of most common signatures) recognize scanning attempts and mitigate recognize common attacks and mitigate
Snake-oil.
Preventing attacks on internal networks or servers, snake oil, LOL FWs typically offer a subset of potential IDS signatures, dedicated appliances or systems offer a higher level of prevention
-Deep packet inspection (application aware inspection for common network services)
Terrible from an availability perspective, snake-oil.
Inspecting application header and data, it will identify/prevent some application >>>attacks? how does that reduce availability?
- Policy based tools for custom traffic classification and filtering
Can be done statelessly, no firewall required.
True, never said this was done statefully, what device are you using to
perform >>>this function?
no firewall required, but part of distributed defense in depth strategy and can be >>>done by a firewall , again a secure architecture is the goal not just a firewall
-Layer 3 segmentation (creates inspection and enforcement points)
Doesn't require a firewall.
No, but segmentation and multiple security enforcements points are essential for >>> a secure architecture,
-Full/Partial Proxy services with authentication
If needed, can be better handled by transparent reverse-proxy farms; auth handled on the servers themselves.
The servers are doing everything in your model, must be quite some servers, are >>>we talking firewalls in general of are we talking datacenter, all companies do not >>>have access to reverse-proxy farms
- Alarm/Logging capabilities providing info on potential attacks -etc ......
NetFlow from the network infrastructure, the OS/apps/services on the server itself do this, etc.
not the same thing , you will need to analyze the data, Netflow does not
perform >>> data analysis, you will need to develop/buy something else for that
Statefull inspection further enhances the security capabilities of a
firewall.
No, it doesn't, not in front of servers where there's no state to inspect, in the first place, given that every incoming packet is unsolicited.
every packet is not unsolicited, webserver to database request ? DB synch >>>between datacenters, administration, remote services, etc ,,,
there is no state for >>>the services you are serving, true, but what about the rest of the network services >>>potentially available and their exploits?
You may choose not to use a firewall or implement a sound security posture utilizing the "Defense in Depth" philosophy, however you chances of being compromised are dramatically increased.
Choosing not to make the mistake of putting a useless, counterproductive firewall in front of a server doesn't mean one isn't employing a sound, multi-faceted opsec strategy.
didn't say it did, I stated several times that a secure architecture should be the >>>goal not just adding a FW, did you fail to read or respond to that part?
I know that all the firewall propaganda denoted above is repeated endlessly, ad nauseam, in the Confused Information Systems Security Professional self-study comic books, but I've found that a bit of real-world operational experience serves as a wonderful antidote, heh.
Again, a firewall has it's place just like any other device in the network, defense in >>> depth is a prudent philosophy to reduce the chances of compromise, it does not >>>eliminate it nor does any architecture you can
think of, period
mike
----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>
Injustice is relatively easy to bear; what stings is justice.
-- H.L. Mencken
On Jan 10, 2010, at 1:22 PM, harbor235 wrote:
Again, a firewall has it's place just like any other device in the network, defense in >>> depth is a prudent philosophy to reduce the chances of compromise, it does not >>>eliminate it nor does any architecture you can think of, period
What a ridiculous statement - of course it does. *The place of the stateful firewall is in front of clients, not servers*. I'm not going to continue the unequal contest of pitting real-world operational experience against Confused Information Systems Security Professional brainwashing. One can spout all the buzzwords and catchphrases one wishes, but at the end of the day, it's all dead wrong - and anyone naive enough to fall for it is setting himself up for a world of hurt. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
On Jan 10, 2010, at 1:32 PM, Dobbins, Roland wrote:
One can spout all the buzzwords and catchphrases one wishes, but at the end of the day, it's all dead wrong - and anyone naive enough to fall for it is setting himself up for a world of hurt.
mike <harbor235@gmail.com>, You deserve a better response than this; I'm sorry for snapping, it just gets a bit wearying going over the same points over and over again. My apologies. Every point you raised has been discussed in detail previously on the two threads encompassing this topic (and previously on cisco-nsp, as well). If you go read through the threads, you'll find the answers to the questions you've asked. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
On 1/9/10 10:32 PM, "Dobbins, Roland" <rdobbins@arbor.net> wrote:
On Jan 10, 2010, at 1:22 PM, harbor235 wrote:
Again, a firewall has it's place just like any other device in the network, defense in >>> depth is a prudent philosophy to reduce the chances of compromise, it does not >>>eliminate it nor does any architecture you can think of, period
What a ridiculous statement - of course it does.
*The place of the stateful firewall is in front of clients, not servers*.
I'm not going to continue the unequal contest of pitting real-world operational experience against Confused Information Systems Security Professional brainwashing. One can spout all the buzzwords and catchphrases one wishes, but at the end of the day, it's all dead wrong - and anyone naive enough to fall for it is setting himself up for a world of hurt.
I certainly understand and agree with your position, in most cases, but there are some instances when a firewall serves an excellent purpose. As an example, we manage hundreds of heterogeneous servers where customers also have administrative access to the devices. As such, we can never be sure they haven't changed something that can negatively impact the security of the server or servers. However, since the firewall is a magic box they don't want anything to do with it. This means that I can keep a server fairly secure from extraneous cruft and have a demarcation point into and out of the customer's environment that I control. I understand this does nothing for SQL injection, XSS, and other application-layer mischief, but it does wonders for keeping all the other stuff blocked, even when an customer "admin" says "why do I need Windows Firewall?" I wish I had a perfect world where I had a homogenous server environment that I controlled all the way through the stack with only one Management Layer to deal with. But, I'm glad I don't because these customers pay my salary. Regards, Mike
I certainly understand and agree with your position, in most cases, but there are some instances when a firewall serves an excellent purpose. As an example, we manage hundreds of heterogeneous servers where customers also have administrative access to the devices. As such, we can never be sure they haven't changed something that can negatively impact the security of the server or servers.
Firewalls do have a purpose and I don't think anyone disputes that. I certainly have firewalls in my network. What I believe the argument here is about is which kinds of traffic does one use a firewall for and which kinds of traffic are best left to other devices to handle access control/management. And I don't believe anyone is necessarily advocating exposing individual servers directly to the internet either. There are other devices that can handle isolation of the servers and protect them against such things as syn floods.
On Jan 10, 2010, at 5:40 PM, George Bonser wrote:
And I don't believe anyone is necessarily advocating exposing individual servers directly to the internet either.
Actually, some of us are.
There are other devices that can handle isolation of the servers and protect them against such things as syn floods.
What is the point of that when the servers can do it themselves? -- bk
And I don't believe anyone is necessarily advocating exposing individual servers directly to the internet either.
Actually, some of us are.
That can be difficult to do when you have maybe 300 or 400 servers that handle one service. Let's say you have a site called www.foobar.com and you have several hundred servers on the front end that handle that domain. You aren't going to put several hundred A records in DNS; at least I hope you aren't. One would probably have a load balancer of some sort in front of those machines. That is the device that would be fielding any DoS.
There are other devices that can handle isolation of the servers and protect them against such things as syn floods.
What is the point of that when the servers can do it themselves?
I have a feeling you are talking about relatively small amounts of traffic.
On Jan 11, 2010, at 12:56 PM, George Bonser wrote:
One would probably have a load balancer of some sort in front of those machines. That is the device that would be fielding any DoS.
Yes, and as you've noted previously, it should be protected via stateless ACLs in hardware capable of handling mpps, S/RTBH, flow-spec, IDMS, whatever. And of course the load-balancer should also be fronted by a reverse-proxy cache farm, if the servers in question are Web servers.
I have a feeling you are talking about relatively small amounts of traffic.
I believe that these comments were more along the lines of 'servers can better handle this that stateful firewalls', not ruling out the use of load-balancers, reverse-proxy caches, etc. as appropriate. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
I believe that these comments were more along the lines of 'servers can better handle this that stateful firewalls', not ruling out the use of load-balancers, reverse-proxy caches, etc. as appropriate.
-----------------------------------------------------------------------
Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>
I read it as the poster linking a service to a server. That would imply to me a small amount of traffic. The notion that a service is somehow related to a server is valid for rates of traffic that a single server can handle but things change once you scale beyond that level. A service provided to the internet does not map to a server except in small scale operation.
On Jan 10, 2010, at 1:32 AM, Dobbins, Roland wrote:
On Jan 10, 2010, at 1:22 PM, harbor235 wrote:
Again, a firewall has it's place just like any other device in the network, defense in >>> depth is a prudent philosophy to reduce the chances of compromise, it does not >>>eliminate it nor does any architecture you can think of, period
Bah, I was trying not to get sucked into the roaring vortex of this thread, but I think that folks are ignoring one of the primary benefits of firewalls: Quite simply, its this: I can now place a checkbox in the "Is there a firewall?" column of the <insert random acronym here> audit. While it may be fun to rail against the stupidity, after the Nth time that you have had the "This is in no way going to help improves security and will actually decrease it" argument, you realize that, if you want to get real work done, you need to choose your battles. In may cases the auditor knows that the firewall may not make thing better, and may make them worse, but he has a set of guidelines that the contracting company he is working for dictates, and he needs to see the widget to sign on the dotted line. I have had auditors cheerfully point out that the way that their specific requirement is worded, a commodity CPE device plugged into port somewhere will fully satisfy their requirements and did I know that BestBuy has them on sale this week? W
What a ridiculous statement - of course it does.
*The place of the stateful firewall is in front of clients, not servers*.
I'm not going to continue the unequal contest of pitting real-world operational experience against Confused Information Systems Security Professional brainwashing. One can spout all the buzzwords and catchphrases one wishes, but at the end of the day, it's all dead wrong - and anyone naive enough to fall for it is setting himself up for a world of hurt.
----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>
Injustice is relatively easy to bear; what stings is justice.
-- H.L. Mencken
On Jan 14, 2010, at 12:37 PM, Warren Kumari wrote:
I can now place a checkbox in the "Is there a firewall?" column of the <insert random acronym here> audit.
mod_security is your friend. ;> ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
On Wed, Jan 13, 2010 at 9:37 PM, Warren Kumari <warren@kumari.net> wrote:
I can now place a checkbox in the "Is there a firewall?" column of the <insert random acronym here> audit.
In most cases, you can check the same box if you use an appropriately designed stateless firewall instead of an inappropriate stateful firewall. (Not always, of course.) And it will keep out some fraction of noise and anklebiters, and optionally give you a place to hang limited intrusion detection, without providing an easy path for attackers to crash your connection. -- ---- Thanks; Bill Note that this isn't my regular email account - It's still experimental so far. And Google probably logs and indexes everything you send it.
Dobbins, Roland wrote:
On Jan 10, 2010, at 1:22 PM, harbor235 wrote:
Again, a firewall has it's place just like any other device in the network, defense in>>> depth is a prudent philosophy to reduce the chances of compromise, it does not>>>eliminate it nor does any architecture you can think of, period
What a ridiculous statement - of course it does.
*The place of the stateful firewall is in front of clients, not servers*.
Servers can also be clients who can benefit from state tracking. The best answer I have to that scenario is to make the client path different than the server path. Joe
On Fri, 08 Jan 2010 08:22:00 EST, bill from home said:
My question is at what size connection does a state table become vulnerable, are we talking 1mb dsl's with a soho firewall?
Security - you're doing it wrong. ;) The question you *should* be asking yourself is "at what size connection am I enough of a network presence that I might attract attention from somebody who might want to attack me?" And that depends more on the *type* of presence than the size of the pipe. If you're a small electrical-components design firm that nobody's heard of, the size of your state table is probably moot. One of your users just drew the attention of some random 4chan /b/tard, the size of the state table is again probably moot. ;) But to answer your question - it's so absurdly easy for a competent(*) attacker to saturate any edge connection smaller than a gigabit or so, that 'state table exhaustion' is only *really* an issue if you have a 10G or bigger pipe. (*) There is of course the case of an incompetent attacker who only has a botnet of a few hundred machines, attacking a small pipe. At that point, it's probably a crap shoot - if your firewall falls over, you've been DDoS'ed. But if it doesn't fall over, you'll probably *still* be DDoS'ed because the machines you're protecting fall over...
All, This thread certainly has been educational, and has changed my perception of what an appropriate outward facing architecture should be. But seldom do I have the luxury of designing this from scratch, and also the networks I administer are "small business's". My question is at what size connection does a state table become vulnerable, are we talking 1mb dsl's with a soho firewall? Or as I suspect we are talking about a larger scale? I know there are variables, I am just looking for a "rule of thumb". I would not want to recommend a change if it is not warranted. But when fatter and fatter pipes become available at what point would a change be warranted.
For small pipes, a simple DoS is trivial enough to jam up the works without worrying about the state table size. However, that doesn't mean it isn't smart to get a handle on it. The biggest question may be pipe size: this variable controls the total possible PPS that can be tossed at the firewall. If you consider a technology such as 10base-T Ethernet, that's 10 megabits per second. When you add up the IFG, MAC preamble, dest/src, MAC type, payload, and CRC, the smallest Ethernet frame is 84 bytes. 10M/(84*8) = 14880 frames per second. Now let's say you want to block a SYN flood from hitting your server (nobody need tell me that there are obvious problems with this; this is an educational exercise). If your firewall is configured to expire state table entries for partially opened connections after 15 seconds, the speed of ethernet combined with the 15 seconds means you need a table that's 225,000 entries large. But wait. If they're blowing 14880 frames per second at you, that Ethernet's quite full. You're already getting DoS'ed by capacity. Further, what happens when the attack moves to simply fully opening connections? Your state table is tiny for that... I know this is NANOG, and it's network-centric. However, fundamentally, a stateful firewall fuzzes the boundary between network and server. It is taking on some duties that have typically been the responsibility of the server. So I'm going to go off on a tangent and say that it may be useful to consider the state of the art in server technology. A good UNIX implementation (OpenBSD, FreeBSD, maybe Linux ;-) ) will be hardened and further-hardenable against these sorts of attacks. The server *already* has to do things such as tracking SYN's, except that they no longer have to - they can issue cookies back and then simply forget about it. If the cookie is returned in the ACK, then a connection is established. A proper implementation of this means that a server using SYN cookies has an infinitely large SYN queue; packets on the network itself are actually the queue. The technique works and scales at 1Mbps as well as it does at 10Gbps. Putting a stateful firewall in front of that would be dumb; the server is completely capable of coping with the superfluous SYN's in a much more competent manner than the firewall. I won't go into this in more detail. You can look to see what the IRC networks are doing to protect themselves. They're commonly beat up and have learned most of the best defense tricks around. A stateless firewall that can implement filtering policies in silicon is absolutely a fantastic thing to have, especially when faced with a DoS for which you can write rules. Put your servers behind one heck of a good stateless firewall, and run a well-tuned OS. You'll be a lot more DoS-resistant. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net "We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
On Fri, Jan 8, 2010 at 10:48 AM, Joe Greco <jgreco@ns.sol.net> wrote:
Putting a stateful firewall in front of that would be dumb; the server is completely capable of coping with the superfluous SYN's in a much more competent manner than the firewall.
The trouble with blanket statements about "all stateful firewalls" and "all servers" is there are lots of different firewall and server platforms. Stateful firewalls can implement SYN cookies, and at least a couple do. Firewalls do not need to build a state entry for partial TCP sessions, there are a few different things that can be done, such as the firewall answering on behalf of the server (using SYN cookies) and negotiating connection with the server after the final ACK. As a result, spoofed TCP packets don't consume state. Multiple IPs they can _receive_ traffic to required, next? Spoofed UDP is a much bigger problem, because there is no connection establishment. And it's probably not sane to put certain public-facing UDP services such as large public DNS service IPs (e.g. 8.8.8.8) behind most forms of stateful filter. But that's not the average case, by any means, most servers are not DNS servers. Servers consume state just like firewalls do.... E.g. A public FTP server that opens a process for each connection, goes down in a connection flood, when kernel process slots are used up, long before the firewall. Servers running a robust OS completely and correctly configured to perfectly protect themsleves (resource limits, etc), no Windows OSes, with unwanted open ports, is a wholly unwarranted assumption for real-world server environments. In the best cases it does hold up (to a great extent). In other cases, it's an operational fantasy; it would be nice if that could be relied upon.... --- -J
On Jan 10, 2010, at 3:48 PM, James Hess wrote:
Firewalls do not need to build a state entry for partial TCP sessions, there are a few different things that can be done, such as the firewall answering on behalf of the server (using SYN cookies) and negotiating connection with the server after the final ACK.
The firewall capacity for doing this can be easily overwhelmed; and again, well-formed traffic can simply 'crowd out' good traffic. The other drawbacks of the stateful firewall further outweigh even this negligible benefit. Fronting one's Web server farms/load-balancers with a tier of transparent reverse-proxy caches is a better way to scale TCP connection capacity, as well as the myriad other benefits offered (described earlier in this thread). ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
On Sun, Jan 10, 2010 at 3:48 AM, James Hess <mysidia@gmail.com> wrote:
there are a few different things that can be done, such as the firewall answering on behalf of the server (using SYN cookies) and negotiating connection with the server after the final ACK.
James, That's called a proxy or sometimes an application-layer gateway. The problem with proxies, aside from the extra computing overhead, is that they radically change the failure semantics of a TCP connection. The sender believes itself connected and has transferred the first window worth of data (which may be all the data he needs to transmit) while the firewall is still trying to connect to the recipient. Often the proxy isn't clever enough to send an RST in stead of a FIN so the remote application thinks it has a successful finish. Even if it does send an RST, most application developers aren't well enough versed in sockets programming to block on the shutdown and check the success status, and even if they do they may report a different error than the basic "failed to connect." Proxies can be a useful tool but they should be used with caution and only when you're absolutely sure that the difference in TCP failure semantics is not important to the protocol you're proxying. Regards, Bill Herrin -- William D. Herrin ................ herrin@dirtside.com bill@herrin.us 3005 Crane Dr. ...................... Web: <http://bill.herrin.us/> Falls Church, VA 22042-3004
On Sun, Jan 10, 2010 at 12:47 PM, William Herrin <bill@herrin.us> wrote:
Even if it does send an RST, most application developers aren't well enough versed in sockets programming to block on the shutdown and check the success status,
Sorry, I got that wrong. shutdown() will succeed without waiting for a FIN or RST from the remote end. So will close(). Instead, after the shutdown() you then have to block on read() waiting for a either 0 bytes read or an error. 0 bytes = FIN, error = RST. Unfortunately, few sockets programmers realize that they have to do this to catch that final possible error. They send what the expect to send and if they don't expect to receive anything back, they shutdown and close the socket without waiting. Regards, Bill Herrin -- William D. Herrin ................ herrin@dirtside.com bill@herrin.us 3005 Crane Dr. ...................... Web: <http://bill.herrin.us/> Falls Church, VA 22042-3004
On Sun, Jan 10, 2010 at 11:47 AM, William Herrin <bill@herrin.us> wrote:
On Sun, Jan 10, 2010 at 3:48 AM, James Hess <mysidia@gmail.com> wrote:
there are a few different things that can be done, such as the firewall answering on behalf of the server (using SYN cookies) and negotiating connection with the server after the final ACK. That's called a proxy or sometimes an application-layer gateway. The
I'm not really referring to ALGs, but to Layer 3 proxies, that are application-agnostic -- simply manipulate the connection setup, and then step 'out of the way' performing only mechanical translation of SEQ numbers / port numbers. However, appliction layer gateways are still stateful firewalls. Content switches and load balancers that track connections and allow access control are also stateful firewalls. They are widely used, for many different kinds of applications.
they radically change the failure semantics of a TCP connection. The sender believes itself connected and has transferred the first window worth of data (which may be all the data he needs to transmit) while
And if the initial window size is 0?
send an RST, most application developers aren't well enough versed in sockets programming to block on the shutdown and check the success status, and even if they do they may report a different error than the basic "failed to connect."
I agree that could be an issue. The proxy might do the wrong thing, the application might do the wrong thing.
Proxies can be a useful tool but they should be used with caution and
I agree they should be used with caution. I don't agree with "You never need a proxy in front of a server, it's only there to fail". -- -J
On Jan 11, 2010, at 4:55 AM, James Hess wrote:
I don't agree with "You never need a proxy in front of a server, it's only there to fail".
Again, reverse proxy *caches* are extremely useful in front of Web farms. Pure proxying makes no sense. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
On Fri, Jan 8, 2010 at 10:48 AM, Joe Greco <jgreco@ns.sol.net> wrote:
Putting a stateful firewall in front of that would be dumb; the server is completely capable of coping with the superfluous SYN's in a much more competent manner than the firewall.
The trouble with blanket statements about "all stateful firewalls" and "all servers" is there are lots of different firewall and server platforms.
Yes, yes there are, but only an idiot buys a Ford Pinto to haul pallets of freight. When you get serious about hauling lots of freight, you buy something appropriate, and there are suddenly a lot fewer combinations.
Stateful firewalls can implement SYN cookies, and at least a couple do. Firewalls do not need to build a state entry for partial TCP sessions, there are a few different things that can be done, such as the firewall answering on behalf of the server (using SYN cookies) and negotiating connection with the server after the final ACK.
And how much of that is done in silicon? Because if it's not in silicon, then it's being done by a CPU, and if it's being done by a CPU, why not just let the server do it? Commodity server hardware is cheap compared to specialized silicon offerings...
As a result, spoofed TCP packets don't consume state. Multiple IPs they can _receive_ traffic to required, next?
Spoofed UDP is a much bigger problem, because there is no connection establishment. And it's probably not sane to put certain public-facing UDP services such as large public DNS service IPs (e.g. 8.8.8.8) behind most forms of stateful filter.
But that's not the average case, by any means, most servers are not DNS servers. Servers consume state just like firewalls do....
E.g. A public FTP server that opens a process for each connection, goes down in a connection flood, when kernel process slots are used up, long before the firewall.
Again, Ford Pinto... you can always design a system that will fail. That's like shooting fish in a barrel. If you're worried about kernel process slots, you *choose* an appropriate service. Like a threaded ftp server.
Servers running a robust OS completely and correctly configured to perfectly protect themsleves (resource limits, etc), no Windows OSes, with unwanted open ports, is a wholly unwarranted assumption for real-world server environments.
And so is a high-performance non-crashable stateful firewall that's so talented that it can keep a poorly configured server operational under all circumstances. Wheee.
In the best cases it does hold up (to a great extent). In other cases, it's an operational fantasy; it would be nice if that could be relied upon....
Some of us are still failing to understand why it is that it's better to buy an extremely expensive stateful firewall device which is likely to collapse under load because the salesman lied; it seems like it'd be easier to go and scale capacity with some cheap stateless firewalls to do packet filtering in silicon, backended by some additional servers and some good admins who have a clue about what they're doing. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net "We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
bill from home wrote:
All, This thread certainly has been educational, and has changed my perception of what an appropriate outward facing architecture should be. But seldom do I have the luxury of designing this from scratch, and also the networks I administer are "small business's". My question is at what size connection does a state table become vulnerable, are we talking 1mb dsl's with a soho firewall?
some numbers, 100Mb/s will carry 220Kpps worth of 64byte packets, if this is a fairly simple syn attack and your firewall can support 100k new connections a second (that's a fairly fast firewall), you need less than 50Mb/s to nuke it... the maximum size of the state table on a linux derived system with 4GB of ram is north of a million connections so assuming the session rate of the dos is trackable your firewall needs to start aging connections out in an accelerated fashion after about 4 seconds otherwise you're similarly hosed... given the same firewall can probably forward 2-3mpps when it comes to small packet you run out of state long before your run out of forwarding horsepower. Some kind of firewall device that you might put in front of a business cable connection, or fractional ethernet is like to support a much lower connection rate embedded Pentium equivalent or low to mid-range mips might support a rate of 2-10k connections per second at which point the thresh-hold for dosing it based on session rate is quite a bit lower (quite a bit lower than that of a webserver or dekstop pc for example). i.e. if 10kpps of dos will take it out that's like 5Mb/s on a device that might other wise be able to forward 300-500Mb/s interface to interface using large packet.
Or as I suspect we are talking about a larger scale? I know there are variables, I am just looking for a "rule of thumb". I would not want to recommend a change if it is not warranted. But when fatter and fatter pipes become available at what point would a change be warranted.
Thanks Bill Kruchas
Dobbins, Roland wrote:
On Jan 8, 2010, at 3:21 PM, Arie Vayner wrote:
Further on, if you want to really protect against a real DDoS you would most likely would have to look at a really distributed solution, where the different geographical load balancing solutions come into play.
GSLB or whatever we want to call it is extremely useful from a general availability standpoint; however, the attackers can always scale up and really distribute their already-DDoS even further (they learned about routeservers and DNS tinkering years ago). Architecture, visibility, and control are key, as are vendor/customer/peer/upstream/opsec community relationships.
----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>
Injustice is relatively easy to bear; what stings is justice.
-- H.L. Mencken
On Wed, 6 Jan 2010 04:53:17 +0000 "Dobbins, Roland" <rdobbins@arbor.net> wrote:
On Jan 6, 2010, at 11:43 AM, George Bonser wrote:
Yes, you have to take some of the things that were done in one spot and do them in different locations now, but the results are an amazing increase in service capacity per dollar spent on infrastructure.
I strongly agree with the majority of your comments, with the caveat that I've seen many, many load-balancers fall over due to state-exhaustion, too; load-balancers need northbound protection from DDoS (S/RTBH, flow-spec, IDMS, et. al.), as well.
And that is the crux of the matter. Any time you maintain state in the network (e.g. stateful firewalls), you're vulnerable to traffic based attacks that can exhaust that state. The Internet is scalable because the (soft) state that it maintains, namely route tables, isn't dependent on or influenced by the traffic that is forwarded through it. Hosts have to maintain state about their connections - there is no choice. However, the more you're able to push state tracking to the hosts, you end up with less consequences of state targeted attacks, and more scalable architectures.
On Tue, 2010-01-05 at 16:24 -0500, Robert Brockway wrote:
On Tue, 5 Jan 2010, Dobbins, Roland wrote:
In the most basic terms, a stateful firewall performs bidirectional classification of communications between nodes, and makes a pass/fail determination on each packet based on a) whether or not a bidirectional communications session is already open between the nodes and b) any policy rules configured on the firewall as to what ports/protocols should be allowed between said nodes.
Stateful firewalls make good sense in front of machines which are primarily clients; the stateful inspection part keeps unsolicited packets away from the clients.
Stateful firewalls make absolutely no sense in front of servers, given that by definition, every packet coming into the server is unsolicited (some protocols like ftp work a bit differently in that there're multiple bidirectional/omnidirectional communications sessions, but the key is that the initial connection is always unsolicited).
Putting firewalls in front of servers is a Really Bad Idea - besides the
Hi Roland. I disagree strongly with this position.
As someone who worked for a startup several years ago working on solving precisely the problem of having a reliable firewall/IDS solution infront of the server, I'm going to have to disagree with your disagreement.
fact that the stateful inspection premise doesn't apply (see above),
The problem is that your premise is wrong. Stateful firewalls (hereafter just called firewalls) offer several advantages. This list is not necessarily exhaustive.
(1) Security in depth. In an ideal world every packet arriving at a server would be for a port that is intended to be open and listening. Unfortunately ports can be opened unintentionally on servers in several ways: sysadmin error, package management systems pulling in an extra package which starts a service, etc. By having a firewall in front of the server we gain security in depth against errors like this.
ACLs in the router hardware handle this. Your average datacentre switch, even a small one can handle stateless ACL checks in hardware. Also ACLs don't protect you from the bad guys, especially if you're incompetent. What my team found was that it was infact -impossible- to sanely do DPI infront of a server and also survive a DDoS attack. DDoS attacks are a big problem these days, in case you didn't notice.
(2) Centralised management of access controls. Many services should only be open to a certain set of source addresses. While this could be managed on each server we may find that some applications don't support this well, and management of access controls is then spread across any number of management interfaces. Using a firewall for network access control reduces the management overhead and chances of error. Even if network access control is managed on the server, doing it on the firewall offers additional security in depth.
ACLs in the router hardware handles this. Doing it on a firewall provides no additional security, and may infact decrease network performance and throughput. Additionally, predictive firewalls can be gamed.
(3) Outbound access controls. In many cases we want to stop certain types of outbound traffic. This may contain an intrusion and prevent attacks against other hosts owned by the organisation or other organisations. Trying to do outbound access control on the server won't help as if the server is compromised the attacker can likely get around it.
ACLs in the router hardware as well as blackholed /32s in the route table of the router handle this. Doing it on a firewall provides no additional security and *will* decrease network performance and throughput. Routers are built for large route tables and make use of RADIX tries and other optimizations that hardware server-oriented firewalls do not typically have.
(4) Rate limiting. The ability to rate limit incoming and outgoing data can prevent certain sorts of DoSes.
I am not sure what makes you believe that. The ability to rate limit incoming data at the server level would definitely not prevent a DoS. The ability to rate limit outgoing data would cause a DoS of anything other than DoS traffic that is hosted on the server. The basic rule here is "you can't filter more than your port speed", and if your port is getting hit with 1.3gbit of DDoS and your port is only 1gbit, you're still offline.
(5) Signature based blocking.
LOL. Signature based blocking is the biggest scam since the 1980s when IDS technology was first invented. It doesn't work. It is snake oil. The only type of 'signature' that would work would be a list of all known botnet IPs, and you're never going to get that.
Modern firewalls can be tied to intrusion prevention systems which will 'raise the shields' in the face of certain attacks. Many exploits require repeated probing and this provides a way to stop the attack before it is successful.
As mentioned above, predictive firewalls can be gamed and cheated to make themselves DoS the downstream network. Keep in mind DoS does not mean the same as "packet flooding", DoS means "denial of service", so if it chokes, service is denied.
rendering the stateful firewall superfluous, even the biggest, baddest firewalls out there can be easily taken down via state-table exhaustion;
Do you have any evidence to support this assertion? You've just asserted that all firewalls have a specific vulnerability. It isn't even possible to know the complete set of architectures (hardware & software) used for firewalls so I don't see how you can assert they all have this vulnerability.
It's a known problem with firewalls in general. You have to keep in mind that all firewalls mechanically do the same thing -- they proxy packets from one port to another port depending on various rules.
In any case, my experience tells me that a DDoS will be successful due to exhaustion of available network capacity before it exhausts the state table on the firewall.
Adding the firewall infront of a server can reduce network throughput to that server by up to 80% depending on what checks it is doing, and how fast it can do the checks. There are no hardware firewalls that can *really* do DPI at 10GE linerate that I know of, for example. William
(4) Rate limiting. The ability to rate limit incoming and outgoing data can prevent certain sorts of DoSes.
I am not sure what makes you believe that. The ability to rate limit incoming data at the server level would definitely not prevent a DoS.
The ability to rate limit outgoing data would cause a DoS of anything other than DoS traffic that is hosted on the server.
It may be good practice to rate limit outgoing ICMP PING replies from your server to the real world. Kind of like being a good neighbor in the event of certain types of attacks on other parties. This can be extended into more specific types of outgoing rate limits. For example, an ISP DNS recurser that normally serves 1Mbps of traffic in aggregate but lives on a 1Gbps ethernet might use a per-destination outgoing limit to restrict the amount of damage that could be inflicted on a remote DNS server (without affecting other destinations); things like FreeBSD ipfw/dummynet and Linux (mumble) have these sorts of capabilities. I can see some usefulness in rate limiting as a form of sanity enforcement. Your average switch cannot do the more complex forms in silicon. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net "We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
On 1/5/10 3:24 PM, Robert Brockway wrote:
On Tue, 5 Jan 2010, Dobbins, Roland wrote:
The problem is that your premise is wrong. Stateful firewalls (hereafter just called firewalls) offer several advantages. This list is not necessarily exhaustive.
Great advantages list, but where's the disadvantages list? Here's mine: 1..n) Stateful firewalls go down. It's the very nature of what they do. If you haven't had this problem, then your application is small. Everyone needs to listen to Roland's mantra: "stateless ACLs in hardware than can handle Mpps". It's more than just a hint.
On Tue, 05 Jan 2010 23:14:05 CST, Ryan Brooks said:
Everyone needs to listen to Roland's mantra: "stateless ACLs in hardware than can handle Mpps". It's more than just a hint.
I suspect that more than a few need to be reminded that "stateless ACLs in switch hardware" is just another name for "switch that also does stateless firewall".
-----Original Message----- From: Valdis.Kletnieks@vt.edu [mailto:Valdis.Kletnieks@vt.edu] Sent: Wednesday, January 06, 2010 3:46 PM To: nanog@nanog.org Subject: Re: I don't need no stinking firewall!
On Tue, 05 Jan 2010 23:14:05 CST, Ryan Brooks said:
Everyone needs to listen to Roland's mantra: "stateless ACLs in hardware than can handle Mpps". It's more than just a hint.
I suspect that more than a few need to be reminded that "stateless ACLs in switch hardware" is just another name for "switch that also does stateless firewall".
I don't think so: "stateless ACLs in switch hardware" != " switch that also does stateless firewall" IMHO... "stateless ACLs in [switch|router] hardware" = ACLs applied to interfaces that filter packets based on source or destination IP addresses and ports, or protocols. Correct me if I'm wrong Roland. - Brian CONFIDENTIALITY NOTICE: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, copying, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Thank you.
On Jan 5, 2010, at 4:24 PM, Robert Brockway wrote:
Do you have any evidence to support this assertion? You've just asserted that all firewalls have a specific vulnerability. It isn't even possible to know the complete set of architectures (hardware & software) used for firewalls so I don't see how you can assert they all have this vulnerability.
Just about every ddos i've ever been involved in mitigation results in some device labeled "firewall" blowing it's brains and crippling the company further than if they had utilized a more distributed model. When combined with various other layers of mitigation that are either integrated or inline with another device we've spent lots of time troubleshooting which exact device was causing the most trouble. I can't cite specific cases unless my customers say I can, but it's somewhat amusing to watch some C* of a company realize they've wasted money on a device/service that actually made the problem worse in the face of an attack. There are those that might say the protection devices were not properly used, configured, etc... and if that's the case, it reflects the sad state of the lack of maturity of the industry/tech. (Or that it's obsolete). - Jared
Le 10-01-05 à 21:29, Dobbins, Roland a écrit :
Stateful firewalls make absolutely no sense in front of servers, given that by definition, every packet coming into the server is unsolicited (some protocols like ftp work a bit differently in that there're multiple bidirectional/omnidirectional communications sessions, but the key is that the initial connection is always unsolicited).
Most hosts are in some measure servers and clients. Sometimes a "server" might want to make an outbound connection for a legitimate reason (say a DNS lookup or zone transfer). Sometimes it might be tricked into doing so for nefarious reasons (like the old reverse telnet trick of binding a shell to an outbound tcp connection). A properly configured firewall will prevent latter. -w
On Jan 6, 2010, at 5:38 PM, William Waites wrote:
A properly configured firewall will prevent latter.
So will stateless ACLs, running in hardware capable of handling mpps. ;> ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
am Mittwoch, 06. Jänner 2010 um 13:43 schrieb Roland Dobbins:
On Jan 6, 2010, at 5:38 PM, William Waites wrote:
A properly configured firewall will prevent latter.
So will stateless ACLs, running in hardware capable of handling mpps.
How do you define "firewall"? I remember something like "a security concept, comprising a device or set of devices designed to prevent unauthorized access to a computer system or network". ACLs would then just be part of the firewall concept. cheers, jutta
On Jan 6, 2010, at 8:25 PM, juttazalud wrote:
How do you define "firewall"?
This threat was about stateful firewalls in particular. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Injustice is relatively easy to bear; what stings is justice. -- H.L. Mencken
On Tue Jan 05, 2010 at 02:16:58PM -0600, Brian Johnson wrote:
I have my own idea of what a firewall is and what it does. I also understand what statefull packet inspection is and what it does. Given this information, and not prejudging any responses, exactly what is a firewall for and when is statefull inspection useful?
Not sure I'd call myself a security guru, but... I'm not a great fan of packet filtering firewalls (as opposed to proxy based or application layer firewalls). Generally, I just use stateless ACLs when I need additional network level security. However, they do have one big disadvantage. Say you've got a server where you want to allow outbound HTTP access to anywhere on the Internet, but only SSH inbound from your home DSL. To do this, you'd build an inbound ACL which looks something like: - Allow from home DSL IP to server port 22 - Allow from anywhere port 80 to server - Deny all other traffic. You need the port 80 rule to allow the return traffic from all those outbound connections. However, an enterprising hacker realises that he can create a TCP connection from port 80 on his own box to port 22 on your server. Now, if you change from stateless to stateful ACLs, you add the intelligence that whenever it sees an connection originating from your server to port 80 on the internet, it automatically adds a rule that allows traffic back from the server you're talking to, but not anywhere else. Therefore, your enterprising hacker can no longer connect in. Of course, the other benefit that a stateful inspection firewall can do is pattern matching on undesirable traffic based on signatures Simon -- Simon Lockhart | * Sun Server Colocation * ADSL * Domain Registration * Director | * Domain & Web Hosting * Internet Consultancy * Bogons Ltd | * http://www.bogons.net/ * Email: info@bogons.net *
Simon Lockhart wrote:
Generally, I just use stateless ACLs when I need additional network level security. However, they do have one big disadvantage. Say you've got a server where you want to allow outbound HTTP access to anywhere on the Internet, but only SSH inbound from your home DSL. To do this, you'd build an inbound ACL which looks something like:
- Allow from home DSL IP to server port 22 - Allow from anywhere port 80 to server
Change the above to: - Allow from anywhere port 80 to server port > 1023 Or better: - Allow from anywhere port 80 to server port > 1023 established
- Deny all other traffic.
You need the port 80 rule to allow the return traffic from all those outbound connections.
Those outbound connections will originate from a random high port, so just allow those as destination ports on your inbound rule.
However, an enterprising hacker realises that he can create a TCP connection from port 80 on his own box to port 22 on your server.
Not with the above rules. -- Jay Hennigan - CCIE #7880 - Network Engineering - jay@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
Doesn't using the established allow any packet with ACK/RST set and wouldn't you have to allow all high ports? Jason -----Original Message----- From: Jay Hennigan [mailto:jay@west.net] Sent: Tuesday, January 05, 2010 3:04 PM To: nanog@nanog.org Subject: Re: I don't need no stinking firewall! Simon Lockhart wrote:
Generally, I just use stateless ACLs when I need additional network level security. However, they do have one big disadvantage. Say you've got a server where you want to allow outbound HTTP access to anywhere on the Internet, but only SSH inbound from your home DSL. To do this, you'd build an inbound ACL which looks something like:
- Allow from home DSL IP to server port 22 - Allow from anywhere port 80 to server
Change the above to: - Allow from anywhere port 80 to server port > 1023 Or better: - Allow from anywhere port 80 to server port > 1023 established
- Deny all other traffic.
You need the port 80 rule to allow the return traffic from all those outbound connections.
Those outbound connections will originate from a random high port, so just allow those as destination ports on your inbound rule.
However, an enterprising hacker realises that he can create a TCP connection from port 80 on his own box to port 22 on your server.
Not with the above rules. -- Jay Hennigan - CCIE #7880 - Network Engineering - jay@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV *** NOTICE--The attached communication contains privileged and confidential information. If you are not the intended recipient, DO NOT read, copy, or disseminate this communication. Non-intended recipients are hereby placed on notice that any unauthorized disclosure, duplication, distribution, or taking of any action in reliance on the contents of these materials is expressly prohibited. If you have received this communication in error, please delete this information in its entirety and contact the Amedisys Privacy Hotline at 1-866-518-6684. Also, please immediately notify the sender via e-mail that you have received this communication in error. ***
Jason Shearer wrote:
Doesn't using the established allow any packet with ACK/RST set
Yes, as would be expected for legitimate return traffic for a TCP connection initiated from a browser inside the firewall.
and wouldn't you have to allow all high ports?
That's what the ">" is for. Cisco syntax "gt" (greater than). The point is that either of these will deny unsolicited new connection attempts from the outside to TCP 22 (and 445, 135, etc.) -- Jay Hennigan - CCIE #7880 - Network Engineering - jay@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
On Tue, Jan 05, 2010 at 13:18:47PM -0800, Jay Hennigan wrote:
Jason Shearer wrote:
Doesn't using the established allow any packet with ACK/RST set
Yes, as would be expected for legitimate return traffic for a TCP connection initiated from a browser inside the firewall.
and wouldn't you have to allow all high ports?
That's what the ">" is for. Cisco syntax "gt" (greater than).
One could also use reflexive access lists, which are much better than static lists, although that takes you back to stateful. It is possible to combine them both to achieve a mostly stateless setup while still having better overall security.
The point is that either of these will deny unsolicited new connection attempts from the outside to TCP 22 (and 445, 135, etc.)
-- Henry Yen Aegis Information Systems, Inc. Senior Systems Programmer Hicksville, New York (800) 234-4700
On Tue, 5 Jan 2010, Brian Johnson wrote:
Given this information, and not prejudging any responses, exactly what is a firewall for and when is statefull inspection useful?
Stateful inspection is useful for breaking things in subtle and hard-to-debug ways. http://fanf.livejournal.com/102206.html http://fanf.livejournal.com/95831.html Tony. -- f.anthony.n.finch <dot@dotat.at> http://dotat.at/ GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS. MODERATE OR GOOD.
Tony Finch wrote:
Stateful inspection is useful for breaking things in subtle and hard-to-debug ways.
http://fanf.livejournal.com/102206.html http://fanf.livejournal.com/95831.html
Is that really stateful inspection? Isn't the SMTP fixup on a PIX an application-level gateway? I *though* most of the world turns SMTP fixup off because it's naff. Peter
On 1/5/10 2:01 PM, Peter Hicks wrote:
Tony Finch wrote:
Stateful inspection is useful for breaking things in subtle and hard-to-debug ways.
http://fanf.livejournal.com/102206.html http://fanf.livejournal.com/95831.html
Is that really stateful inspection? Isn't the SMTP fixup on a PIX an application-level gateway?
I *though* most of the world turns SMTP fixup off because it's naff.
It is a ALG, and a completely braindead one at that. Nothing like trying to figure out what an error message means when its just... XXX ****************************************************** The PIX's fixup for DNS packets have been causing issues on my end too in one setup. -- Brielle Bruns The Summit Open Source Development Group http://www.sosdg.org / http://www.ahbl.org
On Tue, 5 Jan 2010, Peter Hicks wrote:
Is that really stateful inspection? Isn't the SMTP fixup on a PIX an application-level gateway?
Well, the bug I described is caused by it not being stateful enough.
I *though* most of the world turns SMTP fixup off because it's naff.
Exactly my point :-) Tony. -- f.anthony.n.finch <dot@dotat.at> http://dotat.at/ GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS. MODERATE OR GOOD.
On Tue, 5 Jan 2010 20:51:47 +0000 Tony Finch <dot@dotat.at> wrote:
On Tue, 5 Jan 2010, Brian Johnson wrote:
Given this information, and not prejudging any responses, exactly what is a firewall for and when is statefull inspection useful?
Stateful inspection is useful for breaking things in subtle and hard-to-debug ways. http://fanf.livejournal.com/102206.html http://fanf.livejournal.com/95831.html
Your second article (with the pointer to "end-to-end arguments in systems design") reminded me of this thread that came up on the Linux networking development mailing list recently. TCP was flaking out, but if the same traffic was tunnelled over the same connection, all was good. Strange TCP behavior over HSDPA http://www.spinics.net/lists/netdev/msg116809.html
Tony. -- f.anthony.n.finch <dot@dotat.at> http://dotat.at/ GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS. MODERATE OR GOOD.
The primary value of a firewall is two-fold: - It enables a network administrator to define his "edge", the interior of which he is responsible for. - It enables a network administrator to isolate his network from externally-originated traffic per his whims and viewpoints. IMHO, it is not a security solution per se; it is comparable perhaps to human skin - keeping certain stuff out to limit the need to use other tools that one uses internally. That said, the tools one uses to create true security are a combination of network-based detection/ analysis equipment like honeypots, router configurations, and sensors, and host-based security technologies. In the final analysis, the hosted application is responsible for its own security (if some attacker threads the needle, it had better be able to handle the attack), and uses host and network facilities as defense-in-depth (the less it has to worry about that the more effective overall security is). On Jan 5, 2010, at 12:16 PM, Brian Johnson wrote:
Security Gurus, et al,
I have my own idea of what a firewall is and what it does. I also understand what statefull packet inspection is and what it does. Given this information, and not prejudging any responses, exactly what is a firewall for and when is statefull inspection useful?
Please respond on-list as I want to have some useful discourse and discussion in the clear. Flamers and Trolls will be disregarded. :)
Thank you.
- Brian
CONFIDENTIALITY NOTICE: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, copying, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Thank you.
On Tue, 5 Jan 2010, Fred Baker wrote:
The primary value of a firewall is two-fold:
- It enables a network administrator to define his "edge", the interior of which he is responsible for. - It enables a network administrator to isolate his network from externally-originated traffic per his whims and viewpoints.
Actually, a firewall is so the "security administrator" can intervene between the network administrator and the system administrator to impose controls on both because they didn't prevent something themselves. It sounds like of beginning of a joke, a network administrator, system administrator and security administrator walk into a bar ... A statefull firewall is most useful for *outbound* traffic, inbound traffic controls usually break things that depend on maintaining state. Of course, if you want outbound traffic from your web server, its no longer just a web server. Its some mongrel type of client/server. Likewise a IDS/IPS/AV/Anti-X box is no longer just a stateful firewall, its some kind of mongrel security device. Simple ACLs can keep stuff out, or keep stuff in. Stateful things are only needed when you want to keep track of things you sent outbound, so you can let (hopefull) the same thing back inbound.
IMHO, it is not a security solution per se; it is comparable perhaps to human skin - keeping certain stuff out to limit the need to use other tools that one uses internally. That said, the tools one uses to create true security are a combination of network-based detection/analysis equipment like honeypots, router configurations, and sensors, and host-based security technologies. In the final analysis, the hosted application is responsible for its own security (if some attacker threads the needle, it had better be able to handle the attack), and uses host and network facilities as defense-in-depth (the less it has to worry about that the more effective overall security is).
Your "simple", "verifiable", "etc" security devices then become something even more complex than the systems they are supposedly are protecting. With that additional complexity comes additional risks that the security device itself has flaws. Adding NAT/PAT/state/DNS proxy creates its own problems and many protocol hacks, often requiring even more complexity to "fix" what you broke. I blame Bellovin & Cheswick for firewalls :-) There are some subtle points in their early papers I'm still learning. Yes, statefull firewalls can be usefull. But too often security professionals suffer from the I have a hammer syndrome. They break everything with a single tool, even stuff that may be better without it. Security should worry about all the letters in C-I-A.
On Tue, Jan 5, 2010 at 12:16 PM, Brian Johnson <bjohnson@drtel.com> wrote:
Security Gurus, et al,
I have my own idea of what a firewall is and what it does. I also understand what statefull packet inspection is and what it does. Given this information, and not prejudging any responses, exactly what is a firewall for and when is statefull inspection useful?
Please respond on-list as I want to have some useful discourse and discussion in the clear. Flamers and Trolls will be disregarded. :)
Thank you.
- Brian
To me - a firewall is just another layer of security to help protect company/personal assets. Firewalls, AV, IPS, OS patches, physical security, educated users etc. etc...all play a part in protecting what you own and what you data you have from 'bad guys'. Where to place firewalls depends on what you are protecting. If regular humans (ie consumers) stateful packet firewalls are smart (although NAT does provide a level of security - and I know there will be arguments against that). If business assets - it depends on scale and traffic. If you have a small to medium business with a T1 - a smart network engineer can us ACL's to protect your assets but stateful firewalls are fairly cheap so why not use them? If you are running gigabits worth of traffic then a stateful firewall is a bad thing but layered protection is still important. DDOS defenses of some form is part of that layered protection (scale to handle DDOS, work w/ upstream providers etc..) . So I guess my answer is - it just depends on the business, traffic patterns, $$, and skill sets of the engineers or consultants you hire. But I do agree - firewalling or protection of assets is a necessity no matter what your size or scale from a practical and most likely regulatory perspective. So now I get to rant - becuase I think that 'security guru's' are one-tracked minded. Often times - in larger organizations the executives are the largest FUD mongers. This lead to hiring a 'Security Guru'. The 'Security Guru' convinces said executives that the sky is falling. Executives fear for their jobs and company assets and the next thing you know - all innovation and flexibility is removed from the employee's in the name of security. It's a really bad thing. Are most users bungholes that require strict security policies - yes. Are they all? No. It's your job to make sure the company is protected enough to continue innovation and making money. You have a tough job I'll give you that - and I wouldn't want it - but you chose your path in life not me! So make it work without stifling the users you are trying to protect! </end_rant> Kenny
On Tue, 5 Jan 2010 14:16:58 -0600 "Brian Johnson" <bjohnson@drtel.com> wrote:
Security Gurus, et al,
I have my own idea of what a firewall is and what it does. I also understand what statefull packet inspection is and what it does. Given this information, and not prejudging any responses, exactly what is a firewall for and when is statefull inspection useful?
First thing to work out is your threat model. Once you've worked out what you're trying to protect, who you're trying to protect it from, and what techniques they're likely use to break that protection, you can then work out if a firewall is the right tool, then work out how many to have and where (network perimeter only, network perimeter + hosts, network perimeter + hosts + in application protection (e.g. authentication like in ssh - remember, it's people that are your real threat, not machines and their IP addresses - those are just the tools people use). The trap is to think technology like firewalls are only thing you need to worry about. Unfortunately they won't stop the building cleaners from lifting things out of the bins under desks.
Please respond on-list as I want to have some useful discourse and discussion in the clear. Flamers and Trolls will be disregarded. :)
Thank you.
- Brian
CONFIDENTIALITY NOTICE: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, copying, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Thank you.
On Tue, Jan 5, 2010 at 2:16 PM, Brian Johnson <bjohnson@drtel.com> wrote:
I have my own idea of what a firewall is and what it does. I also
A firewall is a term for a class of device (or software program). Ask different people and you should get different answers, depending on who you ask. Windows firewall... bpf.. ipfw.. iptables... broadband router.. It's not one specific thing, but the description of a role...... The essential characteristic of a firewall is: "a control point", and usually a "central control point" at that. A computer network virtual equivalent of real-world "sally ports" and the access control vestibules you will find at airports, with metal detectors, security guards, etc. You probably won't find security booths at each row of seats on airplanes themselves -- but that's the equivalent of a "host firewall" (a much less useful more expensive control point to maintain, if you have many servers, all subject to attack through the shared network, you definitely want some checkpoints and common firewalls, in addition to host firewalls) A control point is a place that provides the core security functions: * Identification/Classification -- determining the nature of traffic crossing the control point into or out from the protected area * Access Control -- implementing a policy against such classified traffic, to determine who/what/when bits may cross into or out from the protected area, or between different protected areas * Auditing -- logging, making a record of what happened, when, in what direction, evidence collection It is just as much about preventing your compromised server from making improper outbound connections, such as 'port 25' connections to send spam (for a server that is not a mail server), or 'port 22' connections to your other servers in different subnets (e.g. attempts to use a successfully compromised server's trusted IP to leapfrog to other servers in different LAN subnets) Some firewalls do not perform auditing functions, or log to /dev/null, anyways: this is just a log retention time of 0 seconds. If the firewall does not produce logs, then how can you validate the firewall blocks the right things and allows other things? Access control policy might be pre-defined by the manufacturer, or set by an administrator. It might not deny anything, unless asked; it's still a firewall. The ability to quickly set a block with flexible parameters is one of the most significant benefits of having firewalls in place. The depth of "Identification" may vary, depending on the type of check point; searching the packet's luggage thoroughly (DPI) versus just passing it through the metal detector ( protocol type / port number based filters).
understand what statefull packet inspection is and what it does. Given this information, and not prejudging any responses, exactly what is a firewall for and when is statefull inspection useful?
Stateful inspection is useful for discarding traffic that is not meaningful in the context of a valid connection. In the absence of a stateful firewall, the destination host has to receive the traffic, and make a decision (hopefully) to drop it. If the host has an exploitable condition in its TCP/IP implementation, a stateful firewall may be able to assist, by refusing packets that are invalid, or do not make any sense in the context of any already established connection. So the host never sees them, it never kernel panics, or blue screens, due to the exploit attempt (which would have happened without the firewall). Some stateful firewalls may re-assemble certain fragments, offer protections against teardrop attacks, common SYN floods, simple floods, and the like. So a Firewall can be a good backup safety net against the exploitation of unpatched vulnerabilities. In addition, by restricting management access, a central firewall protects against password guessing attacks, or even, an outsider who (somehow) stole the server password. You might say your system admins are perfect, and always apply all vendor patches, with no mistakes ever, always keep all ports closed except the absolutely required ones, so none of your servers should ever be vulnerable to 15-year-old TCP stack, application issues, or remote exploit of software you didn't know was installed. But reminder: without a safety net, only one mistake is required, for an attacker to (eventually) become a successful intruder. As for patching.. this is only useful, if the vendor knows about the vulnerability, admits the problem, and makes a patch available, before your adversary can take advantage of it. New (as of yet unknown) vulnerabilities may exist in hosts, and thorough validation by the firewall device looking for any funny business can be beneficial, in almost all situations. As long as you size, configure your firewalls appropriately, and protect against problems like state table exhaustion. Size does not equal just bits per second of traffic, but worst-case peak packets per second cleartext+encrypted, worst-case peak connections per second, etc. Probably if you have 100 mail or web servers, you do not want just 2 or 3 firewalls handling all that all by themselves, depending on how much traffic they get, and the need/price of downtime.. -- -J
A firewall is another layer in a defense-in-depth strategy, but tends to only be truly effective if the first rule in it is deny all from any to any which of course does not happen much of the time in the real world, with predictable results. Moreover, stateful packet inspection is not the end-all be-all: there's a lot to be said for application-level proxying, and for quasi-realtime traffic analysis. I think of my firewalls as tools which reduce the overwhelming flood of malicious and garbage traffic to a trickle -- which does not necessarily reduce the attack surface or the threats to it, but may at least allow me a better chance of seeing the threats and doing something useful about them. ---Rsk
On Tue, Jan 5, 2010 at 9:20 PM, Rich Kulawiec <rsk@gsp.org> wrote:
A firewall is another layer in a defense-in-depth strategy, but tends to only be truly effective if the first rule in it is
deny all from any to any
Not surprisingly, good network security starts with and incorporates the protected users as its most important element. Start with "deny all" and not only won't they work with you, the more creative among them will teach the others how to work around you. I've seen it over and over again and the faulty design always starts with a deny-all mentality. Can you imagine a deny-all mentality in physical security? I'm sorry sir, you can't leave your house until you justify your need to walk down the street. Regards, Bill Herrin -- William D. Herrin ................ herrin@dirtside.com bill@herrin.us 3005 Crane Dr. ...................... Web: <http://bill.herrin.us/> Falls Church, VA 22042-3004
- A firewall is a partition structure that normally consists of two side walls with a fire retardant material between them. - A firewall does not prevent a fire. - A firewall does not extinguish a fire. - A firewall only delays the propagation of a fire event to the other side. - The characteristics and properties of each side are independent and not necessary equal. - A firewall eventually gets on fire. - Fire retardant performance is dependent on type of fire and subject to change without notice. - A firewall does not prevent against monkeys with matchboxes. - In the event of a fire locate the nearest exit. - Use the exit and wait in a safe place for the arrival of trained personnel to extinguish the fire. - Clean up the mess and get rid of the monkey. - Apply to computer networks as necessary. Cheers Jorge
Poking the dragon a bit, aren't you? Fun. If you really look at it, there is no quantitative difference between statefull and non-statefull. A non-stateful firewall can prevent a TCP session from entering the SYN_RECEIVED state by blocking the SYN packet, so it strongly impacts session state without really trying. A statefull firewall will venture a bit deeper into the state diagram with a few more rules, but this is mostly a quantitative difference when viewed at a behavioral level (disregarding the internal implementation, of course). Coders and marketeers will burn me in effigy over this, but there's already a lot of people in that line... In the most general terms, a firewall attempts to permit desired traffic flows and block undesirable traffic flows. Statefull ones attempt to do so using knowledge of the protocols' state machines. The work performed by a statefull firewall must be done, either by the ultimate endpoints (your servers, etc) or by a central enforcement point (a firewall). In other words, desirable traffic (like spice) must flow, and undesirable traffic must not impede the former. The rationale for the existence of firewalls is that you can enforce the rules of the protocols more cheaply if you move some of the enforcement into a specialized device (a statefull firewall). If you care to engineer every end node such that they can enforce the protocols' state machines in every case at every possible traffic level, you have no need for firewalls at all. At the current triple-point of threat, product, and protocol, separation of function is currently a useful method, nothing more. David On Tue, Jan 5, 2010 at 12:16 PM, Brian Johnson <bjohnson@drtel.com> wrote:
Security Gurus, et al,
I have my own idea of what a firewall is and what it does. I also understand what statefull packet inspection is and what it does. Given this information, and not prejudging any responses, exactly what is a firewall for and when is statefull inspection useful?
Please respond on-list as I want to have some useful discourse and discussion in the clear. Flamers and Trolls will be disregarded. :)
Thank you.
- Brian
CONFIDENTIALITY NOTICE: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, copying, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Thank you.
-----Original Message----- From: David Hiers [mailto:hiersd@gmail.com] Sent: Wednesday, January 06, 2010 10:50 AM To: Brian Johnson Cc: nanog@nanog.org Subject: Re: I don't need no stinking firewall!
Poking the dragon a bit, aren't you? Fun.
If you really look at it, there is no quantitative difference between statefull and non-statefull. A non-stateful firewall can prevent a TCP session from entering the SYN_RECEIVED state by blocking the SYN packet, so it strongly impacts session state without really trying. A statefull firewall will venture a bit deeper into the state diagram with a few more rules, but this is mostly a quantitative difference when viewed at a behavioral level -snip-
David
+1 As mentioned before, the line has substantially blurred with what current devices (routers/load balancers) can do in hardware. Brandon L. On Tue, Jan 5, 2010 at 12:16 PM, Brian Johnson <bjohnson@drtel.com> wrote:
Security Gurus, et al,
I have my own idea of what a firewall is and what it does. I also understand what statefull packet inspection is and what it does. Given this information, and not prejudging any responses, exactly what is a firewall for and when is statefull inspection useful?
Please respond on-list as I want to have some useful discourse and discussion in the clear. Flamers and Trolls will be disregarded. :)
Thank you.
- Brian
CONFIDENTIALITY NOTICE: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, copying, use, disclosure, or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Thank you.
participants (44)
-
Arie Vayner -
bill from home -
Bill Stewart -
Brandon M. Lapointe -
Brian Johnson -
Brian Keefer -
Brielle Bruns -
Bruce Curtis -
David Hiers -
Dobbins, Roland -
Fred Baker -
George Bonser -
harbor235 -
Henry Yen -
James Hess -
Jared Mauch -
Jason Shearer -
Jay Hennigan -
Joe Greco -
Joe Maimon -
Joel Jaeggli -
Jonathan Lassoff -
Jorge Amodio -
juttazalud -
Kenny Sallee -
Kevin Oberman -
Mark Foster -
Mark Smith -
Michael K. Smith -
Peter Hicks -
Randy Bush -
Rich Kulawiec -
Robert Brockway -
Ryan Brooks -
Sean Donelan -
Simon Lockhart -
Tim Durack -
Tony Finch -
Valdis.Kletnieks＠vt.edu -
Warren Kumari -
William Herrin -
William Herrin -
William Pitcock -
William Waites