On Sep 13, 2012, at 7:37 AM, Jeroen Massar <jeroen@unfix.org> wrote:

On 2012-09-13 16:29 , Jay Ashworth wrote: [..]

...

If not, do any of the people who've already done have 5 minutes to chime in on what they did and what they learned?

You might want to go through the network presentations given for IETF, NANOG/ARIN and last but definitely not least: CCC congress + camps.

eg: http://events.ccc.de/camp/2007/Fahrplan/events/2043.en.html

Typically though it requires people who have done it before with proper equipment to get a network up and running properly ;)

Greets, Jeroen

I know someone who did Interop's networking for a number of years and does it for various non-Worldcon conventions. His short summary was to stage and label and debug and test extensively beforehand, even if the reassembly might introduce more bugs in the field. George William Herbert Sent from my iPhone

-----Original Message----- From: Jay Ashworth [mailto:jra@baylink.com] Sent: Thursday, September 13, 2012 11:44 AM To: NANOG Subject: Re: Big Temporary Networks ----- Original Message -----

From: "George Herbert" <george.herbert@gmail.com>

I know someone who did Interop's networking for a number of years and does it for various non-Worldcon conventions. His short summary was to stage and label and debug and test extensively beforehand, even if the reassembly might introduce more bugs in the field.

Excellent advice. But how do you load a wireless network and an uplink with 12-14k attachments for testing purposes? I can see how to test the uplink, but testing the WLAN seems ... well, next to impossible, to me, which is why I'm querying the list. :-) Cheers, -- jra I'm not sure if this is obvious for this list or not, but with your WiFi nodes, a good practice for that kind of density is more nodes, lower power. Keep the client connection load per AP as low as possible to improve overall performance. Jacking up the power in a small area like that will just step on the adjacent APs and cause issues. Dylan

----- Original Message -----

From: "Måns Nilsson" <mansaxel@besserwisser.org>

04:05:41PM +0000 Quoting Dylan Bouterse (dylan@corp.power1.com):

...

I'm not sure if this is obvious for this list or not, but with your WiFi nodes, a good practice for that kind of density is more nodes, lower power. Keep the client connection load per AP as low as possible to improve overall performance. Jacking up the power in a small area like that will just step on the adjacent APs and cause issues.

It was. :-) Of course, the propery may (read: probably does) have its own conference areas and residential floors wifi, and those may or may not be V-WLAN capable.

An enterprisey AP flock that perhaps even can talk to eachother about power levels is a must.

At all possible cost, avoid login or encryption for the wireless.

Yes, and no.

Captive portals suck, especially if they try to be clever and keep an eye on the link-state to each client. Tablets and smartphones turn their radios off to conserve battery, and that means having to login all the time.

My plan is to have 3 VWLANs: worldcon-guests, which will have one-time captive portal; I want the controller to remember the MAC address everywhere, all week worldcon-dealers, no captive portal (for credit card and other embedded machines), and worldcon-staff, which may have some relaxed outbound security compared to the other networks. (For example, I have no problems blocking outbound port 25 and redirecting recursive DNS -- though I do want a system that permits me to whitelist MACs on request. But I would do those on the guest and dealer nets, and not on the staff one.)

While things have become much better, doing 802.1x on conference wireless probably is a bit daring. OTOH eduroam does it all over Europe.

If I did try to do that, it would probably only be on the staff network; it's a much more contrained environment.

Get lots of IP addresses. A /16 probably still can be borrowed for this kind of event. I know RIPE had rules and addresses for this kind of use a couple years ago, at least.

Indeed? I did not see that coming. Hell, perhaps Interop could be talked into loaning me a /16. :-)

And get v6.

Yeah, I assumed that, though it will be interesting to see how much play it actually gets; these are SF geeks, not networking geeks.

Do not NAT. When all those people want to do social networking to the same furry BBS while also frequenting three social app sites simultaneously you are going to get Issues if you NAT. So don't. (Keep in mind that the 5-tuple for each TCP connection more often will become a 3-tuple if the demographic of the user base is skewed towards a focus group and NAT is in use. )

This, right here, is the kind of gritty advice that brought me to ask this question in the first place. You're right; NAT is Right Out; forget what I said earlier. :-)

Lots of IP adresses will also enable you to set sensible DHCP lease times on the failover-connected (because they are, right?) DHCP servers. Nothing is so detrimental to connectivity experience as lost leases from either crashed DHCP servers or short lease times. Be very thorough and careful in setting DHCP up. It'll pay off.

Oh yeah. I'm fond of leases as short as 30 minutes, though if I have a /16, I won't care as much.

Have DNS resolvers locally. Unbound is good. As is BIND.

Yep, with lots of RAM on the boxes.

It might be a good idea to have reverse DNS delegation set up, perhaps via the BIND $GENERATE directive; just something like wireless-node-47-11.world.con will do.

Hmmm.

Make sure that the whois contacts for the address block are proper.

Well, I do have 3 years to plan. :-)

Try setting some monitoring up; it is good to be able to keep an eye on client count per AP etc. This is also much easier if the wireless solution is enterprisey.

I was planning on having a NOC, yes, albeit small. Very nice, Måns; thanks. Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274

Subject: Re: Big Temporary Networks Date: Fri, Sep 14, 2012 at 12:20:33AM -0700 Quoting Octavio Alvarez (alvarezp@alvarezp.ods.org):

I'd have expected someone to have QoS mentioned already, mainly to put FTP and P2P traffic on the least important queues and don't hog up the net.

As long as there is no multicast entering the wlan this is best solved by getting more bandwidth. -- Måns Nilsson primary/secondary/besserwisser/machina MN-1334-RIPE +46 705 989668 ... the HIGHWAY is made out of LIME JELLO and my HONDA is a barbequeued OYSTER! Yum!

Once upon a time, Jay Ashworth <jra@baylink.com> said:

Well, we'll be on the *sending* end of the Hugo's, but... ;-)

You might want to talk to whoever did this year's WorldCon networking. I'm a Dragon*Con volunteer, and I know there was a some type of direct connection between Chicago (WorldCon) and Atlanta (Dragon*Con) so that we could show things like the Hugo ceremony (and I think we fed some video the other way as well, like the D*C Parade). I don't know how they did the "regular" Internet stream (except that it went through Ustream, who shut down the Hugo feed because of a DMCA complaint). -- Chris Adams <cmadams@hiwaay.net> Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble.

Once upon a time, Jay Ashworth <jra@baylink.com> said:

My understanding was that Dragon *took its main feed* for the Hugos via Ustream, and the entire room got left standing; no?

I don't know; I wasn't in there, and I didn't find out about the Ustream cut until I was home. I would think I would have heard if the feed was cut though (I wasn't involved, but I work Techops and know the people involved). -- Chris Adams <cmadams@hiwaay.net> Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble.

Once upon a time, Jay Ashworth <jra@baylink.com> said:

Noted. How big is that crew for Dragon; you were, what, 30k attendees?

The estimate I heard was 52,000-55,000 paid attendees this year (plus another 3,000+ for volunteers, guests+spouse/agent/etc., press, etc.). Our Techops staff was around 240-250 this year. -- Chris Adams <cmadams@hiwaay.net> Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble.

----- Original Message -----

From: "Måns Nilsson" <mansaxel@besserwisser.org>

...

It would still be nice to multicast them inside our network (and out to whomever wants to watch), but what the heck's the consumer-level client side of multicast video streaming look like these days?

IIRC a number of IETF meetings were badly maimed in the wireless side until people remembered that the WLAN was no place for multicast video. VLC does mcast really nice. We're testing it for replacing antenna distribution systems and DVB-T receivers at work.

Then you want to know that the HD HomeRun people, Silicon Dust, have versions of their tuners that will generate multicast, I would suspect. You're saying that *receiving* multicast streams over WLAN works poorly? Can you expand on that? Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274

----- Original Message -----

From: "William Herrin" <bill@herrin.us>

On Sat, Sep 15, 2012 at 9:18 PM, Jay Ashworth <jra@baylink.com> wrote:

...

You're saying that *receiving* multicast streams over WLAN works poorly?

I don't have any experience with it, but here's what Google told me:

http://www.wireless-nets.com/resources/tutorials/802.11_multicasting.html

"When any single wireless client associated with an access point has 802.11 power-save mode enabled, the access point buffers all multicast frames and sends them only after the next DTIM (Delivery Traffic Indication Message) beacon, which may be every one, two, or three beacons (referred to as the “DTIM interval”). [...] default 100 millisecond beacon interval"

Thanks for doing my googling for me, Bill. :-) I'll do some more; I would sort've expect that might be something the firmware in enterprise-class APs would handle better (where by better, I mean "not permitting one client to manhandle the entire network" :-). Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274

----- Original Message -----

From: "Masataka Ohta" <mohta@necom830.hpcl.titech.ac.jp>

Jay Ashworth wrote:

...

You're saying that *receiving* multicast streams over WLAN works poorly?

Multicast/broadcast over congested WLAN works poorly, because there can be no ACK.

That is, multicast/broadcast packets lost by collisions are never sent again.

Well, yes, but that wasn't what Bill was talking about. He was talking about AP's being "nice" to associated clients who are in powersave mode, at the expensive of all the other connected clients, by buffering multicast packets until one or more DTIM frames are sent. We expect live streams to drop a packet here and there; that's what buffering is for... and why television proper still exists. Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274

On 16/09/2012 19:30, Masataka Ohta wrote:

Thus, protocols heavily depending on broadcast/multicast, such as ND, will suffer.

ok, you've trolled me enough to ask why ND is worse than ARP on a wavelan network - in your humble opinion? Nick

On 17/09/2012 00:42, Masataka Ohta wrote:

OTOH, IPv6 requires many multicast received by STAs: RA and NS for DAD, for example.

Worse, minimum intervals of ND messages are often very large, which means a lot of delay occurs when a message is lost.

So, what you're saying here is that a wifi network with lots of packet loss will cause connectivity problems with ipv6? Nick

On Sun, Sep 16, 2012 at 7:42 PM, Masataka Ohta <mohta@necom830.hpcl.titech.ac.jp> wrote:

ARP and DHCP usually work.

For an unusual case of ARP for other STAs, collisions do increase initial latencies, but as refreshes are attempted several times, there will be no latter latencies.

OTOH, IPv6 requires many multicast received by STAs: RA and NS for DAD, for example.

Worse, minimum intervals of ND messages are often very large, which means a lot of delay occurs when a message is lost.

Hi Masataka, Where do things go wrong? As I understand it from your description, we're mostly talking about data between a wifi station and remote servers somewhere off the wired side of the network. Wifi station to station communications comprises a relatively minor portion of wifi's use so we don't burn a lot of worry on them in the general analysis. In the wifi to remote wired case, all IPv4 traveling the wifi network is subject to layer-2 error recovery except for the ARP packet from the default gateway to the station, requesting the station's MAC address. This works out OK because the default gateway is somewhat noisy about resending that arp request until it gets a response from the station and then it caches the response for a long time. In IPv6, the station sends an ICMPv6 router solicitation instead of an ARP for the default gateway. This is a multicast message but since it's from the station to the AP it's subject to layer 2 error recovery by the 802.11 protocol. The default gateway sends back a router advertisement (unicast since its responding to a solicitation) with prefix info, its MAC, its IP address, etc. Unicast = layer 2 error recovery. It then configures its address and sends packets to the default gateway. In the reverse direction, the gateway sends a neighbor solicitation via multicast looking for the MAC association with the station's IP. Like the ARP broadcast this is not subject to layer-2 error recovery. When the station eventually receives one of the repeated solicitations, it responds with a neighbor advertisement to the default gateway (station to AP, error recovered) which the default gateway caches for a while. In terms of the number and nature of packets sent without wifi's layer 2 error recovery, they look the same, at least in theory. What did I miss? Where does IPv6 take the bad turn that IPv4 avoided? Thanks, Bill Herrin -- William D. Herrin ................ herrin@dirtside.com bill@herrin.us 3005 Crane Dr. ...................... Web: <http://bill.herrin.us/> Falls Church, VA 22042-3004

On Tue, Sep 18, 2012 at 8:16 AM, Masataka Ohta <mohta@necom830.hpcl.titech.ac.jp> wrote:

William Herrin wrote:

...

In IPv6, the station sends an ICMPv6 router solicitation instead of an ARP for the default gateway. This is a multicast message but since it's from the station to the AP it's subject to layer 2 error recovery by the 802.11 protocol. The default gateway sends back a router advertisement (unicast since its responding to a solicitation)

Unicast since its responding to a solicitation?

RFC4861 states:

A router MAY choose to unicast the response directly to the soliciting host's address (if the solicitation's source address is not the unspecified address), but the usual case is to multicast the response to the all-nodes group.

Ah, okay. So the IPv6 router usually responds to router discovery with multicast where arp would have responded with unicast. This multicast message is not subject to 802.11's layer 2 error recovery so as previously discussed it has a high probability of being lost during some relatively ordinary wifi usage scenarios. But correct me if I'm wrong: the router advertisement daemon could be altered to reply with unicast without changing the standard, right? What do the radvd and rtadvd developers say about this when confronted with the 802.11 multicast problem? Are there any Internet drafts active in the IETF to replace that "MAY" with a "SHOULD," noting that replying with multicast can defeat layer 2 error recovery needed for the successful use of some layer 1 media?

...

What did I miss? Where does IPv6 take the bad turn that IPv4 avoided?

If you still want to defend IPv6, you must say multicast RA and DAD are unnecessary features of IPv6, which means the current IPv6 is broken.

I have no interest in defending IPv6. We're network operators here. You just told us (and offered convincing reasoning) that when selecting a router vendor for use with an IPv6 wifi network, one of our evaluation check boxes should should be, "Responds to ICMPv6 router solicitation with a unicast message? Yes or Fail." And when we provide the list of deficiencies to our vendor and wave the wad of cash around, one of them should be, "Responds to ICMPv6 router solicitations with a multicast packet - unreliable in a wifi environment." That's strikes me as something valuable to know. Far more valuable than, "Dood, IPv6 has problems on wifi networks." So, let's keep going. IPv6 falls down compared to IPv4 on wifi networks when it responds to a router solicitation with a multicast (instead of unicast) router advertisement. Where else does it fall down compared to the equivalent behavior in an IPv4 wifi network? Regards, Bill Herrin -- William D. Herrin ................ herrin@dirtside.com bill@herrin.us 3005 Crane Dr. ...................... Web: <http://bill.herrin.us/> Falls Church, VA 22042-3004

On Tue, Sep 18, 2012 at 4:31 PM, Nick Hilliard <nick@foobar.org> wrote:

On 18/09/2012 21:24, William Herrin wrote:

...

IPv6 falls down compared to IPv4 on wifi networks when it responds to a router solicitation with a multicast (instead of unicast) router advertisement.

You mean it has one extra potential failure mode in situations where radio retransmission doesn't deal with the packet loss - which will cause RA to retry. "Fall down" is a slight overstatement.

Potayto, potahto. Like I said, I have no interest in defending IPv6. But I'm very interested in how to implement an IPv6 network that's as or more reliable than the equivalent IPv4 network. That makes me interested in the faults which get in the way. Regards, Bill Herrin -- William D. Herrin ................ herrin@dirtside.com bill@herrin.us 3005 Crane Dr. ...................... Web: <http://bill.herrin.us/> Falls Church, VA 22042-3004

Seth Mos wrote:

Yes, radvd has a configuration option to send unicast packets. But I think the effects are slightly overstated.

A senario considered by IEEE11ai is that a very crowded train arrives at a station and all the smart phones of passengers try to connect to APs. Then, it is essential to reduce the number of control packet exchanges.

Also note that some wifi networks eat DHCPv4 broadcasts too,

As I already stated, DHCP discover/request from STA to AP is unicast.

And if the wifi network is so bad that you have icmp and udp dropping

I'm afraid you don't understand CSMA/CA at all. Masataka Ohta

Sean Harlow wrote:

...

As I already stated, DHCP discover/request from STA to AP is unicast.

This didn't sound right, so I decided to test.

Your test is invalid.

With the three clients available to me (laptop running OS X 10.7.4, phone running Android 4.0, and iPod running iOS 4.1.2) all client->server DHCP was broadcast

Of course. However, at WiFi L2, it is first unicast to AP and then broadcast by the AP. Masataka Ohta

William Herrin wrote:

...

Unicast since its responding to a solicitation?

RFC4861 states:

A router MAY choose to unicast the response directly to the soliciting host's address (if the solicitation's source address is not the unspecified address), but the usual case is to multicast the response to the all-nodes group.

Ah, okay. So the IPv6 router usually responds to router discovery with

Don't ignore how is the implementations in the real world: : and a comment in rtadvd on the solicited advertisement: : : /* : * unicast advertisements : * XXX commented out. reason: though spec does not forbit it, unicast : * advert does not really help

But correct me if I'm wrong: the router advertisement daemon could be altered to reply with unicast without changing the standard, right?

See above.

What do the radvd and rtadvd developers say about this when confronted with the 802.11 multicast problem?

I reported the problem to IPv6 (or IPng?) WG more than 10 years ago (before rtadvd was developed) and Christian Huitema acknowledged that the problem does exist. Since then, nothing happened.

Are there any Internet drafts active in the IETF to replace that "MAY" with a "SHOULD," noting that replying with multicast can defeat layer 2 error recovery needed for the successful use of some layer 1 media?

Didn't you say "without changing the standard"?

...

...

What did I miss? Where does IPv6 take the bad turn that IPv4 avoided?

You still miss DAD. DupAddrDetectTransmits should be 3, 5 or maybe 10 (depending on level of congestion), which means even more time is wasted. Worse, increasing DupAddrDetectTransmits increases level of congestion, which means congestion collapse occurs with use case senario of IEEE802.11ai.

I have no interest in defending IPv6. We're network operators here. You just told us (and offered convincing reasoning) that when selecting a router vendor for use with an IPv6 wifi network, one of our evaluation check boxes should should be, "Responds to ICMPv6 router solicitation with a unicast message? Yes or Fail." And when we provide the list of deficiencies to our vendor and wave the wad of cash around, one of them should be, "Responds to ICMPv6 router solicitations with a multicast packet - unreliable in a wifi environment."

That's strikes me as something valuable to know. Far more valuable than, "Dood, IPv6 has problems on wifi networks."

The only thing operators have to know about IPv6 is that IPv6, as is currently specified, is not operational. Then, let IETF bother. Masataka Ohta

TJ wrote:

...

The only thing operators have to know about IPv6 is that IPv6, as is currently specified, is not operational.

I think it is safe to say that this is provably false.

You failed to do so.

Are there opportunities for increased efficiency, perhaps ... however:

Congestion collapse is not a matter of mere efficiency.

I get native IPv6 at home via my standard residential cable connection using off the shelf CPE gear and standard OSes. I get native IPv6 via my standard LTE devices, again - off the shelf - no customization required.

That IPv6 works fine sometimes in some environment is not a valid proof that IPv6 is operational. Purposelessly bloated specification of IPv6 cause problems in some environment, against which removal of features is the only cure. It's like not using IP options, even though they are defined in RFC791. Masataka Ohta

William Herrin wrote:

I think Masataka meant to say (and said previously) that the DHCP request from the wifi station is, like all packets from the wifi station to the AP, subject to wifi's layer 2 error recovery. It's not unicast but its subject to error recovery anyway.

Mostly correct. But, as I already wrote: 1) broadcast/multicast from a STA attacked to an AP is actually unicast to the AP and reliably received by the AP (and relayed unreliably to other STAs). That is, a broadcast ARP request from the STA to the AP is reliably received by the AP. Because of hidden terminals, L2 broadcast/multicast is transmitted only from AP.

...

However, at WiFi L2, it is first unicast to AP and then broadcast by the AP.

Your use of nomenclature is incorrect. It'd be like saying my ethernet

Ethernet?

card unicasts a packet to the switch and then the switch broadcasts it out all ports. Or like saying that a packet with an explicit MAC destination

Do you know MAC header of 802.11 contains four, not just source and destination, MAC addresses? Because of hidden terminals and because of impossibility of collision detection, WLAN is a little more complex than your guess.

No offense, but it is not for you or I or Owen Delong to declare that IPv6 is or isn't operational.

A single counter example is enough to deny IPv6 operational.

whether and when IPv6 is sufficiently operational for their use.

The scope is not "their use" but "as a protocol for the entire Internet". Masataka Ohta

TJ wrote:

...

A single counter example is enough to deny IPv6 operational.

Really?

With the Internet wide scope, yes, of course. In general, as IPv6 was designed to make "ND uber Alles", not "IP uber Alles", and ND was designed by a committee with only ATM, Ethernet and PPP in mind, ND can not be an adaptation mechanism to run IP over various link with link specific properties. Thus, even though people only using Ethernet and PPP might think ND is good enough, a single example of a link is enough to deny "ND uber Alles". Though you wrote:

I think it is safe to say that this is provably false.

it is impossible because it is "probatio diabolica". Instead, a single counter example is enough to totally deny "probatio diabolica". Or, if you need another example on how poorly ND behaves under some environment, it's timing constraints are specified mostly in units of "second", not "millisecond", because the IPv6 committee silently assumed that hosts are immobile. Thus, latency imposed by ND is often too large for links with quickly moving objects. Never claim IPv6 operational with your narrowly scoped experiences, because what you are attempting to do is "probatio diabolica".

That is what the ~"IPv6 over Foo" series of documents is all about, accommodating those needs ...

Because "ND uber Alles" is impossible, "IPv6 over Foo" series specifying ND parameters are not helpful. Masataka Ohta

David Miller wrote:

So, a single example of IPv4 behaving in a suboptimal manner would be enough to declare IPv4 not operational?

For example? Masataka Ohta

TJ wrote:

...

...

So, a single example of IPv4 behaving in a suboptimal manner would be enough to declare IPv4 not operational?

For example?

"Heavy reliance on broadcast for a wide range of instances where the traffic is really only destined for a single node" would seem to be rather sub-optimal.

It's not sub-optimal w.r.t. link bandwidth, if the link is a broadcast domain. Moreover, broadcast is no worse than all-node multicast. And, given the CATENET model of the Internet to connect broadcast domains including small number of devices by routers, over which there is no broadcast, that is a sub-optimal operation. In this thread, there is an example of such an operation to have a lot of WiFi base stations with omnidirectional antennas at full power. No protocol can be fool proof against sub-optimal operations. Masataka Ohta

Tony Hain wrote:

...

...

So, a single example of IPv4 behaving in a suboptimal manner would be enough to declare IPv4 not operational?

For example?

Your own example ---

...

... that a very crowded train arrives at a station and all the smart phones of passengers try to connect to APs ...

IPv4 has a train load of devices unicasting and retransmitting all the dropped packets,

IPv4 merely need a broadcast ARP request and broadcast DHCP discover to be piggy backed in a single IEEE802.11ai frame reliably unicast to an AP.

where an IPv6 multicast RA allows all the devices to configure based on reception of a single packet.

You miss multicast storm caused by DAD.

Just because you personally want IPv6 to be nothing more than IPv4 in every aspect is no reason to troll the nanog list and create confusion that causes others to delay their IPv6 deployment.

Just because IPv6 is working without much problem somehow somewhere is not a valid reason to say others should use IPv6. As IP is so essential to the Internet, IP protocol must be perfect w.r.t. the scale and diversity of the Internet. IPv4 has evolved so, as the Internet evolve. IPv6 could have been so, if it were a exact copy of IPv4 save address length. But, it is not, which is why IPv6 failed on various points which are different from IPv4.

Your complaints about IPv6 behavior on wifi ignore the point that IPv6 ND behavior was defined before or in parallel as wifi was defined by a different committee.

The problem is "ND uber Alles" approach, which makes it impossible to make "IP uber Alles".

There will always be newer L2 technologies that arrive after an L3 protocol is defined, and the behavior of the L3 will be 'suboptimal' for the new L2. When the issue is serious enough to warrant documentation, addendum documents are issued.

Because of "ND uber Alles" approach, the document just says "IPv6 works suboptimal" without solving the issue. OTOH with IPv4, the document can solve the problem by having a new adaptation mechanism much different from ARP or PPP. Masataka Ohta

William Herrin wrote:

...

You miss multicast storm caused by DAD.

Second, in the hotspot scenarios where this is likely to be a problem (in IPv4 -or- IPv6) it's addressed by the "AP isolation" feature

As you stated : I think Masataka meant to say (and said previously) that the DHCP : request from the wifi station is, like all packets from the wifi : station to the AP, subject to wifi's layer 2 error recovery. that is not a problem for IPv4 ARP and DHCP.

that's getting close to omnipresent even in the low end APs. With this feature enabled, stations are not allowed to talk to each other over the wlan; they can only talk to hosts on the wired side of the lan. The DAD packets are simply never sent to the other stations.

You are saying to disable DAD, which is a violation of SLAAC.

In theory there are some problems with this. In practice, it's in wide deployment and has been demonstrated to work just fine.

Tell it to IETF to modify SLAAC to exclude DAD. Masataka Ohta

William Herrin wrote:

...

You are saying to disable DAD, which is a violation of SLAAC.

We do that on some wired ethernets too.

You are calling such a link Ethernet. OK. Fine.

The Cisco configuration command is "switchport protected." It helps control virus outbreaks if machines designated clients can't talk to each other at layer 2, regardless of how that annoys layer 3.

It means IPv6 is broken over not only WiFi but also Ethernet.

Does this bother you?

No, not me, not at all. Masataka Ohta

On Sun, Sep 23, 2012 at 3:50 PM, JÁKÓ András <jako.andras@eik.bme.hu> wrote:

...

Second, in the hotspot scenarios where this is likely to be a problem (in IPv4 -or- IPv6) it's addressed by the "AP isolation" feature that's getting close to omnipresent even in the low end APs. With this feature enabled, stations are not allowed to talk to each other over the wlan; they can only talk to hosts on the wired side of the lan.

Not related to the original subject, neither to IPv6 usability on WLANs, just a small comment: As far as I understand "AP isolation" doesn't work if you don't have a WLAN controller but do have more than one APs. E.g. in the following setup

ap1--sw1--sw2--ap2

with "AP isolation" turned on, clients associated to ap1 cannot communicate directly with other clients associated to ap1, however they can communicate directly with those associated to ap2. Broadcast from ap1's clients does also get to all clients at ap2.

Hi András, This is one place where Cisco's "switchport protected" comes in handy. Plug both APs into switches where the port is set to protected mode and neither they nor the associated clients will be able to talk to each other. You can get the same effect with other brands. For example, in one on-the-cheap 5-AP hotspot I did, I vlaned the APs (using an older 802.1q capable switch) back to a Linux bridge with "ebtables --insert FORWARD --jump DROP". The Linux bridge was also the default router out of the wlan, so anything *to* the router worked but anything that would be forwarded was dropped instead. Works great. Regards, Bill Herrin -- William D. Herrin ................ herrin@dirtside.com bill@herrin.us 3005 Crane Dr. ...................... Web: <http://bill.herrin.us/> Falls Church, VA 22042-3004

Subject: Re: Big Temporary Networks Date: Thu, Sep 13, 2012 at 05:45:55PM -0400 Quoting Jay Ashworth (jra@baylink.com):

----- Original Message -----

...

At all possible cost, avoid login or encryption for the wireless.

Yes, and no.

<snip> Just keep in mind that every action you make the visitors have to perform to get Internet connectivity is a support workload.

(For example, I have no problems blocking outbound port 25 and redirecting recursive DNS -- though I do want a system that permits me to whitelist MACs on request. But I would do those on the guest and dealer nets, and not on the staff one.)

Remember that DNSSEC breaks quite easily if you redirect DNS and since this is three years in the future, the uptake on DNSSEC may well have hit the point where there is visual feedback on validation in client UI.

...

While things have become much better, doing 802.1x on conference wireless probably is a bit daring. OTOH eduroam does it all over Europe.

If I did try to do that, it would probably only be on the staff network; it's a much more contrained environment.

It'll work much better there, and FWIW, will be a little yet perhaps effective speedbump for intruders.

...

And get v6.

Yeah, I assumed that, though it will be interesting to see how much play it actually gets; these are SF geeks, not networking geeks.

Again, even in North America, the uptake may well have accelerated enough that it is To Be Expected. Besides, IME, SF geeks are computer savvy more than others.

Oh yeah. I'm fond of leases as short as 30 minutes, though if I have a /16, I won't care as much.

A couple hours will get the user over a lunch break if not overnight, which means that long TCP sessions survive on Proper Computers (that don't tear down TCP on link loss. I'm looking at you, Microsoft!). This is Really Nice. Open up computer from sleep and press enter in xterm and ssh session is up. (my personal record is for telnet, an untouched connection survived two taxi trips, one night, some NATed wlan at the hotel and when i got back to the right network I just plugged the cable in and continued in the same session. But I cheated and had fixed addresses.)

Very nice, Måns; thanks.

My pleasure. -- Måns Nilsson primary/secondary/besserwisser/machina MN-1334-RIPE +46 705 989668 He is the MELBA-BEING ... the ANGEL CAKE ... XEROX him ... XEROX him --

----- Original Message -----

From: "Måns Nilsson" <mansaxel@besserwisser.org>

05:45:55PM -0400 Quoting Jay Ashworth (jra@baylink.com):

...

----- Original Message -----

...

At all possible cost, avoid login or encryption for the wireless.

Yes, and no.

<snip>

Just keep in mind that every action you make the visitors have to perform to get Internet connectivity is a support workload.

I understand entirely. That was the reason for my "remember each MAC address for the entire event" approach to captive portal. I forsee the guests entering a code from their event badge the first time they use each device. Unlike most events, I also forsee a single page "How to use our Internet connectivity" sheet that actually tells you what you need to know. :-)

...

(For example, I have no problems blocking outbound port 25 and redirecting recursive DNS -- though I do want a system that permits me to whitelist MACs on request. But I would do those on the guest and dealer nets, and not on the staff one.)

Remember that DNSSEC breaks quite easily if you redirect DNS and since this is three years in the future, the uptake on DNSSEC may well have hit the point where there is visual feedback on validation in client UI.

Good point.

...

...

While things have become much better, doing 802.1x on conference wireless probably is a bit daring. OTOH eduroam does it all over Europe.

If I did try to do that, it would probably only be on the staff network; it's a much more contrained environment.

It'll work much better there, and FWIW, will be a little yet perhaps effective speedbump for intruders.

Was my plan, yes. This isn't, really, defcon. :-)

...

...

And get v6.

Yeah, I assumed that, though it will be interesting to see how much play it actually gets; these are SF geeks, not networking geeks.

Again, even in North America, the uptake may well have accelerated enough that it is To Be Expected. Besides, IME, SF geeks are computer savvy more than others.

I've heard that asserted. I'm not certain to what extent it's actually true.

...

Oh yeah. I'm fond of leases as short as 30 minutes, though if I have a /16, I won't care as much.

A couple hours will get the user over a lunch break if not overnight, which means that long TCP sessions survive on Proper Computers (that don't tear down TCP on link loss. I'm looking at you, Microsoft!).

Well, I'm a firm believer in Least Recently Used, so as long as my DHCP block is larger than my userbase, everyone will have the same address all weekend anyway.

This is Really Nice. Open up computer from sleep and press enter in xterm and ssh session is up. (my personal record is for telnet, an untouched connection survived two taxi trips, one night, some NATed wlan at the hotel and when i got back to the right network I just plugged the cable in and continued in the same session. But I cheated and had fixed addresses.)

Nice. :-) Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274

On Fri, 14 Sep 2012, Brandon Ross wrote:

On Thu, 13 Sep 2012, Jay Ashworth wrote:

...

...

Get lots of IP addresses. A /16 probably still can be borrowed for this kind of event. I know RIPE had rules and addresses for this kind of use a couple years ago, at least.

Indeed? I did not see that coming. Hell, perhaps Interop could be talked into loaning me a /16. :-)

<dons Interop hat>You might think you are joking, but if it doesn't overlap with an existing commitment, we can probably make that happen.</dons>

I don't know about last /8 policy and how that will change this, but so far I have seen little problem getting a temporary /16 or alike for events from RIPE. -- Mikael Abrahamsson email: swmike@swm.pp.se

On 14 September 2012 11:16, Nick Hilliard <nick@foobar.org> wrote:

On 13/09/2012 21:32, Måns Nilsson wrote:

...

Get lots of IP addresses. A /16 probably still can be borrowed for this kind of event. I know RIPE had rules and addresses for this kind of use a couple years ago, at least.

yes, you can get a bunch of IP addresses from the ripe ncc if you only need them on a temporary basis:

http://www.ripe.net/ripe/docs/ripe-526

We tried to apply using this policy to get address space for EMFCamp, no good in reality. The RIPE hostmaster would only allocate us address space 7 days before the event started, needed longer than this to begin building out the network which span multiple data centres. Especially with time, access and change freeze constraints due to the Olympics this year. They didn't seem to want to budge on this, easier in my opinion to borrow some off a friendly organisation or ISP than jump hoops with RIPE. Only other option would be to build your infra out in an existing spare /24 you can get hold of - put router loopbacks, point to points etc in there. Then a week before the event attempt to get the larger /19 assignment from RIPE to put your clients in. I wouldn't be happy doing that though, as in my opinion it doesn't leave enough time for any reachability testing / debugging. -- Nat http://natmorris.co.uk http://twitter.com/natmorris

On 14 September 2012 11:54, Nick Hilliard <nick@foobar.org> wrote:

On 14/09/2012 11:50, Nat Morris wrote:

...

The RIPE hostmaster would only allocate us address space 7 days before the event started, needed longer than this to begin building out the network which span multiple data centres. Especially with time, access and change freeze constraints due to the Olympics this year. They didn't seem to want to budge on this, easier in my opinion to borrow some off a friendly organisation or ISP than jump hoops with RIPE.

I'm in the process of trying to get this changed. To be completely fair on the RIPE NCC, they don't have flexibility on this issue - the original policy was broken.

This is good news Nick :) I have spoken to others in the past few weeks who were hoping to raise it at the next RIPE meeting. I am happy to share our ticket details with you off list if it'll help. -- Nat http://natmorris.co.uk http://twitter.com/natmorris

On Fri, 14 Sep 2012, Tore Anderson wrote:

It's actually a /13 (151.216.0.0/13).

It used to be in another place (I don't remember exactly, this was 5-8 years ago). Nice that they have a /13 nowadays anyway, I'd imagine there are more temporary events nowadays. I've used it a couple of times and then a week was sufficient (start rigging on monday, everything done by thursday morning where 5000 people show up with their computers (this was mainly 10/100 ports, people brought their own cables), teardown and turning off the network, and then returning the space to RIPE on monday. -- Mikael Abrahamsson email: swmike@swm.pp.se

On Fri, 14 Sep 2012, Nick Hilliard wrote:

Also, 1 week is not suitable for debogonisation.

Could you please elaborate on this aspect? Who would be treating this space as a bogon, and why? -- Mikael Abrahamsson email: swmike@swm.pp.se

On 14/09/2012 12:38, Paul Thornton wrote:

Veering slightly off-topic for NANOG, but is this worth taking onto the address policy mailing list ahead of RIPE65 to ensure people who aren't in the WG session are aware of the issue - and can therefore support (or question) any proposed changes?

I just posted to ap-wg about this:

http://www.ripe.net/ripe/mail/archives/address-policy-wg/2012-September/0071...

Nick

Subject: Re: Big Temporary Networks Date: Fri, Sep 14, 2012 at 09:22:01PM +0900 Quoting Masataka Ohta (mohta@necom830.hpcl.titech.ac.jp):

Måns Nilsson wrote:

...

And get v6.

Do not NAT. When all those people want to do social networking to the same furry BBS while also frequenting three social app sites simultaneously you are going to get Issues if you NAT. So don't.

Don't?

Considering that, ten years ago, some computers were still often shared by thousands of people distinguished by their port numbers and that, today, pseudo ISPs are using NAT, it is not only wrong but also impossible to identify a user only by his IP address without port numbers.

Ohta-san, I am not suggesting that. I'm just trying to point out that there might be a bunch of assumptions that aren't as true anymore when a lot of client connections share both source and destination address, and perhaps also destination port. If this happens simultaneously when a large amount of other tcp connections are NATed through the same box, resource starvation will occur. If public address space is available, it is better to use that. Also, no NAT means there will be no session timers for things like long lived low bandwidth tcp sessions. -- Måns Nilsson primary/secondary/besserwisser/machina MN-1334-RIPE +46 705 989668 I think my career is ruined!

On 9/15/12, Masataka Ohta <mohta@necom830.hpcl.titech.ac.jp> wrote: > Mans Nilsson wrote:

...

I am not suggesting that. I'm just trying to point out that there might be a bunch of assumptions that aren't as true anymore when a lot of client connections share both source and destination address, and perhaps also destination port. If this happens simultaneously when a large amount of other tcp connections are NATed through the same box, resource starvation will occur.

Assumptions that are already broken in Enterprise networks where 100+ users may share an IP What you can use is an edge device with a large NAT table, 30000 entries at least; setup a policy that limits each private IP address to 50 simultaneous TCP connections, to protect NAT device against malware-infected laptops, You might also use a wireless AP and Ethernet switches with PVLAN (or protected port bridging) and DHCP snooping functionality, then configure to restrict attached devices from sending or receiving any kind of IP or Ethernet broadcast traffic with other hosts on the LAN, And ensure the NAT/Firewall device will not route traffic from the Inside interface back to the Inside interface, so the NAT device and DHCP server are the only units that attached nodes may communicate with. Use a short NAT timeout period for UDP (30 seconds), so there is a certain number of users that your NAT device can service, depending on its CPU power and NAT table size. You can calculate an upper bound for required NAT table capacity, based on number of users, and number of allowed connections per user. Then take your total number of users, Divide by say 20, and build a NAT pool with that many public IP addresses, For example: one /24 of public IP addresses per 5000 users. to accomplish on average 20 users per public IP, providing you pick a NAT/Firewall device balances internal private to public IP usage fairly evenly. -- -JH

WLAN in large conferences certainly is a challenge. You basically want to get as many people on 5GHz as possible due to more available channels. 2.4GHz becomes quite noisy. Also, configuring your access points for high density helps. This means disabling the lowest data rates. You also don't want to run full Tx power. Basically this will ensure high data rates and quicker handoff to a nearer AP when roaming. You don't want a client that is far away from an AP connecting to it at a 1 Megabit data rate tying up the radio. This also is key in high density seating open floorplan office situations. Sean Lazar On 9/13/12 8:44 AM, Jay Ashworth wrote:

----- Original Message -----

...

From: "George Herbert" <george.herbert@gmail.com> I know someone who did Interop's networking for a number of years and does it for various non-Worldcon conventions. His short summary was to stage and label and debug and test extensively beforehand, even if the reassembly might introduce more bugs in the field. Excellent advice.

But how do you load a wireless network and an uplink with 12-14k attachments for testing purposes? I can see how to test the uplink, but testing the WLAN seems ... well, next to impossible, to me, which is why I'm querying the list.

:-)

Cheers, -- jra

On 2012-09-13 18:32 , Tim Franklin wrote:

...

You'll need a beefy NAT box. Linux with Xeon CPU and 4GB RAM minimum.

Or not. The CCC presentation is showing *real* Internet for everyone, unless I'm very much mistaken...

No NAT was involved there indeed. Typically conferences can get a temporary prefix from their local RIR for conference-alike setups. Of course that does require one to arrange uplinks who will announce that prefix, a friendly LIR etc etc etc. Thus this all boils down on how large your setup will be and how good your want your network to perform. Greets, Jeroen

On 13 Sep 2012, at 17:32, Tim Franklin <tim@pelican.org> wrote:

...

You'll need a beefy NAT box. Linux with Xeon CPU and 4GB RAM minimum. Or not. The CCC presentation is showing *real* Internet for everyone, unless I'm very much mistaken…

Absolutely. NAT is too fragile/expensive/non-performant for these setups. CGN boxes are too new to be economically borrowed/rented, maybe one day it will be possible, but for now we can still get the address space required (Timespan issues notwithstanding) On 13 Sep 2012, at 21:03, Chris Boyd <cboyd@gizmopartners.com> wrote:

If you know of an ISP in Central Texas that can deploy a 10Mbit plus connection along with a /22 of v4 address space for a 1 day event, please let me know. TWCable has been pretty easy to work with for special events, but I'd be really surprised to see them be able to do that.

I suggest either getting a L2 circuit or else IPIP/GRE tunnel to somewhere with a functioning internet market. It is far preferable to tunnel than it is to have session state in the network. I've been part of the team deploying networking to various leafy parts of the Netherlands (e.g. HAR2009), ex-soviet airbases (CCC Camp), a park in Milton Keynes, UK (EMF2012). With some thought and creative planning it is possible to bring in a useful uplink in the 300M-10G+ range. [I'm not sure I remember those DS3s and OC3s that other posters are talking about, something these days used only in developing countries i thought ;-)] As a network engineer, these events are a great way to meet people with different experience, talk to eager young folk, do things in a different way and generally have a reset on your professional life. You might even get some sun too :-)

-----Original Message----- From: Josh Baird [mailto:joshbaird@gmail.com] Sent: Thursday, September 13, 2012 4:03 PM To: Donald Eastlake Cc: NANOG Subject: Re: Big Temporary Networks We have been using Unifi (a Ubiquiti WIFI product) for local conventions and festivals. The product is fairly cheap, robust, and their access points have very good range. We have deployed it at several commercial businesses as well with great success. The deployment is very easy. We run the controller on a VM at our NOC, but you can also run it locally at the event as well. Besides this, we have a fairly beefy box that handles DNS and DHCP and basic firewalling. Josh The UniFi line is hard to beat for the price. The controller software is free and the base access points are <$100, in fact you can get a 3-pack for <$200. Deployment of the APs would be 95% of the work. Configuration via the software would take minutes. Dylan

Yes, we backhaul our own bandwidth to it; either using Cambium or Ubiquiti unlicensed 5Ghz backhauls. Depending on the distance and type of backhaul, we can get 50-150mbps to the event. Josh On Thu, Sep 13, 2012 at 5:13 PM, Jay Ashworth <jra@baylink.com> wrote:

----- Original Message -----

...

From: "Josh Baird" <joshbaird@gmail.com>

...

We have been using Unifi (a Ubiquiti WIFI product) for local conventions and festivals. The product is fairly cheap, robust, and their access points have very good range. We have deployed it at several commercial businesses as well with great success. The deployment is very easy. We run the controller on a VM at our NOC, but you can also run it locally at the event as well.

Besides this, we have a fairly beefy box that handles DNS and DHCP and basic firewalling.

Have you had to/been able to haul in your own bandwidth to feed it? What class? (Real DS3/OC1/OC3, FiOS/HFC, something else?)

Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274

Thanks for sharing that. I just got my CCNA and find this stuff interesting. Derek On Sep 13, 2012, at 7:18 PM, Nat Morris <nat@nuqe.net> wrote:

On 13 September 2012 22:13, Jay Ashworth <jra@baylink.com> wrote:

...

----- Original Message -----

...

From: "Josh Baird" <joshbaird@gmail.com> Besides this, we have a fairly beefy box that handles DNS and DHCP and basic firewalling.

Have you had to/been able to haul in your own bandwidth to feed it? What class? (Real DS3/OC1/OC3, FiOS/HFC, something else?)

Two weekends ago EMFCamp took place north of London in Milton Keynes, the UK’s first maker weekend long festival, ran along the same lines as CCC / HAR2009 etc.

A small team of us designed the infrastructure for it, we started at the end of May, 3 months in advance. The CCC noc team in Germany were kind enough to lend us their event /19 + /48 + ASN, we built a temporary network spanning from Telehouse East in London Docklands up to a local data centre (Pulsant) in Milton Keynes.

Pulsant sponsored us with a 1gb/s L2 circuit from Telehouse to Milton Keynes, we placed a router (c7202+npe-g2) in each decenter. We took on transit in both sites and had temporary membership to LONAP in Telehouse where we connected to their route server for v4,v6 peering and even multicast.

Biggest cost was the 2 mile link from the dc back to the festival site, we rented 2 portable 30m trailer mounted masts. A microwave company loaned us some DragonWave kit which ran on 18ghz at 385mb full duplex, this was our primary link and they applied for a UK OFCOM temp telco license for this on our behalf. We also bought a pair of Ubiquiti Nanobridge M5’s for backup, running at about 100mb.

We didn’t firewall anything, users were made aware what they were connecting to, there were no passwords on the SSID’s, we had no agenda to monitor traffic. We published abuse email addresses and a number that people could call if required and we would act on it (the RIR contacts for the address space were updated too)

Our NOC: http://www.flickr.com/photos/nottinghack/7929611918/ https://dl.dropbox.com/u/74717/2012-08-30%2017.40.26.jpg http://www.flickr.com/photos/russss/7909193016/ http://www.flickr.com/photos/nottinghack/7929909834/

Onsite core and servers http://www.flickr.com/photos/nottinghack/7929611592/ http://www.flickr.com/photos/andy_d/7902260210/

For wireless we deployed a pair of Cisco wireless controllers, all the APs were lightweight and RF allocation was easily managed centrally. https://twitter.com/emfnoc/status/241944863887749121/photo/1

Just like CCC + HAR we deployed portaloo’s / datenklo around the campsite and campers connected up to them for power and Ethernet: http://www.flickr.com/photos/ne0hack3r/7924490940/ http://www.flickr.com/photos/je4d/7924689482/

Sort out kit configuration out well in advance, really glad we did as we spent far longer getting the mast and microwave kit aligned that we thought. Switches, servers were all configured before arriving so we just unloaded and connected things up according to the plan. Avoid NAT’ing anything, speak to a friendly ISP and borrow some address space. We split DNS resolvers, DHCP, monitoring VMs across 3 separate VM hosts just in case one had a hardware failure, don't rely on a single server box.

Do it properly and attendees will be happy: https://twitter.com/Ash_Force/status/242067006537474048 https://twitter.com/markphelan/status/241896897290309633 https://twitter.com/je4d/status/242386884276396032

Our slides are here (warning 50mb)… http://www.natmorris.co.uk/camp_network.pdf

Get a team on board to help out, ours rocked!

-- Nat

http://natmorris.co.uk http://twitter.com/natmorris

On Thu, Sep 13, 2012 at 10:55 PM, Donald Eastlake <d3e3e3@gmail.com> wrote:

The 2015 WorldCon site selection is contested. There is a group seeking selection for the Disney Coronado Spring Resort in Florida but also competing groups seeking Spokane, Washington, and Helsinki, Finland.

Finland..we have one (1/2 hobbyist) group here that does Big Event Networking: NetCrew. They build the network needed for The Assembly series of events every year. (http://www.assembly.org) Typical event spec: 1-2*10G uplink, appr. 3000 100-1000baseT connections, WLAN, NOC with cool tracking tools to pinpoint a system behaving badly on a BIG switch network, switch configuration bulders etc. They have most of the distribution gear on store and rent out equipment (especially switch gear) between events. Surprise surprise, the core group of the team consists of networking veterans with 10-15+ year experience on the field. Netcrew is a cool way of being able to build some funky designs and test stuff on a very hostile environment compared to their $dayjob. --Kauto FICORA/CERT-FI (kauto.huopio@ficora.fi) -- Kauto Huopio - kauto@huopio.fi

Subject: Re: Big Temporary Networks Date: Tue, Sep 18, 2012 at 01:03:00PM -0700 Quoting Jo Rhett (jrhett@netconsonance.com):

On Sep 13, 2012, at 7:29 AM, Jay Ashworth wrote:

...

I'm talking to the people who will probably be, in 2015, running the first Worldcon I can practically drive to, in Orlando, at -- I think -- the Disney World Resort. I've told them how critical the issue is for this market; they, predictably, replied "We look forward to your patch". :-}

So I just want to point out that this is an utterly irrelevant topic. Worldcon is full to the brim with really smart people who can build good networks, but in every place large enough to host a Worldcon the owners of the building make money selling Internet access and don't want competition. The very best we've been able to do was create an Internet Lounge with good connectivity, and even that isn't acceptable at most locations.

All the IETF and RIPE meetings I've been to have had excellent custom networks. How come? -- Måns Nilsson primary/secondary/besserwisser/machina MN-1334-RIPE +46 705 989668 How do you explain Wayne Newton's POWER over millions? It's th' MOUSTACHE ... Have you ever noticed th' way it radiates SINCERITY, HONESTY & WARMTH? It's a MOUSTACHE you want to take HOME and introduce to NANCY SINATRA!