Re: Reducing Usenet Bandwidth
Simon Lyall <simon.lyall@ihug.co.nz> writes:
On Sun, 3 Feb 2002, Marco d'Itri wrote:
The major drawback of that protocol is that it limits articles size to 64KB, so it does not reduce binary traffic, which is the largest part of a newsfeed.
In which case you just modify the protocol (or roll your own) to have articles spread across multiple packets. It's not that hard and our guys did this and I expect it's been done (at least) half a dozen times by other people.
It was trivial when I did it as a testbed for my previous boss (sorry, not open-source). If you compress the article before sending (we used libz, and at the time a 1 GHz PIII (the fastest machine we could get) kept up with the 28mbps full feed just fine), a truly startling percentage of articles (I want to say 78%) fit in a single packet on the ethernet... 1500 bytes minus udp encap and non-compressed header data, which includes stuff like local article sequence number, part number for multipacket articles, md5 checksum of article, and message-id. Of course, those little articles are not the ones that are eating your bandwidth, but even so I found that to be quite interesting. Our software was nowhere near as complex as the UUnet software (just point-to-multipoint), and probably quite similar to Cidera's software. Worked pretty well over satellite too. Of course, the dirty little not-so-secret is that for all the hacks that people have done over the years, inter-AS multicast that doesn't get hosed at the drop of a hat remains an elusive goal(1). Thinking about doing multicast Usenet feeds as a way to cut down the bandwidth in and out of your ISP overlooks the fact that reliable transport for such things doesn't exist. I'm sorry if I've offended anyone with this (uncharitable) assessment. ---Rob (1): http://beaconserver.accessgrid.org:9999/
Thus spake "Robert E. Seastrom" <rs@seastrom.com>
Of course, the dirty little not-so-secret is that for all the hacks that people have done over the years, inter-AS multicast that doesn't get hosed at the drop of a hat remains an elusive goal(1).
But if this service works for most of the people most of the time, it's achieved enough to justify deployment, no?
Thinking about doing multicast Usenet feeds as a way to cut down the bandwidth in and out of your ISP overlooks the fact that reliable transport for such things doesn't exist.
Ah, but the point of Muse was not to provide reliable news service, but to reduce the average propagation time and provide "good enough" delivery which could be supplanted by a traditional reliable feed. S
On Sat, Feb 02, 2002 at 08:34:52PM -0500, Robert E. Seastrom wrote:
Of course, the dirty little not-so-secret is that for all the hacks that people have done over the years, inter-AS multicast that doesn't get hosed at the drop of a hat remains an elusive goal(1). Thinking about doing multicast Usenet feeds as a way to cut down the bandwidth in and out of your ISP overlooks the fact that reliable transport for such things doesn't exist. I'm sorry if I've offended anyone with this (uncharitable) assessment.
I don't take any immediate offense but the number of multicast connected sites is (slowly) going up. Sprint customers can toggle their (bgp) session from nlri unicast -> unicast multicast in order to get mbgp routes and enable pim to do forwarding. Obviously one needs to chat with someone to get msdp going. The routers don't tend to have very multicast related bugs anymore, it's all people who can't configure their routers. Of the 100+ sessions that are in sdr/sap these days I can typically reach (at least) 65%+ of them without any problems. The problem is that the "tier 2" and whatnot providers have [mostly] missed the boat on it and don't have people that are able to configure their routers for mbgp. (this is not to say all but most). There are a lot of resources for getting help in fixing and deploying multicast these days. It'd be nice to see more people turning it on. - Jared
---Rob
-- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.
Once upon a time, Jared Mauch <jared@puck.Nether.net> said:
The problem is that the "tier 2" and whatnot providers have [mostly] missed the boat on it and don't have people that are able to configure their routers for mbgp. (this is not to say all but most).
There are a lot of resources for getting help in fixing and deploying multicast these days. It'd be nice to see more people turning it on.
Okay, I'm with a (decent sized, although probably small by NANOG measures) ISP. Why do I want to turn on multicast (what does it get me that I don't have today)? And if I want to turn it on, where can I find the resources you mention? These are meant as honest questions (not intended as rhetorical or argumentative questions). I want to understand why I should do this. -- Chris Adams <cmadams@hiwaay.net> Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble.
On Sun, Feb 03, 2002 at 03:00:03PM -0600, Chris Adams wrote:
Okay, I'm with a (decent sized, although probably small by NANOG measures) ISP. Why do I want to turn on multicast (what does it get me that I don't have today)?
Do you want a technical spin or the www.showmethemoney.com spin ? The basis of the technical spin is that ip multicast is as much ip as ip unicast and that as such it need to be provided for applications to leverage all the benefits of the ip service. The basis of the money spin is that you as an ISP are making money (hopefully) by shuffling bits out onto links (to customers) who are paying for the links, and you are paying for traffic on backbone links where you connect to the core, and ip multicast allows you to shuffle more bits out from your network on the links to customers without linearly increasing the amount of bits on your uplinks. Practically speaking, there a small but fair amount of ip multicast content out on the network that you could have your customers receive, mostly entertainment oriented, but you're not really limited by this. Given that the primary purpose of ip multicast for you is to give you a savings in bandwidth, you might as well look into encouraging your customers to put up content - especially if there are likely candidates that are interesting to your regional customer space - after all, live content today often does not exist at good quality because the need to use some existing splitter technology drives up the cost to the content producer in such a way that it's not feasible. Also, you could convert unicast content at the uplink of your network into multicast so as to save pulling it in multiple times into your network.
And if I want to turn it on, where can I find the resources you mention?
There's a couple of books out there, but the config recommendation would mostly come from your gear vendor like they'll probably come too for most of the other features you've got on your routers. If you use cisco gear, check out "www.cisco.com/go/ipmulticast", or ask on cisco-multicast@andrew.cmu.edu. Also, check out with your uplink ISPs, you should easily be able to get up and running by having an ip multicast enabled isp give you the config recommendation. Getting ip multicast to run this way is not particularily challenging, it is more the problem of getting the right cooking recipe and the three pointers above should lead you there pretty well. Cheers Toerless
On Sun, Feb 03, 2002 at 03:00:03PM -0600, Chris Adams wrote:
Once upon a time, Jared Mauch <jared@puck.Nether.net> said:
The problem is that the "tier 2" and whatnot providers have [mostly] missed the boat on it and don't have people that are able to configure their routers for mbgp. (this is not to say all but most).
There are a lot of resources for getting help in fixing and deploying multicast these days. It'd be nice to see more people turning it on.
Okay, I'm with a (decent sized, although probably small by NANOG measures) ISP. Why do I want to turn on multicast (what does it get me that I don't have today)? And if I want to turn it on, where can I find the resources you mention?
These are meant as honest questions (not intended as rhetorical or argumentative questions). I want to understand why I should do this.
As a sprint customer, you can start to turn on pim on your igp interfaces, set your routes for nlri unicast multicast as well as your bgp sessions. here's a pointer to sprint webpage: http://www.sprintlink.net/multicast/faq.html Verio customers can contact me for the support/request address (want to keep it out of the list archives to reduce spam and also incase it changes). This is also not a endorsement of Multicast Technologies but they have some info here: http://www.multicasttech.com/faq/ There are a lot of other resources out there. I suggest doing a few searches in google/yahoo and you can find some. - Jared -- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.
In the referenced message, Jared Mauch said: <snip>
I don't take any immediate offense but the number of multicast connected sites is (slowly) going up. Sprint customers can toggle their (bgp) session from nlri unicast -> unicast multicast in order to get mbgp routes and enable pim to do forwarding. Obviously one needs to chat with someone to get msdp going.
The routers don't tend to have very multicast related bugs anymore, it's all people who can't configure their routers.
Of the 100+ sessions that are in sdr/sap these days I can typically reach (at least) 65%+ of them without any problems.
The problem is that the "tier 2" and whatnot providers have [mostly] missed the boat on it and don't have people that are able to configure their routers for mbgp. (this is not to say all but most).
There are a lot of resources for getting help in fixing and deploying multicast these days. It'd be nice to see more people turning it on.
- Jared
I'm wondering how may folks are having issues with the oddities related to Cisco doing RPF checks based on administrative distance, rather than longest-match against the MRIB. With this, a shorter EBGP-learned route wins out over a longer IBGP-learned route. I know it is possible to turn on (via undocumented command) longest-match. But, it still matches across both the URIB and MRIB equally, such that a longer unicast prefix wins over a shorter multicast prefix. Ideally, the RPF checks would be longest-match against the MRIB first, and falling back to longest-match against the URIB (which is exactly what the MSDP RPF checks presently do.) I find it really weird when the "sh ip rpf" lists an ebgp-unicast route as the rpf source, when an ibgp-multicast route is available. Essentially makes even exchanging multicast nlri somewhat pointless, when it is disregarded frequently. The workaround I've been suggested to use is the undocumented command and lots of static mroutes (to beat out unicast bgp), whenever problems surface.
I think you are refering to cisco bugid: CSCdv47188 -- snip -- If the first entry in the Multicast Border Gateway Protocol (MBGP) routing table is supernet of the destination IP address or the MBGP route exists but does not have the best path, Reverse Path Forwarding (RPF) lookup will fail or return a unicast Border Gateway Protocol (BGP) route if a unicast BGP route exists. Workaround: Remove the first entry or add a dummy route that is smaller than the first entry. In case of a MBGP route without a best path, change the network configuration to ensure that the specified destination address has the best path. -- snip --
I'm wondering how may folks are having issues with the oddities related to Cisco doing RPF checks based on administrative distance, rather than longest-match against the MRIB. With this, a shorter EBGP-learned route wins out over a longer IBGP-learned route.
I know it is possible to turn on (via undocumented command) longest-match. But, it still matches across both the URIB and MRIB equally, such that a longer unicast prefix wins over a shorter multicast prefix.
Ideally, the RPF checks would be longest-match against the MRIB first, and falling back to longest-match against the URIB (which is exactly what the MSDP RPF checks presently do.)
I find it really weird when the "sh ip rpf" lists an ebgp-unicast route as the rpf source, when an ibgp-multicast route is available. Essentially makes even exchanging multicast nlri somewhat pointless, when it is disregarded frequently.
The workaround I've been suggested to use is the undocumented command and lots of static mroutes (to beat out unicast bgp), whenever problems surface.
-- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.
In the referenced message, Jared Mauch said:
I think you are refering to cisco bugid: CSCdv47188
-- snip -- If the first entry in the Multicast Border Gateway Protocol (MBGP) routing table is supernet of the destination IP address or the MBGP route exists but does not have the best path, Reverse Path Forwarding (RPF) lookup will fail or return a unicast Border Gateway Protocol (BGP) route if a unicast BGP route exists.
Workaround: Remove the first entry or add a dummy route that is smaller than the first entry. In case of a MBGP route without a best path, change the network configuration to ensure that the specified destination address has the best path. -- snip --
This does seem to resemble the issue, but in my discussion with cisco it didn't seem they intended on deploying resources on the corner cases I described. I'll check to see how the above bug relates to those issues.
In the referenced message, Stephen Griffin said:
In the referenced message, Jared Mauch said:
I think you are refering to cisco bugid: CSCdv47188
-- snip -- If the first entry in the Multicast Border Gateway Protocol (MBGP) routing table is supernet of the destination IP address or the MBGP route exists but does not have the best path, Reverse Path Forwarding (RPF) lookup will fail or return a unicast Border Gateway Protocol (BGP) route if a unicast BGP route exists.
Workaround: Remove the first entry or add a dummy route that is smaller than the first entry. In case of a MBGP route without a best path, change the network configuration to ensure that the specified destination address has the best path. -- snip --
This does seem to resemble the issue, but in my discussion with cisco it didn't seem they intended on deploying resources on the corner cases I described. I'll check to see how the above bug relates to those issues.
Actually, after confirming with Cisco, the above bug is different. Essentially, if a router has two links, and a route in both iBGP and iMBGP with same igp distance, local pref, mask, etc, the unicast BGP route is chosen for RPF, rather than the multicast route. This would lead to using the "wrong" interface. Although it is related in that, if the RPF check was done against the MRIB first, it would solve this bug.
On Tue, Feb 05, 2002 at 04:28:39PM -0500, Stephen Griffin wrote:
Actually, after confirming with Cisco, the above bug is different. Essentially, if a router has two links, and a route in both iBGP and iMBGP with same igp distance, local pref, mask, etc, the unicast BGP route is chosen for RPF, rather than the multicast route. This would lead to using the "wrong" interface.
Although it is related in that, if the RPF check was done against the MRIB first, it would solve this bug.
does "ip multicast multipath" help you at all either? that should cause it to rpf down both paths. - jared -- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.
In the referenced message, Jared Mauch said:
On Tue, Feb 05, 2002 at 04:28:39PM -0500, Stephen Griffin wrote:
Actually, after confirming with Cisco, the above bug is different. Essentially, if a router has two links, and a route in both iBGP and iMBGP with same igp distance, local pref, mask, etc, the unicast BGP route is chosen for RPF, rather than the multicast route. This would lead to using the "wrong" interface.
Although it is related in that, if the RPF check was done against the MRIB first, it would solve this bug.
does "ip multicast multipath" help you at all either?
that should cause it to rpf down both paths.
- jared
Based upon what I've previously been told, "ip multicast multipath" doesn't actually work yet. I've not tested it. The problems I see are related to short EBGP unicast routes beating long IBGP multicast routes due to the default checks based on administrative distance (ebgp has lower admin distance). (EBGP unicast routes beat IBGP multicast routes, even at the same prefix-length) if the knob is twiddled to do length-based checks, then the problem turns to long unicast routes beat short multicast routes, which can be a problem if someone is doing TE with no-export tagged more-specifics, such that you would have multicast connectivity via the shorter prefix, but follow the path where there is no multicast connectivity. Cisco is aware of my concerns, and the proposed solution of doing multicast RPF checks in the same manner as the MSDP checks (of sorts) with longest-match across the mrib, and falling back to longest-match across the urib (if there's no covering prefix in the mrib) but don't have any timeframe as to when that might be undertaken.
participants (6)
-
Chris Adams -
Jared Mauch -
rs＠seastrom.com -
Stephen Griffin -
Stephen Sprunk -
Toerless Eckert