do folk have experience with platforms where ifIndexes are not stable across reboots etc? how do you deal with it? do some of those platforms trap on change? randy, who hates ifIndex changes
Most platforms I've worked with have a method to make the indexes persistent, often by additional command-line options. -Steve On Fri, Oct 12, 2018 at 2:08 PM Randy Bush <randy@psg.com> wrote:
do folk have experience with platforms where ifIndexes are not stable across reboots etc? how do you deal with it? do some of those platforms trap on change?
randy, who hates ifIndex changes
Once upon a time, Randy Bush <randy@psg.com> said:
do folk have experience with platforms where ifIndexes are not stable across reboots etc? how do you deal with it? do some of those platforms trap on change?
Is there any good excuse that SNMP client software can't handle a basic design of SNMP - indexed tables? ifIndex is far from the only index in SNMP, and many of them still change today at various times. It isn't that hard to fetch the indexed field in a bulk get, rewalking the table if you don't get what you expected. Cricket did this in 1999. -- Chris Adams <cma@cmadams.net>
I see this all the time. Especially in module chassis. It seems like sometimes it has to do with when each board goes to a ready state as the system boots. We also see renumbering due to virtual interface and board additions. While you are running they seem to get the next ifindex available but when you reboot the seem to be in the order they come up or the order they are in the configuration. It is a real pain and some software allows us to rescan a device and other software we have no easy way other than to delete and the re-add the device. I feel your pain on this one. I have no idea why most NMS systems can't seem to understand this and just rescan at a set interval or after an up/down device event. Steven Naslund Chicago IL
do folk have experience with platforms where ifIndexes are not stable across reboots etc? how do you deal with it? do some of those platforms trap on change?
Cisco has a feature you can enable called “Interface Index Persistence”: https://www.cisco.com/c/en/us/support/docs/ip/simple-network-management-prot... This solves the problem, at least with Cisco gear. -mel beckman On Oct 12, 2018, at 1:33 PM, Naslund, Steve <SNaslund@medline.com<mailto:SNaslund@medline.com>> wrote: I see this all the time. Especially in module chassis. It seems like sometimes it has to do with when each board goes to a ready state as the system boots. We also see renumbering due to virtual interface and board additions. While you are running they seem to get the next ifindex available but when you reboot the seem to be in the order they come up or the order they are in the configuration. It is a real pain and some software allows us to rescan a device and other software we have no easy way other than to delete and the re-add the device. I feel your pain on this one. I have no idea why most NMS systems can't seem to understand this and just rescan at a set interval or after an up/down device event. Steven Naslund Chicago IL do folk have experience with platforms where ifIndexes are not stable across reboots etc? how do you deal with it? do some of those platforms trap on change?
On Fri, 12 Oct 2018 at 21:40, Chris Adams <cma@cmadams.net> wrote:
Is there any good excuse that SNMP client software can't handle a basic design of SNMP - indexed tables? ifIndex is far from the only index in SNMP, and many of them still change today at various times.
It isn't that hard to fetch the indexed field in a bulk get, rewalking the table if you don't get what you expected. Cricket did this in 1999.
It's never going to be provably correct, depending on what stability means. You fetch relation at t0, then at t1 you fetch data. Was the relation same at t0 and t1? You can gain some confidence by fetching relation again at t2 and disregard data if t0 != t2. But this becomes polling expensive quite fast, and still not provably correct. This may be nitpicking, but I've always felt uneasy about the lack of guarantee. I wonder if those who have stable indeces, have them for all cases, all logical interfaces and virtual interfaces? -- ++ytti
Saku, The issue isn't that ifindexes change during operation. That would truly make SNMP useless. The issue is that they change across reboots. That's where features such as Cisco's Interface Index Persistence helps out. -mel via cell
On Oct 13, 2018, at 2:59 AM, Saku Ytti <saku@ytti.fi> wrote:
On Fri, 12 Oct 2018 at 21:40, Chris Adams <cma@cmadams.net> wrote:
Is there any good excuse that SNMP client software can't handle a basic design of SNMP - indexed tables? ifIndex is far from the only index in SNMP, and many of them still change today at various times.
It isn't that hard to fetch the indexed field in a bulk get, rewalking the table if you don't get what you expected. Cricket did this in 1999.
It's never going to be provably correct, depending on what stability means.
You fetch relation at t0, then at t1 you fetch data. Was the relation same at t0 and t1? You can gain some confidence by fetching relation again at t2 and disregard data if t0 != t2. But this becomes polling expensive quite fast, and still not provably correct. This may be nitpicking, but I've always felt uneasy about the lack of guarantee.
I wonder if those who have stable indeces, have them for all cases, all logical interfaces and virtual interfaces?
-- ++ytti
Cisco tries very hard to make such useless data occur in XR. If you have a gigE SFP in an SFP+ port, a new ifindex will appear for the resulting GigabitEthernetX port, then it remains even if both the config and SFP have been removed. Automated systems will keep querying it as if it were a downed port, but wait, reboot, and suddenly it vanishes. I went back and forth with TAC for weeks explaining that SNMP interfaces should not disappear as a result of a reboot, I should either be able to remove it, or it's stuck there forever, but a reboot should not cause a change. They didn't care; it is 'by design'. On 10/13/18, 8:47 AM, "NANOG on behalf of Mel Beckman" <nanog-bounces@nanog.org on behalf of mel@beckman.org> wrote: Saku, The issue isn't that ifindexes change during operation. That would truly make SNMP useless. The issue is that they change across reboots. That's where features such as Cisco's Interface Index Persistence helps out. -mel via cell > On Oct 13, 2018, at 2:59 AM, Saku Ytti <saku@ytti.fi> wrote: > >> On Fri, 12 Oct 2018 at 21:40, Chris Adams <cma@cmadams.net> wrote: >> >> Is there any good excuse that SNMP client software can't handle a basic >> design of SNMP - indexed tables? ifIndex is far from the only index in >> SNMP, and many of them still change today at various times. >> >> It isn't that hard to fetch the indexed field in a bulk get, rewalking >> the table if you don't get what you expected. Cricket did this in 1999. > > It's never going to be provably correct, depending on what stability means. > > You fetch relation at t0, then at t1 you fetch data. Was the relation > same at t0 and t1? You can gain some confidence by fetching relation > again at t2 and disregard data if t0 != t2. But this becomes polling > expensive quite fast, and still not provably correct. This may be > nitpicking, but I've always felt uneasy about the lack of guarantee. > > I wonder if those who have stable indeces, have them for all cases, > all logical interfaces and virtual interfaces? > > -- > ++ytti
David, All you have to do is turn on IFindex persistence: https://www.cisco.com/c/en/us/td/docs/routers/crs/software/crs_r4-2/system_m... We do this on our XRs and it works perfectly. -mel via cell On Oct 13, 2018, at 9:20 AM, David Hubbard <dhubbard@dino.hostasaurus.com<mailto:dhubbard@dino.hostasaurus.com>> wrote: Cisco tries very hard to make such useless data occur in XR. If you have a gigE SFP in an SFP+ port, a new ifindex will appear for the resulting GigabitEthernetX port, then it remains even if both the config and SFP have been removed. Automated systems will keep querying it as if it were a downed port, but wait, reboot, and suddenly it vanishes. I went back and forth with TAC for weeks explaining that SNMP interfaces should not disappear as a result of a reboot, I should either be able to remove it, or it's stuck there forever, but a reboot should not cause a change. They didn't care; it is 'by design'. On 10/13/18, 8:47 AM, "NANOG on behalf of Mel Beckman" <nanog-bounces@nanog.org<mailto:nanog-bounces@nanog.org> on behalf of mel@beckman.org<mailto:mel@beckman.org>> wrote: Saku, The issue isn't that ifindexes change during operation. That would truly make SNMP useless. The issue is that they change across reboots. That's where features such as Cisco's Interface Index Persistence helps out. -mel via cell On Oct 13, 2018, at 2:59 AM, Saku Ytti <saku@ytti.fi<mailto:saku@ytti.fi>> wrote: On Fri, 12 Oct 2018 at 21:40, Chris Adams <cma@cmadams.net<mailto:cma@cmadams.net>> wrote: Is there any good excuse that SNMP client software can't handle a basic design of SNMP - indexed tables? ifIndex is far from the only index in SNMP, and many of them still change today at various times. It isn't that hard to fetch the indexed field in a bulk get, rewalking the table if you don't get what you expected. Cricket did this in 1999. It's never going to be provably correct, depending on what stability means. You fetch relation at t0, then at t1 you fetch data. Was the relation same at t0 and t1? You can gain some confidence by fetching relation again at t2 and disregard data if t0 != t2. But this becomes polling expensive quite fast, and still not provably correct. This may be nitpicking, but I've always felt uneasy about the lack of guarantee. I wonder if those who have stable indeces, have them for all cases, all logical interfaces and virtual interfaces? -- ++ytti
I do that too, but I’m referring to XR when you use different speed optics in a multi-speed port; if you have a SFP+ port and 10gig SFP, you’ll get one ifindex. New use case requires swapping to a gigE SFP and you’ll get a new ifindex. Take the port out of service, remove the GigE SFP and the related config, yet both ifindexes remain; until the device is reloaded. At that the gigE ifindex goes away leaving just the native-speed ifindex. It’s a pain for management because we’re forced to make exclusions in our NMS for ifindex’s that may disappear at some point, because they show as down with no way to make that not the case. Worse, if that port is put to use again at the non-native speed, and has such an exclusion in place, we don’t auto learn the new usage because of the exclusion. I tried to argue with TAC that if the gigE SFP has been removed from the SFP+ port, and its config has been deleted, the corresponding ifindex and related counters should be gone; it no longer exists in any form. If you reload, it will disappear, but that’s the only way. From: Mel Beckman <mel@beckman.org> Date: Saturday, October 13, 2018 at 4:46 PM To: David Hubbard <dhubbard@dino.hostasaurus.com> Cc: "nanog@nanog.org" <nanog@nanog.org> Subject: Re: ifIndex David, All you have to do is turn on IFindex persistence: https://www.cisco.com/c/en/us/td/docs/routers/crs/software/crs_r4-2/system_m... We do this on our XRs and it works perfectly. -mel via cell On Oct 13, 2018, at 9:20 AM, David Hubbard <dhubbard@dino.hostasaurus.com<mailto:dhubbard@dino.hostasaurus.com>> wrote: Cisco tries very hard to make such useless data occur in XR. If you have a gigE SFP in an SFP+ port, a new ifindex will appear for the resulting GigabitEthernetX port, then it remains even if both the config and SFP have been removed. Automated systems will keep querying it as if it were a downed port, but wait, reboot, and suddenly it vanishes. I went back and forth with TAC for weeks explaining that SNMP interfaces should not disappear as a result of a reboot, I should either be able to remove it, or it's stuck there forever, but a reboot should not cause a change. They didn't care; it is 'by design'. On 10/13/18, 8:47 AM, "NANOG on behalf of Mel Beckman" <nanog-bounces@nanog.org<mailto:nanog-bounces@nanog.org> on behalf of mel@beckman.org<mailto:mel@beckman.org>> wrote: Saku, The issue isn't that ifindexes change during operation. That would truly make SNMP useless. The issue is that they change across reboots. That's where features such as Cisco's Interface Index Persistence helps out. -mel via cell On Oct 13, 2018, at 2:59 AM, Saku Ytti <saku@ytti.fi<mailto:saku@ytti.fi>> wrote: On Fri, 12 Oct 2018 at 21:40, Chris Adams <cma@cmadams.net<mailto:cma@cmadams.net>> wrote: Is there any good excuse that SNMP client software can't handle a basic design of SNMP - indexed tables? ifIndex is far from the only index in SNMP, and many of them still change today at various times. It isn't that hard to fetch the indexed field in a bulk get, rewalking the table if you don't get what you expected. Cricket did this in 1999. It's never going to be provably correct, depending on what stability means. You fetch relation at t0, then at t1 you fetch data. Was the relation same at t0 and t1? You can gain some confidence by fetching relation again at t2 and disregard data if t0 != t2. But this becomes polling expensive quite fast, and still not provably correct. This may be nitpicking, but I've always felt uneasy about the lack of guarantee. I wonder if those who have stable indeces, have them for all cases, all logical interfaces and virtual interfaces? -- ++ytti
participants (7)
-
Chris Adams -
David Hubbard -
Mel Beckman -
Naslund, Steve -
Randy Bush -
Saku Ytti -
Steve Meuse