The tale of a single MAC
Hi there, I encountered an interesting issue today and I found it so bizarre so I thought I would share it. I brought online a spare server to help offload some of the recent VMs that I have been deploying. Around the same time this new machine (we¹ll call it Server-B) came online, another machine which has been online for about a year now stopped responding to our monitoring (and we¹ll name this Server-A). I logged into the switch and saw that the machine that stopped responding was in the same VLAN as this newly deployed, and then quickly noticed that Server-A¹s MAC address was now on Server-B¹s switch port. ³What the ...² was my initial response. I went ahead and moved Server-B¹s to another VLAN, updated the switchport, cleared the ARP, and Server-A came back to life. Happy new year to me. So here is the interesting part... Both servers are HP Proliant DL380 G4s, and both of their NIC1 and NIC2 MACs addresses are exactly the same. Not spoofd and the OS drivers are not mucking with them ... They¹re burned-in I triple checked them in their respective BIOS screen. I acquired these two machines at different times and both were from the grey market. The ³What the ...² is sitting fresh in my mind ... How can this be? In the last 15 years of being in IT, I have never encountered a ³burned-in² duplicated MACs across two physically different machines. What are the odds, that HP would dup¹d them and that both would eventually end up at my shop? Or maybe this type of thing isn¹t big of deal... ? -graham
I've seen duplicate MAC addresses but only on no name made in china NICs installed on cheap (assembled from parts) PCs at a school computer lab. On Sun, Jan 2, 2011 at 9:03 AM, Graham Wooden <graham@g-rock.net> wrote:
In the last 15 years of being in IT, I have never encountered a ³burned-in² duplicated MACs across two physically different machines. What are the odds, that HP would dup¹d them and that both would eventually end up at my shop? Or maybe this type of thing isn¹t big of deal... ?
-- Suresh Ramasubramanian (ops.lists@gmail.com)
Seen this on six-figure gateways. -RR On 1/1/11, Suresh Ramasubramanian <ops.lists@gmail.com> wrote:
I've seen duplicate MAC addresses but only on no name made in china NICs installed on cheap (assembled from parts) PCs at a school computer lab.
On Sun, Jan 2, 2011 at 9:03 AM, Graham Wooden <graham@g-rock.net> wrote:
In the last 15 years of being in IT, I have never encountered a ³burned-in² duplicated MACs across two physically different machines. What are the odds, that HP would dup¹d them and that both would eventually end up at my shop? Or maybe this type of thing isn¹t big of deal... ?
-- Suresh Ramasubramanian (ops.lists@gmail.com)
-- Sent from my mobile device
On Jan 2, 2011, at 10:33 AM, Graham Wooden wrote:
What are the odds, that HP would dup’d them and that both would eventually end up at my shop?
There may be some setting you're overlooking or a bug which needs an update to fix, or you may simply have purchased HP ProLiant *cases*, rather than actual *servers*. ;> Note that search engine results for 'proliant dl380 duplicate mac' returns multiple links. ------------------------------------------------------------------------ Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Most software today is very much like an Egyptian pyramid, with millions of bricks piled on top of each other, with no structural integrity, but just done by brute force and thousands of slaves. -- Alan Kay
No - these are Genuine HP Servers. Both servers have the latest BIOSs and firmware applied to the board as well as cards. The search results that I have seen didn't apply to the actual bios, rather to guest Oss mucking or teamming. On 1/1/11 9:56 PM, "Dobbins, Roland" <rdobbins@arbor.net> wrote:
On Jan 2, 2011, at 10:33 AM, Graham Wooden wrote:
What are the odds, that HP would dup¹d them and that both would eventually end up at my shop?
There may be some setting you're overlooking or a bug which needs an update to fix, or you may simply have purchased HP ProLiant *cases*, rather than actual *servers*.
;>
Note that search engine results for 'proliant dl380 duplicate mac' returns multiple links.
------------------------------------------------------------------------ Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com>
Most software today is very much like an Egyptian pyramid, with millions of bricks piled on top of each other, with no structural integrity, but just done by brute force and thousands of slaves.
-- Alan Kay
On 1/1/11 8:33 PM, Graham Wooden wrote:
So here is the interesting part... Both servers are HP Proliant DL380 G4s, and both of their NIC1 and NIC2 MACs addresses are exactly the same. Not spoofd and the OS drivers are not mucking with them ... They¹re burned-in I triple checked them in their respective BIOS screen. I acquired these two machines at different times and both were from the grey market. The ³What the ...² is sitting fresh in my mind ... How can this be?
From the same grey market supplier? I know HP has a disc they put out which updates all the firmware/bios in a specific server model, its not too far fetched that a vendor might have a modified version that also either purposely or accidentally changes the MAC address. Off the top of my head, I'm not sure where the MAC is stored - maybe an eeprom or a portion of the bios flash. Or, it could be botched flashing that blew away the portion of memory where that was stored and the system defaulted to a built in value. Excellent example is, IIRC, the older sparc stuff, where the ethernet cards didn't have MAC addresses as part of the card, but were stored in non-volatile or battery backed memory. Memory goes poof, and you'll have problems. Some WRT54G/WAP54Gs suffer from the same problem when throwing third party firmware on there. -- Brielle Bruns The Summit Open Source Development Group http://www.sosdg.org / http://www.ahbl.org
On Sat, 01 Jan 2011 20:59:16 -0700 Brielle Bruns <bruns@2mbit.com> wrote:
On 1/1/11 8:33 PM, Graham Wooden wrote:
So here is the interesting part... Both servers are HP Proliant DL380 G4s, and both of their NIC1 and NIC2 MACs addresses are exactly the same. Not spoofd and the OS drivers are not mucking with them ... They¹re burned-in I triple checked them in their respective BIOS screen. I acquired these two machines at different times and both were from the grey market. The ³What the ...² is sitting fresh in my mind ... How can this be?
From the same grey market supplier?
I know HP has a disc they put out which updates all the firmware/bios in a specific server model, its not too far fetched that a vendor might have a modified version that also either purposely or accidentally changes the MAC address. Off the top of my head, I'm not sure where the MAC is stored - maybe an eeprom or a portion of the bios flash. Or, it could be botched flashing that blew away the portion of memory where that was stored and the system defaulted to a built in value.
Excellent example is, IIRC, the older sparc stuff, where the ethernet cards didn't have MAC addresses as part of the card, but were stored in non-volatile or battery backed memory.
This was actually the intended way to use "MAC" addresses, to used as host addresses rather than as individual interface addresses, according to the following paper - "48-bit Absolute Internet and Ethernet Host Numbers" Yogan K. Dalal and Robert S. Printis, July 1981 http://ethernethistory.typepad.com/papers/HostNumbers.pdf That paper also discusses why 48 bits were chosen as the size, despite "Ethernet systems" being limited to 1024 hosts. I think things evolved into MAC per NIC because when add-in NICs were invented there wasn't any appropriate non-volatile storage on the host to store the address. Regards, mark.
i have seen dups in 3com, dell, and hp kit over the years. the best was moving mac addresses btwn 802,3 and 802.5 cards. --bill On Sun, Jan 02, 2011 at 03:03:24PM +1030, Mark Smith wrote:
On Sat, 01 Jan 2011 20:59:16 -0700 Brielle Bruns <bruns@2mbit.com> wrote:
On 1/1/11 8:33 PM, Graham Wooden wrote:
So - here is the interesting part... Both servers are HP Proliant DL380 G4s, and both of their NIC1 and NIC2 MACs addresses are exactly the same. Not spoofd and the OS drivers are not mucking with them ... They9re burned-in - I triple checked them in their respective BIOS screen. I acquired these two machines at different times and both were from the grey market. The 3What the ...2 is sitting fresh in my mind ... How can this be?
From the same grey market supplier?
I know HP has a disc they put out which updates all the firmware/bios in a specific server model, its not too far fetched that a vendor might have a modified version that also either purposely or accidentally changes the MAC address. Off the top of my head, I'm not sure where the MAC is stored - maybe an eeprom or a portion of the bios flash. Or, it could be botched flashing that blew away the portion of memory where that was stored and the system defaulted to a built in value.
Excellent example is, IIRC, the older sparc stuff, where the ethernet cards didn't have MAC addresses as part of the card, but were stored in non-volatile or battery backed memory.
This was actually the intended way to use "MAC" addresses, to used as host addresses rather than as individual interface addresses, according to the following paper -
"48-bit Absolute Internet and Ethernet Host Numbers" Yogan K. Dalal and Robert S. Printis, July 1981 http://ethernethistory.typepad.com/papers/HostNumbers.pdf
That paper also discusses why 48 bits were chosen as the size, despite "Ethernet systems" being limited to 1024 hosts.
I think things evolved into MAC per NIC because when add-in NICs were invented there wasn't any appropriate non-volatile storage on the host to store the address.
Regards, mark.
On Jan 1, 2011, at 11:33 24PM, Mark Smith wrote:
On Sat, 01 Jan 2011 20:59:16 -0700 Brielle Bruns <bruns@2mbit.com> wrote:
On 1/1/11 8:33 PM, Graham Wooden wrote:
So here is the interesting part... Both servers are HP Proliant DL380 G4s, and both of their NIC1 and NIC2 MACs addresses are exactly the same. Not spoofd and the OS drivers are not mucking with them ... They¹re burned-in I triple checked them in their respective BIOS screen. I acquired these two machines at different times and both were from the grey market. The ³What the ...² is sitting fresh in my mind ... How can this be?
From the same grey market supplier?
I know HP has a disc they put out which updates all the firmware/bios in a specific server model, its not too far fetched that a vendor might have a modified version that also either purposely or accidentally changes the MAC address. Off the top of my head, I'm not sure where the MAC is stored - maybe an eeprom or a portion of the bios flash. Or, it could be botched flashing that blew away the portion of memory where that was stored and the system defaulted to a built in value.
Excellent example is, IIRC, the older sparc stuff, where the ethernet cards didn't have MAC addresses as part of the card, but were stored in non-volatile or battery backed memory.
This was actually the intended way to use "MAC" addresses, to used as host addresses rather than as individual interface addresses, according to the following paper -
"48-bit Absolute Internet and Ethernet Host Numbers" Yogan K. Dalal and Robert S. Printis, July 1981 http://ethernethistory.typepad.com/papers/HostNumbers.pdf
Yup.
That paper also discusses why 48 bits were chosen as the size, despite "Ethernet systems" being limited to 1024 hosts.
I think things evolved into MAC per NIC because when add-in NICs were invented there wasn't any appropriate non-volatile storage on the host to store the address.
On really old Sun gear, the MAC address was stored on a separate ROM chip; if the motherboard was replaced, you'd just move the ROM chip to the new board. I'm not sure what you mean, though, when you say "when add-in NICs were invented" -- the Ethernet cards I used in 1982 plugged into Unibus slots on our VAXen, so that goes back quite a ways... --Steve Bellovin, http://www.cs.columbia.edu/~smb
Hi, On Sun, 2 Jan 2011 08:50:42 -0500 Steven Bellovin <smb@cs.columbia.edu> wrote:
On Jan 1, 2011, at 11:33 24PM, Mark Smith wrote:
On Sat, 01 Jan 2011 20:59:16 -0700 Brielle Bruns <bruns@2mbit.com> wrote:
On 1/1/11 8:33 PM, Graham Wooden wrote:
<snip>
Excellent example is, IIRC, the older sparc stuff, where the ethernet cards didn't have MAC addresses as part of the card, but were stored in non-volatile or battery backed memory.
This was actually the intended way to use "MAC" addresses, to used as host addresses rather than as individual interface addresses, according to the following paper -
"48-bit Absolute Internet and Ethernet Host Numbers" Yogan K. Dalal and Robert S. Printis, July 1981 http://ethernethistory.typepad.com/papers/HostNumbers.pdf
Yup.
That paper also discusses why 48 bits were chosen as the size, despite "Ethernet systems" being limited to 1024 hosts.
I think things evolved into MAC per NIC because when add-in NICs were invented there wasn't any appropriate non-volatile storage on the host to store the address.
On really old Sun gear, the MAC address was stored on a separate ROM chip; if the motherboard was replaced, you'd just move the ROM chip to the new board.
I'm not sure what you mean, though, when you say "when add-in NICs were invented" -- the Ethernet cards I used in 1982 plugged into Unibus slots on our VAXen, so that goes back quite a ways...
More that as add-in cards supplied their own "storage" for the MAC address, rather than expecting it from the host (e.g. something like MAC addresses set by init scripts at boot or the ROM chip you mentioned on Suns), this has now evolved into an expected model of a MAC address tightly bound to an Ethernet interface and supplied by the Ethernet interface e.g. by an add-in board if one is added. Now that this model as been around for a long time, people find it a bit strange when MAC addresses aren't as tightly bound to a NIC/Ethernet interface. This is all speculation on my part though, I'd be curious if the reasons are different. When I first read that paper, it was really quite surprising that "MAC" addresses were designed to be more general host addresses/identifiers that were also to be used as Ethernet addresses. One example they talk about is using them as unique host identifiers when sharing files via floppy disk. Regards, mark.
On Jan 2, 2011, at 5:15 54PM, Mark Smith wrote:
Hi,
On Sun, 2 Jan 2011 08:50:42 -0500 Steven Bellovin <smb@cs.columbia.edu> wrote:
On Jan 1, 2011, at 11:33 24PM, Mark Smith wrote:
On Sat, 01 Jan 2011 20:59:16 -0700 Brielle Bruns <bruns@2mbit.com> wrote:
On 1/1/11 8:33 PM, Graham Wooden wrote:
<snip>
Excellent example is, IIRC, the older sparc stuff, where the ethernet cards didn't have MAC addresses as part of the card, but were stored in non-volatile or battery backed memory.
This was actually the intended way to use "MAC" addresses, to used as host addresses rather than as individual interface addresses, according to the following paper -
"48-bit Absolute Internet and Ethernet Host Numbers" Yogan K. Dalal and Robert S. Printis, July 1981 http://ethernethistory.typepad.com/papers/HostNumbers.pdf
Yup.
That paper also discusses why 48 bits were chosen as the size, despite "Ethernet systems" being limited to 1024 hosts.
I think things evolved into MAC per NIC because when add-in NICs were invented there wasn't any appropriate non-volatile storage on the host to store the address.
On really old Sun gear, the MAC address was stored on a separate ROM chip; if the motherboard was replaced, you'd just move the ROM chip to the new board.
I'm not sure what you mean, though, when you say "when add-in NICs were invented" -- the Ethernet cards I used in 1982 plugged into Unibus slots on our VAXen, so that goes back quite a ways...
More that as add-in cards supplied their own "storage" for the MAC address, rather than expecting it from the host (e.g. something like MAC addresses set by init scripts at boot or the ROM chip you mentioned on Suns), this has now evolved into an expected model of a MAC address tightly bound to an Ethernet interface and supplied by the Ethernet interface e.g. by an add-in board if one is added. Now that this model as been around for a long time, people find it a bit strange when MAC addresses aren't as tightly bound to a NIC/Ethernet interface. This is all speculation on my part though, I'd be curious if the reasons are different.
When I first read that paper, it was really quite surprising that "MAC" addresses were designed to be more general host addresses/identifiers that were also to be used as Ethernet addresses. One example they talk about is using them as unique host identifiers when sharing files via floppy disk.
If you read the XNS specs, you'll see that they liked 64-bit addresses -- a 16-bit network number and a 48-bit host address. In other words, they had id/locator separation... --Steve Bellovin, http://www.cs.columbia.edu/~smb
Date: Mon, 3 Jan 2011 08:45:54 +1030 From: Mark Smith <nanog@85d5b20a518b8f6864949bd940457dc124746ddc.nosense.org>
Hi,
On Sun, 2 Jan 2011 08:50:42 -0500 Steven Bellovin <smb@cs.columbia.edu> wrote:
On Jan 1, 2011, at 11:33 24PM, Mark Smith wrote:
On Sat, 01 Jan 2011 20:59:16 -0700 Brielle Bruns <bruns@2mbit.com> wrote:
On 1/1/11 8:33 PM, Graham Wooden wrote:
<snip>
Excellent example is, IIRC, the older sparc stuff, where the ethernet cards didn't have MAC addresses as part of the card, but were stored in non-volatile or battery backed memory.
This was actually the intended way to use "MAC" addresses, to used as host addresses rather than as individual interface addresses, according to the following paper -
"48-bit Absolute Internet and Ethernet Host Numbers" Yogan K. Dalal and Robert S. Printis, July 1981 http://ethernethistory.typepad.com/papers/HostNumbers.pdf
Yup.
That paper also discusses why 48 bits were chosen as the size, despite "Ethernet systems" being limited to 1024 hosts.
I think things evolved into MAC per NIC because when add-in NICs were invented there wasn't any appropriate non-volatile storage on the host to store the address.
On really old Sun gear, the MAC address was stored on a separate ROM chip; if the motherboard was replaced, you'd just move the ROM chip to the new board.
I'm not sure what you mean, though, when you say "when add-in NICs were invented" -- the Ethernet cards I used in 1982 plugged into Unibus slots on our VAXen, so that goes back quite a ways...
More that as add-in cards supplied their own "storage" for the MAC address, rather than expecting it from the host (e.g. something like MAC addresses set by init scripts at boot or the ROM chip you mentioned on Suns), this has now evolved into an expected model of a MAC address tightly bound to an Ethernet interface and supplied by the Ethernet interface e.g. by an add-in board if one is added. Now that this model as been around for a long time, people find it a bit strange when MAC addresses aren't as tightly bound to a NIC/Ethernet interface. This is all speculation on my part though, I'd be curious if the reasons are different.
When I first read that paper, it was really quite surprising that "MAC" addresses were designed to be more general host addresses/identifiers that were also to be used as Ethernet addresses. One example they talk about is using them as unique host identifiers when sharing files via floppy disk.
My Ethernet experience goes back before VAXen and the DEUNA to the original DIX Ethernet 3Com and InterLAN cards. They had the MAC in a ROM on the card set. (Yes, they were 2 card sets with a top of the card ribbon cable between them.) Don't confuse this with the REALLY old Ethernet V1 3Com and Wang 1 and 10 Mbps Ethernet, which I did not personally deal with. I worked with early Ethernet on quite a few systems and the only one I ever ran into that implemented the single per-system hardware MAC was Sun, though others (notably Digital, SGI and Xerox) would re-write all MACs with a single value derived from the network address (DECnet or XNS) at boot time. I seem to remember that Tektronix systems also did this before they bought the rights to the CMU TCP-IP stack and moved to IP. -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: oberman@es.net Phone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751
On Sun, 2 Jan 2011, Steven Bellovin wrote:
This was actually the intended way to use "MAC" addresses, to used as host addresses rather than as individual interface addresses, according to the following paper -
"48-bit Absolute Internet and Ethernet Host Numbers" Yogan K. Dalal and Robert S. Printis, July 1981 http://ethernethistory.typepad.com/papers/HostNumbers.pdf
Yup.
That paper also discusses why 48 bits were chosen as the size, despite "Ethernet systems" being limited to 1024 hosts.
I think things evolved into MAC per NIC because when add-in NICs were invented there wasn't any appropriate non-volatile storage on the host to store the address.
On really old Sun gear, the MAC address was stored on a separate ROM chip; if the motherboard was replaced, you'd just move the ROM chip to the new board.
And I'm sure many will remember that Suns of a certain vintage with multiple ethernet interfaces would use that same "host" MAC address on all those interfaces, unless you weaved some magic in the eeprom to use the (presumably) burned-in MAC address of the interface itself. I have long forgotten precisely what the incantation was now ... Jethro. . . . . . . . . . . . . . . . . . . . . . . . . . Jethro R Binks, Network Manager, Information Services Directorate, University Of Strathclyde, Glasgow, UK The University of Strathclyde is a charitable body, registered in Scotland, number SC015263.
Two different suppliers - one was out of Wisconsin (I believe; it's been some time), and the other of Phoenix for the most recent batch. I have lots and lots of HP server gear - and never encountered such bizarre issue. On 1/1/11 9:59 PM, "Brielle Bruns" <bruns@2mbit.com> wrote:
On 1/1/11 8:33 PM, Graham Wooden wrote:
So here is the interesting part... Both servers are HP Proliant DL380 G4s, and both of their NIC1 and NIC2 MACs addresses are exactly the same. Not spoofd and the OS drivers are not mucking with them ... They¹re burned-in I triple checked them in their respective BIOS screen. I acquired these two machines at different times and both were from the grey market. The ³What the ...² is sitting fresh in my mind ... How can this be?
From the same grey market supplier?
I know HP has a disc they put out which updates all the firmware/bios in a specific server model, its not too far fetched that a vendor might have a modified version that also either purposely or accidentally changes the MAC address. Off the top of my head, I'm not sure where the MAC is stored - maybe an eeprom or a portion of the bios flash. Or, it could be botched flashing that blew away the portion of memory where that was stored and the system defaulted to a built in value.
Excellent example is, IIRC, the older sparc stuff, where the ethernet cards didn't have MAC addresses as part of the card, but were stored in non-volatile or battery backed memory. Memory goes poof, and you'll have problems. Some WRT54G/WAP54Gs suffer from the same problem when throwing third party firmware on there.
So along simlar lines, Ubiquiti sell routerstation pro boards with sequential MAC addresses. The trouble is they've allocated a single MAC for the first port - the second ethernet port (also attached to the bridge) doesn't get a second MAC. So in a purchase of a few hundred boards, we had plenty that were sequential. Since the FreeBSD driver allocated MAC+1 to the second NIC, this caused duplcate MAC addresses and this caused hilarity to ensue. The "fix" was to just get this company to apply for some MAC space and then use -that- on the second NIC and the bridge interfaces. Ah, vendors.. :-) Adrian On Sat, Jan 01, 2011, Graham Wooden wrote:
Hi there,
I encountered an interesting issue today and I found it so bizarre ? so I thought I would share it.
I brought online a spare server to help offload some of the recent VMs that I have been deploying. Around the same time this new machine (we?ll call it Server-B) came online, another machine which has been online for about a year now stopped responding to our monitoring (and we?ll name this Server-A). I logged into the switch and saw that the machine that stopped responding was in the same VLAN as this newly deployed, and then quickly noticed that Server-A?s MAC address was now on Server-B?s switch port. ?What the ...? was my initial response.
I went ahead and moved Server-B?s to another VLAN, updated the switchport, cleared the ARP, and Server-A came back to life. Happy new year to me.
So ? here is the interesting part... Both servers are HP Proliant DL380 G4s, and both of their NIC1 and NIC2 MACs addresses are exactly the same. Not spoofd and the OS drivers are not mucking with them ... They?re burned-in ? I triple checked them in their respective BIOS screen. I acquired these two machines at different times and both were from the grey market. The ?What the ...? is sitting fresh in my mind ... How can this be?
In the last 15 years of being in IT, I have never encountered a ?burned-in? duplicated MACs across two physically different machines. What are the odds, that HP would dup?d them and that both would eventually end up at my shop? Or maybe this type of thing isn?t big of deal... ?
-graham
-- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $24/pm+GST entry-level VPSes w/ capped bandwidth charges available in WA -
On 02/01/2011, at 2:33 PM, Graham Wooden wrote:
I encountered an interesting issue today and I found it so bizarre – so I thought I would share it.
Had a fun one with D-Link ADSL modems a few years ago. The MAC address used to source PPPoE frames from the ADSL interface was the same in a batch of modems. This wasn't a problem when using our wholesaler's ATM based DSLAMs, but when we moved users to our own Ethernet based systems, we ran into some fun. The DSLAMs were configured to rewrite the user MAC address into a constructed address including their DSLAM port and PVC details toward the BRAS. Even though this rewrite was occurring correctly, within a single DSLAM unit two users with the same real MAC address would have five minutes of connectivity each, alternating between the two ports. Rgds, - I.
On 1/1/11 7:33 PM, Graham Wooden wrote:
So here is the interesting part... Both servers are HP Proliant DL380 G4s, and both of their NIC1 and NIC2 MACs addresses are exactly the same. Not spoofd and the OS drivers are not mucking with them ... They¹re burned-in I triple checked them in their respective BIOS screen. I acquired these two machines at different times and both were from the grey market. The ³What the ...² is sitting fresh in my mind ... How can this be?
In the last 15 years of being in IT, I have never encountered a ³burned-in² duplicated MACs across two physically different machines. What are the odds, that HP would dup¹d them and that both would eventually end up at my shop? Or maybe this type of thing isn¹t big of deal... ?
None of the HP servers I have contain duplicate MAC addresses. (I just looked through all the iLO2 cards to make sure I wasn't lying.) I'll send you some details offlist. ~Seth
Hey Seth, thanks for the reply. I don't use the iLO port, so I didn't look at it's MAC within the BIOS, however my issue isn't that the MACs are the same within a physical machine. They're different, just like all the other HP gear ... It's that I have two machines that the MACs are identical. Like Server-A's NIC1 matches Server-B's NIC1 ... And the same goes for NIC2. Heck, maybe even their iLO matches too. I just re-read my post and I can see where maybe I didn't explain it properly. Yesterday was a long day ... I guess it's not that big of deal now, I resolved it rather quickly by putting Server-B on another VLAN. On 1/2/11 12:56 AM, "Seth Mattinen" <sethm@rollernet.us> wrote:
On 1/1/11 7:33 PM, Graham Wooden wrote:
So here is the interesting part... Both servers are HP Proliant DL380 G4s, and both of their NIC1 and NIC2 MACs addresses are exactly the same. Not spoofd and the OS drivers are not mucking with them ... They¹re burned-in I triple checked them in their respective BIOS screen. I acquired these two machines at different times and both were from the grey market. The ³What the ...² is sitting fresh in my mind ... How can this be?
In the last 15 years of being in IT, I have never encountered a ³burned-in² duplicated MACs across two physically different machines. What are the odds, that HP would dup¹d them and that both would eventually end up at my shop? Or maybe this type of thing isn¹t big of deal... ?
None of the HP servers I have contain duplicate MAC addresses. (I just looked through all the iLO2 cards to make sure I wasn't lying.) I'll send you some details offlist.
~Seth
About 11-12 years ago, we ghosted Compaq Prosignia 330? desktops with Intel NICs. When we ghosted them, some of the desktops ended up with the same MAC addresses on the NICs. It turned out that there were two different models of Intel NICs in the desktops and ghosting the desktop with the second type of NIC resulted in the MAC address from the original Ghost computer put on that computer. Updating the NIC driver resolved the issue. Eric
---------- Original Message ----------- From: Graham Wooden <graham@g-rock.net>
Hi there,
I encountered an interesting issue today and I found it so bizarre so I thought I would share it.
I brought online a spare server to help offload some of the recent VMs that I have been deploying. Around the same time this new machine (we¹ll call it Server-B) came online, another machine which has been online for about a year now stopped responding to our monitoring (and we¹ll name this Server-A). I logged into the switch and saw that the machine that stopped responding was in the same VLAN as this newly deployed, and then quickly noticed that Server- A¹s MAC address was now on Server-B¹s switch port. ³What the ...² was my initial response.
Fresh OS install from scratch or did you load an image from an existing server? What make/model of on-board NICs? -- Randy M.
I should note -- this isn't that surprising. The IPv6 stateless autoconfig RFCs have always assumed that this could happen, which is why duplicate address detection is mandatory.
In the early 90's a friend of mine got a box of 10 HP cards with all the same MAC address. ----- Original Message ----- From: "Graham Wooden" <graham@g-rock.net> To: nanog@nanog.org Sent: Sunday, 2 January, 2011 4:33:46 PM Subject: The tale of a single MAC Hi there, I encountered an interesting issue today and I found it so bizarre so I thought I would share it. I brought online a spare server to help offload some of the recent VMs that I have been deploying. Around the same time this new machine (we¹ll call it Server-B) came online, another machine which has been online for about a year now stopped responding to our monitoring (and we¹ll name this Server-A). I logged into the switch and saw that the machine that stopped responding was in the same VLAN as this newly deployed, and then quickly noticed that Server-A¹s MAC address was now on Server-B¹s switch port. ³What the ...² was my initial response.
On Jan 2, 2011, at 1:24 PM, Franck Martin wrote:
In the early 90's a friend of mine got a box of 10 HP cards with all the same MAC address.
In my early days of network admining, a coworker told me a (apocryphal) story of 3com shipping a batch of 80K cards with identical MAC addresses, which they then had to recall. Unfortunately a cursory Google turns up nothing, so I suppose he was either misinformed or pulling my leg. -- Corey Quinn / KB1JWQ
On Jan 2, 2011, at 8:39 PM, Corey Quinn wrote:
On Jan 2, 2011, at 1:24 PM, Franck Martin wrote:
In the early 90's a friend of mine got a box of 10 HP cards with all the same MAC address.
In my early days of network admining, a coworker told me a (apocryphal) story of 3com shipping a batch of 80K cards with identical MAC addresses, which they then had to recall.
Unfortunately a cursory Google turns up nothing, so I suppose he was either misinformed or pulling my leg.
I have also heard such stories, again from the '90s. Can cause odd failure modes. Regards Marshall
-- Corey Quinn / KB1JWQ
On 1/2/2011 6:00 PM, Marshall Eubanks wrote:
On Jan 2, 2011, at 8:39 PM, Corey Quinn wrote:
On Jan 2, 2011, at 1:24 PM, Franck Martin wrote:
In the early 90's a friend of mine got a box of 10 HP cards with all the same MAC address.
In my early days of network admining, a coworker told me a (apocryphal) story of 3com shipping a batch of 80K cards with identical MAC addresses, which they then had to recall.
Unfortunately a cursory Google turns up nothing, so I suppose he was either misinformed or pulling my leg.
I have also heard such stories, again from the '90s. Can cause odd failure modes.
Google does NOT know all. I was there. I have had to deal with a building full of such wickedness. I administered DNS (in my copious spare time) for two subdomains, and managed the network in the building (a not inconsiderable /22, and also in my spare time), and started getting frantic calls from people who were getting knocked off the network because their machine had the same MAC address as another. I had trouble believing it at first, but after dealing with five of them (all Gateways, and yes, all with the same MAC address), I directed the local sysadmins to disable the nic that came with them, and to replace it with a spare. I understand that there were 30,000 of them, all with the same address. My guess is that you'll never find it on Google, since it happened around 1993-4 or so. -- A picture is worth 10K words -- but only those to describe the picture. Hardly any sets of 10K words can be adequately described with pictures.
On Jan 3, 2011, at 10:31 AM, Lynda wrote:
My guess is that you'll never find it on Google, since it happened around 1993-4 or so.
I remember that there were several high-profile instances of duplicate MAC addresses being burnt into NICs during the 1990s - once every 2-3 years, IIRC. And those were just the ones that were discussed publicly. Not to mention the old ARCNet NICs, which all came set to the same ARCNet address by default (one changed the address assignment via DIP switches on the cards themselves). ;> ------------------------------------------------------------------------ Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Most software today is very much like an Egyptian pyramid, with millions of bricks piled on top of each other, with no structural integrity, but just done by brute force and thousands of slaves. -- Alan Kay
On Mon, 3 Jan 2011, Dobbins, Roland wrote:
I remember that there were several high-profile instances of duplicate MAC addresses being burnt into NICs during the 1990s - once every 2-3 years, IIRC. And those were just the ones that were discussed publicly.
D-Link shipped NAT-boxes around 2003-2004 or so with identical MAC addresses (and a "clone your PC mac address to the WAN interface"-functionality). I checked my then employer ADSL network and 5% of the customer ports had the same MAC address, D-Link support alledgedly said something about the MAC address not being "unique enough" and directed their customers to the cloning functionality to "solve" the problem. -- Mikael Abrahamsson email: swmike@swm.pp.se
Subject: Re: The tale of a single MAC
On Mon, 3 Jan 2011, Dobbins, Roland wrote:
I remember that there were several high-profile instances of duplicate MAC addresses being burnt into NICs during the 1990s - once every 2-3 years, IIRC. And those were just the ones that were discussed publicly.
D-Link shipped NAT-boxes around 2003-2004 or so with identical MAC addresses (and a "clone your PC mac address to the WAN interface"- functionality). I checked my then employer ADSL network and 5% of the customer ports had the same MAC address, D-Link support alledgedly said something about the MAC address not being "unique enough" and directed their customers to the cloning functionality to "solve" the
On Mon, jan 03, 2011 at 07:05:24, Mikael Abrahamsson wrote: problem.
-- Mikael Abrahamsson email: swmike@swm.pp.se
Years ago D-link and Linksys and maybe other vendors used the source MAC of 00:00:00:00:00:00 which isn't very nice and could cause interesting issues. At my current job we used to have a routine to find these MACs and tell the users to change to a valid address.
On Sun, 2 Jan 2011, Lynda wrote:
Google does NOT know all. I was there. I have had to deal with a building full of such wickedness. I administered DNS (in my copious spare time) for two subdomains, and managed the network in the building (a not inconsiderable /22, and also in my spare time), and started getting frantic calls from people who were getting knocked off the network because their machine had the same MAC address as another.
I had trouble believing it at first, but after dealing with five of them (all Gateways, and yes, all with the same MAC address), I directed the local sysadmins to disable the nic that came with them, and to replace it with a spare. I understand that there were 30,000 of them, all with the same address. My guess is that you'll never find it on Google, since it happened around 1993-4 or so.
You will now. . . . . . . . . . . . . . . . . . . . . . . . . . Jethro R Binks, Network Manager, Information Services Directorate, University Of Strathclyde, Glasgow, UK The University of Strathclyde is a charitable body, registered in Scotland, number SC015263.
On Sat, Jan 01, 2011 at 09:33:46PM -0600, Graham Wooden wrote:
Hi there,
I encountered an interesting issue today and I found it so bizarre � so I thought I would share it.
I brought online a spare server to help offload some of the recent VMs that I have been deploying. Around the same time this new machine (we�ll call it Server-B) came online, another machine which has been online for about a year now stopped responding to our monitoring (and we�ll name this Server-A). I logged into the switch and saw that the machine that stopped responding was in the same VLAN as this newly deployed, and then quickly noticed that Server-A�s MAC address was now on Server-B�s switch port. �What the ...� was my initial response.
I went ahead and moved Server-B�s to another VLAN, updated the switchport, cleared the ARP, and Server-A came back to life. Happy new year to me.
So � here is the interesting part... Both servers are HP Proliant DL380 G4s, and both of their NIC1 and NIC2 MACs addresses are exactly the same. Not spoofd and the OS drivers are not mucking with them ... They�re burned-in � I triple checked them in their respective BIOS screen. I acquired these two machines at different times and both were from the grey market. The �What the ...� is sitting fresh in my mind ... How can this be?
In the last 15 years of being in IT, I have never encountered a �burned-in� duplicated MACs across two physically different machines. What are the odds, that HP would dup�d them and that both would eventually end up at my shop? Or maybe this type of thing isn�t big of deal... ?
We got a batch of NICS that had duplicate MACs in several pallets of IBM desktops, about 15 years back. We noticed this only when two of the machines were shipped to the same field office location. I've heard other state agencies talk about the same sort of problem with IBM and several other vendors. -- Mike Andrews, W5EGO mikea@mikea.ath.cx Tired old sysadmin
On 01/01/2011 09:33 PM, Graham Wooden wrote:
In the last 15 years of being in IT, I have never encountered a ³burned-in² duplicated MACs across two physically different machines. What are the odds, that HP would dup¹d them and that both would eventually end up at my shop? Or maybe this type of thing isn¹t big of deal... ?
Out of curiosity, have you checked if there's a sticker on the board with the MAC address(es)? I know a lot of vendors do that. Jima
One interesting aside to all of this... HP Lefthand SAN actually licenses the SAN/IQ software off of the NIC1 MAC address. I can't help but wonder if the MAC address is set to that specific address to possibly get around that (perhaps a leaked license or something). If nothing else... You can license both boxes with the same license of SAN/IQ, just don't put them in the same cluster. ;) Tom
participants (24)
-
Adrian Chadd -
bmanning＠vacation.karoshi.com -
Brielle Bruns -
Corey Quinn -
Daniel Dib -
Dobbins, Roland -
Eric Tow -
Express Web Systems -
Franck Martin -
Graham Wooden -
Ian Henderson -
Jethro R Binks -
Jima -
Kevin Oberman -
Lynda -
Mark Smith -
Marshall Eubanks -
Mikael Abrahamsson -
mikea -
Randy McAnally -
Raul Rodriguez -
Seth Mattinen -
Steven Bellovin -
Suresh Ramasubramanian