Network failures and the Dell R620

So we recently installed a Dell R620 server with IP Office Server Edition 9.1. While troubleshooting an unrelated issue we came up with a big problem: The device could no longer be reached on the network! Well that's a huge problem since we have IP phones and all the bells and whistles on this server. Avaya support wasn't able to resolve the issue because...guess what? They couldn't get into it remotely...

So we replaced the server and I took a look once we had it back in the lab. It turns out that the devices had been renamed. ETH0 switched with ETH2 and ETH1 switched with ETH3. By plugging in to Port 3 (which should have been ETH2) and enabling the port (#ip link set up) I was able to get connectivity right away. But how could this have happened, and what can I do to make it work with the correct network port? It turns out there is an issue with the way CentOS uses UDEV to create network interfaces. So how do you fix it? Well it's not too hard. I was able to make it work exactly as it did before by looking at a few files and changing one.

In the root folder there are four interesting files. They are ifcfg-eth0, ifcfg-eth1, ifcfg-eth2, and ifcfg-eth3. These files each contain a single line of text with the MAC address of the server:

HWADDR="b8:2a:72:dc:17:28"

Trust these files. They are the gospel. I was also able to find the MAC address of eth0 (Port 1) printed on the circuit board inside the server. The four addresses should be sequential, but keep in mind that they are sequential in HEXADECIMAL. So...29 does not go to 30, 29 goes to 2a.

So you may wonder, now that we have these MAC addresses, what are we going to do with them?

That's easy. Use your favourite text editor (I personally like NANO) to edit the file /etc/udev/rules.d/70-persistent-net.rules. You'll see your interfaces and, I bet, incorrect MAC addresses. Or at least MAC addresses that are not associated with the correct device name. Simply change the MAC addresses to the correct address for each device and you're on your way.

For example, change:

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="b8:2a:72:dc:17:2a", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

 to:

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="b8:2a:72:dc:17:28", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"