Upgrading the IP Office

I'm not going to tell you how to upgrade your IP Office, that's what the official training is for. I am, however, happy to discuss the various issues that I have experienced while performing upgrades, and the steps I take to ensure that the upgrade goes smoothly.

File Write Errors

I often see people struggling when transferring files to the IP Office during an upgrade. More times than I'd like to count I've seen the system pop up with an error stating "405 Method Not Allowed".  While there are a number of things that can cause this issue the most common, by far, is the File Writer IP field in the system tab. If you're getting this error just check there and see if there is an address populated. I usually set the address to 0.0.0.0, which allows the writing of files from any address. While this may present a security risk, as long as you follow strong password management and don't leave any of the default passwords in place the risk should be next to zero. Alternatively you could specify the address of the PC being used to perform the upgrade, restricting the ability to write to the memory card to just that device. The third option is to set the address to 255.255.255.255, which will cause the system to update the address in this field the next time embedded file management is used to reflect the address of the system that is using embedded file management.

In some cases, even after making the change to the File Writer IP address, some installations still don't proceed. In these cases I've seen a couple of solutions. In some instances the system will allow the completion of the upgrade to the core equipment, and not the update of the files on the SD Card. When this happens it is often possible to simply log in to Embedded File Management and upload the system files from there. If that doesn't work, however, the cause is usually a corrupted SD Card. This happens most often on systems that have been through a number of upgrades, or potentially if you've forgotten that you have to step up from any release below 8.1(65) to 8.1(65) or 9.0 prior to upgrading to the current release. When this happens the best plan is to pop out the SD Card, put it in your laptop, and recreate the SD Card. Once this is complete pop it back in to the system and push your backed up config (and any embedded voicemail files, if applicable) back to the system. Once this is complete you should be able to access the system just fine.

Server Edition and Application Server Upgrades

Server Edition and Application Server upgrades appear to take forever, frequently appearing to stop around the 92% complete mark. This has caused panic for so many of my colleagues over the years. The first time I encountered this I panicked and rebooted the server. Let me tell you, this is a mistake! Now I've learned a thing or two and I have found a better way to keep an eye on the progress than simply looking at that 92% for what seems like an eternity. Log in to the server as root and type the following command:

tail -f /var/log/yum.log

This command will continuously scroll the output of the YUM application. YUM, or the Yellowdog Updater, Modified, is the Linux program that the IP Office uses to manage the installation of software. The yum.log file will scroll frequently even if the Web Manager progress indicator hasn't moved. This should keep your mind at ease while you wait.

It is possible that the updater may fail. Avaya has documented issues with the upgrade failing, and has even written code into the update software to identify issues with YUM failing to complete the upgrade. If you see the error "yum process died before completing its job" you have experienced an issue with the /boot/ partition being full. This happens, again, on systems that have been upgraded multiple times. If your system has been upgraded to at least 9.1.7 you should not experience this issue. If you do have this issue after 9.1.7 engage Avaya support right away, and anticipate a rebuild. If you are at a release older than 9.1.7 there is a quick fix. Download the file UpgradeKernelFix_v2.zip from Avaya. This zip file contains a patch that will allow you to expand the /boot/ partition. Extract UpgradeKernelFix_v2.sh from the zip file and upload it to your server using WinSCP, or your favorite SFTP client. Browse to the location of the file as root and type chmod +x UpgradeKernelFix_v2.sh - this makes the file executable. Run the file with ./UpgradeKernelFix_v2.sh and you should be off to the races. Of course, you will have to start the upgrade again. This issue should only affect systems running on VMWare, however I wouldn't rule it out for bare metal servers. 

You can download the fix from Avaya's web site here.

I can not stress enough the importance of making a full backup before you begin. For an application server this means assessing the applications used and ensuring that all application data is backed up off the system before you begin. For systems using embedded voicemail this means ensuring that you save all the dynamic contents of the SD Card before you even think of starting. 

Let me know if you've experienced any other errors, and if so, what is your resolution. I'll try to help out anyone who is stuck if I can!