VMWare vMotion not working on ESX VDI (VMWare View)

After clean installs of my 2 ESX VDI systems (now VMWare View) I was unable to get a successful vMotion from one host to the next. I was also having issues getting HA properly configure although after a bit of moving it around, creating a new cluster with a new name and moving the host into that it finally configured. All that said, vMotion still did not work.

After a bit of googling, I found that removing the Virtual Center Agent from the ESX Host and re-installing it might solve the problem. It worked.

I first removed both of my Hosts from virtual center, then logged into the console of the hosts to find the agent version:
rpm -qa | grep -i vmware-vpxa

Then, to remove the agent (replacing the x's with version from above):
rpm -e VMware-vpxa-2.0.x.xxxxx

I repeated this for my second host and then rebooted them both. I had the luxuray of being able to shut off my virtual machines for this process. Once both ESX Hosts were back up I simply went through the process of adding them to Virtual Center, Creating a Cluster and adding them to the cluster.

Upgrade VMware ESX 3.0.1 to ESX 3.0.2

Pretty easy & smooth process to upgrade from VMware Infrastructure ESX 3.0.1 to VMware ESX 3.0.2. Since I'm running my systems off a SAN and using VMWare vmotion all I needed to do was put my first ESX server in maintenance mode and shut it down. This automatically moved all my running VM's via vmotion over to the other ESX server. I then put in the 3.0.2 install / upgrade CD and booted from it. CD Upgrade went pretty quickly, approx 10 minutes and then rebooted the system. After reboot my Virtual Center picked the system with no problems and I turned on a test machine to make sure it was working ok. I then moved over a couple production VM's without problems.

Since the upgrade appeared to work just fine I repeated the process on my second ESX server.

VMWare Upgrade ESX 3.0.0 to ESX 3.0.1

Finally decided to upgrade my VMWare ESX Servers from the original ESX 3.0.0 that I installed back in August to VMWare ESX 3.0.1. Since I have (2) ESX machines running I moved all my production boxes over to server#2 and turned off my test boxes. This gave 8 production systems running on Server#2, but it barley noticed them. I then put Server#1 in maintenance mode and decided to use the .tar upgrade file rather than the CD. Checksum was correct so I went ahead and took a quick backup of a few key directories "/etc" and "/vmware" and then ran the esxupdate command to kick off the upgrade.

Uh oh, failed on VM-Kernal file not matching. After finding other users on the VMTN Discussion Forums with the same problem...but no solutions...I decided to download the ISO file and try an upgrade from CD. Went through the CD Upgrade and about half way through it failed to read the VM-Kernel file from the CD. Seriously? The CD Media checked out fine! So I burned a new CD (at 8x speed just in case) checked it again and started the upgrade process...same thing. Wow, this is lame. So I decided I might as well just do a clean install as I'm sure my system is hosed by now anyway.

Clean install worked fine off the CD Media but now I had to re-setup all my LUN Paths, Networks, DNS, IP's, etc. No biggie, its not too hard and I have Server#2 to look at as a reference. After deleting the new old Server#1 from Virtual Infrastructure and re-adding the new Server#1 I find that I can't light up VMotion and all my test boxes have been (orphaned).
Ok, I'm starting to get worried as I can't really upgrade Server#2 if I can't move my 8 production VM's over to Server#1 and I'm not really sure I want to after this smooth upgrade I've just had.

Time to call VMWare Tech Support. I have the gold support, which means I call them... they take my info and call me back the next day. Not ideal...but at least my production VM's were still running. After a couple days of phone tag with VMWare support I finally get to talk to somebody and he quickly shows me a command to fix my orphaned test VMs ( vmware-cmd -s register /vpx/vms/_/ ). Nice, all my test VM's are back to normal status. Now on to why VMotion wouldn't work. I missed a key step when setting up my Network again, that little check box next to Enable VMotion in the network config. Wow, that was easy. Re-configure for HA and were ready to roll.

Upgrade for Server#2 planned for next week...I hope the upgrade goes much smoother.

VMWare Setup

I just finished my 3rd Academic quarter with 3 of my production servers running as virtual machines. I setup VMWare in the fall and went into production just before classes started fall quarter.

My setup includes (2) Dell 2950's with (2) each Dual Core 3.73 ghz cpu, 16 GB RAM, dual fiber cards, 4 gig nics, and dual power supplies. These 2 servers are directly connected to a Network Appliance 3050c Cluster SAN. (the direct connection, without a fiber switch, was a little tricky to setup but works well as long as I don't ad any more Servers). I setup VMWare ESX 3.0 on both servers and hooked them together with VMotion.

VMotion is sweet. Had a switch go out that was connected to my ESX Server 1 and VMotion auto moved all running servers to ESX Server 2 within seconds. If I hadn't got the email about it from VMotion I would of never known as the systems all stayed in production.