BVLog Bryan Voss’ mental synchronization point


Silent install of VMWare Tools under a Windows guest

In vCenter Server, right-click on a virtual machine, click Guest | Install/Upgrade VMware Tools, and enter "/S /v /qn REBOOT=R" in the Advanced field. Reboot will be supressed.


VMWare: HOWTO clone a Windows VM

Care must be taken when cloning a Windows VM to ensure that a new SID is generated in order to properly assign access controls. There are also steps necessary to ensure the new VM has a unique UUID and MAC address. Here is the proper way to clone a Windows VM for hosting under VMWare Server:

  1. Shut down the source VM
  2. Copy the VM directory and rename
  3. Edit the target .vmx file and remove the lines that begin with: "uuid.location", "uuid.bios", "ethernet0.generatedAddress", and "ethernet0.generatedAddressOffset"
  4. Start the target VM and login as local admin
  5. Change the IP address
  6. Run NewSID to replace the existing SID with a generated one. Choose to rename the PC and enter the new name.
  7. Reboot the target VM
  8. Join the domain (if applicable)
  9. You may now boot the source VM

1.27%, baby!

Average CPU usage

We had VMWare run an analysis of 43 servers in our datacenter (about 1/3 of our total) that we thought would be good candidates for virtualization. They ran a monitoring box that collected performance stats for a couple of weeks and compiled the results to report back to us. The final results proved that we are in an absolutely ridiculous state right now. They told us we can consolidate those 43 servers down to 2 or 3 ESX servers running at around 15-20% CPU utilization. Given the way we have to space servers in the racks now due to power and cooling limitations, we could potentially consolidate 4 racks down to a few servers.

How is it that we have so many servers sitting there practically idle sucking up power and cooling 24x7? It comes down the the specs provided by our vendors and the fact that we generally purchase hardware and software as a package deal from the vendor. The vendors spec out the latest and greatest hardware and we just blindly accept what they suggest. After all, they're the experts, right?

We left a lot of servers out of the analysis. Primarily database servers and servers with special hardware like Brooktrout fax boards that can't be virtualized. There are also several systems that the vendors specifically said they would not support under virtual environments. That's another hurdle that we have to overcome: virtualization acceptance. There is some movement in that direction from our vendors, but many are still clueless when we ask about it. There are also a couple of cases that I am aware of where vendors say they cannot support a virtual environment due to licensing restrictions on third party code that they include with their products. That should subside over time as virtualization becomes a standard deployment platform.

One day, enterprise applications will be provided as self-contained virtual appliances that we deploy on a virtualization layer. The hypervisor is becoming the OS and the OS is becoming merely a set of APIs between the hypervisor and the application. Sure, there is a lot of friction from companies like Microsoft that have made their monopolies on operating systems, but the times they are a changin'.


The case of the disappearing eth0

There have been a couple of occasions in the past week that I have lost an ethernet interface when swapping machines around. Looking back into my murky past, I can recall a couple of other times that I probably encountered the same issue. Don't recall how I resolved the issue before, but I have a definite solution now. I figured I should note it here so I can look it up later and so others can benefit from it.

Scenario 1: I build a Debian virtual machine using VMWare Workstation on my laptop. I later move the VM to a VMWare Server box. On first boot, VMWare Server asks if I want to assign a new UUID and I select yes. It turns out that the MAC address assigned to the virtual ethernet device is affiliated with the VMWare UUID. When the UUID changes, the MAC address changes. Debian assigns eth devices based on MAC address and therefore eth0 is lost after the MAC changes. The issue shows up when I try to start networking on the VM and eth0 doesn't come up.

Scenario 2: I install Debian on a PC-class box and tinker with it a while. It breaks (something to do with heat, probably a fan failure). I move the hard drive to an identical box and it boots fine, but eth0 doesn't come up. Same as above. Since Debian assigns the eth devices based on MAC address and the new ethernet device has a different MAC address, I get no eth0.

Solution: A comment on this post pointed me down the path of enlightenment.

/etc/udev/rules.d/z25_persistent-net.rules contains the MAC address to eth device mappings. Delete the lines like below, noting the module name on the "# PCI device" line:

# PCI device xxxxxx:xxxxxx ([module])
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{address}=="xx:xx:xx:xx:xx:xx", NAME="eth0"

This removes the MAC to eth device mapping info. Now we need to restart udev to allow the change to take effect:

/etc/init.d/udev restart

Next step is to "bounce" the kernel module for the ethernet device. Use the module name from the z25_persistent-net.rules file noted above:

modprobe -r [module]
modprobe [module]

"ifconfig" should now show the eth0 interface as up and running. If not, try "ifup eth0" and check "ifconfig" again. That rascally ethernet interface can't hide for long!

Update 2009/03/11: This post details a method that does not require modifying individual VMs. Probably a better solution for template VMs or virtual appliances.


Fixing VMWare Server directory permissions on Debian hosts

I often create a VMWare virtual machine using VMWare Workstation on my laptop, then later move it to a Debian machine running VMWare Server. As a result of the move from Windows to Linux, permissions on the VM directory end up being wrong and I get a blank black screen when connecting the VMWare Server Console to the VM. I normally just fix the permissions manually when I notice the problem.

Today, I was helping my father deal with the same situation via email and started typing out all the steps to create a group and fix permissions on the VM directory when I thought, "Why not be a good little sysadmin and write a script to do this?" So, here's the resulting BASH script:

# Fix VMWare Server permissions on Debian host
# Bryan Voss 2007/07/19

# *** Config items ***
# Group that all VMWare users will belong to
# *** /Config items ***

# Find directory where VMs are stored
VMDIR=`grep vmdir /etc/vmware/config | cut --delimiter=' ' -f 3 | cut --delimiter='"' -f 2`

# Add group that VMs should belong to
addgroup --system $VMGROUP

# Fix permissions on directories under VMDIR
chgrp -R $VMGROUP *
find . -type d -exec chmod g+rwxs \{\} \;

# Fix permissions on vmx files
find . -name *.vmx -exec chmod -R +x,g+rwx \{\} \;
# Fix permissions on all other files
chmod -R g+rw *

Make sure any users who will be connecting via VMWare Server Console are members of whatever group you set VMGROUP to ("vmware" by default).

I have saved this as /usr/local/sbin/vmware-fixperms on my VMWare Server boxes and will probably use it pretty often. Maybe others will benefit from it.