vCenter: Removing VSC custom attributes

Well, yesterday I got pissed of those Virtual Storage Console custom attributes.

Currently we don’t use the Provisioning & Cloning feature of the VSC, thus we don’t need the custom attributes. After poking around, I decided to write a short PowerCLI script to do the task.

It’s really rather simple, so here goes:

VMs in Alarm state after scheduled maintainance

Well, I’m back at work after three weeks of vacation (some pictures to follow) and the provider hosting our disaster datacenter had their annual (or is it monthly now?) SAN maintainance, so we shut down everything over there by 9:00 am.

After things were back up around 5pm, I booted the ESX hosts, however the VMs we’re all displaying the alert state – as if either the VMs had an HA event or we’re using to much CPU time. It didn’t matter whether or not the VM was running or not, the state persisted.

vCenter - VM in alarm state
vCenter – VM in alarm state

Lucky me, someone else already ran into this issue. So, after simply vMotion’ing the VMs to another host and the VM would no longer be in that alert state.

VMware Update Manager issues

Well, I recently (last Wednesday) had a lot of trouble with Update Manager.

First I thought, upgrading vCenter and modules to 5.0U1 would solve my troubles, however it did not. Update Manager was still complaining about something. Since neither in the vCenter Update Manager nor the vCenter log itself were having any useful information I enabled SSHd and the ESXi Shell via the vCenter client:

SSH’ed into the ESX host and looked at /var/log/esxupdate.log, and found this particular log:

As you can see from the above log, for whatever reason, the Update Manager is trying to install an updated version of the VMware HA agent (vmware-fdm). Now, since that isn’t the job of Update Manager (afaik, vCenter handles that separately), I figured what the hell, let’s try and remove it as the ESX host was already in Maintainance Mode:

And guess what: after removing the package and then rerunning the Remediate task on the host, my troubles were gone.

Microsoft Cluster on VMware and Devices

Well, once again the Microsoft Cluster on VMware bit my ass … As you might know, MSCS on VMware is a particular kind of pain, with each upgrade you end up with the same problem over and over again (SCSI reservations on the RDM-LUNs being one, and the passive node not booting being the other).

So I opened up another support case with VMware, and the responded like this:

Please see this kb entry: http://kb.vmware.com/kb/1016106

This doesn’t completely fit my case, but since the only active cluster-node failed yesterday evening (it’s only our internal file-share server, thus no worries), I thought I’d try to set the options.

And guess what ? My damn cluster works again 🙂

Reconfiguring NTP settings vCenter-wide

I recently started reinstalling all my ESX hosts, so I wrote up a short script that is reconfiguring all hosts and sets the NTP configuration according to my wish:

As you can see, the script takes the vCenter hostname and two NTP servers and basically applies it to each host in your vCenter environment.

Rename a Standard Port Group on all hosts in a cluster

Well, I recently decided to rename a bunch of my Standard Port Groups, since they did no longer reflect the network they were providing. Since I’m a lazy bastard (well lazy as in click lazy), I wrote this little PowerCLI script:

This script basically takes a vCenter instance and a single cluster, then creates a new Port Group on each host, after which it reconfigures all VMs possessing a virtual NIC with that Port Group and then deletes the old Port Group.

Empty Port SSL after ADAM installation

I’ve been meaning to post this, but never actually got around to doing that. When installing vCenter 5.0, an instance of ADAM is installed, which stores all the configration data for Linked Mode.

It basically boils down to running this script and rebooting the box:

This is no new invention of myself, just writing it down for myself from here or here.

Changing the vCenter hostname

I recently reinstalled the vCenter Server at work, and in my never ending wisdom *cough*, I decided to do that on new hardware. That entitled using the same host name plus the appendix _NEW. Now, I know this isn’t conforming with DNS naming schemes (iirc underscore isn’t a valid DNS char), however it worked …

So once I had everything installed on the new hardware, I switched the ESX servers from the old vCenter to the freshly installed one. Once that was finished, I shut down the old vCenter server, changed IP address and host name of the new one and rebooted. That basically worked, even though suddenly every ESX in my inventory was disconnected. After reconnecting all ESX servers everything was back online.

Then I tried sorting the list of All VMs in my vCenter and got the error message “Unable to look up vcenter_NEW.home.barfoo.org” when doing so .. I was like *WTF*, searched the registry found a few ones, and after a quick reboot tried again … No luck, still the same error. After looking through the VMware Knowledgebase, I actually found one that worked for me, even if it didn’t apply to my environment 100 percent (we don’t use Linked Mode) …

After fixing the vmw-vc-URL in each property, the sorting actually works!

Create an offline snapshot of a VM

We’re currently thinking about automating Windows Updates and the involved disaster snapshot-copy to a degree, where we don’t need to intervene anymore.

Right now, we already have a rudimentary scheduler in place, which does the reboots for some (200 ..) systems already. Now, we’d like to extend it to also cover the bi-weekly Windows Update spree.

Since PowerShell (and PowerCLI) work quite well with vSphere automation, I cooked up the below script to first shutdown a virtual machine (for snapshot consistency reasons), then take a snapshot and power on the virtual machine again afterwards.

Modified SnapReminder

Well, PowerCLI makes my life a little bit easier. Believe it or not, each of us vCenter infrastructure admins has one of these: a Windows admin, thinking a snapshot is also a backup. Thankfully, Alan Renouf over at virtu-al.net wrote the SnapReminder, which already helped me a lot! However, occasionally the script isn’t finding the snapshot author (for whatever reason).

Since I want a notification in that case, I modified the script a little bit to suit my needs.