Fix Path Selection Policy for a whole vCenter Cluster

These last few weeks, I’ve been toying with PowerCLI (and PowerShell for that matter). One thing I do have to say, is that Microsoft finally did it right! It’s a useable, program-able command line interface for Windows after all! Thanks to Ivo Beerens and his post “Best practices for HP EVA, vSphere 4 and Round Robin multi-pathing“, I was able to come up with the below:

This works great, however you could make it work on the whole vCenter inventory, which I don’t want. We usually add LUNs to a single cluster at one time. Only thing you might need to change, is the canonical name. Mine simply says “find all SVC LUNs” and you might need to change it, if you’re using a different storage.

Custom certificates in VMware vSphere

Finally, after about 6 months (I last talked about that on February 25th, when Virtual Center 2.5U4 was released) our troubles with our “custom” certificates seems to be resolved! As it turns out, it really was our fault and not VMware’s.

When generating the pfx from the signed certificate and the key-file, you need to supply a password, otherwise the vCenter service is unable to utilize the private key of the pfx, since it’s unable to access the PFX with the default password (testpassword is the default for Virtual Center as well as vSphere).

As noted in the Replacing VirtualCenter Server Certificates document for Virtual Infrastructure 3, as well as through our Customer support, you need to specify the password when exporting the signed crt/Private key into the pfx:

After successfully doing so, you just need to replace the original files (hopefully move them away beforehand) with the ones generated. And afterwards, you should be able to utilize your new certificates! When you now try to clone a template and customize it using an existing customization spec, you’re gonna see this:

vCenter: Cannot decrypt password
vCenter: Cannot decrypt password

After clicking on “OK“, you’re gonna get the normal customization specification edit frame, where you should be able to skip ahead to “Workgroup or Domain“, where you’re gonna have to reenter the domain administrator password.

VMware: New VirtualCenter 2.5 Update 4

As many people on the VM-Planet already blogged about this, I ain’t gonna write just about it. Let’s turn the clock back a few months, to January 2008.

As the institution I work for, is part of the DFN we took the opportunity to be a part of the “I want you to run our RA“-gang. In January 2008 we thought about changing the vCenter certificate. Now, apparently there’s a slight difference between the DFN-PCA and what VMware considers common practice.

The DFN-PCA states, that only CSR’s with a key length of 2048 bits are allowed (as outlined in 6.1.5 of the DFN-PKI Certificate Policy). Now VMware apparently didn’t actually think customers would use this “feature” (that is changing the SSL certificates).

Customization Specifications Created in Previous Releases Can Be Used in VirtualCenter 2.5 Update 4 to Clone or Deploy Virtual Machine with Customized Guest Operating Systems
This release resolves an issue where, if you clone or deploy a virtual machine using a customization specification that was created prior to upgrading the VirtualCenter, the VirtualCenter Server might display the error message The VirtualCenter server is unable to decrypt the passwords stored in the customization specification in the following scenarios:

  1. VirtualCenter Server is uninstalled first, and then re-installed and/or upgraded afterwards.
  2. Custom SSL certificate are deployed, but the instruction in http://www.vmware.com/pdf/vi_vcserver_certificates.pdf are not followed in a verbatim manner.

Well, and apparently it ain’t fixed yet. At least not for us 😕

VMware vCenter: is not connected

Well, today I once again had the case where a virtual machine (in my case a Virtual Machine Template) was kinda stuck. You couldn’t remove the template (as in the entries for “Remove from inventory” was grayed out) and you couldn’t re-add the Virtual Machine’s VMX from the datastore browser either.

VI Client - Disconnected templates
VI Client – Disconnected templates

Though, a simple putting the host into maintenance mode and rebooting helped that problem. Maybe there is a simpler solution for this, I just don’t know about it.

Thanks to Sven in #1, I now know that simple solution for my problem!

Half a minute, and a heart-stopping moment later (all VM’s on that host turn grey after the first update) the VM’s are accessible again. Thanks again to Sven!

VI Client: Changing the language from the system default

Well, as I am in fact running a german Windows XP, the VI Client started displaying all menus and operations in German when I updated to 2.5u2. Normally, I wouldn’t have much of a problem with that, but recently it started to annoy me, since the translation is a bit off from the real meaning of much of the operations.

So today, in the morning I started looking for ways to revert the VI Client back to displaying everything in English. And guess what. There’s no way to switch the language from the VI Client itself. There’s just a workaround.

Simply rename the folder in “%ProgramFiles%VMwareInfrastructureVirtual Infrastructure Client2.5“, %ProgramFiles%VMwareInfrastructureVIUpdate” “%ProgramFiles%VMwareInfrastructureVirtual Infrastructure ClientPluginsConverter Enterprise 4.0.2” and “%ProgramFiles%VMwareInfrastructureVirtual Infrastructure ClientPluginsUpdate Manager 1.0 Update3” named “de” to something else. Tada, your VI Client is back in English.

More VirtualCenter troubles (fini)

Well, today the support request came back. Seems one of the originally linked VMTN dicussions really is the only way:

  1. Export the customization specification
  2. Edit the XML file
  3. Import it again

The related part inside the customization specification should then look like this:

So if you ever think about switching the default VirtualCenter certificate (for whatever reason), make sure you use the above workaround. Otherwise VirtualCenter is gonna fail miserably during the customization phase of the cloning process.

More VirtualCenter troubles

Well, after my co-worker switched the VirtualCenter certificates with one produced by our RA a few days ago, I can’t clone anything using a customization specification anymore.

Unable to decrypt passwords in customization specification
Unable to decrypt passwords in customization specification

Guess, we’re shit outa luck. At least both of those linked VMTN discussions don’t contain any (that is for us) workable solution (well besides storing the password in cleartext in the spec — which ain’t sooo good). Gonna bug him tomorrow to open up a VMware support request, maybe that’ll help somewhat. I sure hope so.

VMware design rules

I’m just got back from four days in Rostock over at S&N, where I was attending a VMware design course and here’s a list of questions I did ask the trainer:

  1. What’s the disadvantage of having a 1016 ported vSwitch ?
  2. Any clues on how to exchange the default certificate of the Virtual Center ?
  3. Are there any tools to stress test the virtual system ?
  4. Are there any performance impacts of having more than 10 users in Virtual Center ?
  5. Any clues and/or guides on how to do time synchronization in VMware guests, especially Linux guests ?
  6. What’s the preferred NIC type for Linux guests ?
  7. Any clues to using Raw Device Mappings with VMotion ?
  8. Is there a way of defining CPU masks on a global level ?

Answers:

  1. There might be a small overhead, though that’s limited to a really, non-measureable amount
  2. Hasn’t done it yet.
  3. Yes, there are free stress test tools like cpubusy.vbs, cpubusy.pl, iometer.exe, ..
  4. Nope, you should only experienece load problems starting at 25 or so users
  5. Select *one* variant, either time synchronization by use of the VMware tools or ntpupdate; if ntpupdate, select a single time source for your whole environment
  6. For ESX 3.5.0 that would be “Flexible” (as per VMware Knowledgebase), as the vmxnet type is a leftover from ESX 3.0
  7. Raw device mappings are *absolutely* supported by VMware, and also work without any troubles (when mapping/zonig is correctly configured)
  8. Currently there’s no known way of doing this
  • When adjusting the CPU afinity of a VM, *always* completely stop the virtual machine afterwards
  • When trying to figure out CPU bottlenecks, check whether or not hyperthreading is enabled. The hyperthreaded (second) core is only giving you a CPU with 15% of the first.

Also, here are some guidelines on how the trainer extended the defaults:

ESX Server:

  • Extend the “/” size to 10GiB
  • Extend the “swap” partition to about 1GiB
  • Extend the “/var/log” partition to about 4 GiB
  • don’t mess around with creating too many vSwitches; just keep it simple
  • set the duplex mode manually if the ESX is giving you any trouble
  • disable the Traffic Shaping, unless you *really* need it

VirtualCenter:

  • There’s two options when installing VirtualCenter: either install it on a physical box or simply put it into a virtual machine itself
  • A problem with putting it into a virtual machine is, when the VM is shutting down or powered off due to isolation of the ESX running it, any ESX Server powering up isn’t going to start any virtual machines as that in return requires the License Server (as Michael pointed out in #c1, the VM is still gonna start as the HA agent is able to start virtual machines on the basis of the 14-day grace period)
  • Only use the SQL Server Express variant if you really have to. It’s limited to 4GB database size, so if your installation grows above say 50 hosts and 2000 VM’s, this is gonna break the limits of SQL Server Express