September 2013

XenServer 6.x: Update all hosts in pool

September 15, 2013September 15, 2013 Christian Leave a comment

Well, what annoyed me in the past was that I had to patch each XenServer patch by patch (no bulk applying) and when used in combination with UCS blades (especially if those have >250GB RAM), it takes ages to keep a pool up-to-date. So I ended up writing yet another script (I know why I hate Citrix XenServer … the XenCenter GUI is lacking sooooo much) which will download new patches from a directory on a HTTP server and then print the lines necessary to apply the patches to all hosts in a pool.

#!/bin/bash

#
# Updates a XenServer to the most current release
#

TMPDIR=$( mktemp -d )

# Figure out the XS release
XS_REL="$( grep PRODUCT_VERSION /etc/xensource-inventory | cut -d\' -f2 )"

#set -x

# Gather the Patchlist/UUID list
UUID="$( xe patch-list | grep ^uuid | awk '{ print $5 }' )"
XSNAME="$( xe patch-list | grep name-label | awk '{ print $4 }' )"

declare -a pUUID=($UUID)
declare -a pXS=($XSNAME)

count=${#pUUID[@]}
count=$((count-1))
for i in $( seq 0 $count ); do
        echo "${pXS[$i]}: ${pUUID[$i]}"
done | sort

# Now get the current patchlist
PATCH_LIST="$( wget -q -O $TMPDIR/digest http://xen.heimdaheim.de/patches/$XS_REL/digest )"

for xs in "${pXS[@]}"; do
        sed -i "/$xs/d" $TMPDIR/digest
done

if [ "$( cat $TMPDIR/digest )" != "" ]; then
	for file in $( cat $TMPDIR/digest ); do
		wget -q -O $TMPDIR/$file http://xen.heimdaheim.de/patches/$XS_REL/$file
	done

	POOL_MASTER="$( xe host-list uuid=$( xe pool-list | grep master | \
			awk '{ print $4 }' ) | grep name-label | awk '{ print $4 }' )"

	while true; do
		read -p "New XenServer updates found. Should we continue ? (y/n) " yn
		case $yn in
			[Yy]* ) break;;
			[Nn]* ) rm -rf $TMPDIR; exit;;
			* ) echo "Please answer yes or no.";;
		esac
	done

	# Upload the patchfiles to the pool master
	for xs in $( grep -v iso $TMPDIR/digest ); do
		echo -n "Uploading XS patch $xs"
		xe patch-upload file-name=$TMPDIR/$xs 1>/dev/null
		echo "..."
	done

	UUID="$( xe patch-list | grep ^uuid | awk '{ print $5 }' )"
	XSNAME="$( xe patch-list | grep name-label | awk '{ print $4 }' )"

	declare -a PUID=($UUID)
	declare -a XS=($XSNAME)

	count=${#PUID[@]}
	count=$((count-1))

	# Write the stuff to a csv, so we can sort it
	for i in $( seq 0 $count ); do
		echo "${XS[$i]};${PUID[$i]}"
	done | sort > $TMPDIR/patchlist.csv

	echo "The following commands need to be issued on the pool master $POOL_MASTER:"
	echo

	for patch in $( cat $TMPDIR/patchlist.csv ); do
		XS="$( echo $patch | cut -d\; -f1 )"
		UUID="$( echo $patch | cut -d\; -f2 )"

		for host in $( xe host-list | grep ^uuid | awk '{ print $5 }' ); do
			echo "xe patch-apply host-uuid=$host uuid=${UUID}"
		done
		echo
	done

	if [ "$( grep iso $TMPDIR/digest )" != "" ] ; then
		echo
		echo "The following commands need to be issued on each host individually:"

		for xs in $( grep iso $TMPDIR/digest ); do
			cp -f $TMPDIR/$xs /root
			echo "$xs needs to be installed manually."
			echo "This can be achieved with the following commands:"
			echo "   mount -o loop /root/$xs /mnt"
			echo "   cd /mnt"
			echo "   ./install.sh"
			echo "   cd"
			echo "   umount /mnt"
		done
	fi
else
	echo "No updates found!"
fi

rm -rf $TMPDIR

100

101

102

103

104

#!/bin/bash

# Updates a XenServer to the most current release

TMPDIR=$( mktemp -d )

# Figure out the XS release

XS_REL="$( grep PRODUCT_VERSION /etc/xensource-inventory | cut -d\' -f2 )"

#set -x

# Gather the Patchlist/UUID list

UUID="$( xe patch-list | grep ^uuid | awk '{ print $5 }' )"

XSNAME="$( xe patch-list | grep name-label | awk '{ print $4 }' )"

declare -a pUUID=($UUID)

declare -a pXS=($XSNAME)

count=${#pUUID[@]}

count=$((count-1))

for i in $( seq 0 $count ); do

echo "${pXS[$i]}: ${pUUID[$i]}"

done | sort

# Now get the current patchlist

PATCH_LIST="$( wget -q -O $TMPDIR/digest http://xen.heimdaheim.de/patches/$XS_REL/digest )"

for xs in "${pXS[@]}"; do

sed -i "/$xs/d" $TMPDIR/digest

done

if [ "$( cat $TMPDIR/digest )" != "" ]; then

for file in $( cat $TMPDIR/digest ); do

wget -q -O $TMPDIR/$file http://xen.heimdaheim.de/patches/$XS_REL/$file

done

POOL_MASTER="$( xe host-list uuid=$( xe pool-list | grep master | \

awk '{ print $4 }' ) | grep name-label | awk '{ print $4 }' )"

while true; do

read -p "New XenServer updates found. Should we continue ? (y/n) " yn

case $yn in

[Yy]* ) break;;

[Nn]* ) rm -rf $TMPDIR; exit;;

* ) echo "Please answer yes or no.";;

esac

done

# Upload the patchfiles to the pool master

for xs in $( grep -v iso $TMPDIR/digest ); do

echo -n "Uploading XS patch $xs"

xe patch-upload file-name=$TMPDIR/$xs 1>/dev/null

echo "..."

done

UUID="$( xe patch-list | grep ^uuid | awk '{ print $5 }' )"

XSNAME="$( xe patch-list | grep name-label | awk '{ print $4 }' )"

declare -a PUID=($UUID)

declare -a XS=($XSNAME)

count=${#PUID[@]}

count=$((count-1))

# Write the stuff to a csv, so we can sort it

for i in $( seq 0 $count ); do

echo "${XS[$i]};${PUID[$i]}"

done | sort > $TMPDIR/patchlist.csv

echo "The following commands need to be issued on the pool master $POOL_MASTER:"

echo

for patch in $( cat $TMPDIR/patchlist.csv ); do

XS="$( echo $patch | cut -d\; -f1 )"

UUID="$( echo $patch | cut -d\; -f2 )"

for host in $( xe host-list | grep ^uuid | awk '{ print $5 }' ); do

echo "xe patch-apply host-uuid=$host uuid=${UUID}"

done

echo

done

if [ "$( grep iso $TMPDIR/digest )" != "" ] ; then

echo

echo "The following commands need to be issued on each host individually:"

for xs in $( grep iso $TMPDIR/digest ); do

cp -f $TMPDIR/$xs /root

echo "$xs needs to be installed manually."

echo "This can be achieved with the following commands:"

echo " mount -o loop /root/$xs /mnt"

echo " cd /mnt"

echo " ./install.sh"

echo " cd"

echo " umount /mnt"

done

else

echo "No updates found!"

rm -rf $TMPDIR

There’s just a little caveat (not because of the script – but of XenServer’s design): if you switch the pool master before the updates are applied to all host, you’ll need to manually delete the patches from the pool (xe patch-delete) and rerun the script on the new pool master …

XenServer 6.x: Quick VM Protection Policy to VM name-label script

September 15, 2013September 15, 2013 Christian Leave a comment

Well, today I ended up writing a short script that’ll give me a list of VMPPs with the VMs that are associated to it.

#!/bin/bash

# Get a list of VMPPs
for vmpp in `xe vmpp-list params=uuid --minimal | sed "s/,/ /g"`; do
        VMPP_NAME=`xe vmpp-list params=name-label uuid=$vmpp --minimal`

        for vm in `xe vmpp-list params=VMs --minimal uuid=$vmpp | sed -e "s/;//g" -e "s/,//g"`; do
                VM=`xe vm-list params=name-label uuid=$vm --minimal`

                echo "$VMPP_NAME: $VM"
        done
done

#!/bin/bash

# Get a list of VMPPs

for vmpp in `xe vmpp-list params=uuid --minimal | sed "s/,/ /g"`; do

VMPP_NAME=`xe vmpp-list params=name-label uuid=$vmpp --minimal`

for vm in `xe vmpp-list params=VMs --minimal uuid=$vmpp | sed -e "s/;//g" -e "s/,//g"`; do

VM=`xe vm-list params=name-label uuid=$vm --minimal`

echo "$VMPP_NAME: $VM"

done

Hetzner, Debian, KVM and IPv6

September 9, 2013September 9, 2013 Christian 3 Comments

Well, I’ve had my share of troubles with Hetzner, Debian, KVM and IPv6 addresses. After figuring out how to get around the IPv6 neighbor stuff (npd6 for teh win!), I battled with the problem that after restarting (rebooting/resetting – doesn’t really matter) a domain it’s IPv6 address would no longer work.

Well, today I decided to take a closer look. After the reboot, the guest comes up with this:

pinguinfuss:(thanatos.heimdaheim.de/webs) PWD:~
Mon Sep 09, 19:01:27 [0] > ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:96:ed:35 brd ff:ff:ff:ff:ff:ff
    inet 78.46.37.114 peer 78.46.37.118/32 brd 78.46.37.114 scope global eth0
    inet6 2a01:4f8:110:3148::5/64 scope global tentative dadfailed
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe96:ed35/64 scope link
       valid_lft forever preferred_lft forever

pinguinfuss:(thanatos.heimdaheim.de/webs) PWD:~

Mon Sep 09, 19:01:27 [0] > ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

inet6 ::1/128 scope host

valid_lft forever preferred_lft forever

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

link/ether 52:54:00:96:ed:35 brd ff:ff:ff:ff:ff:ff

inet 78.46.37.114 peer 78.46.37.118/32 brd 78.46.37.114 scope global eth0

inet6 2a01:4f8:110:3148::5/64 scope global tentative dadfailed

valid_lft forever preferred_lft forever

inet6 fe80::5054:ff:fe96:ed35/64 scope link

valid_lft forever preferred_lft forever

A quick peek into ip 6 neigh show reveals this:

pinguinfuss:(kvm.heimdaheim.de/KVM) PWD:~
Mon Sep 09, 19:02:27 [0] > sudo ip -6 neigh show
2a01:4f8:110:3148::5 dev eth0  FAILED

pinguinfuss:(kvm.heimdaheim.de/KVM) PWD:~

Mon Sep 09, 19:02:27 [0] > sudo ip -6 neigh show

2a01:4f8:110:3148::5 dev eth0 FAILED

At this point I had no idea were to look (I haven’t used IPv6 much), so thanks to a friend I ended up googling whatever dadfailed meant … as it turns out dadfailed indicates that a duplicate address had been detected. A short peek into kern.log/dmesg fuelled that idea:

pinguinfuss:(thanatos.heimdaheim.de/webs) PWD:/var/log
Mon Sep 09, 19:33:46 [0] > sudo grep eth kern.log
Sep  9 19:03:25 thanatos kernel: [    9.150549] eth0: IPv6 duplicate address 2a01:4f8:110:3148::5 detected!

pinguinfuss:(thanatos.heimdaheim.de/webs) PWD:/var/log

Mon Sep 09, 19:33:46 [0] > sudo grep eth kern.log

Sep 9 19:03:25 thanatos kernel: [ 9.150549] eth0: IPv6 duplicate address 2a01:4f8:110:3148::5 detected!

So, I went on googling IPv6, KVM and duplicate address, and guess what .. I don’t seem to be the only one that has this issue … I haven’t found the root cause of this, but I have a quick fix … I usually don’t assign duplicate IPv6 addresses to multiple domains (each domain has it’s on block of IPv6 addresses), so I ended up writing a short puppet class, that’ll disable the Duplicate Adress Detection for all my KVM guests!

class kvm-ipv6-domain {
        file { 'kvm-ipv6.conf':
                path    =>      '/etc/sysctl.d/kvm-ipv6.conf',
                ensure  =>      'present',
                mode    =>      '0644',
                content =>      'net.ipv6.conf.eth0.accept_dad=0',
        }
}

class kvm-ipv6-domain {

file { 'kvm-ipv6.conf':

path => '/etc/sysctl.d/kvm-ipv6.conf',

ensure => 'present',

mode => '0644',

content => 'net.ipv6.conf.eth0.accept_dad=0',

}

vmware-config-tools.pl finished with “Could not find Parent Node”

September 9, 2013September 9, 2013 Christian Leave a comment

Well, today I encountered a old problem (or so I thought). Basically a specific udevadm version causes the vmware-config-tools.pl script to error out like this:

The vmblock enables dragging or copying files between host and guest in a
Fusion or Workstation virtual environment.  Do you wish to enable this feature?
[no]

!!! [EXPERIMENTAL] !!!
VMware automatic kernel modules enables automatic building and installation of
VMware kernel modules at boot that are not already present.  By selecting yes,
you will be enabling this experimental feature.  You can always disable this
feature by re-running vmware-config-tools.pl.

Would you like to enable VMware automatic kernel modules?
[no]

No X install found.

Could not find Parent Node...

The vmblock enables dragging or copying files between host and guest in a

Fusion or Workstation virtual environment. Do you wish to enable this feature?

[no]

!!! [EXPERIMENTAL] !!!

VMware automatic kernel modules enables automatic building and installation of

VMware kernel modules at boot that are not already present. By selecting yes,

you will be enabling this experimental feature. You can always disable this

feature by re-running vmware-config-tools.pl.

Would you like to enable VMware automatic kernel modules?

[no]

No X install found.

Could not find Parent Node...

I’ve had encountered this before in the past, and before there was some explanation on the VMware forums, which I couldn’t locate. Lucky me, the VMware Tools updater keeps modified versions of vmware-tools-config.pl around. So I ended up creating this short diff, so that I may find it in the future – if I still need it:

--- /usr/bin/vmware-config-tools.pl     2013-09-09 09:43:13.000000000 +0200
+++ vmware-config-tools.pl      2013-09-09 09:43:04.000000000 +0200
@@ -6880,7 +6880,7 @@
       my $txt = $1;
       if ($txt =~ m/{vendor}=="VMware/is) {
          # Get the udev device path from this node
-         if ($txt =~ m/'(.*?)':/s) {
+         if ($txt =~ m/^'(.*?)':/s) {
             unshift(@scsiNodes, $1);
             unshift(@nodeText, $txt);
          } else {

--- /usr/bin/vmware-config-tools.pl 2013-09-09 09:43:13.000000000 +0200

+++ vmware-config-tools.pl 2013-09-09 09:43:04.000000000 +0200

@@ -6880,7 +6880,7 @@

my $txt = $1;

if ($txt =~ m/{vendor}=="VMware/is) {

# Get the udev device path from this node

- if ($txt =~ m/'(.*?)':/s) {

+ if ($txt =~ m/^'(.*?)':/s) {

unshift(@scsiNodes, $1);

unshift(@nodeText, $txt);

} else {