Work – Page 5

Automating qual_devices updates

May 30, 2012August 8, 2014 Christian Leave a comment

Well, once again I was presented with a nice AutoSupport warning once I logged into my NOW account. Since we don’t have CIFS and/or NFS licensed on our filers, I wrote a cute little script that does the whole work for me.

; html-script: false  ]#!/bin/bash

TMPDIR="$( mktemp -d )"
KEY_FILE="/root/.ssh/netapp.dsa"
SSH_OPTS="/root/.ssh/netapp-ssh_config"
FAS_CTRL=$1
QUALDEVICES=$2

ssh_fas() {
  # $@: commands for Data ONTAP
  COMMANDS="$@"
  /usr/bin/ssh -i $KEY_FILE -l root -F $SSH_OPTS $COMMANDS
}
#set -x

echo "Updating qual_devices on $FAS_CTRL"

# Enable ftpd if it isn't enabled already
echo -n "Checking FTP subsystem"

FTPD_INITIAL_STATE="$( ssh_fas $FAS_CTRL options ftpd.enable | 
  awk '{ print $2 }' )"
if [ $FTPD_INITIAL_STATE == "off" ] ; then
  ssh_fas $FAS_CTRL options ftpd.enable on
  echo "   .... ok"
else
  echo "   .... ok"
fi

echo
read -s -p "Please supply the root password for $FAS_CTRL: "
ROOT_PASSWD=$REPLY

echo
echo "Checking qual_devices:"
echo -n "     Running: "
mkdir $TMPDIR/fas-version
# Check the old version.
ftp -n $FAS_CTRL >/dev/null <<END_SCRIPT
prompt
user root $ROOT_PASSWD
lcd $TMPDIR/fas-version
cd /etc
mget qual_devices*
quit
END_SCRIPT

FAS_VERSION="$( grep Datecode $TMPDIR/fas-version/qual_devices_v3 | 
  head -n1 | cut -d  -f3 )"

echo "$FAS_VERSION"
# Unzip the qual_devices.zip file and compare it.
ORIGPWD=$PWD
mkdir $TMPDIR/new-version
cd $TMPDIR/new-version
unzip $QUALDEVICES >/dev/null
echo -n "         New: "
NEW_VERSION="$( grep Datecode $TMPDIR/new-version/qual_devices_v3 | 
  head -n1 | cut -d  -f3 )"
echo "$NEW_VERSION"
echo

# Upload the supplied version if the new file doesn't match the one running
# on the controller
if [ $NEW_VERSION != $FAS_VERSION ] ; then
  echo "Uploading qual_devices to $FAS_CTRL"
  ftp -n $FAS_CTRL >/dev/null <<END_SCRIPT
prompt
user root $ROOT_PASSWD
lcd $TMPDIR/new-version
cd /etc
mput qual_devices*
quit
END_SCRIPT

  # Send an asup message, that the issue is corrected
  echo "Generating AutoSupport message"
  ssh_fas $FAS_CTRL options autosupport.doit qual_devices_fixed_$NEW_VERSION
fi

#set +x

# Enable ftpd
if [ $FTPD_INITIAL_STATE == "off" ] ; then
  ssh_fas $FAS_CTRL options ftpd.enable off
fi

rm -r $TMPDIR

; html-script: false ]#!/bin/bash

TMPDIR="$( mktemp -d )"

KEY_FILE="/root/.ssh/netapp.dsa"

SSH_OPTS="/root/.ssh/netapp-ssh_config"

FAS_CTRL=$1

QUALDEVICES=$2

ssh_fas() {

# $@: commands for Data ONTAP

COMMANDS="$@"

/usr/bin/ssh -i $KEY_FILE -l root -F $SSH_OPTS $COMMANDS

}

#set -x

echo "Updating qual_devices on $FAS_CTRL"

# Enable ftpd if it isn't enabled already

echo -n "Checking FTP subsystem"

FTPD_INITIAL_STATE="$( ssh_fas $FAS_CTRL options ftpd.enable |

awk '{ print $2 }' )"

if [ $FTPD_INITIAL_STATE == "off" ] ; then

ssh_fas $FAS_CTRL options ftpd.enable on

echo " .... ok"

else

echo " .... ok"

echo

read -s -p "Please supply the root password for $FAS_CTRL: "

ROOT_PASSWD=$REPLY

echo

echo "Checking qual_devices:"

echo -n " Running: "

mkdir $TMPDIR/fas-version

# Check the old version.

ftp -n $FAS_CTRL >/dev/null <<END_SCRIPT

prompt

user root $ROOT_PASSWD

lcd $TMPDIR/fas-version

cd /etc

mget qual_devices*

quit

END_SCRIPT

FAS_VERSION="$( grep Datecode $TMPDIR/fas-version/qual_devices_v3 |

head -n1 | cut -d -f3 )"

echo "$FAS_VERSION"

# Unzip the qual_devices.zip file and compare it.

ORIGPWD=$PWD

mkdir $TMPDIR/new-version

cd $TMPDIR/new-version

unzip $QUALDEVICES >/dev/null

echo -n " New: "

NEW_VERSION="$( grep Datecode $TMPDIR/new-version/qual_devices_v3 |

head -n1 | cut -d -f3 )"

echo "$NEW_VERSION"

echo

# Upload the supplied version if the new file doesn't match the one running

# on the controller

if [ $NEW_VERSION != $FAS_VERSION ] ; then

echo "Uploading qual_devices to $FAS_CTRL"

ftp -n $FAS_CTRL >/dev/null <<END_SCRIPT

prompt

user root $ROOT_PASSWD

lcd $TMPDIR/new-version

cd /etc

mput qual_devices*

quit

END_SCRIPT

# Send an asup message, that the issue is corrected

echo "Generating AutoSupport message"

ssh_fas $FAS_CTRL options autosupport.doit qual_devices_fixed_$NEW_VERSION

#set +x

# Enable ftpd

if [ $FTPD_INITIAL_STATE == "off" ] ; then

ssh_fas $FAS_CTRL options ftpd.enable off

rm -r $TMPDIR

The whole thing is surly based, that FTP is configured (as I described previously).

TSM and NetApp – Another Quick Hint

May 27, 2012June 21, 2013 Christian Leave a comment

Well, we’ve been trying to come up with a decent way to backup NetApp snapshots to tape (SnapMirror To Tape), so we evaluated all the available methods of using NDMP backups.

There’s Image Backup in two different variants – FULL and DIFFerntial
There’s SnapMirror To Tape

So the Image Backup is one of the ways. However the DIFFerntial backup only works for CIFS and NFS shares (which we don’t use). We only have FC luns (or rather FCoE luns), so there’s only a single (or in case of the boot luns more than one) file in each volume. With that however, each run of the Image Backup with the DIFFerential option, it’s gonna backup the full size of the volume (plus the deduplicated amount).

The SnapMirror To Tape option presents another problem: We intend to use SnapManager for SQL/Oracle, which creates “consistent” snapshots of the database luns. However the SnapMirror To Tape backup doesn’t have an option to use an already existing snapshot, but creates another one. Which puts the whole SnapManager business down the curb. So we either do use a SnapMirror To Disk from one database lun to another controller and then run the SnapMirror To Tape from the second controller or come up with another way to back them up to TSM.

vm-online-backup – Another day, another PowerCLI script

April 30, 2012June 21, 2013 Christian Leave a comment

Well, on Friday I had a short chat with someone from one of our application departments, stating he wanted a backup copy of a VM (ain’t to hard), but a) they don’t want any downtime and b) it has to be identical to the original.

So I sat down today, googled for a bit and actually found something that pretty much does what I want, though I had to fix it up a bit … So find attached a script, which creates a hot-clone from a snapshot and then only if the latest clone was successful deletes the old one.

param( [string] $vCenter )

# Add the VI-Snapin if it isn't loaded already
if ( (Get-PSSnapin -Name "VMware.VimAutomation.Core" -ErrorAction SilentlyContinue) -eq $null )
{
	Add-PSSnapin -Name "VMware.VimAutomation.Core"
}

if ( !($vCenter) ) {
	Write-Host ""
	Write-Host "vm-create-snapshot-clone: <vcenter> <VM-Name> <Target Datastore>"
	Write-Host "    <vcenter>          - Hostname of the vCenter Server instance for this script"
	Write-Host "                         to work on."
	Write-Host ""
	exit 1
}
# Based on http://www.simonlong.co.uk/blog/2010/05/05/powercli-a-simple-vm-backup-script/

# Import Backup CSV
$backupinfo =  Import-Csv C:ScriptsTEMPbackupvms.csv

#Set Date format for clone names
$date = Get-Date -Format "yyyyMMdd"

#Set Date format for emails
$time = (Get-Date -f "HH:MM")

#Connect to vCenter
Connect-VIServer $vCenter

foreach ($customer in $backupinfo)
{
    $vm = Get-VM $customer.MasterVM
	$SCRIPTS = "C:ScriptsTEMP$( $customer.MasterVM ).txt"

    # Create new snapshot for clone
    $vm | New-Snapshot -Name "Clone Snapshot" -Description (get-date -format "'Created: 'yyyy-MM-dd HH:mm")

    # Get managed object view
    $vmView = $vm | Get-View

    # Get folder managed object reference
    $cloneFolder = $vmView.parent

    # Build clone specification
    $cloneSpec = new-object Vmware.Vim.VirtualMachineCloneSpec
    $cloneSpec.Snapshot = $vmView.Snapshot.CurrentSnapshot

    # Make linked disk specification
    $cloneSpec.Location = new-object Vmware.Vim.VirtualMachineRelocateSpec
    $cloneSpec.Location.Datastore = (Get-Datastore -Name $customer.BackupDS | Get-View).MoRef
    $cloneSpec.Location.Transform =  [Vmware.Vim.VirtualMachineRelocateTransformation]::sparse

    $cloneName = "$vm-$date"

    # Create clone
    $vmView.CloneVM( $cloneFolder, $cloneName, $cloneSpec )

	if (( Test-Path -Path $SCRIPTS) -ne $true ) {
		$cloneName | Out-File $SCRIPTS
	} else {
		$old_vm = Get-Content $SCRIPTS
		$cloneName | Out-File $SCRIPTS

		Get-VM $old_vm | Remove-VM -DeletePermanently -Confirm:$false
	}

    # Remove Snapshot created for clone
    Get-Snapshot -VM $vm -Name "Clone Snapshot" | Remove-Snapshot -confirm:$False
}
#Disconnect from vCenter
Disconnect-VIServer -Server $vCenter -Confirm:$false

param( [string] $vCenter )

# Add the VI-Snapin if it isn't loaded already

if ( (Get-PSSnapin -Name "VMware.VimAutomation.Core" -ErrorAction SilentlyContinue) -eq $null )

{

Add-PSSnapin -Name "VMware.VimAutomation.Core"

}

if ( !($vCenter) ) {

Write-Host ""

Write-Host "vm-create-snapshot-clone: <vcenter> <VM-Name> <Target Datastore>"

Write-Host " <vcenter> - Hostname of the vCenter Server instance for this script"

Write-Host " to work on."

Write-Host ""

exit 1

}

# Based on http://www.simonlong.co.uk/blog/2010/05/05/powercli-a-simple-vm-backup-script/

# Import Backup CSV

$backupinfo = Import-Csv C:ScriptsTEMPbackupvms.csv

#Set Date format for clone names

$date = Get-Date -Format "yyyyMMdd"

#Set Date format for emails

$time = (Get-Date -f "HH:MM")

#Connect to vCenter

Connect-VIServer $vCenter

foreach ($customer in $backupinfo)

{

$vm = Get-VM $customer.MasterVM

$SCRIPTS = "C:ScriptsTEMP$( $customer.MasterVM ).txt"

# Create new snapshot for clone

$vm | New-Snapshot -Name "Clone Snapshot" -Description (get-date -format "'Created: 'yyyy-MM-dd HH:mm")

# Get managed object view

$vmView = $vm | Get-View

# Get folder managed object reference

$cloneFolder = $vmView.parent

# Build clone specification

$cloneSpec = new-object Vmware.Vim.VirtualMachineCloneSpec

$cloneSpec.Snapshot = $vmView.Snapshot.CurrentSnapshot

# Make linked disk specification

$cloneSpec.Location = new-object Vmware.Vim.VirtualMachineRelocateSpec

$cloneSpec.Location.Datastore = (Get-Datastore -Name $customer.BackupDS | Get-View).MoRef

$cloneSpec.Location.Transform = [Vmware.Vim.VirtualMachineRelocateTransformation]::sparse

$cloneName = "$vm-$date"

# Create clone

$vmView.CloneVM( $cloneFolder, $cloneName, $cloneSpec )

if (( Test-Path -Path $SCRIPTS) -ne $true ) {

$cloneName | Out-File $SCRIPTS

} else {

$old_vm = Get-Content $SCRIPTS

$cloneName | Out-File $SCRIPTS

Get-VM $old_vm | Remove-VM -DeletePermanently -Confirm:$false

}

# Remove Snapshot created for clone

Get-Snapshot -VM $vm -Name "Clone Snapshot" | Remove-Snapshot -confirm:$False

}

#Disconnect from vCenter

Disconnect-VIServer -Server $vCenter -Confirm:$false

The backupvms.csv looks pretty simple:

; html-script: false ]MasterVM,BackupDS
sles11-sp1,datastore2-nfs
sles10-sp4,datastore1-nfs

; html-script: false ]MasterVM,BackupDS

sles11-sp1,datastore2-nfs

sles10-sp4,datastore1-nfs

TSM and NetApp – Quick Hint

April 25, 2012June 21, 2013 Christian Leave a comment

Well, to save everyone else the trouble (since it isn’t documented anywhere – and I just spent about an hour finding the cause for this), if you need to configure NDMP on your NetApp Filer, make sure you also configure an interface other than e0M.

Apparently the necessary controlport for NDMP (10000) is being blocked on e0M, thus ndmp may be configured and running, however TSM is gonna complain that it is unable to connect to the specified data mover.

ANR4728E Server connection to file server 172.31.76.100 failed. Please check
the attributes of the file server specified during definition of the datamover.
ANR2146E DEFINE DATAMOVER: Node NAS_DM is not registered.
ANS8001I Return code 11.

ANR4728E Server connection to file server 172.31.76.100 failed. Please check

the attributes of the file server specified during definition of the datamover.

ANR2146E DEFINE DATAMOVER: Node NAS_DM is not registered.

ANS8001I Return code 11.

SLES11.1 and updated multipath-tools

April 7, 2012June 21, 2013 Christian Leave a comment

Well, after I scripted the installation the other day, I tried installing SLES11.1-Updates to the freshly installed systems. Guess what ? The thing broke. Initially (it was late Friday afternoon – like 6 PM – before my one week vacation) I didn’t have much time to debug the issue, so I sat down last week and looked at the issue.

During the installation, when first starting multipath via command line, the scsi-mpatha device appears, and each and every occurance of this is subsequentially being used (and other stuff replaced by this actually) during the whole installation phase.

But what is this multipath-tools update doing ? No clue what exactly, however after installing the update the system is bricked. The system is basically looking for /dev/scsi/by-id/scsi-disk-mpatha and waiting for this device to appear. But since the update robbed the device, the system is no longer starting.

So I went ahead and digged around in the /dev/disk/by-id directory. Turns out /dev/disk/by-id/scsi- is actually pointing to the right device, and thus I ended up using it. So I rewrote all my scripts (profile and chroot/post-chroot adjustments) as you can see below, and for now at least, I have a working installation that lets you install updates! (careful it gots electrolytes)

#!/bin/bash

cat << EOF > /etc/multipath.conf
defaults {
	user_friendly_names		no
	bindings_file			/etc/multipath_bindings
}

devices {
	device {
		vendor			"NETAPP"
		product			"LUN"
		getuid_callout		"/lib/udev/scsi_id -g -u --device=/dev/%n"
		features		"1 queue_if_no_path"
		hardware_handler	"1 alua"
		path_checker		tur
		path_selector		"round-robin 0"
		path_grouping_policy	group_by_prio
		failback		immediate
		rr_weight		uniform
		rr_min_io		128
		prio			alua
		max_fds			max
        }
}
EOF

sleep 1

# Disable the resume kernel
sed -e "s,resume=/dev/mapper/.*_part[0-9],noresume,g" -i 
	/etc/sysconfig/bootloader /boot/grub/menu.lst 

# Figure out the multipath ID
MD_ID="$( /sbin/multipath -l | head -n1 | cut -d  -f1 )"

# Fix wrong root-path in /etc/fstab
sed -i "s/mapper/.*_part/disk/by-id/scsi-${MD_ID}-part/" /etc/fstab

# Fix grub root-path and wrong root-partition
sed -i -e "s/mapper/.*_part/disk/by-id/scsi-${MD_ID}-part/" 
	-e "s,scsi-${MD_ID}-part2,scsi-${MD_ID}-part3," /boot/grub/menu.lst

# Fix the device.map
echo -e "(hd0)t/dev/disk/by-id/scsi-${MD_ID}" > /boot/grub/device.map

sleep 1
mkinitrd -f multipath

#!/bin/bash

cat << EOF > /etc/multipath.conf

defaults {

user_friendly_names no

bindings_file /etc/multipath_bindings

}

devices {

device {

vendor "NETAPP"

product "LUN"

getuid_callout "/lib/udev/scsi_id -g -u --device=/dev/%n"

features "1 queue_if_no_path"

hardware_handler "1 alua"

path_checker tur

path_selector "round-robin 0"

path_grouping_policy group_by_prio

failback immediate

rr_weight uniform

rr_min_io 128

prio alua

max_fds max

}

EOF

sleep 1

# Disable the resume kernel

sed -e "s,resume=/dev/mapper/.*_part[0-9],noresume,g" -i

/etc/sysconfig/bootloader /boot/grub/menu.lst

# Figure out the multipath ID

MD_ID="$( /sbin/multipath -l | head -n1 | cut -d -f1 )"

# Fix wrong root-path in /etc/fstab

sed -i "s/mapper/.*_part/disk/by-id/scsi-${MD_ID}-part/" /etc/fstab

# Fix grub root-path and wrong root-partition

sed -i -e "s/mapper/.*_part/disk/by-id/scsi-${MD_ID}-part/"

-e "s,scsi-${MD_ID}-part2,scsi-${MD_ID}-part3," /boot/grub/menu.lst

# Fix the device.map

echo -e "(hd0)t/dev/disk/by-id/scsi-${MD_ID}" > /boot/grub/device.map

sleep 1

mkinitrd -f multipath

Now what’s left to do for tomorrow is “fixing” the already (previous to those changes) installed systems, so we can install security updates on those too!

VMware Update Manager issues

March 31, 2012June 21, 2013 Christian Leave a comment

Well, I recently (last Wednesday) had a lot of trouble with Update Manager.

First I thought, upgrading vCenter and modules to 5.0U1 would solve my troubles, however it did not. Update Manager was still complaining about something. Since neither in the vCenter Update Manager nor the vCenter log itself were having any useful information I enabled SSHd and the ESXi Shell via the vCenter client:

SSH’ed into the ESX host and looked at /var/log/esxupdate.log, and found this particular log:

2012-03-28T16:18:26Z esxupdate: HostImage: INFO: Attempting to download VIB vmware-fdm
2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR: An esxupdate error exception was caught:
2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR: Traceback (most recent call last):
2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR:   File "/usr/sbin/esxupdate", line 216, in main
2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR:     cmd.Run()
2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR:   File "/build/mts/release/bora-515841/bora/build/esx/release/python-2.6-lib-zip-stage/515841/visor/pylib/python2.6/site-packages/vmware/esx5update/Cmdline.py", line 144, in Run
2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR:   File "/build/mts/release/bora-515841/bora/build/esx/release/python-2.6-lib-zip-stage/515841/visor/pylib/python2.6/site-packages/vmware/esximage/Transaction.py", line 243, in InstallVibsFromSources
2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR:   File "/build/mts/release/bora-515841/bora/build/esx/release/python-2.6-lib-zip-stage/515841/visor/pylib/python2.6/site-packages/vmware/esximage/Transaction.py", line 345, in _installVibs
2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR:   File "/build/mts/release/bora-515841/bora/build/esx/release/python-2.6-lib-zip-stage/515841/visor/pylib/python2.6/site-packages/vmware/esximage/Transaction.py", line 388, in _validateAndInstallProfile
2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR:   File "/build/mts/release/bora-515841/bora/build/esx/release/python-2.6-lib-zip-stage/515841/visor/pylib/python2.6/site-packages/vmware/esximage/HostImage.py", line 630, in Stage
2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR:   File "/build/mts/release/bora-515841/bora/build/esx/release/python-2.6-lib-zip-stage/515841/visor/pylib/python2.6/site-packages/vmware/esximage/HostImage.py", line 414, in _download_and_stage
2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR: VibDownloadError: (<vmware.esximage.Vib.ArFileVib object at 0x86c79ac>, 'Unable to download VIB from any of the URLs ')
2012-03-28T16:18:26Z esxupdate: esxupdate: DEBUG: <<<

2012-03-28T16:18:26Z esxupdate: HostImage: INFO: Attempting to download VIB vmware-fdm

2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR: An esxupdate error exception was caught:

2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR: Traceback (most recent call last):

2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR: File "/usr/sbin/esxupdate", line 216, in main

2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR: cmd.Run()

2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR: File "/build/mts/release/bora-515841/bora/build/esx/release/python-2.6-lib-zip-stage/515841/visor/pylib/python2.6/site-packages/vmware/esx5update/Cmdline.py", line 144, in Run

2012-03-28T16:18:26Z esxupdate: esxupdate: ERROR: VibDownloadError: (<vmware.esximage.Vib.ArFileVib object at 0x86c79ac>, 'Unable to download VIB from any of the URLs ')

2012-03-28T16:18:26Z esxupdate: esxupdate: DEBUG: <<<

As you can see from the above log, for whatever reason, the Update Manager is trying to install an updated version of the VMware HA agent (vmware-fdm). Now, since that isn’t the job of Update Manager (afaik, vCenter handles that separately), I figured what the hell, let’s try and remove it as the ESX host was already in Maintainance Mode:

esxcli software vib remove -n=vmware-fdm

1	esxcli software vib remove -n=vmware-fdm

And guess what: after removing the package and then rerunning the Remediate task on the host, my troubles were gone.

NetApp FAS/Data ONTAP public key authentification with CIFS/NFS license

March 7, 2012June 21, 2013 Christian 2 Comments

Well as the title says, sadly we bought our FAS6210 without CIFS/NFS license. Thus, in order to create the folder structure/add the authorized_keys file, you’ll have to work for your money a little bit.

First, you need to run cifs setup / cifs passwd somewhere. I did it on our Data ONTAP simulator, which comes in handy for things like that.

You’ll get a cryptic looking password (no clue which format that is), looking like this: _OnWddr)xa.

Now, in order for the ftpd process to work, you need to create a /etc/passwd file. Usually the cifs setup would take care of that, but since we don’t own a CIFS license and I didn’t wanna add a trial license, I simply did what I described above on our simulator.

Now, open a SSH session with your filer. Create a new /etc/passwd file using wrfile. The new passwd file should look like this:

root:_OnWddr)xa.:0:1::/:
pcuser::65534:65534::/:
nobody::65535:65535::/:
ftp::65533:65533:FTP Anonymous:/home/ftp:

root:_OnWddr)xa.:0:1::/:

pcuser::65534:65534::/:

nobody::65535:65535::/:

ftp::65533:65533:FTP Anonymous:/home/ftp:

Now make sure, to replace the whole string in between the double dots with the one you got from the output of cifs passwd. After that is done, enable the FTP daemon using the options command:

; html-script: false ]options ftpd.enable on

1	; html-script: false ]options ftpd.enable on

Now, create your authorized_keys file somewhere (I exported my Public Key using PuTTygen), and from there open a ftp sessions with your root user on the filer. In the ftp shell run this:

cd /etc/sshd
mkdir root
cd root
mkdir .ssh
cd .ssh
lcd D:userschrischieDesktop
prompt
mput authorized_keys

cd /etc/sshd

mkdir root

cd root

mkdir .ssh

cd .ssh

lcd D:userschrischieDesktop

prompt

mput authorized_keys

The above example asumes that you created the authorized_keys file in the folder Desktop (that’s where my Desktop folder is, so replace it to suit your needs). Afterwards, disable the FTP daemon again:

options ftpd.enable off

1	options ftpd.enable off

And, tada … enjoy SSH password-less with your shiny public key.

UCS blades w/ Boot-from-SAN and AutoYaST

March 3, 2012August 8, 2014 Christian 1 Comment

As I wrote before about enabling multipathing for the AutoYaST installation it’s about time I write this one here.

Sadly AutoYaST needs a little push in the right direction (as to where to actually put the root device), so here’s part of my AutoYaST profile for such a Cisco blade:

<profile xmlns="http://www.suse.com/1.0/yast2ns" xmlns:config="http://www.suse.com/1.0/configns">

	<bootloader>
		<device_map config:type="list">
			<device_map_entry>
				<firmware>hd0</firmware>
				<linux>/dev/sda</linux>
			</device_map_entry>
		</device_map>
	</bootloader>

	<partitioning config:type="list">
		<drive>
			<device>/dev/sda</device>
		</drive>
	</partitioning>

	<scripts>
		<pre-scripts config:type="list">

			<script>
				<debug config:type="boolean">false</debug>
				<feedback config:type="boolean">false</feedback>
				<filename>config-ucs.sh</filename>
				<interpreter>shell</interpreter>
				<source><![CDATA[
cat /tmp/profile/autoinst.xml | sed "s,/dev/sda,/dev/mapper/`/sbin/multipath -ll | grep dm-0 | cut -d  -f1`," > /tmp/profile/modified.xml
]]>
				</source>
			</script>

		</pre-scripts>

		<chroot-scripts config:type="list">

			<script>
				<chrooted config:type="boolean">true</chrooted>
				<debug config:type="boolean">true</debug>
				<feedback config:type="boolean">true</feedback>
				<filename>config-ucs-chroot.sh</filename>
				<interpreter>shell</interpreter>
				<location>http://install.home.barfoo.org/autoyast/scripts/config-ucs-chroot.sh</location>
			</script>

		</chroot-scripts>
	</scripts>

	<software>
		<packages config:type="list">
			<package>multipath-tools</package>
		</packages>
	</software>

</profile>

<device_map config:type="list">

<device_map_entry>

</device_map_entry>

</device_map>

</bootloader>

<drive>

</drive>

</partitioning>

<pre-scripts config:type="list">

<debug config:type="boolean">false</debug>

<feedback config:type="boolean">false</feedback>

<filename>config-ucs.sh</filename>

<interpreter>shell</interpreter>

<source><![CDATA[

cat /tmp/profile/autoinst.xml | sed "s,/dev/sda,/dev/mapper/`/sbin/multipath -ll | grep dm-0 | cut -d -f1`," > /tmp/profile/modified.xml

]]>

</source>

</script>

</pre-scripts>

<chroot-scripts config:type="list">

<filename>config-ucs-chroot.sh</filename>

<interpreter>shell</interpreter>

<location>http://install.home.barfoo.org/autoyast/scripts/config-ucs-chroot.sh</location>

</script>

</chroot-scripts>

</scripts>

<package>multipath-tools</package>

</packages>

</software>

</profile>

Now, the profile addition takes care of the placement of the root-device now (simply parses multipath -ll) and adjusts the pulled profile accordingly (/tmp/profile/modified.xml), which AutoYaST then re-reads.

Now, after installing the system, it’s gonna come up broken and shitty. That’s what the chroot-script above is for. This script looks like this (the original idea was here, look through the attachments):

#!/bin/bash

echo "defaults {
	user_friendly_names yes
	bindings_file /etc/multipath_bindings
}" > /etc/multipath.conf

sleep 1

# Fix wrong root-path in /etc/fstab
sed -i 's/mapper/.*_part/disk/by-id/scsi-mpatha-part/' /etc/fstab

# Fix grub root-path and wrong root-partition
sed -i -e 's/mapper/.*_part/disk/by-id/scsi-mpatha-part/' 
	-e 's,scsi-mpatha-part2,scsi-mpatha-part3,' /boot/grub/menu.lst

# Fix the device.map
echo -e "(hd0)t$( ls /dev/disk/by-id/scsi-* | grep -v part )" > /boot/grub/device.map

sleep 1
mkinitrd -f multipath

#!/bin/bash

echo "defaults {

user_friendly_names yes

bindings_file /etc/multipath_bindings

}" > /etc/multipath.conf

sleep 1

# Fix wrong root-path in /etc/fstab

sed -i 's/mapper/.*_part/disk/by-id/scsi-mpatha-part/' /etc/fstab

# Fix grub root-path and wrong root-partition

sed -i -e 's/mapper/.*_part/disk/by-id/scsi-mpatha-part/'

-e 's,scsi-mpatha-part2,scsi-mpatha-part3,' /boot/grub/menu.lst

# Fix the device.map

echo -e "(hd0)t$( ls /dev/disk/by-id/scsi-* | grep -v part )" > /boot/grub/device.map

sleep 1

mkinitrd -f multipath

What the script does, is 1) fix the occurances of /dev/mapper/, since this isn’t the proper way 2) create a multipathing initrd. Without the creation of the multipath-aware initrd the system will also not boot!

Enabling multipathing in autoyast installations

February 29, 2012June 21, 2013 Christian 1 Comment

As I mentioned before, we’re starting to utilize Boot-from-SAN as a means to strip the blades of their local disk. As the title says, after trying a manual installation of SLES 11.1 via CD/HTTP I wanted to automate the process, in order to get a reproducible, consistent installation method. As you might have figured, AutoYaST doesn’t have any built in support for configuring multipathing (hey, that’s what Novell says here). Now, they also provide a comprehensive how-to on how to “add” this to your AutoYaST, using a DUD (or Driver Update Disk).

Now, you can download the provided Driver Update Disk to any Linux box and unpack it using cpio. As the KB states, do the following:

cd /tmp/
gunzip multipath.DUD.gz
cd /Installsource/SLES11SP1-x86_64/
cpio -i </tmp/multipath.DUD

cd /tmp/

gunzip multipath.DUD.gz

cd /Installsource/SLES11SP1-x86_64/

cpio -i </tmp/multipath.DUD

And you might get the same as I do:

chrischie:(sles-vm.home.barfoo.org) PWD:/srv/instsrc/sles/11.1/x64
Wed Feb 29, 21:13:42 [0] > sudo cpio -i < /tmp/multipath.DUD
cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux: Cannot mkdir: No such file or directory
cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse: Cannot mkdir: No such file or directory
cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11: Cannot mkdir: No such file or directory
cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys: Cannot mkdir: No such file or directory
cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr: Cannot mkdir: No such file or directory
cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr/lib: Cannot mkdir: No such file or directory
cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr/lib/YaST2: Cannot mkdir: No such file or directory
cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr/lib/YaST2/startup: Cannot mkdir: No such file or directory
cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr/lib/YaST2/startup/hooks: Cannot mkdir: No such file or directory
cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr/lib/YaST2/startup/hooks/preFirstStage-old: Cannot mkdir: No such file or directory
cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr/lib/YaST2/startup/hooks/preFirstStage-old/S01start_multipath.sh: Cannot open: No such file or directory
cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr/lib/YaST2/startup/hooks/preFirstCall: Cannot mkdir: No such file or directory
cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr/lib/YaST2/startup/hooks/preFirstCall/S01start_multipath.sh: Cannot open: No such file or directory
8 blocks

chrischie:(sles-vm.home.barfoo.org) PWD:/srv/instsrc/sles/11.1/x64

Wed Feb 29, 21:13:42 [0] > sudo cpio -i < /tmp/multipath.DUD

cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux: Cannot mkdir: No such file or directory

cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse: Cannot mkdir: No such file or directory

cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11: Cannot mkdir: No such file or directory

cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys: Cannot mkdir: No such file or directory

cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr: Cannot mkdir: No such file or directory

cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr/lib: Cannot mkdir: No such file or directory

cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr/lib/YaST2: Cannot mkdir: No such file or directory

cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr/lib/YaST2/startup: Cannot mkdir: No such file or directory

cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr/lib/YaST2/startup/hooks: Cannot mkdir: No such file or directory

cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr/lib/YaST2/startup/hooks/preFirstStage-old: Cannot mkdir: No such file or directory

cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr/lib/YaST2/startup/hooks/preFirstStage-old/S01start_multipath.sh: Cannot open: No such file or directory

cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr/lib/YaST2/startup/hooks/preFirstCall: Cannot mkdir: No such file or directory

cpio: /NETSTORE/Installsource/SLES11SP1-x86_64/linux/suse/x86_64-sles11/inst-sys/usr/lib/YaST2/startup/hooks/preFirstCall/S01start_multipath.sh: Cannot open: No such file or directory

8 blocks

Now, after creating the path in question (just mkdir -p /NETSTORE/Installsource/SLES11SP1-x86_64) and retrying the cpio, you’ll get the following:

chrischie:(sles-vm.home.barfoo.org) PWD:/srv/instsrc/sles/11.1/x64
Wed Feb 29, 21:15:44 [0] > sudo cpio -i < /tmp/multipath.DUD
8 blocks

chrischie:(sles-vm.home.barfoo.org) PWD:/srv/instsrc/sles/11.1/x64

Wed Feb 29, 21:15:44 [0] > sudo cpio -i < /tmp/multipath.DUD

8 blocks

Next, according to the KB article is moving the linux directory to your actual install location (in my case /srv/instsrc/sles/11.1/x64) and then booting your system in question. And guess what you get ? Nil (as in the setup starts, but /sbin/multipath isn’t being called — which is nothing in my setup).

Next I tried copying the cpio-image (/tmp/multipath.DUD) to my install location as driverupdate (I know the SLES setup is pulling that file), which produced a warning about an unsigned driverupdate file (as it isn’t in ./content) — which can be circumvented by adding Insecure: 1 to your Info file (or passing insecure=1 as linuxrc parameter) — but after pressing “Yes” produced yet again Nil (still no call to /sbin/multipath).

After about three hours of fiddling with the original DUD (sadly the UCS blades are painfully slow to reboot — takes them about six minutes each), I decided to repack the Driver Update Disk. The Update Media HOWTO explains the structure/layout of the DUD pretty well, but fails to mention what kind of image it actually is or how to create it. Luckily there’s Google and the Internets.

The guys over at OPS East Blog, posted something that helped me create the DUD.

Basically, create /tmp/update-media and copy/move the linux folder into this folder.

mkdir /tmp/driver-update
cp -r /NETSTORE/Installsource/SLES11SP1-x86_64/linux /tmp/update-media
chown -R root.root /tmp/update-media

mkdir /tmp/driver-update

cp -r /NETSTORE/Installsource/SLES11SP1-x86_64/linux /tmp/update-media

chown -R root.root /tmp/update-media

After this, we create a Driver Update Disk configration.

echo "UpdateName: Enable multipathing config on AutoYaST boot" 
  > /tmp/update-media/linux/suse/x86_64-sles11/dud.config
echo "UpdateID: autoyast_multipath_1" 
  >> /tmp/update-media/linux/suse/x86_64-sles11/dud.config

echo "UpdateName: Enable multipathing config on AutoYaST boot"

> /tmp/update-media/linux/suse/x86_64-sles11/dud.config

echo "UpdateID: autoyast_multipath_1"

>> /tmp/update-media/linux/suse/x86_64-sles11/dud.config

Now, we create the DUD package.

mkfs.cramfs /tmp/update-media /tmp/driverupdate

1	mkfs.cramfs /tmp/update-media /tmp/driverupdate

This produces a CramFS image named /tmp/driverupdate (which you can view using mount -o loop). After moving this image to my install location and keeping the filename driverupdate, /sbin/multipath is actually being called as you can see below.

WDS and multi-architecture boot images

February 24, 2012June 21, 2013 Christian Leave a comment

Well, I recently stumbled upon another cute bug/feature with Windows Deployment Services. When you already have 32bit boot images (as we do) and then add an 64bit boot image (which we needed, since the drivers for UCS firmware v2.0 only support Windows Server 2008 R2) you still only see the 32bit images.

Why ? Because apparently the client (in my case a UCS blade) isn’t reporting it’s architecture correctly in the PXE phase. Microsoft actually has a KB article for this. You only need to enable architecture discovery.

wdsutil /set-server /architecturediscovery:yes

1	wdsutil /set-server /architecturediscovery:yes

That’s it. Once you boot the next client via PXE, you’ll see the 64bit boot image.