Enabling multipathing in autoyast installations

As I mentioned before, we’re starting to utilize Boot-from-SAN as a means to strip the blades of their local disk. As the title says, after trying a manual installation of SLES 11.1 via CD/HTTP I wanted to automate the process, in order to get a reproducible, consistent installation method. As you might have figured, AutoYaST doesn’t have any built in support for configuring multipathing (hey, that’s what Novell says here). Now, they also provide a comprehensive how-to on how to “add” this to your AutoYaST, using a DUD (or Driver Update Disk).

Now, you can download the provided Driver Update Disk to any Linux box and unpack it using cpio. As the KB states, do the following:

And you might get the same as I do:

Now, after creating the path in question (just mkdir -p /NETSTORE/Installsource/SLES11SP1-x86_64) and retrying the cpio, you’ll get the following:

Next, according to the KB article is moving the linux directory to your actual install location (in my case /srv/instsrc/sles/11.1/x64) and then booting your system in question. And guess what you get ? Nil (as in the setup starts, but /sbin/multipath isn’t being called — which is nothing in my setup).

Next I tried copying the cpio-image (/tmp/multipath.DUD) to my install location as driverupdate (I know the SLES setup is pulling that file), which produced a warning about an unsigned driverupdate file (as it isn’t in ./content) — which can be circumvented by adding Insecure: 1 to your Info file (or passing insecure=1 as linuxrc parameter) — but after pressing “Yes” produced yet again Nil (still no call to /sbin/multipath).

After about three hours of fiddling with the original DUD (sadly the UCS blades are painfully slow to reboot — takes them about six minutes each), I decided to repack the Driver Update Disk. The Update Media HOWTO explains the structure/layout of the DUD pretty well, but fails to mention what kind of image it actually is or how to create it. Luckily there’s Google and the Internets.

The guys over at OPS East Blog, posted something that helped me create the DUD.

Basically, create /tmp/update-media and copy/move the linux folder into this folder.

After this, we create a Driver Update Disk configration.

Now, we create the DUD package.

This produces a CramFS image named /tmp/driverupdate (which you can view using mount -o loop). After moving this image to my install location and keeping the filename driverupdate, /sbin/multipath is actually being called as you can see below.

SMT: Disable unmirrorable catalogs

Today I got this report from my SMT:

However, if you try and disable that repository with smt-catalogs -d, SMT is gonna quit your action with “0 repositories disabled“. Since I didn’t want the error to show up again, here’s a quick way on how to disable it.

Open up a mysql shell (mysql -u root -p preferably) and enter those queries:

Afterwards, if you rerun smt-mirror the warning message should be gone.

Creating a custom RPM repository for SMT

I spent some time yesterday figuring out ways on how to assign custom (as in self-built) RPMs to a installation using SMT. First you obviously need a “external” repository, that can be integrated into the SMT.

So we need to create repository someplace, where the SMT can go and grab it. I ain’t gonna cover the sharing part, since that is your job! I’m just gonna cover the steps on how to create the custom repository and how to integrate it into the SMT.

That’s it, you just created a RPM-repository which you could use with YaST/zypper … But since you already invested time into getting SMT up and running, you might want to assign this repository based on the product that is being installed.

Say we have RPMs for SLES10 SP1/SP2/SP3 in this repository for the architectures i586 and x86_64. First you would take a look at the output of smt-list-products and get the ID for the products/architecture you want to assign this to. But since we’re lazy, you could just use this:

And likewise for SLES11:

Now, after smt-mirror has been executed the next time (either by yourself or via the predefined crontab entry), SMT is able to assign this repository to clients. While this isn’t completely true — SMT is able to assign this custom catalog before running smt-mirror, but it just doesn’t make sense, since it doesn’t contain any data — it still works.

Now, once you install the next SLES10/SLES11 (hopefully you enabled suseRegister, that actually gathers the channels), SMT will assign this “update channel” (jesus, why does Novell use so many words for the same damn thing ?), on top of all the others, to your system.

The only trouble with this is, that if you want to install packages from this repository during setup, it’s not gonna work. That’s because YaST (or AutoYaST) first install packages, preps the environment and after prebooting the new system, then runs suseRegister/customer_center … Screwed. Again.

Guess the only way is to add the original repository (no need to automatically assign this, since we can’t install during setup) into the add-ons section of my AutoYaST file.

Active Directory authentification for Samba on SLES11

I recently “redesigned” the PXE-installation server, which comes with a Samba service to easily move files on/off the box. The old one had the restriction, you need to create local user accounts. Since I also did an distribution upgrade, I wanted to try the integration of SLES11 into Active Directory.

And as it turns out, it really is simple. Just follow the steps outlined in the handbook.

  1. Open the Windows Domain Membership module, yast samba-client (or yast, then Network Services -> Windows Domain Membership) and enter your Domain information
  2. Open the Samba Server Module, yast samba-server (or yast, then Network Services -> Samba Server) and also enter your Domain information

Just make sure, you also check the box labeled Also Use SMB Information for Linux Authentication, otherwise it won’t work — don’t ask me why …

SLES11 and AutoYaST

After the first week passed awfully quick, the last week I worked on refining the way on how we are doing openSuSE / SUSE Linux Enterprise Server installations. Up till now, they were done by hand (without a predefined schema) and were getting ugly to maintain. Working my way through the Novell documentation on AutoYaST was pretty straight forward, but the little details were getting hairy. So I decided to write them down, in case someone was gonna end up in the same situation like me.

1. Understanding the folder layout
This isn’t a debate about how you should organize your file system(s), this is a reminder on how to design the AutoYaST profile … If (and probably when) you’re using rules, the layout of the profile directory mentioned in your autoyast PXE line should look like this:

AutoYast folder structure

Keep in mind, that profiles mentioned in your rules.xml, need to be placed in the parent folder, not in the rules folder itself.

2. Rules might not work as you think (ie installed_product)
If you work your way through AutoYaST, you are gonna get to the point where you can’t get around rules. Rules match specific parameters of the target system to a (sub) set of configuration parameters. Now, say you want to create a rule for systems on which you are about to install SUSE Linux Enterprise Server 11. The documentation tells us, to use something along the lines like this:

You’re quickly gonna notice that you’re installation is gonna fail. Why ? Because Novell decided to drop the defined schema and use something else … The tag install_product, when performing an SLES11 installation doesn’t contain ‘SUSE Linux Enterprise Server‘ as you might think, but rather ‘SUSE Linux Enterprise Server 11‘ … notice the difference ? That and the default, that if no match-tag is given, it matches to the exact phrase. So the correct AutoYaST snippet should look likes this:

Another hint when trying to match for SUSE Linux Enterprise Server 10 — say you wanna deploy SLES10SP2. Make sure you make a regex match for that, since — as you might have guessed, I didn’t at first — the install_product tag seems to contain SUSE Linux Enterprise Server 10 SP2 ❗

3. Asking questions during the installation
If you’re in need of asking questions (like “Please enter the hostname” or “Please supply the root password”), you’re gonna need a bit of understanding as well as a few tricks:

4. Working with scripts
Once you get to the point, where you want to embed scripts into the AutoYaST profile, you might up in a situation where YaST literally shoots you in the foot. If you (like me) utilize different profile, and in each profile define a script, YaST is gonna overwrite the tags from the other profile. This sucks, and I ended up placing a dummy script into my master profile.

4. Debugging the rules merging process

Well this is quite troublesome. There is no easy way to do this, no GUI or script telling you what the resulting AutoYaST xml is gonna look like. There are a few helps (like the xsltproc call mentioned in the FAQ — or looking at the output of /var/log/YaST2/y2log concerning your rules.xml) but other than that there really is just fiddling with it.

Right now, I’m stuck working on the determination whether or not the system YaST is about to install happens to be inside a VMware virtual machine or not (simply looking at the MAC address should suffice).

But again, as mentioned in the second point, Novell apparently lacks some consistency. The handbook states the rules tag mac would contain the MAC address … apprently not, the mac-tag is empty … Which leads us to using a custom rules script fiddling with ip, a bit of grep and a simple echo, which works quite well. Just make sure, that you move the output of your script to /dev/null, otherwise you’re gonna sit there and end up trying to figure out why the hell the script works on the command line, but not within an AutoYaST installation …

Loooong time

It’s been very quiet around here, I’ve been rather busy with my real life. During that busy time, a lot of things happened. I switched jobs starting on October 1st, I’m now working in Karlsruhe (as compared to the 870km northern Greifswald). It may sound far, but it’s actually quite pleasant. You know, I was born down here (well not exactly here — 70 kilometers afar) and I still had the feeling that this is my home.

My tasks haven’t changed that much, I’m still doing

  • VMware Virtual Infrastructure (as compared to vSphere)
  • IBM Storage / Brocade SAN (was IBM Storage / Cisco SAN)
  • Storage Virtualisation Controller (we were just buying that before I left)
  • SUSE Linux Enterprise Server 10/11 – Deployment and Management (is pretty much the same as before)

What I don’t do any longer is Windows. That is, per se not completely right, since Virtual Center only runs on Windows boxen, but pretty much my whole work focuses on Linux and Storage. It’s as I argumented in the the interview a step ahead, since I’m specializing myself into a certain direction (whether or not that works out — I can’t tell yet. Time is gonna tell me that).

In my first week I spent some time getting to know the co-workers, working my head into the SVC (I already had a somewhat theroretical — and practical — insight, but not deep enough to actually make do with it). Next on my list is the AutoYaST environment for SLES-boxen / Kickstart for ESX(i), which (hopefully) enables us to standardize server installations using common schemas and partitions layout.

Also on the list is building a two-node test environment for the SVC so we don’t break the live environment with some tests we might be doing. Next on the list is some accouting to bring the settlements for the resource utilization based on vCPU/vMem upon a solid, up to date foundation.

New vmware-tools-kmp

Disclaimer: I don’t take any responsibility for faults within the software, I just provide the RPM’s! Feel free to ask me about stuff concerning these RPM’s, but I ain’t accountable if your stuff goes kaboom! Oh, and those RPM’s aren’t recommended or supported by VMware!

Since we recently upgraded our VMware Infrastructure to VMware vSphere, I finally had a chance to refresh the RPM’s for the KMP for 2.6.16.60-0.39.3-0.1 and 2.6.27.21-0.1. You can find the source RPM here.

  • vmware-tools-kmp-bigsmp (SLES10: i586)
  • vmware-tools-kmp-debug (SLES10: i586/x86_64; SLES11: i586/x86_64)
  • vmware-tools-kmp-default (SLES10: i586/x86_64; SLES11: i586/x86_64)
  • vmware-tools-kmp-kdump (SLES10: i586/x86_64)
  • vmware-tools-kmp-kdumppae (SLES10: i586)
  • vmware-tools-kmp-pae (SLES11: i586)
  • vmware-tools-kmp-smp (SLES10: i586/x86_64)
  • vmware-tools-kmp-trace (SLES11: i586/x86_64)
  • vmware-tools-kmp-vmi (SLES10: i586; SLES11: i586)
  • vmware-tools-kmp-vmipae (SLES10: i586)
  • vmware-tools-kmp-xen (SLES10: i586/x86_64; SLES11: i586/x86_64)
  • vmware-tools-kmp-xenpae (SLES10: i586)

Just a heads-up: while the modules itself work, VMware did something to the init-script which prevents it to load the modules unless you run vmware-config-tools.pl. I’m still in the process of sorting that out.

Novell KMP: vmware-tools-kmp and ibm-lin_tape-kmp

Disclaimer: I don’t take any responsibility for faults within the software, I just provide the RPM’s! Feel free to ask me about stuff concerning these RPM’s, but I ain’t accountable if your stuff goes kaboom … Oh, and those RPM’s aren’t recommended or supported by Novell or IBM!

After working with the novell-kmp solution, I think it’s actually rather easy to create a “Kernel Module Package“. In the end, I created two additional KMP’s, one for the tools component of the VMware-Tools shipped with VMware ESX, and another for the lin_tape SCSI driver, used by our IBM TS3400 as well as the IBM TS7530.

Some parts (especially the build system used within the VMware kernel modules) took some figuring out/playing around, but I actually got it working. Now each time I update the VMware-Tools I just need to install the new RPM, tada! No need for a fully fledged build environment on every box.

SUSE Linux Enterprise Server 10:

  • ibm-lin_tape-1.24.0_2.6.16.60_0.37_f594963d-0.1 (i586, x86_64, SRPM)
    • ibm-lin_tape-kmp-bigsmp (i586)
    • ibm-lin_tape-kmp-debug (i586, x86_64)
    • ibm-lin_tape-kmp-default (i586, x86_64)
    • ibm-lin_tape-kmp-kdump (i586, x86_64)
    • ibm-lin_tape-kmp-kdumppae (i586)
    • ibm-lin_tape-kmp-smp (i586, x86_64)
    • ibm-lin_tape-kmp-vmi (i586)
    • ibm-lin_tape-kmp-vmipae (i586)
  • vmware-tools-kmp-3.5.0_153875_2.6.16.60_0.37_f594963d-0.1 (SRPM)
    • vmware-tools-kmp-bigsmp (i586)
    • vmware-tools-kmp-debug (i586, x86_64)
    • vmware-tools-kmp-default (i586, x86_64)
    • vmware-tools-kmp-kdump (i586, x86_64)
    • vmware-tools-kmp-kdumppae (i586)
    • vmware-tools-kmp-smp (i586, x86_64)
    • vmware-tools-kmp-vmi (i586)
    • vmware-tools-kmp-vmipae (i586)
    • vmware-tools-kmp-xen (i586, x86_64)
    • vmware-tools-kmp-xenpae (i586)

SUSE Linux Enterprise Server 11:

  • ibm-lin_tape-1.24.0_2.6.27.21_0.1-0.1 (i586, x86_64, SRPM)
    • ibm-lin_tape-kmp-debug (i586, x86_64)
    • ibm-lin_tape-kmp-default (i586, x86_64)
    • ibm-lin_tape-kmp-pae (i586)
    • ibm-lin_tape-kmp-trace (i586)
    • ibm-lin_tape-kmp-vm (i586, x86_64)
  • vmware-tools-kmp-3.5.0_153875_2.6.27.21_0.1-0.1 (SRPM)
    • vmware-tools-kmp-debug (i586, x86_64)
    • vmware-tools-kmp-default (i586, x86_64)
    • vmware-tools-kmp-pae (i586)
    • vmware-tools-kmp-trace (i586, x86_64)
    • vmware-tools-kmp-vmi (i586)
    • vmware-tools-kmp-xen (i586, x86_64)