Tivoli Storage Manager Server 5.5.3

I spent yesterday afternoon upgrading our TS7530, and in my fad I also upgraded TSM to 5.5.3. Now, once I started TSM it quickly started complaining about the paths to the drives.

I thought maybe this is a mere device problem (we have had them before), so I rebooted the boxes. But still no luck and I went home after about an hour of trying without any luck. In the morning, my co-worker called our trustworthy IBM service partner, and the TSM consultant said he had the exact, same problem yesterday. We would have two options:

  1. Enable the option SANDISCOVERY, with the (completely undocumented) Passive setting (setopt SANDISCOVERY PASSIVE)
  2. Downgrade back to 5.5.2

For now, we implemented the first option, in the hope that’ll solve our troubles. And it actually does.

Mass-updating Tivoli Storage Manager drive status

I was fighting with our VTL again, and TSM was thinking all the drives were offline. In order to update the drive status, you’d need to go into the ISC and select each drive and set them to ONLINE. Since I’m a bit click-lazy, I wrote a simple nested for-loop, which gives me the output to update all the drives at once:

Result is a list like this:

The same goes for mass-updating the path status:

Result is a list like this:

TSM: Restoring the database/recovery log to a point-in-time

Well, my co-worker just called on my cell (it’s Friday, 16:00), and asked me which start-up script he needed to change in order to restore the database. My first response was, “ummm, that’s gonna be hard, we’re using heartbeat”.

Okay, so after a bit of asking I got out of him what he wanted to achieve by changing the start-up script. Apparently he did something to crash Tivoli Storage Manager (or rather repeatedly crash it) and wanted to restore the database. He talked to one of the systems partner we do have (and I’m happy we have them most of the time), who in return told him how to do it, but forgot a minute after he hung up the phone.

So, I went digging while he still was telling me how he got Tivoli to kick his own ass … After a bit, I thought “hrrrrrm, shouldn’t this be covered in the Tivoli documentation ?”, and surprisingly it’s actually covered in the documentation.

It’s actually rather simple.

  1. Stop the dsmserv Linux-HA cluster service (tsm-control ha stop tsm1)
  2. Setup the environment (since we’re running multiple instances of Tivoli Storage Manager – export DSMSERV_DIR, export DSMSERV_CONFIG)
  3. Enter the path of the server
  4. Run dsmserv restore db
  5. Wait some time (took about half an hour to restore the 95G database and the 10G recovery log)
  6. Start the dsmserv Linux-HA cluster service (tsm-control ha start tsm1)
  7. Update the server-to-server communication, since the restore db changes the communication verification token

TSM client: Backing up files with umlauts on SLES

In the past, I always had problems with SLES and our Tivoli Storage Manager client’s when backing up files with german umlauts. Well, today I looked a bit harder, and quite quickly found a solution.

As you can see from the above, SLES9/10 ain’t setting LANG or LC_ALL (which I searched for first), but is setting LC_CTYPE.

So, simply changing the LC_CTYPE in the init-script and/or prepending the dsmc command line with a new LC_CTYPE fixes my umlauts problems!

Well, I had a long’ish talk with one of my trustworthy IBM senior consultants the day after writing this …

He told me something along the lines of this:

If you would like to back up files with names containing characters with a code > 127 please ensure that you have chosen a SBCS character set for your locale. The default code page C or the code page POSIX supports characters up to 127 only. Files whose names contain special characters will be skipped if C or POSIX is used. It is strongly recommended to perform a system backup by using a SBCS character set to prevent any file or directory from being skipped. This behavior for different locales is intended.

And this:

The UTF-8 locale is default on some Linux platforms. However, TSM Client currently does not support running under UTF-8 locales (such as en_US.UTF-8 and ja_JP.UTF-8). Export your LANG and LC_ALL environment variables to the iso8859-1 or EUC versions of your locale and then start a new xterm (or mlterm) session prior to running TSM Client.

That basically means, at least for using the TSM Client Java Interface (dsmj) and the scheduler/client acceptor daemon you have to switch your locales to something _not_ UTF-8 capable.

He also mentioned, that IBM doesn’t have a real solution for this problem, as well that there is no real workaround. You need to invest some time into figuring out the “right” locale setting for your system(s), since after writing the above I came to the result that it ain’t enough ..

You need to do the following:

After doing so, the scheduler and the command-line client works …

Updating path information for TSM

As I did some switching today (between the new lin_tape version by IBM and our own lin_tape version), I ended up writing those lines a dozen times. Here is (just for me, if you don’t care .. skip ahead) on how to generate a list of commands:

which should get you a list like this:

Tivoli Storage Manager Client and Microsoft Cluster Services

Well, I just had another look at our client scheduler services on our Microsoft Cluster. A while back we noticed that those scheduler services were going nuts after some time. Well, as it turns out, I can tell why. Microsoft Cluster Services have a feature called registration replication, which replicates a given key, if changed when the resource is online, to all connected cluster nodes.

Now, we added the obvious registry key to the settings of our cluster resources for the scheduler services (SOFTWAREIBMADSMCurrentVersionBackupClientNodes<TSM NODE NAME>) and the scheduler service would use the same registry key to store it’s passwords. But it seems we were far off with that assumption.

The scheduler service uses another registry key, it’s quite similar to the one the GUI is using, but it’s different enough (SOFTWAREIBMADSMCurrentVersionNodes<TSM NODE NAME>).

Setting up Linux-HA

Well, initially I thought writing the OCF resource agent for Tivoli Storage Manager was the hard part. But as it turns out, it really ain’t. The hard part, is getting the resources into the heartbeat agent (or whatever you wanna call it). The worst part about it, is that the hb_gui is completely worthless if you want to do a configuration without quorum.

First of all, we need to setup the main Linux-HA configuration file (/etc/ha.d/ha.cf). Configuring that, is rather simple. For me, as I do have two network devices, over which both nodes see each other (one is an adapter bond of comprising of two simple, plain, old 1G copper ports; the other is the 1G fibre cluster port), the configuration looks like this:

After configuring the service itself is done, one just needs to start the heartbeat daemon on both nodes. Afterwards, we should be able to configure the cluster resources.

I find it particularly easier to just update the corresponding sections with cibadmin (the man-page really has some good examples). So here are my configuration files for two resource groups (crm_mon doesn’t difference between resources and grouped resources, it’ll just show you that you configured two resources).
Read More