Today, I had a rather troublesome morning. Once I got to work, Nagios was already complaining about the lin_taped on one of our TSM servers, which apparently failed due to too many SCSI resets. Additionally, I can’t login using the VE console (I can login however using SSH) so I ended up opening up a IBM Electronic Service Call (ESC+).
Using SSH, I can get some information on the VE’s status:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
vetapeservice@VTL-A:~> sudo /ve/bin/ve status IBM VE for Tape Server v3.00 (Build 1465) Copyright (c) 2001-2008 FalconStor Software. All Rights Reserved. Status of VE for Tape SNMPD Module [RUNNING] Status of VE for Tape Configuration Module [RUNNING] Status of VE for Tape Base Module [RUNNING] Status of VE for Tape HBA Module [RUNNING] Status of VE for Tape Authentication Module [RUNNING] Status of VE for Tape Server (Compression) Module [RUNNING] Status of VE for Tape Server (Hifn HW Compression) Module [RUNNING] Status of VE for Tape Server (Application Upcall) Module [RUNNING] Status of VE for Tape Server (FSNBase) Module [RUNNING] Status of VE for Tape Server (Upcall) Module [RUNNING] Status of VE for Tape Server (Application) Module [RUNNING] Status of VE for Tape Server (Application IOCTL) Module [RUNNING] Status of VE for Tape Server (User) [STOPPED] Status of VE for Tape Target Module [RUNNING] Status of VE for Tape Server IMA Daemon [RUNNING] Status of VE for Tape Server RDE Daemon [RUNNING] Status of VE for Tape Communication Module [RUNNING] Status of VE for Tape Logger Module [RUNNING] Status of VE for Tape Call Home Module [RUNNING] Status of VE for Tape Local Client (VBDI) [RUNNING] Status of VE for Tape Self Monitor Module [RUNNING] Status of VE for Tape Failover Module [RUNNING] vetapeservice@VTL-B:~> sudo /ve/bin/ve status IBM VE for Tape Server v3.00 (Build 1465) Copyright (c) 2001-2008 FalconStor Software. All Rights Reserved. Status of VE for Tape SNMPD Module [RUNNING] Status of VE for Tape Configuration Module [RUNNING] Status of VE for Tape Base Module [RUNNING] Status of VE for Tape HBA Module [RUNNING] Status of VE for Tape Authentication Module [RUNNING] Status of VE for Tape Server (Compression) Module [RUNNING] Status of VE for Tape Server (Hifn HW Compression) Module [RUNNING] Status of VE for Tape Server (Application Upcall) Module [RUNNING] Status of VE for Tape Server (FSNBase) Module [RUNNING] Status of VE for Tape Server (Upcall) Module [RUNNING] Status of VE for Tape Server (Application) Module [RUNNING] Status of VE for Tape Server (Application IOCTL) Module [RUNNING] Status of VE for Tape Server (User) [RUNNING] Status of VE for Tape Target Module [RUNNING] Status of VE for Tape Server IMA Daemon [RUNNING] Status of VE for Tape Server RDE Daemon [RUNNING] Status of VE for Tape Communication Module [RUNNING] Status of VE for Tape Logger Module [RUNNING] Status of VE for Tape Call Home Module [RUNNING] Status of VE for Tape Local Client (VBDI) [RUNNING] Status of VE for Tape Self Monitor Module [RUNNING] Status of VE for Tape Failover Module [RUNNING] |
After looking a bit deeper, it seems that none of the two TSM server is able to see the IBMchanger devices for the first VTL. The second is perfectly visible, just not the first. After putting both VE nodes into suspended failover, gathering support data for the IBM support from both VE’s and the Brocade SAN switches, apparently everything works again. I guess the library does have “self healing” properties.