Beware of Backup Tape Drive Shoe-Shining
In the area of disk to disk backups, we have customers using tapes for archival or retention purposes.
With fast, modern tape drives the following symptoms may occur more often:
- Poor Tapes
- Bad Drives
- Stuck Tapes
- Dirty Drives
- Cleaning cartridges expiring quickly
- Backups failing with no clear reason
- Vendors blaming other vendors
You may have issues even through you did your implementation right, nothing has changed, drives were replaced, no common pattern, all tapes are new and all firmware, drivers and versions are up-to-date.
These are the symptoms of chronic issue commonly known as shoe-shining or back-hitching effect. This problem is more common in the modern area of faster tape drives. This is a very interesting performance issue.
Backup tape drives like LTO4, LTO5 and LTO6 have a buffer to keep the data that it is writing. A bottleneck somewhere in the path from NAS / SAN / Disk storage to the tape drive may not keep up with the tape drive writing speed, the buffer repeatedly empties causing the tape drive to stop and rewind. This process repeats at an alarming rate causing this shoe-shining or back-hitching effect.
A lot more details can be given why and how it is happening but I will focus only on the symptoms.
- Poor Tapes - Due to wear and tear caused by shoe shining
- Bad Drives - Mechanical failure caused by shoe shining
- Stuck Tapes - Mechanical failure caused by shoe shining
- Dirty Drives - Residue from tapes due to shoe shining
- Cleaning cartridges expiring quickly - Auto cleaning is performed more often due to dirty drives caused by shoe shining.
- Backups failing with no clear reason - The end of file (EOF) could not be written due to dirt / poor tape / failed drive causing the backup to fail abruptly and mark tape as poor.
- Vendors blaming other vendors - Backup software and hardware are not equipped to log performance issues caused by slow network/fiber/disk. Thus vendors blame each other.
Lesson learned: Just getting faster drives does not necessarily provide faster backups. You have to keep up with speed on your production and backup systems and network, as well.
Would you like help with backups and retention? Let's Talk.
Author: Khaja Moizuddin
Equilibrium IT Solutions, Inc.